Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Building a REST API with JAXB, Spring Boot and Spring Data

Project JAXB If someone asked you to develop a REST API on the JVM, which frameworks would you use? I was recently tasked with such a project. My client asked me to implement a REST API to ingest requests from a 3rd party. The project entailed consuming XML requests, storing the data in a database, then exposing the data to internal application with a JSON endpoint. Finally, it would allow taking in a JSON request and turning it into an XML request back to the 3rd party.

With the recent release of Apache Camel 2.14 and my success using it, I started by copying my Apache Camel / CXF / Spring Boot project and trimming it down to the bare essentials. I whipped together a simple Hello World service using Camel and Spring MVC. I also integrated Swagger into both. Both implementations were pretty easy to create (sample code), but I decided to use Spring MVC. My reasons were simple: its REST support was more mature, I knew it well, and Spring MVC Test makes it easy to test APIs.

Camel's Swagger support without web.xml
As part of the aforementioned spike, I learned out how to configure Camel's REST and Swagger support using Spring's JavaConfig and no web.xml. I made this into a sample project and put it on GitHub as camel-rest-swagger.

This article shows how I built a REST API with Java 8, Spring Boot/MVC, JAXB and Spring Data (JPA and REST components). I stumbled a few times while developing this project, but figured out how to get over all the hurdles. I hope this helps the team that's now maintaining this project (my last day was Friday) and those that are trying to do something similar.

[Read More]

Posted in Java at Oct 29 2014, 05:52:37 AM MDT 2 Comments

Developing with AngularJS - Part III: Services

This is the 3rd article in a series on my experience developing with AngularJS. I used AngularJS for several months to create a "My Dashboard" feature for a client and learned a whole bunch of Angular goodness along the way. For previous articles, please see Part I: The Basics and Part II: Dialogs and Data.

Angular offers several ways to interact with data from the server. The easiest way is to use the $resource factory, which lets you interact with RESTful server-side data sources. When we started the My Dashboard project, we were hoping to interact with a REST API, but soon found out that it didn't have all the data we needed. Rather than loading the page and then making another request to get its data, we decided to embed the JSON in the page. For communication back to the server, we used our tried-and-true Ajax solution: DWR.

In Angular-speak, services are singletons that carry out specific tasks common to web apps. In other words, they're any $name object that can be injected into a controller or directive. However, as a Java Developer, I tend to think of services as objects that communicate with the server. Angular's documentation on Creating Services shows you various options for registering services. I used the angular.Module api method.

[Read More]

Posted in The Web at Jun 25 2013, 07:03:26 AM MDT 10 Comments

Secure JSON Services with Play Scala and SecureSocial

AntwerpTownSquare Last November, I traveled to Antwerp to speak at Devoxx. After my talk on HTML5 with Play Scala, Mattias Karlsson approached me and we had a chat about doing the same talk at Jfokus in Stockholm. I agreed and we began talking details after Trish and I returned to the US.

Jfokus

I wrote this article on a plane between Denver and Seattle and will be hopping over the North Pole to Stockholm via Iceland tonight. For the past couple of weeks, I've been updating my Play More! HTML5/mobile app to add some new features. Most notably, I wanted to upgrade to Play 2.0, create JSON services and add authentication.

Upgrading to Play 2.0
My attempt to upgrade to Play 2.0 involved checking out the source from GitHub, building and installing the RC1 snapshot. As I tried to upgrade my app and started getting failed imports, I turned to the internet (specifically StackOverflow) to see if it was a good idea. The first answer for that question suggested I stay with 1.x.

If it's a critical project, to be finished before next March 2012, I would go with Play 1.x. If it's a less important project, which could be delayed, and that in any case won't be released before March 2012, try Play 2.0.

While I didn't plan on releasing Play More! before Jfokus, I decided upgrading didn't add a whole lot to the talk. Also, I couldn't find a Play Scala 0.9.1 to Play 2.0 upgrade guide and I didn't have enough time to create one. So I decided to stick with Play 1.2.4 and add some JSON services for my iPhone client.

JSON Servers
I found Manuel Bernhardt's Play! Scala and JSON. This led me to Jerkson, built by the now infamous @coda. I was able to easily get things working fairly quickly and wrote the following WorkoutService.scala:

package controllers.api

import play.mvc.Controller
import models._
import com.codahale.jerkson.Json._

object WorkoutService extends Controller {

  def workouts = {
    response.setContentTypeIfNotSet("application/json")
    generate(Workout.find().list())
  }
  def edit(id: Long) = {
    generate(Workout.byIdWithAthleteAndComments(id))
  }

  def create() = {
    var workout = params.get("workout", classOf[Workout])
    Workout.create(workout)
  }

  def save(id: Option[Long]) = {
    var workout = params.get("workout", classOf[Workout])
    Workout.update(workout)
  }

  def delete(id: Long) = {
    Workout.delete("id={id}").on("id" -> id).executeUpdate()
  }
}

Next, I added routes for my new API to conf/routes:

GET     /api/workouts               api.WorkoutService.workouts
GET     /api/workout/{id}           api.WorkoutService.edit
POST    /api/workout                api.WorkoutService.create
PUT     /api/workout/{id}           api.WorkoutService.save
DELETE  /api/workout/{id}           api.WorkoutService.delete

Then I created an ApiTest.scala class that verifies the first method works as expected.

import play.test.FunctionalTest
import play.test.FunctionalTest._
import org.junit._

class ApiTests extends FunctionalTest {
  
    @Test
    def testGetWorkouts() {
        var response = GET("/api/workouts");
        assertStatus(200, response);
        assertContentType("application/json", response)
        println(response.out)
    }
}

I ran "play test", opened my browser to http://localhost:9000/@tests and clicked ApiTests -> Start to verify it worked. All the green made me happy.

Play More API Tests

Finally, I wrote some CoffeeScript and jQuery to allow users to delete workouts and make sure delete functionality worked.

$('#delete').click ->
  $.ajax
    type: 'POST'
    url: $(this).attr('rel')
    error: ->
      alert('Delete failed, please try again.')
    success: (data) ->
      location.href = "/more"

I was very impressed with how easy Play made it to create JSON services and I smiled as my CoffeeScript skills got a refresher.

The Friday before we left for Devoxx, I saw the module registration request for SecureSocial.

SecureSocial with Play Scala
From SecureSocial's README:

SecureSocial allows you to add an authentication UI to your app that works with services based on OAuth1, OAuth2, OpenID and OpenID+OAuth hybrid protocols.

It also provides a Username and Password mechanism for users that do not wish to use existing accounts in other networks.

The following services are supported in this release:

  • Twitter (OAuth1)
  • Facebook (OAuth2)
  • Google (OpenID + OAuth Hybrid)
  • Yahoo (OpenID + OAuth Hybrid)
  • LinkedIn (OAuth1)
  • Foursquare (OAuth2)
  • MyOpenID (OpenID)
  • Wordpress (OpenID)
  • Username and Password

In other words, it sounded like a dream come true and I resolved to try it once I found the time. That time found me last Monday evening and I sent a direct message to @jaliss (the module's author) via Twitter.

Does Secure Social work with Play Scala? I'd like to use it in my Play More! project.

Jorge responded 16 minutes later saying that he hadn't used Play Scala and he'd need to do some research. At 8 o'clock that night (1.5 hours after my original DM), Jorge had a sample working and emailed it to me. 10 minutes later I was adding a Secure trait to my project.

package controllers

import play.mvc._
import controllers.securesocial.SecureSocial

/*
 * @author Jorge Aliss <[email protected]> of Secure Social fame.
 */
trait Secure {
  self: Controller =>

  @Before def checkAccess() {
    SecureSocial.DeadboltHelper.beforeRoleCheck()
  }

  def currentUser = {
    SecureSocial.getCurrentUser
  }
}

I configured Twitter and Username + Password as my providers by adding the following to conf/application.conf.

securesocial.providers=twitter,userpass

I also had to configure a number of securesocial.twitter.* properties. Next, I made sure my routes were aware of SecureSocial by adding the following to the top of conf/routes:

  *       /auth               module:securesocial

Then I specified it as a dependency in conf/dependencies.yml and ran "play deps".

    - play -> securesocial 0.2.4

After adding "with Secure" to my Profile.scala controller, I tried to access its route and was prompted to login. Right off the bat, I was shown an error about a missing jQuery 1.5.2 file in my "javascripts" folder, so I added it and rejoiced when I was presented with a login screen. I had to add the app on Twitter to use its OAuth servers, but I was pumped when both username/password authentication worked (complete with signup!) as well as Twitter.

The only issue I ran into with SecureSocial was that it didn't find the default implementation of SecureSocial's UserService.Service when running in prod mode. I was able to workaround this by adding a SecureService.scala implementation to my project and coding it to talk to my Athlete model. I didn't bother to hook in creating a new user when they logged in from Twitter, but that's something I'll want to do in the future. I was also pleased to find out customizing SecureSocial's views was a breeze. I simply copied them from the module into my app's views and voila!

package services

import play.db.anorm.NotAssigned
import play.libs.Codec
import collection.mutable.{SynchronizedMap, HashMap}
import models.Athlete
import securesocial.provider.{ProviderType, UserService, SocialUser, UserId}

class SecureService extends UserService.Service {
  val activations = new HashMap[String, SocialUser] with SynchronizedMap[String, SocialUser]

  def find(userId: UserId): SocialUser = {
    val user = Athlete.find("email={email}").on("email" -> userId.id).first()

    user match {
      case Some(user) => {
        val socialUser = new SocialUser
        socialUser.id = userId
        socialUser.displayName = user.firstName
        socialUser.email = user.email
        socialUser.isEmailVerified = true
        socialUser.password = user.password
        socialUser
      }
      case None => {
        if (!userId.provider.eq(ProviderType.userpass)) {
          var socialUser = new SocialUser
          socialUser.id = userId
          socialUser
        } else {
          null
        }
      }
    }
  }

  def save(user: SocialUser) {
    if (find(user.id) == null) {
      val firstName = user.displayName
      val lastName = user.displayName
      Athlete.create(Athlete(NotAssigned, user.email, user.password, firstName, lastName))
    }
  }

  def createActivation(user: SocialUser): String = {
    val uuid: String = Codec.UUID()
    activations.put(uuid, user)
    uuid
  }

  def activate(uuid: String): Boolean = {
    val user: SocialUser = activations.get(uuid).asInstanceOf[SocialUser]
    var result = false

    if (user != null) {
      user.isEmailVerified = true
      save(user)
      activations.remove(uuid)
      result = true
    }

    result
  }

  def deletePendingActivations() {
    activations.clear()
  }
}

Jorge was a great help in getting my authentication needs met and he even wrote a BasicAuth.scala trait to implement Basic Authentication on my JSON services.

package controllers

import _root_.securesocial.provider.{UserService, ProviderType, UserId}
import play._
import play.mvc._
import play.libs.Crypto

import controllers.securesocial.SecureSocial

/*
 * @author Jorge Aliss <[email protected]> of Secure Social fame.
 */
trait BasicAuth {
  self: Controller =>

  @Before def checkAccess = {
    if (currentUser != null) {
      // this allows SecureSocial.getCurrentUser() to work.
      renderArgs.put("user", currentUser)
      Continue
    }

    val realm =
      Play.configuration.getProperty("securesocial.basicAuth.realm", "Unauthorized")

    if (request.user == null || request.password == null) {
      Unauthorized(realm)
    } else {
      val userId = new UserId
      userId.id = request.user
      userId.provider = ProviderType.userpass
      val user = UserService.find(userId)

      if (user == null ||
        !Crypto.passwordHash(request.password).equals(user.password)) {
        Unauthorized(realm)
      } else {
        // this allows SecureSocial.getCurrentUser() to work.
        renderArgs.put("user", user)
        Continue
      }
    }
  }

  def currentUser = {
    SecureSocial.getCurrentUser()
  }
}

Summary
My latest pass at developing with Scala and leveraging Play to build my app was a lot of fun. While there were issues with class reloading every-so-often and Scala versions with Scalate, I was able to add the features I wanted. I wasn't able to upgrade to Play 2.0, but I didn't try that hard and figured it's best to wait until its upgrade guide has been published.

I'm excited to describe my latest experience to the developers at Jfokus this week. In addition, the conference has talks on Play 2.0, CoffeeScript, HTML5, Scala and Scalate. I hope to attend many of these and learn some new tricks to improve my skills and my app.

Update: The Delving developers have written an article on Migration to Play 2. While it doesn't provide specific details on what they needed to change, it does have good information on how long it took and things to watch for.

Posted in Java at Feb 12 2012, 04:02:43 PM MST 4 Comments

Integrating OAuth with AppFuse and its REST API

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth 

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

AppFuse WS Homepage

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Posted in Java at Jul 05 2011, 10:56:48 AM MDT 4 Comments

How's the ol' Team Doing?

Back in March, I wrote about How We Hired a Team of 10 in 2 Months:

This week, we on-boarded 3 of our final 4 developers. I breathed a big sigh of relief that the hiring was over and we could get back to slinging code and making things happen. As luck would have it, I received an e-mail from my boss on Tuesday that the hiring engine is starting up again and we need to hire 6 more developers. While I'm not anxious to start the Hiring Engine again, I am glad to know it works well and it has helped us build a great team.

We never ended up hiring those additional 6 developers, but we've had quite a ride since March. One of the first commenters on my original post wrote:

I hope that whirlwind of hiring works out for you Matt. But, don't you think it's a bit early to be declaring success?

At that time, we still had a lot of work to do to become a successful team. However, by the end of March, we'd finished our first deliverable - converting a somewhat slow ColdFusion/jQuery/Video webapp into a fast Java/jQuery/Video webapp. The slowness wasn't due to ColdFusion, but mostly performance, caching and YSlow-type optimizations. At that time, we surprised the folks that were in charge of our app. They didn't think we'd finish so fast and it took them awhile to decide what to do with our work.

While we were waiting for the product folks to launch our app in April/May, we decided to experiment with developing some new clients. So we wrote an iPhone app, an Android App, an iPhone/Android/iPad/HTML5 version of our webapp and a Blu-ray client. All of these applications used our webapp's backend RESTful services and we were able to learn the SDKs and implement all the apps in a matter of weeks. In May, we demoed all our clients and got rave reviews from executives. We celebrated that afternoon with a big sigh of relief.

The glowing from our many-clients demo was short-lived. A week later, we were asked to enhance our iPad app to include TV-Remote type features, namely channel-changing and DVR functionality. After freaking out and trying to figure out how to deliver such an app in a week, the demo was rescheduled and we were afforded 2 weeks to build the app. After much frantic development, we were able to complete the app in time and the demo was published to YouTube a couple months later.

In June, if you asked me if we were a successful team, I would've definitely said "Yes!" We'd been asked to develop apps for 3 different demos and we delivered on-time.

The remainder of June and July we slipped into a bit of limbo where we weren't asked to develop anything, but simply maintain and enhance the stuff we'd already developed. After a few weeks of doing this, several of us began to wonder if the apps we'd developed would ever see the light of day. We expressed this concern to our VP and a new idea was hatched: The 60-Day Push.

The 60-Day Push was designed to eliminate politics and meetings and allow us to develop a 3-screen (PC, iPad, TV) experience for our video content. We decided to aim high and try to complete most of the work in 30-days, so we could do executive demos of our progress. We started all our applications from scratch, split into Services, Portal, TV and iPad teams and worked with Method to implement a slick design for all our apps. I'm proud to say we delivered yet again and there were many proud moments as we demoed to the top executives of the company.

In early September, after doing several demos, we were approved to launch and we've been working towards that goal ever since. We feel we have several weeks of work to coax our DemoWare into RealWorldWare, but the momentum is there and the end of the tunnel is in sight.

As a further sign of our success, we're moving into a new office in LoDo next week. This also means the End of an Era, where the Raible Designs' office across from Forest Room 5 will cease to exist. We (Goodwill, Scotty, Country Bry and I) first moved into this office while working for Evite in April 2009. It's been a fantastic location, a facilitator of extensive collaboration and host to many dart games, FACs, bird-dog spottings and way too many Mom Jokes.

As part of our transition, I'll be looking for renters to fill out the rest of my lease (through March 2010). If you're looking for a sweet location for 5-6 people in Denver's Highlands (near the REI store downtown), please let me know.

Also, there's a good chance my team will continue to grow as we move our apps into production and start ramping up for millions of users. If you're a strong Web or Java developer with social skills and a Ditchdigger attitude, I'd love to hear from you. We probably won't be hiring until January, but that doesn't mean we can't start talking now.

Posted in Java at Oct 21 2010, 09:00:41 AM MDT 5 Comments

Developing Rich Web Service APIs with Java

This afternoon, I attended Ryan Heaton's talk on Developing Rich Web Service APIs with Java. I've always admired Ryan's work and what he's done with Enunciate. Below are my notes from his talk.

We've come a long way from the WS (SOAP) <-> EJB days. There are many tools that you can use to develop web services today. A Web Service is an API that's accessible over a network via platform-independent protocol. Historical examples of web services (in chronological order):

  • CORBA
  • DCOM
  • RMI
  • SOAP
  • REST

A Web Service is composed of code wrapped with a container and it's bound to a contract. The table below shows the types of web services and their equivalent Java standard.

StandardTechnology
SOAP JAX-WS
RESTJAX-RS
XMLJAXB
JSON???

JAXB is the standard technology used to marshal XML <-> Java. It's annotation-driven, using @XmlRootElement, @XmlElement and @XmlAttribute to translate Java properties to XML.

JAX-WS is a Java standard, outputs classes as SOAP endpoints and it's annotation-driven. @WebService and @WebMethod are the main annotations. @WebMethod is only needed if you're trying to expose a non-public method.

JAX-RS is a Java standard used to expose REST services. Not surprisingly, it's annotation-driven. The class itself needs to be annotated with @Path. Methods are annotated with @GET, @PUT, @POST or @DELETE. If you need to pass in a parameter to a method, you can add a path parameter like the following:

@Path("/{id}")
@GET
public Person readPerson(@PathParam("id") String id);

To specify the possible mime-type outputs of a method, you can use the @Produces annotation. For example:

@Produces({"application/xml", "application/json"})

You can also do this for input methods with the @Consumes annotation.

For JSON, there's no Java standard. However, there's a number of libraries available, including Jackson, Jettison, GSON, XStream. Personally, I've used Jackson and highly recommend it.

The major players in exposing Java code as JAX-WS and JAX-RS services are Oracle, Spring and JBoss. Oracle (formerly Sun) has implemented the JAX-WS reference implementation and it's called Metro (a.k.a. JAX-WS RI). For JAX-RS, Oracle has the Jersey project. For Spring, the JAX-WS and JAX-RS implementation is Apache CXF. JBoss has JBoss-WS and RESTEasy.

There's also a number of custom containers for exposing web services. For example, AMF (implementations: BlazeDS, GraniteDS), GWT-RPC (GWT), JSON-PRC (DWR), Atom (Abdera), OpenSocial (Shindig).

The contract is a very important part of a web service API. It's composed of an Interface Definition Language (WSDL, WADL JSON Schema), Developer Docs (text, context, preconditions, restrictions), example code and client libraries.

Enunciate allows you to create your contract for your web services. It's a build-time WS Development tool. It generates developer documentation and client-side libraries (for Java, C/C++, Objective C, ActionScript and Ruby). It leverages existing specs (JAX-*) and fails fast at compile time.

From there, Ryan gave us a demo of Enunciate and how it could easily create an API website based on JAX-RS annotations.

I liked Ryan's talk and I'm definitely a fan of Enunciate. While I didn't learn anything new, I think there's a lot of Java developers that don't know about the various standards and how easy it is to develop web services. Hopefully by taking notes from Ryan's talk, I'll get the word out a bit more and make more folks aware of Enunciate. On a related note, Sonatype has a good post on how they documented the Nexus API with Enunciate.

Posted in Java at Mar 18 2010, 05:51:00 PM MDT 1 Comment

The Cloud Computing Continuum with Bob McWhirter

This afternoon, I sat in a Keynote by Bob McWhirter at TSSJS 2010. Bob is the Chief Architect of Cloud Computing at Red Hat. Bob is "The Despot" at Codehaus, helped start Groovy and founded Drools. Below are my notes from his talk.

The cloud is not an either/or question. A scorched Earth strategy is not necessary to be in the cloud. The cloud is a continuum of technologies, many of those that you're already familiar with. You're used to web, messaging and storage. You can add just a little bot of cloud to your application at a time.

So what is the cloud?

It's the next logical step in application delivery.

How did we get to the cloud? First, we jammed a lot of servers into a closet (back in the .com days). That's a lot of stuff to deal with if all you want to do is deliver an internet-based service. Then we did some colo. The problem with colo was that it was still our stuff. Then we leased managed servers. With the cloud, we lease what appears to be servers (or services). This is a path we've all been taking to abstract the annoying bits away from hosting an internet-based service.

The typical diagram of a cloud has IaaS, PaaS and PaaS. IaaS abstracts away hardware. PaaS abstracts away services. SaaS abstracts away software.

Tenants of the cloud:

  • The illusion of infinite resources.
  • On-demand acquisition and provision.
  • Someone else's responsibility.

Magicians provide the illusion of pulling a card out of your ear, but they don't actually do it. Because there are limits of what we can do. With on-demand requisitioning, there's no more purchase orders. There's no more waiting for delivery. No waiting for IT to rack it. Or install it. By having someone else manage your hardware, it's not your problem. There's clear lines of responsibility. This intentionally limits visibility into the implementation. Hopefully it's best-of-breed.

The Cloud is really just a mindset. Virtualization makes it affordable to make mistakes. Repeatedly. Each service is managed as an autonomous entity. Services are designed for scalability.

Architecture 101 give us a presentation layer, a business logic layer and a persistence layer. You can scale it by putting many instances on many servers. However, the truth is that most apps are a big ball of mud. You need to get your application in order, then figure out which parts really needs to scale. Is the web tier really where we have trouble? Painting HTML isn't all that hard.

You don't have to put everything in the cloud. The cloud is not a revolution. We still need containers for web things, messaging things, storage of things and other things. Containers still exist, they're just in the cloud.

From container to cloud
When you have an application and a database, there's not much to abstract away into the cloud. However, as soon as you notice your DB is slow, you'll add caching. Once you hit your machine's memory limit, you discover distributed caching systems. Once you've done this, you'll discover that the network isn't all that slow, especially compared to spinning disks. Then you realize that you don't even need the storage and you can keep everything in cache. From there, you move to a RESTful caching service. Then you call it a NoSQL Service so you can be buzzword-compliant.

As far as caching services, there's many available, but a lot of them are startups. However, the cloud does not have to mean external 3rd-party providers. Remember DBAs? It was their job to view the database as a service and to protect it. There's a good chance that we'll end up with DBAs for services. By turning caching into an independent service, it becomes subject to economies of scale.

The cloud is a mindset. It's an SOA approach that allows groups to specialize and optimize for scale.

So how do you implementing a local cloud? We already have great caching technologies. Bob, with his JBoss hat on, recommends Infinispan and its REST API. To get around network latency, you can still run your cloud on your LAN. You're essentially trading disk traffic for network traffic. You can use many hands (e.g. MapReduce) to make things happen quickly.

Besides caching, there's a number of other services that can be put in the cloud. For example:

  • Messaging (REST-MQ
  • Scheduling
  • Security & Identity (OASIS, etc)
  • Computation (query & otherwise)
  • Transactions (REST-TX)
  • Business Process Management
  • Telephony (VOIP, SMS)

Your application becomes a stack of services, instead of a stack of software.

Higher-order business-logic services are also subject to cloudification. Ultimately, going into the cloud is not rocket science. The cloud does not turn everything on its head. We don't have to wait for new companies to develop new technologies. Startups may be the best consumers of the cloud, but might not be the best providers. We'll let the startups assume the risks for now. Until the 3rd party services have been proven, it's probably best to build your own cloud of services.

More than anything, the cloud has caused a return to fundamentals. Once your app is a collection of services, you can easily pick and choose what pieces you want to put in the cloud. This is where we're headed, whether you like it or not. As more apps become global-scape, you're going to run into these issues. By adopting a cloud mindset and architecture, your app can be prepared for scalability issues.

For more communication with Bob and his Cloud Evangelism, checkout StormGrind or follow him on Twitter.

Personally, I really enjoyed Bob's talk. It was by far the best keynote today, mostly because he told a story and did it elegantly with a nice-looking presentation. Not only that, but he provided the audience with a practical view of the cloud and ways that we can start using it in our applications today.

Update: You can find the slides from this session on SlideShare.

Posted in Java at Mar 17 2010, 04:20:26 PM MDT 1 Comment

Developing and Testing GWT Client Services

Earlier this week, Hiram Chirino released RestyGWT, a GWT generator for REST services and JSON encoded data transfer objects. You can read more about it in Hiram's post RestyGWT, a Better GWT RPC??. First of all, I'm impressed with RestyGWT because provides something I've always wanted with GWT: the ability to call RESTful services and get a populated POJO in your callback, much like AsyncCallback provides for RPC services.

RestyGWT also allows you to easily create services using only interfaces and JAX-RS annotations. For example:

import javax.ws.rs.POST;
...
public interface PizzaService extends RestService {
    @POST
    public void order(PizzaOrder request, MethodCallback<OrderConfirmation> callback);
}

After taking a brief look at RestyGWT, I thought it'd be interesting to share how I develop and test GWT client services.

Developing GWT Client Services
Writing services in a GWT application can be helpful when you're using MVP, especially since you can EasyMock them in a test. On my GWT projects, I've often used overlay types because they allow me to write less code and they make parsing JSON super simple. I've had issues testing my presenters when using overlay types. The good news is I think I've figured out a reasonable solution, but it does require using GWTTestCase. If RestyGWT supported overlay types, there's a good chance I'd use it, especially since its integration tests seem to require GWTTestCase too.

Rather than using callbacks in my presenters, I try to only use them in my service implementations. That way, my presenters don't have to worry about overlay types and can be tested in a JUnit-only fashion. The callbacks in my services handle JSON parsing/object population and fire events with the populated objects.

GWT's RequestBuilder is one option for communicating with RESTful services. The Development Guide for HTTP Requests explains how to use this class. To simplify REST requests and allow multiple callbacks, I'm using a RestRequest class, and a number of other utility classes that make up a small GWT REST framework (created by a former colleague). RestRequest wraps RequestBuilder and provides a Fluent API for executing HTTP requests. Another class, Deferred, is a GWT implementation of Twisted's Deferred.

As part of my service implementation, I inject an EventBus (with GIN) into the constructor and then proceed to implement callbacks that fire Events to indicate loading, saving and deleting has succeeded. Here's an example service:

public class ConversationServiceImpl implements ConversationService {
    private EventBus eventBus;

    @Inject
    public ConversationServiceImpl(EventBus eventBus) {
        this.eventBus = eventBus;
    }

    public void getConversation(String name) {
        Deferred<Representation> d =
                RestRequest.get(URLs.CONVERSATION + "/" + URL.encode(name)).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceLoadedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void saveConversation(Conversation conversation) {
        Deferred<Representation> d = RestRequest.post(URLs.CONVERSATION)
                .setRequestData(conversation.toJson()).build();
        
        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceSavedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void deleteConversation(Long id) {
        Deferred<Representation> d =
                RestRequest.post(URLs.CONVERSATION + "/" + id).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                eventBus.fireEvent(new ResourceDeletedEvent());
            }
        });

        d.run();
    }

    /**
     * Convenience method to populate object in one location
     *
     * @param result the result of a resource request.
     * @return the populated object.
     */
    private Conversation convertResultToConversation(Representation result) {
        JSOModel model = JSOModel.fromJson(result.getData());
        return new Conversation(model);
    }
}

In the saveConversation() method you'll notice the conversation.toJson() method call. This method uses a JSON class that loops through an objects properties and constructs a JSON String.

public JSON toJson() {
    return new JSON(getMap());
}

Testing Services
In my experience, the hardest part about using overlay types is writing your objects so they get populated correctly. I've found that writing tests which read JSON from a file can be a great productivity boost. However, because of overlay types, you have to write a test that extends GWTTestCase. When using GWTTestCase, you can't simply read from the filesystem. The good news is there is a workaround where you can subclass GWTShellServlet and overwrite GWT's web.xml to have your own servlet that can read from the filesystem. A detailed explanation of how to do this was written by Alex Moffat in Implementing a -noserver flag for GWTTestCase.

Once this class is in place, I've found you can easily write services using TDD and the server doesn't even have to exist. When constructing services, I've found the following workflow to be the most productive:

  1. Create a file with the expected JSON in src/test/resources/resource.json where resource matches the last part of the URL for your service.
  2. Create a *ServiceGwtTest.java and write tests.
  3. Run tests to make sure they fail.
  4. Implement the service and run tests to ensure JSON is getting consumed/produced properly to/from model objects.

Below is the code for my JsonReaderServlet.java:

public class JsonReaderServlet extends GWTShellServlet {

    public void service(ServletRequest servletRequest, ServletResponse servletResponse)
            throws ServletException, IOException {

        HttpServletRequest req = (HttpServletRequest) servletRequest;
        HttpServletResponse resp = (HttpServletResponse) servletResponse;

        String uri = req.getRequestURI();
        if (req.getQueryString() != null) {
            uri += "?" + req.getQueryString();
        }

        if (uri.contains("/services")) {
            String method = req.getMethod();
            String output;

            if (method.equalsIgnoreCase("get")) {
                // use the part after the last slash as the filename
                String filename = uri.substring(uri.lastIndexOf("/") + 1, uri.length()) + ".json";
                System.out.println("loading: " + filename);
                String json = readFileAsString("/" + filename);
                System.out.println("loaded json: " + json);
                output = json;
            } else {
                // for posts, return the same body content
                output = getBody(req);
            }

            PrintWriter out = resp.getWriter();
            out.write(output);
            out.close();

            resp.setStatus(HttpServletResponse.SC_OK);
        } else {
            super.service(servletRequest, servletResponse);
        }
    }

    private String readFileAsString(String filePath) throws IOException {
        filePath = getClass().getResource(filePath).getFile();
        BufferedReader reader = new BufferedReader(new FileReader(filePath));
        return getStringFromReader(reader);
    }

    private String getBody(ServletRequest request) throws IOException {
        BufferedReader reader = new BufferedReader(new InputStreamReader(request.getInputStream()));
        return getStringFromReader(reader);
    }

    private String getStringFromReader(Reader reader) throws IOException {
        StringBuilder sb = new StringBuilder();
        char[] buf = new char[1024];
        int numRead;
        while ((numRead = reader.read(buf)) != -1) {
            sb.append(buf, 0, numRead);
        }
        reader.close();
        return sb.toString();
    }
}

This servlet is mapped to <url-pattern>/*</url-pattern> in a web.xml file in src/test/resources/com/google/gwt/dev/etc/tomcat/webapps/ROOT/WEB-INF.

My Service Test starts by getting an EventBus from GIN and registering itself to handle the fired events.

public class ConversationServiceGwtTest extends AbstractGwtTestCase
        implements ResourceLoadedEvent.Handler, ResourceSavedEvent.Handler, ResourceDeletedEvent.Handler {
    ConversationService service;
    ResourceLoadedEvent<Conversation> loadedEvent;
    ResourceSavedEvent<Conversation> savedEvent;
    ResourceDeletedEvent deletedEvent;

    @Override
    public void gwtSetUp() throws Exception {
        super.gwtSetUp();
        DesigntimeGinjector injector = GWT.create(MyGinjector.class);
        EventBus eventBus = injector.getEventBus();
        service = new ConversationServiceImpl(eventBus);
        eventBus.addHandler(ResourceLoadedEvent.ENGINE, this);
        eventBus.addHandler(ResourceSavedEvent.ENGINE, this);
        eventBus.addHandler(ResourceDeletedEvent.ENGINE, this);
    }

    @SuppressWarnings("unchecked")
    public void onLoad(ResourceLoadedEvent event) {
        this.loadedEvent = event;
    }

    @SuppressWarnings("unchecked")
    public void onSave(ResourceSavedEvent event) {
        this.savedEvent = event;
    }

    public void onDelete(ResourceDeletedEvent event) {
        this.deletedEvent = event;
    }
}

After this groundwork has been done, a test can be written that loads up the JSON file and verifies the objects are populated correctly.

public void testGetConversation() {

    service.getConversation("test-conversation");

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceLoadedEvent not received", loadedEvent);
            Conversation conversation = loadedEvent.getResource();
            assertEquals("Conversation name is incorrect","Test Conversation", conversation.getName());

            assertNotNull("Conversation has no channel", conversation.getChannel());
            assertEquals("Conversation has incorrect task size", 3, conversation.getTasks().size());

            convertToAndFromJson(conversation);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

private void convertToAndFromJson(Conversation fromJsonModel) {
    Representation json = fromJsonModel.toJson();
    assertNotNull("Cannot convert empty JSON", json.getData());

    // change back into model
    JSOModel data = JSOModel.fromJson(json.getData());
    Conversation toJsonModel = new Conversation(data);
    verifyModelBuiltCorrectly(toJsonModel);
}

private void verifyModelBuiltCorrectly(Conversation model) {
    assertEquals("Conversation name is incorrect", "Test Conversation", model.getString("name"));
    assertEquals("Conversation has incorrect task size", 3, model.getTasks().size());
    assertEquals("Conversation channel is incorrect", "Web", model.getChannel().getString("type"));
}

For more information on the usage of the Timer, finishTest() and delayTestFinish(), see GWTTestCase's javadoc.

The tests for saving and deleting a resource look as follows:

public void testSaveConversation() {
    Conversation conversation = new Conversation().setName("Test").setId("1");

    List<Task> tasks = new ArrayList<Task>();
    for (int i = 1; i < 4; i++) {
        tasks.add(new Task().setName("Task " + i));
    }
    conversation.setTasks(tasks);

    System.out.println("conversation.toJson(): " + conversation.toJson());

    assertTrue(conversation.toJson().toString().contains("Task 1"));

    service.saveConversation(conversation);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceSavedEvent not received", savedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}
  
public void testDeleteConversation() {
    service.deleteConversation(1L);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceDeletedEvent not received", deletedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

Summary
This article has shown you how I develop and test GWT Client Services. If RestyGWT supported overlay types, there's a good chance I could change my service implementation to use it and I wouldn't have to change my test. Robert Cooper, author of GWT in Practice, claims he has a framework that does this. Here's to hoping this article stimulates the GWT ecosystem and we get a GWT REST framework that's as easy to use as GWT RPC.

Update: Today I enhanced this code to use Generics-based classes (inspired by Don't repeat the DAO!) for the boiler-plate CRUD code in a service. In a nutshell, a service interface can now be written as:

public interface FooService extends GenericService<Foo, String> {
 
}

The implementation class is responsible for the URL and converting the JSON result to an object:

public class FooServiceImpl extends GenericServiceImpl<Foo, String> implements FooService {

    @Inject
    public FooServiceImpl(EventBus eventBus) {
        super(eventBus, "/services/foo");
    }

    @Override
    protected Foo convertResultToModel(Representation result) {
        return new Foo(JSOModel.fromJson(result.getData()));
    }
}

I'm sure this can be further enhanced to get rid of the need to create classes altogether, possibly leveraging GIN or some sort of factory. The parent classes referenced in this code can be viewed at the following URLs:

There's also a GenericServiceGwtTest.java that proves it all works as expected.

Posted in Java at Oct 21 2009, 06:55:17 AM MDT 6 Comments

My Experience with Java REST Frameworks (specifically Jersey and CXF)

Recently I was tasked with developing a server-side REST strategy for a client. When I started working with them, they were using GWT RPC for exposing services. They wanted to move to RESTful services to allow for a more open platform, that multiple types of clients could talk to. There were interested in supporting SOAP and GWT RPC as well, but it wasn't required. They are using Spring, a custom namespace (for easily creating remote services) and an HTML documentation generator to expose their API in human readable form.

When I first starting developing, I chose to try Enunciate. From Enunciate's homepage:

Enunciate is an engine for creating, maintaining, and deploying your rich Web service API on the Java platform. If that sounds complicated, it's not. All you have to do is define your service interfaces in Java source code. ... Then invoke Enunciate.

Sounds pretty sweet, eh? At first glance, the things I liked about Enunciate were:

  1. The ability to generate multiple endpoints (and clients).
  2. Generates nice-looking documentation.
  3. Allows selecting different frameworks (for example, CXF instead of JAX-WS RI).

Initially, the hardest part of using Enunciate was integrating it into my project. This was somewhat related to having a multi-module project, but moreso related to getting the settings right in my enunciate.xml file. After getting everything working, I encountered a few Spring wiring issues with the GWT Endpoints and with Jersey's Spring support.

The good news is I believe most of these issues were related to my project and how it proxies Spring beans that Jersey couldn't find. Jersey's Spring support only supports annotations at this time, so I was unable to tell it the proxied bean names via XML (or even its own annotations). I'm sure this problem is solvable, but after struggling with it for a day or two, I decided to give up on Enunciate. I had to get something working, and fast.

At this point, I was back to the drawing board. I knew there were plenty of good Java REST frameworks available, but Spring and CXF (for SOAP) were already available dependencies in my project, so I chose that route. I was able to get something up and running fairly quickly with this tutorial and CXF's JAX-RS documentation. I ran into an Invalid JSON Namespace issue, but was able to solve it by adding a custom namespace.

It's not all rosy though, there are still a couple CXF issues I haven't solved:

While CXF does take a bit of XML for each service, it's kinda slick in that it only requires you to annotate your service interfaces. It also generates documentation for your services (WSDL and WADL), but it's not as pretty as Enunciate's. To solve this, I added Enunciate for documentation only. To make Enunciate work, I did have to add some annotations to my service implementation classes and parse the generated HTML to fix some links to CXF's services, but ultimately it works pretty well.

In the end, I recommended my client not use Enunciate for generating endpoints. This was primarily related to their unique Spring configuration, but also because I was able to easily use the same classes for REST, SOAP and GWT Endpoints. However, I will continue to keep my eye on Enunciate. It could be very useful for the upcoming appfuse-ws archetype. I'm also eager to see better GWT support and the ability to generate Overlay types in future releases.

Posted in Java at Aug 27 2009, 02:20:51 PM MDT 15 Comments

Enhancing Evite.com with GWT and Grails

Evite.com On my LinkedIn Profile, it says my current gig is a SOFEA consultant at a stealth-mode startup.

SOFEA Consultant, Stealth Mode Startup, Los Angeles, CA. December 2008 -- Present.

OK, I lied. It's not a startup, it's a well-known company that helps you plan parties. For the last 5+ months, my UI team from LinkedIn has been working with Evite.com to enhance portions of their site with a SOFEA architecture.

In January, we started evaluating Ajax Frameworks and came to the conclusion that GWT was right for us. After we chose the UI framework, other team members chose Grails and memcached to develop scalable RESTful services. The architecture we implemented involves using GWT's RequestBuilder to talk to Grails' services, which cache almost all their JSON output in memcached.

To see an example of a feature we developed with GWT, see Evite's Design Gallery. I personally worked on this feature and very much enjoyed becoming a GWT aficionado in the process. GWT's zero-turnaround feature made doing pure client-side work a lot of fun. It's definitely something I'd like to continuing doing at my next gig.

Everyone from Evite is very happy with what we've been able to do with GWT and Grails. We have a stateless architecture and are quickly able to develop both client-side and server-side features. We've learned to scale the client by using out-of-the-box GWT components. We've scaled Grails by caching as much as possible. We serve up Ads and Analytics using the same JavaScript mechanisms that traditional server-side frameworks use.

At the end of this month, my gig with Evite comes to an end. I'll be spending a few weeks at my family's cabin in Montana and then it's on to the next big thing. What's the next big thing? I'm not sure yet, hence the reason for writing this. If you're looking to develop a GWT application, introduce a SOFEA architecture at your company, or simply adopt some open source frameworks, I'd love to help out. Drop me a line and let's start a conversation.

Posted in Java at Jun 15 2009, 07:41:37 AM MDT 9 Comments