Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Concurrency on the JVM Using Scala with Venkat Subramaniam

This evening, I attending the Denver JUG where Venkat Subramaniam was speaking about Scala. Unfortunately, I arrived halfway through his Programming Scala talk and didn't get a chance to learn as much as I wanted to. What I did see made Scala look very powerful and (possibly) easier to learn than Java. Below are my notes from Venkat's talk.

Concurrency is important these days because we're in a world of multiple processors. When you have multiple threads running at one time, it can become painful. Before Java, you had to learn the API for multi-threading for each different platform. With Java's "Write once, debug everywhere", you only had to learn one API. Unfortunately, it's pretty low level: how to start a thread, manage it, stop it, etc. You also have to remember where to put synchronize in your code.

With Scala, immutability and its Actors make it easy to program concurrent systems. For example, here's a web service that retrieves stock prices in sequential order:

def getyearEndClosing(symbol : String, year : Int) = {
  val url = "http://ichart.finance.yahoo.com/table.csv?s=" + 
    symbol + "&a=11&b=01&c" + year + "&d=11&e=31&f=" + year + "&g=m"
  val data = io.Source.fromURL(url).mkString
  val price = data.split("\n")(1).split(",")(4).toDouble
  Thread.sleep(1000); // slow down internet
  (symbol, price)
}

val symbols = List("APPL", "GOOG", "IBM", "JAVA", "MSFT")

val start = System.nanoTime

val top = (("", 0.0) /: symbols) { (topStock, symbol) => 
  val (sym, price) = getYearEndClosing(symbol, 2008)

  if (topStock._2 < price) (sym, price) else topStock
}

val end = System.nanoTime

println("Top stock is " + top._1 + " with price " + top._2)
println("Time taken " + (end - start)/10000000000.0)

To make this concurrent, we create Actors. Actors are nothing but Threads with a built-in message queue. Actors allow spawning separate threads to retrieve each stock price. Instead of doing:

symbols.foreach { symbol => 
  getYearEndClosing(symbol, 2008)
}

You can add actors:

val caller = self

symbols.foreach { symbol => 
  actor { caller ! getYearEndClosing(symbol, 2008) }
}

Then remove val (sym, price) = getYearEndClosing(symbol, 2008) and replace it with:

  receive {
    case(sym: String, price: Double) =>
      if (topStock._2 < price) (sym, price) else topStock
  }

After making this change, the time to execute the code dropped from ~7 seconds to ~2 seconds. Also, since nothing is mutable in this code, you don't have to worry about concurrency issues.

With Scala, you don't suffer the multiple-inheritance issues you do in Java. Instead you can use Traits to do mixins. For example:

import scala.actors._
import Actor._

class MyActor extends Actor {
  def act() {
    for(i <- 1 to 3) {
    receive {
      case msg => println("Got " + msg)
    }
  }
}

When extending Actor, you have to call MyActor.start to start the Actor. Writing actors this way is not recommended (not sure why, guessing because you have to manually start them).

Venkat is now showing an example that counts prime numbers and he's showing us how it pegs the CPU when counting how many exist between 1 and 1 million (78,499). After adding actor and receive logic, he shows how his Activity Monitor shows 185% CPU usage, indicating that both cores are being used.

What happens when one of the threads crashes and burns? The receive will wait forever. Because of this, using receive is a bad idea. It's much better to use receiveWithin(millis) to set a timeout. Then you can catch the timeout in the receiveWithin block using:

case TIMEOUT => println("Uh oh, timed out")

A more efficient way to use actors is using react instead of receive. With react, threads leave after putting the message on the queue and new threads are started to execute the block when the message is "reacted" to. One thing to remember with react is any code after the react block will never be executed. Just like receiveWithin(millis), you can use reactWithin(millis) to set a timeout.

The major thing I noticed between receive and react is Venkat often had to change the method logic to use react. To solve this, you can use loop (or better yet, loopWhile(condition)) to allow accessing the data outside the react block. In conclusion, reactWithin(millis) is best to use, unless you need to execute code after the react block.

Conclusion
This was a great talk by Venkat. He used TextMate the entire time to author and execute all his Scala examples. Better yet, he never used any sort of presentation. All he had was a "todo" list with topics (that he checked off as he progressed) and a sample.scala file.

Personally, I don't plan on using Scala in the near future, but that's mostly because I'm doing UI development and GWT and JavaScript are my favorite languages for that. On the server-side, I can see how it reduces the amount of Java you need to write (the compiler works for you instead of you working for the compiler). However, my impression is its sweet spot is when you need to easily author an efficient concurrent system.

If you're looking to learn Scala, I've heard Scala by Example (PDF) is a great getting-started resource. From there, I believe Programming in Scala and Venkat's Programming Scala are great books.

Posted in Java at Sep 09 2009, 09:12:48 PM MDT 8 Comments

My Experience with Java REST Frameworks (specifically Jersey and CXF)

Recently I was tasked with developing a server-side REST strategy for a client. When I started working with them, they were using GWT RPC for exposing services. They wanted to move to RESTful services to allow for a more open platform, that multiple types of clients could talk to. There were interested in supporting SOAP and GWT RPC as well, but it wasn't required. They are using Spring, a custom namespace (for easily creating remote services) and an HTML documentation generator to expose their API in human readable form.

When I first starting developing, I chose to try Enunciate. From Enunciate's homepage:

Enunciate is an engine for creating, maintaining, and deploying your rich Web service API on the Java platform. If that sounds complicated, it's not. All you have to do is define your service interfaces in Java source code. ... Then invoke Enunciate.

Sounds pretty sweet, eh? At first glance, the things I liked about Enunciate were:

  1. The ability to generate multiple endpoints (and clients).
  2. Generates nice-looking documentation.
  3. Allows selecting different frameworks (for example, CXF instead of JAX-WS RI).

Initially, the hardest part of using Enunciate was integrating it into my project. This was somewhat related to having a multi-module project, but moreso related to getting the settings right in my enunciate.xml file. After getting everything working, I encountered a few Spring wiring issues with the GWT Endpoints and with Jersey's Spring support.

The good news is I believe most of these issues were related to my project and how it proxies Spring beans that Jersey couldn't find. Jersey's Spring support only supports annotations at this time, so I was unable to tell it the proxied bean names via XML (or even its own annotations). I'm sure this problem is solvable, but after struggling with it for a day or two, I decided to give up on Enunciate. I had to get something working, and fast.

At this point, I was back to the drawing board. I knew there were plenty of good Java REST frameworks available, but Spring and CXF (for SOAP) were already available dependencies in my project, so I chose that route. I was able to get something up and running fairly quickly with this tutorial and CXF's JAX-RS documentation. I ran into an Invalid JSON Namespace issue, but was able to solve it by adding a custom namespace.

It's not all rosy though, there are still a couple CXF issues I haven't solved:

While CXF does take a bit of XML for each service, it's kinda slick in that it only requires you to annotate your service interfaces. It also generates documentation for your services (WSDL and WADL), but it's not as pretty as Enunciate's. To solve this, I added Enunciate for documentation only. To make Enunciate work, I did have to add some annotations to my service implementation classes and parse the generated HTML to fix some links to CXF's services, but ultimately it works pretty well.

In the end, I recommended my client not use Enunciate for generating endpoints. This was primarily related to their unique Spring configuration, but also because I was able to easily use the same classes for REST, SOAP and GWT Endpoints. However, I will continue to keep my eye on Enunciate. It could be very useful for the upcoming appfuse-ws archetype. I'm also eager to see better GWT support and the ability to generate Overlay types in future releases.

Posted in Java at Aug 27 2009, 02:20:51 PM MDT 15 Comments

The 2009 Rich Web Experience

Rich Web Experience 2009 Late last year, I decided to take a year off from speaking at conferences. My reason was simple: I wanted to hunker down and do some learning. Ever since November 2007, I'd been interested in getting into developing SOFEA applications. This year, I was able to accomplish that and I've learned a lot about GWT and RESTful architectures in the process. Now I'm happy to announce I'll be sharing my experiences at this year's Rich Web Experience.

I look forward to speaking on the following topics:

The first one might look familiar since I used the same title at OSCON 2008. The difference is this time I plan to dive much deeper into the frameworks and help the audience learn more. I also plan to re-visit developing with Flex and Rails in the near future. If there's anything in particular you'd like to see covered in the above talks, please leave a comment.

This is sure to be a fun conference. First of all, it's pretty close to Disney World. A few years back, TSS hosted the "Java In Action" conference in Disney World and while the conference didn't do too well, it was awesome being there with my family. I plan on taking my kids and having some fun when I'm not speaking. Secondly, this conference is the first to offer all-inclusive pricing for attendees - registration, flight (continental U.S.) and 3 nights lodging. This should allow you to get approval without having to jump through the traditional travel hoops. Lastly, the JSF Summit will be held concurrently and you get access to it at no additional cost.

So come on down to Florida in December, listen to all the great speakers, have some fun with your family and look me up for a beer or two. You know it'll be a good time! ;-)

Posted in Java at Aug 11 2009, 10:16:24 PM MDT 1 Comment

My attempt at browser-based username/password autocomplete with GWT

Last night, I did a quick spike to try and implement username/password autocomplete in my GXT application. By "autocomplete", I don't mean Ajax-style autocomplete, but rather browser-based autocomplete. The best information I found on doing this is in the following post:

http://osdir.com/ml/GoogleWebToolkit/2009-04/msg01838.html

I didn't use this technique (wrapping an existing form with a FormPanel) because I'm using GXT and didn't want to lose the look-and-feel of my login form.

I was successful in getting everything to work in Firefox (it populates both the username and password). In IE, it only populates the username, not the password. In Safari/Chrome, it doesn't work at all. Here's how I did it:

1. Created a hidden HTML form on my HTML page that embeds GWT.

<form method="post" action="javascript:void(0)" style="display: none">
    <input type="text" id="username" name="username" value=""/>
    <input type="password" id="password" name="password" value=""/>
    <input type="submit" value="Login" id="login"/>
</form>

2. When a user clicks on the "Login" button in my GWT application, populate the fields in this hidden form and "click" on the Login button (which will do nothing since the action="javascript:void(0)").

// Set the hidden fields to trigger the browser to remember
DOM.getElementById("username").setAttribute("value", username.getValue());
DOM.getElementById("password").setAttribute("value", password.getValue());
clickFormLogin();

...

public static native void clickFormLogin() /*-{
$doc.getElementById("login").click();
}-*/;

This works in Firefox 3.5 and prompts me to save the user/pass at the top of the screen. Why doesn't this work in Safari/Chrome? My guess is because the form's action doesn't go anywhere and the form is not submitted. If I change the action to be an actual URL and show the form, clicking on the form's Login button will save it in those browsers.

Grabbing the actual populated values when the screen loads did turn out to be a bit tricky. With IE, I was able to grab the username, but not the password. It seems that keyboard or mouse interaction (tabbing off or focusing/blurring) with the username field is necessary to get the password field populated. In Firefox, I was unable to grab the values if I did it straight away. The solution turned out to be wrapping everything in a Timer and waiting a 1/2 second.

// grab user/pass from browser and hidden login form (if user has stored them)
// runs 1/2 second after page loads so cursor goes at end of username and works in Firefox
Timer t = new Timer() {
    public void run() {
        username.setValue(getElementValue("username"));
        password.setValue(getElementValue("password"));
        if (username.getValue() != null) {
            password.focus();
        }
        validate();
    }
};

t.schedule(500);

...

public static native String getElementValue(String domId) /*-{
    return $doc.getElementById(domId).value;
}-*/;

While I'm glad I got it working in Firefox, I'm disappointed with IE's lack of password autocompletion. More than anything, I can't help but think there's a way to make this work in WebKit-based browsers. If you've successfully implemented cross-browser username/password autocomplete in GWT, please share your experience. If you share your experience about successfully doing it in Safari/Chrome, I might even buy you a beer or two. ;-)

Posted in Java at Aug 07 2009, 08:09:11 AM MDT 2 Comments

Integrating GWT with Spring Security

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
 

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

Posted in Java at Aug 06 2009, 08:50:15 AM MDT 10 Comments

How to do cross-domain GWT RPC with a ProxyServlet

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Posted in Java at Aug 05 2009, 04:06:12 PM MDT 17 Comments

The good ol' Job Hunt

Just over two years ago, I wrote about the good ol' job hunt. Today, I find myself in a very similar situation. My Evite gig ends in a couple hours and I'm heading to The Cabin on Monday for a month of vacation. Similar to last time, there are opportunities out there, but most of them come through recruiters.

I have to admit - it seems like I got lucky the last couple times I tried to find a gig. I simply blogged I was looking and found myself negotiating with companies a few days later. Getting laid off from LinkedIn was awesome in that a few companies contacted me about hiring the whole team the next day. The ability to blog-and-get-a-gig was a great way to cut out the middle-man.

Unfortunately, I think I got spoiled by my blog-and-get-a-gig success. I figured it would happen again this time...

Not so much.

I don't know if it's because of the down economy or because I took the year off from speaking at conferences. Regardless, I'm in an interesting situation since I signed a 1-year lease on an office in downtown Denver. It's easy to market to companies in Denver, but if I land a gig here, there's a good chance the client is going to want me in the office everyday. That leaves me wondering: what's the best way to market to companies outside of Colorado? My ideal contract is one that 1) allows me to work remotely and 2) only requires me to travel once or twice a month.

I believe my ideal gigs are out there, but I think it's difficult to convince companies I'm a good remote worker. In an attempt to convince them otherwise, I'd like to offer recommendations from my last two clients.

"Matt contributed dramatically to the engineering practice at Evite, both directly and indirectly. Directly, he lead the effort to define and prove a successful new web UI architecture for us. Indirectly, he brought senior engineering talent into the organization, and energized the existing team. We are significantly better positioned to deliver on our Product goals as a result of having worked with Matt. I'd love to work with him again someday." David Thomas, VP of Technology, Evite

"Matt has unique abilities in the realm of software engineering. He brings great energy, an amazing breadth of knowledge in UI technologies, along with a can-do attitude that's refreshing to interact with. He is a great leader and communicator. In addition, Matt can apply his deep understanding of the technologies in question pragmatically to the engineering problem at hand, leading his team in execution and delivery. He's an excellent technical visionary to complement any team." Arnold Goldberg, Vice President Platform Engineering, LinkedIn

If you're looking to hire someone with my skills, let me know - there's a good chance you'll be glad you did. ;-)

Posted in Java at Jun 26 2009, 03:30:42 PM MDT 9 Comments

How to use GWT 2.0 with Maven and Generate SOYC Reports

One of the most interesting features coming in GWT 2.0 is code splitting and the ability to use GWT.runAsync() to reduce the size of your application's initial download. This week, I learned how to use GWT 2.0 with my GWT 1.6/Maven project. Below are instructions on how to build and use the latest GWT with Maven.

  • Checkout GWT and setup GWT_TOOLS.
  • Set a GWT_VERSION environment variable to 2.0.0-SNAPSHOT (export GWT_VERSION=2.0.0-SNAPSHOT).
  • Build GWT with the ant command.
  • After building completes, install the GWT artifacts into your local Maven repository using the following commands:
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-user -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-user.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-servlet -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-servlet.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac -Dpackaging=jar -Dfile=build/lib/gwt-dev-mac.jar
    
    mkdir temp
    tar -zxf build/dist/gwt-mac-2.0.0-SNAPSHOT.tar.gz -C temp
    cd temp/gwt-mac-2.0.0-SNAPSHOT
    zip -0 gwt-mac-2.0.0-SNAPSHOT.zip lib*.jnilib
    cd ../..
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac-libs -Dpackaging=zip \
    -Dfile=temp/gwt-mac-2.0.0-SNAPSHOT/gwt-mac-2.0.0-SNAPSHOT.zip
    
    Thanks to Jason for his help with this script.
  • Modify the pom.xml of your GWT project to use the the gwt-maven-plugin from Codehaus. Of course, you'll need to modify the <runTarget> to fit your project.
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>gwt-maven-plugin</artifactId>
        <version>1.1</version>
        <configuration>
            <runTarget>org.appfuse.gwt.mvc.${entry.point}/${entry.point}.html</runTarget>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>compile</goal>
                    <goal>test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
    
  • Modify your dependencies to match the ones below. With the Codehaus plugin, dependencies are much more concise.
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-servlet</artifactId>
        <version>${gwt.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-user</artifactId>
        <version>${gwt.version}</version>
        <scope>provided</scope>
    </dependency>
    
  • Add <gwt.version>2.0.0-SNAPSHOT</gwt.version> to the properties section of your pom.xml.
  • At this point, you should be able to compile your project with mvn gwt:compile and run it in hosted mode using mvn gwt:run.

Generate SOYC Reports
In Google's code splitting documentation, it mentions The Story of Your Compile (SOYC). From the documentation:

To obtain a SOYC report for your application, there are two steps necessary. First, add -soyc to the compilation options that are passed to the GWT compiler. This will cause the compiler to emit raw information about the compile to XML files in an -aux directory beside the rest of the compiled output. In that directory, you will see an XML file for each permutation and a manifest.xml file that describes the contents of all the others.

The second step is to convert that raw information into viewable HTML. This is done with the SoycDashboard tool.

The first step is not currently possible with the gwt-maven-plugin, so I created a patch for it.

If you patch the gwt-maven-plugin and install it locally, make sure and change the version in your pom.xml to 1.2-SNAPSHOT.

To use the SoycDashboard tool, you'll need to install the gwt-soyc-vis.jar.

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-soyc-vis -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-soyc-vis.jar

Now you can generate SOYC reports with mvn gwt:compile -Dgwt.compiler.soyc=true. You can also add <soyc>true</soyc> to the <configuration> section of the gwt-maven-plugin.

The second step (converting the raw information into viewable HTML) is possible using java from the command-line, or by using the exec-maven-plugin. Here's the (lengthy) command-line version:

java -Xmx1024m -cp /Users/mraible/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:/Users/mraible/.m2/repository/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-mac.jar com.google.gwt.soyc.SoycDashboard -resources ~/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar -out target/soyc-report target/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz

In this example, I'm using the files from stories0.xml.gz, dependencies0.xml.gz, splitPoints0.xml.gz. In the soycReport output directory, there's 5 of each these files and I'm not sure what the difference between reports is. Hopefully someone on the GWT team can elaborate. The exec-maven-plugin version is as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <executable>java</executable>
        <arguments>
            <argument>-cp</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:${settings.localRepository}/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-${platform}.jar
            </argument>
            <argument>com.google.gwt.soyc.SoycDashboard</argument>
            <argument>-out</argument>
            <argument>target/soyc-report</argument>
            <argument>-resources</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar
            </argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz</argument>
        </arguments>
    </configuration>
</plugin>

After configuring this plugin in your project, you should be able to run mvn gwt:compile exec:exec and open the generated report (at target/soyc-report/SoycDashboard-index.html). Currently, there doesn't seem to be much documentation on SOYC. Fred Sauer's recent presentation talks a bit about SOYC and GWT.runAsync(), but that's about it.

To figure out how to use GWT 2.0 with Maven, I used my GWT MVC Example project. The first SOYC report I generated said the initial download was 108,967 KB. To integrate GWT.runAsync(), I modified all the project's controllers so their handleEvent() methods changed from this:

public void handleEvent(AppEvent event) {
    onViewHome(event);
}

To this:

public void handleEvent(final AppEvent event) {
    GWT.runAsync(new RunAsyncCallback() {
        public void onFailure(Throwable throwable) {
            Window.alert(throwable.getMessage());
        }

        public void onSuccess() {
            onViewHome(event);
        }
    });
}

When I generated a new SOYC report, the initial download size was reduced to 56,718 KB. Furthermore, I was able to see that my "Leftovers code" consisted of 63,175 KB. I'm sure there's better ways to split my project using GWT.runAsync(), but I'm happy to see I was able to reduce the initial download by 50%.

If you'd like to try GWT 2.0, you can can download my gwt-mvc example project. To build/run this project, you'll need to 1) build and install GWT, 2) patch gwt-maven-plugin and 3) run mvn gwt:compile exec:exec to generate the SOYC report. In an ideal world, the gwt-maven-plugin can be enhanced to generate the SOYC report (rather than using the exec-maven-plugin). In the meantime, I think it's pretty cool that you can try out GWT 2.0 features while they're still being developed.

Posted in Java at Jun 25 2009, 11:45:04 PM MDT 11 Comments

JSON Parsing with JavaScript Overlay Types in GWT

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Posted in Java at Jun 24 2009, 09:52:49 AM MDT 10 Comments

Implementing OAuth with GWT

I've heard about OAuth for quite some time, but never had an opportunity to implement it on a project. For a good explanation of what OAuth is, see its Introduction. Here's an excerpt:

...it allows you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). While OpenID is all about using a single identity to sign into many sites, OAuth is about giving access to your stuff without sharing your identity at all (or its secret parts).

The reason I needed OAuth was to interact with the Google Contacts API. I've always hated how sites make you import all your contacts from Gmail. I wanted to develop a system that'd let you simply read your contacts from Google in real-time.

Since the application I'm working on uses GWT, I chose to implement an OAuth client in GWT. After googling for "gwt oauth", I found two examples. Unfortunately, neither worked out-of-the-box.

The good news is I did manage to create a working solution. The bad news is it only seems to work at random. That's right folks, I created a solution that only works 50% of the time. I'm somewhat embarrassed to post it here, but I also realize the power of open source and community. By sharing, I hope we can find the flaws in my logic and come up with a solution for all GWT applications.

The best project for OAuth libraries seems to be oauth on Google Code. However, you'll notice that there is no JavaScript implementation listed on the homepage. I did look at the Java implementation, but quickly realized it wouldn't be usable in GWT. Therefore, I opted for the JavaScript implementation.

OAuth consists of several steps. The following diagram explains the authentication flow nicely.

OAuth Authentication Flow

In a nutshell, you have to complete the following steps:

  1. Get a token from the service provider.
  2. Redirect user to service provider to grant access and redirect back to application.
  3. Request access token to access protected resources.
  4. Access protected resources and pull/push data.

To access a service provider's OAuth service, you'll likely need to start by registering your application. For Google, OAuth Authentication for Web Applications is an excellent resource. Google's OAuth Playground is a great way to with the Google Data APIs after you've registered.

Now that you know how OAuth works, let's look at how I implemented it with GWT. I started by adding the necessary JavaScript references to my *.gwt.xml file.

<script src="//oauth.googlecode.com/svn/code/javascript/oauth.js"/>
<script src="//oauth.googlecode.com/svn/code/javascript/sha1.js"/>

Next, I needed a way to sign the request. I tried to use Sergi Mansilla's OAuth.java for this, but discovered issues with how the parameters were being written with GWT 1.6. I opted for Paul Donnelly's makeSignedRequest function instead. By adding this to my application's HTML page, I'm able to call it using the following JSNI method:

public native static String signRequest(String key, String secret, String tokenSecret, String url) /*-{
    return $wnd.makeSignedRequest(key, secret, tokenSecret, url);
}-*/;

After the URL is signed, it needs to be sent to the provider to get a request token. To do this, I used GWT's RequestBuilder and created a send() method:

protected void send(RequestCallback cb, String URL) {
    RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL);
    builder.setTimeoutMillis(10000);
    builder.setCallback(cb);
    
    Request req = null;
    try {
        req = builder.send();
    } catch (RequestException e) {
        cb.onError(req, e);
    }
}

If you try this with Google's Request Token URL in GWT's hosted mode, nothing will happen. Compile/browse to Safari and you'll still see nothing. Try it in Firefox and you'll see the following.

SOP Error

To workaround browsers' Same Origin Policy, I added a proxy servlet to send the requests. I started with Jason Edwards's ProxyServlet and modified it to fit my needs. I then registered it in both *.gwt.xml and web.xml.

<servlet path="/google/" class="org.appfuse.gwt.servlet.AlternateHostProxyServlet"/>

Now, before calling the send() method, I replace the start of the URL so the request would be routed through the servlet.

public void getToken(RequestCallback cb) {
    String url = signRequest(provider.getConsumerKey(), 
                             provider.getConsumerSecret(), 
                             "", provider.getRequestTokenURL());
    url = url.replace("https://www.google.com/", "/google/");
    send(cb, url);
}

When the request returns, I create two cookies by calling a createOAuthCookies() method with the payload returned:

public static String[] createOAuthCookies(String data) {
    String oauth_token = data.substring(data.indexOf("oauth_token=") + 12);
    oauth_token = oauth_token.substring(0, oauth_token.indexOf("&"));

    String oauth_token_secret = data.substring(data.indexOf("oauth_token_secret=") + 19);

    Cookies.setCookie("oauth_token", URL.decode(oauth_token));
    Cookies.setCookie("oauth_token_secret", URL.decode(oauth_token_secret));
    return new String[]{oauth_token, oauth_token_secret};
}

The next step is to authorize the token. This is where things got tricky with my proxy servlet and I had to add some special logic for GWT. Google was sending back a 302 with a Location header, but it wasn't hitting the onResponseReceived() method in my callback. For this reason, I had to change it to a 200 status code and add the redirect location to the body. I also discovered that sometimes they'd return an HTML page with a <meta http-equiv="refresh" ...> tag. When using Twitter, I discovered the full HTML for the allow/deny page was returned. Below is the callback I'm using. WindowUtils is a class I got from Robert Hanson and the gwt-widget project.

public void onResponseReceived(Request request, Response response) {
    String text = response.getText();
    if (response.getStatusCode() == 200 && response.getText().startsWith("http")) {
        WindowUtils.changeLocation(response.getText());
    } else {
        // look for meta-tag that refreshes and grab its URL
        if (text.contains("";
            String url = text.substring(text.indexOf(tokenToStartWith) + tokenToStartWith.length());
            url = url.substring(0, url.indexOf(tokenToEndWith) + tokenToEndWith.length());
            WindowUtils.changeLocation(url);
        } else {
            // Twitter returns a full HTML page, so redirect to the authorize URL manually
            if (provider instanceof Twitter) {
                String url = provider.getAuthorizeTokenURL();
                url = url.replace("$1", OAuthRequest.getAuthToken());
                url = url.replace("$2", DefaultRequest.getCurrentLocation());
                WindowUtils.changeLocation(url);
            } else {
                onError(request, new RequestException(text));
            }
        }
    }
}

public void onError(Request request, Throwable caught) {
    Window.alert("Calling authorize token failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + caught.getMessage());
}

The 3rd step is to get an access token. The most important thing to remember when you do this is to include the "oauth_token_secret" value when signing the request.

signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
            getAuthTokenSecret(), url);

After this completes with a 200, I create the cookies again (since oauth_token and oauth_token_secret are returned in the body), then call the API to get a list of contacts. The ContactsRequests class is responsible for making the call. The DefaultRequest class contains the send() method as well as utility methods to get the cookie values of the oauth tokens.

public class ContactsRequest extends DefaultRequest {
    private static final String GOOGLE_CONTACTS_URL = 
        "http://www.google.com/m8/feeds/contacts/default/thin?oauth_token=$1";
    private OAuthProvider provider;

    public ContactsRequest(OAuthProvider provider) {
        this.provider = provider;
    }

    public void getContacts(RequestCallback cb) {
        String url = GOOGLE_CONTACTS_URL.replace("$1", getAuthToken());
        url = signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
                          getAuthTokenSecret(), url);

        String proxiedURLPrefix = "/contacts/";
        // allow for deploying at /gwt-oauth context
        if (WindowUtils.getLocation().getPath().contains("gwt-oauth")) {
            proxiedURLPrefix = "/gwt-oauth" + proxiedURLPrefix;
        }

        url = url.replace("http://www.google.com/", proxiedURLPrefix);

        send(cb, url);
    }
}

If all goes well, the response contains the data you requested and it's used to populate a textarea (at least in this demo application). Of course, additional processing needs to occur to parse/format this data into something useful.

This all sounds pretty useful for GWT applications, right? I believe it does - but only if it works consistently. I sent a message to the OAuth Google Group explaining the issues I've had.

I'm trying to use the JavaScript API to authenticate with OAuth from a GWT application. I've got it working with both Google and Twitter's OAuth implementations. However, it seems to fail to sign the URL at random. In other words, it works 1 out of 3 times. ... Any idea why this could be happening?

I received a response with a cleaner makeSignedRequest() function. I tried it and, unfortunately, it seems to be equally unreliable. I suspect the problem is with the OAuth JavaScript implementation, GWT's interpretation of it, or that OAuth isn't as mature as it appears to be. I'd like to think one of the first two causes the problem.

To make it easier to create a robust example of GWT and OAuth, I created a gwt-oauth project you can download or view online. Please keep in mind the demo is likely to be flakey. If you're persistent and try enough times, it's likely to work. Firefox seems to succeed moreso than Safari or Chrome. If you have any suggestions for improving this example, please let me know.

Posted in Java at Jun 18 2009, 01:59:13 PM MDT 13 Comments