Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

My attempt at browser-based username/password autocomplete with GWT

Last night, I did a quick spike to try and implement username/password autocomplete in my GXT application. By "autocomplete", I don't mean Ajax-style autocomplete, but rather browser-based autocomplete. The best information I found on doing this is in the following post:

http://osdir.com/ml/GoogleWebToolkit/2009-04/msg01838.html

I didn't use this technique (wrapping an existing form with a FormPanel) because I'm using GXT and didn't want to lose the look-and-feel of my login form.

I was successful in getting everything to work in Firefox (it populates both the username and password). In IE, it only populates the username, not the password. In Safari/Chrome, it doesn't work at all. Here's how I did it:

1. Created a hidden HTML form on my HTML page that embeds GWT.

<form method="post" action="javascript:void(0)" style="display: none">
    <input type="text" id="username" name="username" value=""/>
    <input type="password" id="password" name="password" value=""/>
    <input type="submit" value="Login" id="login"/>
</form>

2. When a user clicks on the "Login" button in my GWT application, populate the fields in this hidden form and "click" on the Login button (which will do nothing since the action="javascript:void(0)").

// Set the hidden fields to trigger the browser to remember
DOM.getElementById("username").setAttribute("value", username.getValue());
DOM.getElementById("password").setAttribute("value", password.getValue());
clickFormLogin();

...

public static native void clickFormLogin() /*-{
$doc.getElementById("login").click();
}-*/;

This works in Firefox 3.5 and prompts me to save the user/pass at the top of the screen. Why doesn't this work in Safari/Chrome? My guess is because the form's action doesn't go anywhere and the form is not submitted. If I change the action to be an actual URL and show the form, clicking on the form's Login button will save it in those browsers.

Grabbing the actual populated values when the screen loads did turn out to be a bit tricky. With IE, I was able to grab the username, but not the password. It seems that keyboard or mouse interaction (tabbing off or focusing/blurring) with the username field is necessary to get the password field populated. In Firefox, I was unable to grab the values if I did it straight away. The solution turned out to be wrapping everything in a Timer and waiting a 1/2 second.

// grab user/pass from browser and hidden login form (if user has stored them)
// runs 1/2 second after page loads so cursor goes at end of username and works in Firefox
Timer t = new Timer() {
    public void run() {
        username.setValue(getElementValue("username"));
        password.setValue(getElementValue("password"));
        if (username.getValue() != null) {
            password.focus();
        }
        validate();
    }
};

t.schedule(500);

...

public static native String getElementValue(String domId) /*-{
    return $doc.getElementById(domId).value;
}-*/;

While I'm glad I got it working in Firefox, I'm disappointed with IE's lack of password autocompletion. More than anything, I can't help but think there's a way to make this work in WebKit-based browsers. If you've successfully implemented cross-browser username/password autocomplete in GWT, please share your experience. If you share your experience about successfully doing it in Safari/Chrome, I might even buy you a beer or two. ;-)

Posted in Java at Aug 07 2009, 08:09:11 AM MDT 2 Comments

Integrating GWT with Spring Security

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
 

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

Posted in Java at Aug 06 2009, 08:50:15 AM MDT 10 Comments

How to do cross-domain GWT RPC with a ProxyServlet

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Posted in Java at Aug 05 2009, 04:06:12 PM MDT 17 Comments

How to use GWT 2.0 with Maven and Generate SOYC Reports

One of the most interesting features coming in GWT 2.0 is code splitting and the ability to use GWT.runAsync() to reduce the size of your application's initial download. This week, I learned how to use GWT 2.0 with my GWT 1.6/Maven project. Below are instructions on how to build and use the latest GWT with Maven.

  • Checkout GWT and setup GWT_TOOLS.
  • Set a GWT_VERSION environment variable to 2.0.0-SNAPSHOT (export GWT_VERSION=2.0.0-SNAPSHOT).
  • Build GWT with the ant command.
  • After building completes, install the GWT artifacts into your local Maven repository using the following commands:
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-user -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-user.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-servlet -Dversion=2.0.0-SNAPSHOT \
    -Dpackaging=jar -Dfile=build/lib/gwt-servlet.jar
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac -Dpackaging=jar -Dfile=build/lib/gwt-dev-mac.jar
    
    mkdir temp
    tar -zxf build/dist/gwt-mac-2.0.0-SNAPSHOT.tar.gz -C temp
    cd temp/gwt-mac-2.0.0-SNAPSHOT
    zip -0 gwt-mac-2.0.0-SNAPSHOT.zip lib*.jnilib
    cd ../..
    
    mvn install:install-file -DgroupId=com.google.gwt \
    -DartifactId=gwt-dev -Dversion=2.0.0-SNAPSHOT \
    -Dclassifier=mac-libs -Dpackaging=zip \
    -Dfile=temp/gwt-mac-2.0.0-SNAPSHOT/gwt-mac-2.0.0-SNAPSHOT.zip
    
    Thanks to Jason for his help with this script.
  • Modify the pom.xml of your GWT project to use the the gwt-maven-plugin from Codehaus. Of course, you'll need to modify the <runTarget> to fit your project.
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>gwt-maven-plugin</artifactId>
        <version>1.1</version>
        <configuration>
            <runTarget>org.appfuse.gwt.mvc.${entry.point}/${entry.point}.html</runTarget>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>compile</goal>
                    <goal>test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
    
  • Modify your dependencies to match the ones below. With the Codehaus plugin, dependencies are much more concise.
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-servlet</artifactId>
        <version>${gwt.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>com.google.gwt</groupId>
        <artifactId>gwt-user</artifactId>
        <version>${gwt.version}</version>
        <scope>provided</scope>
    </dependency>
    
  • Add <gwt.version>2.0.0-SNAPSHOT</gwt.version> to the properties section of your pom.xml.
  • At this point, you should be able to compile your project with mvn gwt:compile and run it in hosted mode using mvn gwt:run.

Generate SOYC Reports
In Google's code splitting documentation, it mentions The Story of Your Compile (SOYC). From the documentation:

To obtain a SOYC report for your application, there are two steps necessary. First, add -soyc to the compilation options that are passed to the GWT compiler. This will cause the compiler to emit raw information about the compile to XML files in an -aux directory beside the rest of the compiled output. In that directory, you will see an XML file for each permutation and a manifest.xml file that describes the contents of all the others.

The second step is to convert that raw information into viewable HTML. This is done with the SoycDashboard tool.

The first step is not currently possible with the gwt-maven-plugin, so I created a patch for it.

If you patch the gwt-maven-plugin and install it locally, make sure and change the version in your pom.xml to 1.2-SNAPSHOT.

To use the SoycDashboard tool, you'll need to install the gwt-soyc-vis.jar.

mvn install:install-file -DgroupId=com.google.gwt \
-DartifactId=gwt-soyc-vis -Dversion=2.0.0-SNAPSHOT \
-Dpackaging=jar -Dfile=build/lib/gwt-soyc-vis.jar

Now you can generate SOYC reports with mvn gwt:compile -Dgwt.compiler.soyc=true. You can also add <soyc>true</soyc> to the <configuration> section of the gwt-maven-plugin.

The second step (converting the raw information into viewable HTML) is possible using java from the command-line, or by using the exec-maven-plugin. Here's the (lengthy) command-line version:

java -Xmx1024m -cp /Users/mraible/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:/Users/mraible/.m2/repository/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-mac.jar com.google.gwt.soyc.SoycDashboard -resources ~/.m2/repository/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar -out target/soyc-report target/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz target/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz

In this example, I'm using the files from stories0.xml.gz, dependencies0.xml.gz, splitPoints0.xml.gz. In the soycReport output directory, there's 5 of each these files and I'm not sure what the difference between reports is. Hopefully someone on the GWT team can elaborate. The exec-maven-plugin version is as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <executable>java</executable>
        <arguments>
            <argument>-cp</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar:${settings.localRepository}/com/google/gwt/gwt-dev/2.0.0-SNAPSHOT/gwt-dev-2.0.0-SNAPSHOT-${platform}.jar
            </argument>
            <argument>com.google.gwt.soyc.SoycDashboard</argument>
            <argument>-out</argument>
            <argument>target/soyc-report</argument>
            <argument>-resources</argument>
            <argument>
                ${settings.localRepository}/com/google/gwt/gwt-soyc/2.0.0-SNAPSHOT/gwt-soyc-2.0.0-SNAPSHOT.jar
            </argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/stories0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/dependencies0.xml.gz</argument>
            <argument>${project.build.directory}/extra/org.appfuse.gwt.mvc.MVC/soycReport/splitPoints0.xml.gz</argument>
        </arguments>
    </configuration>
</plugin>

After configuring this plugin in your project, you should be able to run mvn gwt:compile exec:exec and open the generated report (at target/soyc-report/SoycDashboard-index.html). Currently, there doesn't seem to be much documentation on SOYC. Fred Sauer's recent presentation talks a bit about SOYC and GWT.runAsync(), but that's about it.

To figure out how to use GWT 2.0 with Maven, I used my GWT MVC Example project. The first SOYC report I generated said the initial download was 108,967 KB. To integrate GWT.runAsync(), I modified all the project's controllers so their handleEvent() methods changed from this:

public void handleEvent(AppEvent event) {
    onViewHome(event);
}

To this:

public void handleEvent(final AppEvent event) {
    GWT.runAsync(new RunAsyncCallback() {
        public void onFailure(Throwable throwable) {
            Window.alert(throwable.getMessage());
        }

        public void onSuccess() {
            onViewHome(event);
        }
    });
}

When I generated a new SOYC report, the initial download size was reduced to 56,718 KB. Furthermore, I was able to see that my "Leftovers code" consisted of 63,175 KB. I'm sure there's better ways to split my project using GWT.runAsync(), but I'm happy to see I was able to reduce the initial download by 50%.

If you'd like to try GWT 2.0, you can can download my gwt-mvc example project. To build/run this project, you'll need to 1) build and install GWT, 2) patch gwt-maven-plugin and 3) run mvn gwt:compile exec:exec to generate the SOYC report. In an ideal world, the gwt-maven-plugin can be enhanced to generate the SOYC report (rather than using the exec-maven-plugin). In the meantime, I think it's pretty cool that you can try out GWT 2.0 features while they're still being developed.

Posted in Java at Jun 25 2009, 11:45:04 PM MDT 11 Comments

JSON Parsing with JavaScript Overlay Types in GWT

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Posted in Java at Jun 24 2009, 09:52:49 AM MDT 10 Comments

Implementing OAuth with GWT

I've heard about OAuth for quite some time, but never had an opportunity to implement it on a project. For a good explanation of what OAuth is, see its Introduction. Here's an excerpt:

...it allows you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). While OpenID is all about using a single identity to sign into many sites, OAuth is about giving access to your stuff without sharing your identity at all (or its secret parts).

The reason I needed OAuth was to interact with the Google Contacts API. I've always hated how sites make you import all your contacts from Gmail. I wanted to develop a system that'd let you simply read your contacts from Google in real-time.

Since the application I'm working on uses GWT, I chose to implement an OAuth client in GWT. After googling for "gwt oauth", I found two examples. Unfortunately, neither worked out-of-the-box.

The good news is I did manage to create a working solution. The bad news is it only seems to work at random. That's right folks, I created a solution that only works 50% of the time. I'm somewhat embarrassed to post it here, but I also realize the power of open source and community. By sharing, I hope we can find the flaws in my logic and come up with a solution for all GWT applications.

The best project for OAuth libraries seems to be oauth on Google Code. However, you'll notice that there is no JavaScript implementation listed on the homepage. I did look at the Java implementation, but quickly realized it wouldn't be usable in GWT. Therefore, I opted for the JavaScript implementation.

OAuth consists of several steps. The following diagram explains the authentication flow nicely.

OAuth Authentication Flow

In a nutshell, you have to complete the following steps:

  1. Get a token from the service provider.
  2. Redirect user to service provider to grant access and redirect back to application.
  3. Request access token to access protected resources.
  4. Access protected resources and pull/push data.

To access a service provider's OAuth service, you'll likely need to start by registering your application. For Google, OAuth Authentication for Web Applications is an excellent resource. Google's OAuth Playground is a great way to with the Google Data APIs after you've registered.

Now that you know how OAuth works, let's look at how I implemented it with GWT. I started by adding the necessary JavaScript references to my *.gwt.xml file.

<script src="//oauth.googlecode.com/svn/code/javascript/oauth.js"/>
<script src="//oauth.googlecode.com/svn/code/javascript/sha1.js"/>

Next, I needed a way to sign the request. I tried to use Sergi Mansilla's OAuth.java for this, but discovered issues with how the parameters were being written with GWT 1.6. I opted for Paul Donnelly's makeSignedRequest function instead. By adding this to my application's HTML page, I'm able to call it using the following JSNI method:

public native static String signRequest(String key, String secret, String tokenSecret, String url) /*-{
    return $wnd.makeSignedRequest(key, secret, tokenSecret, url);
}-*/;

After the URL is signed, it needs to be sent to the provider to get a request token. To do this, I used GWT's RequestBuilder and created a send() method:

protected void send(RequestCallback cb, String URL) {
    RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL);
    builder.setTimeoutMillis(10000);
    builder.setCallback(cb);
    
    Request req = null;
    try {
        req = builder.send();
    } catch (RequestException e) {
        cb.onError(req, e);
    }
}

If you try this with Google's Request Token URL in GWT's hosted mode, nothing will happen. Compile/browse to Safari and you'll still see nothing. Try it in Firefox and you'll see the following.

SOP Error

To workaround browsers' Same Origin Policy, I added a proxy servlet to send the requests. I started with Jason Edwards's ProxyServlet and modified it to fit my needs. I then registered it in both *.gwt.xml and web.xml.

<servlet path="/google/" class="org.appfuse.gwt.servlet.AlternateHostProxyServlet"/>

Now, before calling the send() method, I replace the start of the URL so the request would be routed through the servlet.

public void getToken(RequestCallback cb) {
    String url = signRequest(provider.getConsumerKey(), 
                             provider.getConsumerSecret(), 
                             "", provider.getRequestTokenURL());
    url = url.replace("https://www.google.com/", "/google/");
    send(cb, url);
}

When the request returns, I create two cookies by calling a createOAuthCookies() method with the payload returned:

public static String[] createOAuthCookies(String data) {
    String oauth_token = data.substring(data.indexOf("oauth_token=") + 12);
    oauth_token = oauth_token.substring(0, oauth_token.indexOf("&"));

    String oauth_token_secret = data.substring(data.indexOf("oauth_token_secret=") + 19);

    Cookies.setCookie("oauth_token", URL.decode(oauth_token));
    Cookies.setCookie("oauth_token_secret", URL.decode(oauth_token_secret));
    return new String[]{oauth_token, oauth_token_secret};
}

The next step is to authorize the token. This is where things got tricky with my proxy servlet and I had to add some special logic for GWT. Google was sending back a 302 with a Location header, but it wasn't hitting the onResponseReceived() method in my callback. For this reason, I had to change it to a 200 status code and add the redirect location to the body. I also discovered that sometimes they'd return an HTML page with a <meta http-equiv="refresh" ...> tag. When using Twitter, I discovered the full HTML for the allow/deny page was returned. Below is the callback I'm using. WindowUtils is a class I got from Robert Hanson and the gwt-widget project.

public void onResponseReceived(Request request, Response response) {
    String text = response.getText();
    if (response.getStatusCode() == 200 && response.getText().startsWith("http")) {
        WindowUtils.changeLocation(response.getText());
    } else {
        // look for meta-tag that refreshes and grab its URL
        if (text.contains("";
            String url = text.substring(text.indexOf(tokenToStartWith) + tokenToStartWith.length());
            url = url.substring(0, url.indexOf(tokenToEndWith) + tokenToEndWith.length());
            WindowUtils.changeLocation(url);
        } else {
            // Twitter returns a full HTML page, so redirect to the authorize URL manually
            if (provider instanceof Twitter) {
                String url = provider.getAuthorizeTokenURL();
                url = url.replace("$1", OAuthRequest.getAuthToken());
                url = url.replace("$2", DefaultRequest.getCurrentLocation());
                WindowUtils.changeLocation(url);
            } else {
                onError(request, new RequestException(text));
            }
        }
    }
}

public void onError(Request request, Throwable caught) {
    Window.alert("Calling authorize token failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + caught.getMessage());
}

The 3rd step is to get an access token. The most important thing to remember when you do this is to include the "oauth_token_secret" value when signing the request.

signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
            getAuthTokenSecret(), url);

After this completes with a 200, I create the cookies again (since oauth_token and oauth_token_secret are returned in the body), then call the API to get a list of contacts. The ContactsRequests class is responsible for making the call. The DefaultRequest class contains the send() method as well as utility methods to get the cookie values of the oauth tokens.

public class ContactsRequest extends DefaultRequest {
    private static final String GOOGLE_CONTACTS_URL = 
        "http://www.google.com/m8/feeds/contacts/default/thin?oauth_token=$1";
    private OAuthProvider provider;

    public ContactsRequest(OAuthProvider provider) {
        this.provider = provider;
    }

    public void getContacts(RequestCallback cb) {
        String url = GOOGLE_CONTACTS_URL.replace("$1", getAuthToken());
        url = signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
                          getAuthTokenSecret(), url);

        String proxiedURLPrefix = "/contacts/";
        // allow for deploying at /gwt-oauth context
        if (WindowUtils.getLocation().getPath().contains("gwt-oauth")) {
            proxiedURLPrefix = "/gwt-oauth" + proxiedURLPrefix;
        }

        url = url.replace("http://www.google.com/", proxiedURLPrefix);

        send(cb, url);
    }
}

If all goes well, the response contains the data you requested and it's used to populate a textarea (at least in this demo application). Of course, additional processing needs to occur to parse/format this data into something useful.

This all sounds pretty useful for GWT applications, right? I believe it does - but only if it works consistently. I sent a message to the OAuth Google Group explaining the issues I've had.

I'm trying to use the JavaScript API to authenticate with OAuth from a GWT application. I've got it working with both Google and Twitter's OAuth implementations. However, it seems to fail to sign the URL at random. In other words, it works 1 out of 3 times. ... Any idea why this could be happening?

I received a response with a cleaner makeSignedRequest() function. I tried it and, unfortunately, it seems to be equally unreliable. I suspect the problem is with the OAuth JavaScript implementation, GWT's interpretation of it, or that OAuth isn't as mature as it appears to be. I'd like to think one of the first two causes the problem.

To make it easier to create a robust example of GWT and OAuth, I created a gwt-oauth project you can download or view online. Please keep in mind the demo is likely to be flakey. If you're persistent and try enough times, it's likely to work. Firefox seems to succeed moreso than Safari or Chrome. If you have any suggestions for improving this example, please let me know.

Posted in Java at Jun 18 2009, 01:59:13 PM MDT 13 Comments

Enhancing Evite.com with GWT and Grails

Evite.com On my LinkedIn Profile, it says my current gig is a SOFEA consultant at a stealth-mode startup.

SOFEA Consultant, Stealth Mode Startup, Los Angeles, CA. December 2008 -- Present.

OK, I lied. It's not a startup, it's a well-known company that helps you plan parties. For the last 5+ months, my UI team from LinkedIn has been working with Evite.com to enhance portions of their site with a SOFEA architecture.

In January, we started evaluating Ajax Frameworks and came to the conclusion that GWT was right for us. After we chose the UI framework, other team members chose Grails and memcached to develop scalable RESTful services. The architecture we implemented involves using GWT's RequestBuilder to talk to Grails' services, which cache almost all their JSON output in memcached.

To see an example of a feature we developed with GWT, see Evite's Design Gallery. I personally worked on this feature and very much enjoyed becoming a GWT aficionado in the process. GWT's zero-turnaround feature made doing pure client-side work a lot of fun. It's definitely something I'd like to continuing doing at my next gig.

Everyone from Evite is very happy with what we've been able to do with GWT and Grails. We have a stateless architecture and are quickly able to develop both client-side and server-side features. We've learned to scale the client by using out-of-the-box GWT components. We've scaled Grails by caching as much as possible. We serve up Ads and Analytics using the same JavaScript mechanisms that traditional server-side frameworks use.

At the end of this month, my gig with Evite comes to an end. I'll be spending a few weeks at my family's cabin in Montana and then it's on to the next big thing. What's the next big thing? I'm not sure yet, hence the reason for writing this. If you're looking to develop a GWT application, introduce a SOFEA architecture at your company, or simply adopt some open source frameworks, I'd love to help out. Drop me a line and let's start a conversation.

Posted in Java at Jun 15 2009, 07:41:37 AM MDT 9 Comments

Creating a Facebook-style Autocomplete with GWT

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Facebook Autocomplete

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

Posted in Java at Jun 05 2009, 07:05:10 AM MDT 25 Comments

Ajax Framework Analysis Results

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

  1. Choose a short list of frameworks to prototype with.
  2. Create an application prototype with each framework.
  3. Document findings and create a matrix with important criteria.
  4. Create presentation to summarize document.
  5. Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Number of Committers

Books on Amazon

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Ranking with equal criteria weights

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Ranking with weighted criteria

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

  • GXT: a GWT version of Ext JS
  • SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.

Posted in Java at Apr 23 2009, 08:34:44 PM MDT 53 Comments

Optimizing a GWT Application with Multiple EntryPoints

Building a GWT application is an easy way for Java Developers to write Ajax applications. However, it can be difficult to release a GWT application to production before it's finished. One of the most important things I've learned in Software Development is to get a new application into production as soon as possible. Not only does getting it from dev → qa → prod verify your process works, it also can do a lot to test the viability of the new application.

One of the biggest issues with GWT applications is size. The project I'm working on compiles Java to JavaScript and creates ~570K *.cache.html files (one for each modern browser). These files end up being around 180K gzipped. I believe this is an OK size for an entire application. However, if you're going to release early, release often with GWT, chances are you'll just want to release one feature at a time.

When the first feature was completed on my project, the *.cache.html files were around 300K. Rather than using branches to release to QA and UAT, bug fixes and new features were developed on trunk. Unfortunately, the QA and UAT process took several weeks longer than expected so by the time the feature was ready to release, the *.cache.html files had grown to around ~570K. The reason the file had grown so much was because it included all of the other features.

Earlier this week, while running to a dentist appointment, I thought of a solution to this problem. The basic idea was to optimize the compilation process so only the to-be-released feature was included. Even better, the solution didn't require more modularization. The results:

Before: *.cache.html -> 569K, gzipped 175K
After: *.cache.html -> 314K, gzipped 100K

According to my calculations, that's a 56% reduction in size. How did I do it?

  1. Created a new FeatureName.java EntryPoint with only the to-be-released features imported.
  2. Created a new FeatureName.gwt.xml that references the new EntryPoint.
  3. Copied old (kitchen-sink) EntryPoint.html to FeatureName.html and changed the reference to the nocache.js file.
  4. Created a Maven profile that allows using -PFeatureName to build a FeatureName-only module.

One downside to doing things this way is it's possible to create a WAR that has the same name and different features. Surely the Maven Overlords would frown upon this. Since this is just a temporary solution to release features incrementally, I'm not too worried about it. A possible workaround is to create different WAR names when a feature's profile is activated. I believe the true "Maven way" would be to make the "kitchen sink" application into a JAR and have several WAR modules with the different EntryPoints. Seems a bit complicated to me.

Other than this Maven publishing issue, the only other issue I can foresee is keeping the two EntryPoints and HTML files in synch. Then again, the separate files allow a feature to be customized for the release and can be deleted when its no longer needed.

What do you think? Do you know of a better way to compile a GWT application so it only contains certain features?

Posted in Java at Mar 25 2009, 04:00:37 PM MDT 12 Comments