Matt RaibleMatt Raible is a writer with a passion for software. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.
You searched this site for "&amp". 294 entries found.

You can also try this same search on Google.

Integrating GWT with Spring Security

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
 

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

Posted in Java at Aug 06 2009, 08:50:15 AM MDT 10 Comments

How to do cross-domain GWT RPC with a ProxyServlet

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Posted in Java at Aug 05 2009, 04:06:12 PM MDT 17 Comments

The good ol' Job Hunt

Just over two years ago, I wrote about the good ol' job hunt. Today, I find myself in a very similar situation. My Evite gig ends in a couple hours and I'm heading to The Cabin on Monday for a month of vacation. Similar to last time, there are opportunities out there, but most of them come through recruiters.

I have to admit - it seems like I got lucky the last couple times I tried to find a gig. I simply blogged I was looking and found myself negotiating with companies a few days later. Getting laid off from LinkedIn was awesome in that a few companies contacted me about hiring the whole team the next day. The ability to blog-and-get-a-gig was a great way to cut out the middle-man.

Unfortunately, I think I got spoiled by my blog-and-get-a-gig success. I figured it would happen again this time...

Not so much.

I don't know if it's because of the down economy or because I took the year off from speaking at conferences. Regardless, I'm in an interesting situation since I signed a 1-year lease on an office in downtown Denver. It's easy to market to companies in Denver, but if I land a gig here, there's a good chance the client is going to want me in the office everyday. That leaves me wondering: what's the best way to market to companies outside of Colorado? My ideal contract is one that 1) allows me to work remotely and 2) only requires me to travel once or twice a month.

I believe my ideal gigs are out there, but I think it's difficult to convince companies I'm a good remote worker. In an attempt to convince them otherwise, I'd like to offer recommendations from my last two clients.

"Matt contributed dramatically to the engineering practice at Evite, both directly and indirectly. Directly, he lead the effort to define and prove a successful new web UI architecture for us. Indirectly, he brought senior engineering talent into the organization, and energized the existing team. We are significantly better positioned to deliver on our Product goals as a result of having worked with Matt. I'd love to work with him again someday." David Thomas, VP of Technology, Evite

"Matt has unique abilities in the realm of software engineering. He brings great energy, an amazing breadth of knowledge in UI technologies, along with a can-do attitude that's refreshing to interact with. He is a great leader and communicator. In addition, Matt can apply his deep understanding of the technologies in question pragmatically to the engineering problem at hand, leading his team in execution and delivery. He's an excellent technical visionary to complement any team." Arnold Goldberg, Vice President Platform Engineering, LinkedIn

If you're looking to hire someone with my skills, let me know - there's a good chance you'll be glad you did. ;-)

Posted in Java at Jun 26 2009, 03:30:42 PM MDT 9 Comments

JSON Parsing with JavaScript Overlay Types in GWT

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Posted in Java at Jun 24 2009, 09:52:49 AM MDT 10 Comments

Creating a Facebook-style Autocomplete with GWT

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Facebook Autocomplete

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

Posted in Java at Jun 05 2009, 07:05:10 AM MDT 25 Comments

Ajax Framework Analysis Results

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

  1. Choose a short list of frameworks to prototype with.
  2. Create an application prototype with each framework.
  3. Document findings and create a matrix with important criteria.
  4. Create presentation to summarize document.
  5. Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Number of Committers

Books on Amazon

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Ranking with equal criteria weights

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Ranking with weighted criteria

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

  • GXT: a GWT version of Ext JS
  • SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.

Posted in Java at Apr 23 2009, 08:34:44 PM MDT 53 Comments

Jason and Holly's Wedding was a blast!

This past weekend, I experienced an incredible 4 days in West Palm Beach, Florida. When Julie and I got married there in April 2000, we had so much fun that we always wanted to do it again. We had a lot of good friends and family fly in for the wedding. If you've ever experienced a beach holiday with many good friends, you'll know what I'm talking about. Julie is Holly's sister and Jason (her new husband) is one of my fraternity brothers. That means I knew both sides of the wedding party and it very much resembled the reunion we always wanted.

I flew down on Thursday morning after a little bit of oversleeping. Because I flew into Orlando, I had to rent a car and drive 2 hours to West Palm. This wasn't so bad since I got hooked up with a convertible and it was a beautiful 80°F on the drive. That night, I picked up Abbie and Jack from "Grammy's" house and they spent the night with me in my hotel.

The next morning, we woke up and headed to Sailfish for a ride on a friend's boat. After a couple hours on the intercostal, we played in the pool, enjoyed the sun and started getting ready for the Rehearsal Dinner.

Jack and Uncle Jason The Hook Kids are ready!

Abbie and Jack on The Hook Crazy Abbie Captain Jack

During the Rehearsal Dinner, a friend (Navarro) and I were bartenders while Abbie was in charge of making sure everyone's drinks were full. Below are some of my favorite pictures from Friday night.

The Bride-To-Be The Crew Julie, Holly and Jack

Daddy and Jack Crazy Kids Give me back that wine opener!

After the rehearsal dinner, the party crowd headed to Rum Bar and had lots of laughs and played many games of "hook". On the wedding day (Saturday), we took another boat ride, hung out on the beach and enjoyed a beautiful ceremony on the beach.

Jack the ringbearer Flower Girls Awesome Wedding

The rest of the night was spent dancing and enjoying everyone's company. The party didn't end that night though. Sunday, Holly and Julie's mom had a brunch at her house where we all laid by the pool, played with the kids and sipped on many cold drinks. At sunset, we walked down to the beach so Mr. and Mrs. Harris could get some sand from their ceremony location.

Mr. and Mrs. Harris

It was a spectacular weekend with lots of good friends and a ton of great memories. Congratulations Jason and Holly!

For more pictures, see Flickr or Facebook.

Posted in General at Apr 21 2009, 09:53:54 AM MDT Add a Comment

New Office and New Bike

Back in January, I moved into a new office to work on my current project. The following week, my bike was stolen. The next day I ran to work and decided to do it for a couple months.

My current goal is to run until April 1st or until I lose 20 pounds, whichever comes first.

The next week, my co-worker's bike was stolen and I knew I had to stick with my goal. The significance of April 1st was that our office lease expires on April 1st and we were planning on moving to a new office. While our office is nice, it is a large one-room office with no windows. The previous office was quite a bit cooler, but also cost twice as much.

Raible Designs HQ 2009 A couple of weeks ago, we found a nice office near downtown. I signed a 1-year lease and moved in over the weekend. For folks in Denver, you might recognize the nice location.

I did my final run to work last Thursday, in the midst the Blizzard of 2009. Almost Whiteout While running to work was a great experiment and I enjoyed telling people I was doing it, it wasn't fun. I've been riding my bike to work for many years (first year was 1999). The one thing I've always enjoyed was the thrill of the ride in the morning. On a beautiful spring day, it's really a fantastic experience. It's easily been the best part of any job I've ever had.

New Trek FX 7.5 With the new office secured and the running mission completed, I walked over to my favorite bike shop and picked up a Trek FX 7.5 yesterday. To research and figure out which bike to buy, I asked my network on LinkedIn.

As luck would have it, my first ride to the new office was today and First Ride on New Trek we woke up to a morning snow storm. Even though the ride was cold and wet, I still had a blast. I've enjoyed riding since I traversed the hills on a BMX bike back in Montana. Getting back in the saddle today was simply awesome and I can't wait to ride again tomorrow, regardless of the weather. I know those nice spring days are just around the corner. :-D

Posted in General at Mar 30 2009, 11:51:39 PM MDT 3 Comments

How To Setup Your Own Software Development Company

This post was originally titled "FTE vs. Contract in this Economy", but it didn't seem to capture the essence of this entry. I wanted to write about why I think contracting is better in this down economy, but I also wanted to write about how you you might go about setting up your own company. Starting a company is relatively easy from a legal standpoint, and hopefully I can provide some resources that'll make it even easier.

First of all, I believe that contracting is better in this economy for a very simple reason:

When you're a contractor, you're prepared to be let go.

There's really nothing like being laid off. It sucks. It often shocks you and makes you depressed. The good part is you usually get a good afternoon's worth of drinking out of it, but that's about it. Severance is cool, but let's face it - you'd much rather be employed.

As a contractor, you're always looking for your next gig. You're prepared for the worst. You're more motivated to learn marketable skills. You're constantly thinking about how you can market yourself better. Writing (blogging, articles, books) is an excellent way to do this and I believe it's rare that FTE are as motivated to do these kinds of things.

Being a contractor forces you to better yourself so you're more marketable.

People's biggest fear of contracting is that they'll have a hard time finding their next gig. In my career, I've rarely had an issue with this. There's always contracts available, it's just a matter of how much you're going to get paid. Yes, I've had to suck-it-up and make $55/hour instead of $125/hour, but that was back in 2003 and $55/hour is still more than I would have made as a FTE.

The other thing that makes me believe contracting is better in this economy is I believe companies are hiring more short-term contractors than employees. I don't know if this is because they consider employees liabilities and contractors expenses, but something about it seems to make the books look better.

So you've decided to take my advice and try your hand at contracting. Should you setup your own Corporation or LLC?

Starting a Company
Yes, you should absolutely start your own company. As a Software Developer, chances are you're going to make enough to put you in the highest tax bracket. If you're a Sole Proprietor (no company), you will pay something like 35% of your income to taxes and you can be sued for everything you own by your clients.

Should you create an LLC or Corporation? I started Raible Designs in May 1998. I started out as an LLC and later converted to an S Corp. For the first few years, I made $30-$55/hour and this seemed to work pretty well. I believe this was similar to having a Sole Proprietorship (because I was the only employee), except that I was protected from lawsuits.

In 2001, I got my first high-paying gig at $90/hour and my Accountant suggested I change to an S Corp to save 10K+ on self-employment tax. I'm certainly not an expert on the different types of business entities, but this path seemed to work well for me. It was $50 to convert from an LLC to an S Corp. I'm not sure if you can go from an S Corp to an LLC. The beauty of an S Corp is the corporation typically gets taxed at 15%, so you can run a lot of things through your business and pay less taxes. Date nights can be business meetings, vacations can be Shareholders Meetings, seasons tickets can be client entertainment and you can write off your car and fuel costs.

There's lots of good resources on the web that describe the different business entity options. My favorite is A List Apart's This Web Business IV: Business Entity Options. Another good resource is How to form an LLC.

The hardest part of starting a new business is coming up with a good name. My advice is to make sure the domain name is available and pick something you like. I chose Raible Designs because I designed web sites at the time. Raible is a pretty unique name, so that's worked well having it as part of my business name. Googlability is important - don't choose a generic name that will make you difficult to find. Potential clients should be able to google your business name and find you easily.

Once you've picked a name, the business establishment part is pretty easy. In Colorado, you can File a Document with the Secretary of State. Their site also allows you to reserve a name if you're not quite ready to make the leap.

You'll also need to get a Federal Employer Identification Number (FEIN) from the IRS. The IRS has a good Starting a Business article and also allows you to Apply for an Employer Identification Number (EIN) Online.

Once you've got all the documents setup, you'll want to create a bank account for your business. I'm currently using Wells Fargo and really like how software-friendly they are. Their online banking is clean and easy to use. They also support QuickBooks for the Mac. They have Payroll Services to allow you to pay your quarterly taxes online as well as setup direct deposit, but I'm not using them.

For payroll, I use PayCycle and have nothing but good things to say about them (see update below). I have the Small Business Package at $42.99 per month. This package allows me to pay myself and employees + up to 5 sub-contractors with direct deposit. It also allows me to pay both Federal and State quarterly taxes online. Of course, if you can also get an Accountant to do this for you.

Having a good Accountant and Financial Advisor (for your retirement plan) will likely be an essential part of your business.. LinkedIn's Service Providers is a good way to find recommended professionals in your area. For example, click here to search for Accountants and then click the change location link in the top right corner to specify your zip code.

Finally, you'll need insurance. The Hartford has a good Small Business package that costs around $500/year. It's liability limits have worked for all of my clients and I'm covered if my laptop ever gets stolen. For Health Insurance, I recommend using eHealthInsurance.com to find a good provider for you. I don't get sick or hurt much, so I typically get a disaster prevention plan with a $5K deductible. For dental insurance, brush your teeth. Vision insurance typically sucks, so I wouldn't buy it. Yes, our health care system in the US needs work and I believe if everyone had a small business, it might get more affordable a lot quicker.

Over the next few days, I'll post some additional advice I've received on retirement plans, deducting a home office, drawing up contracts and how to come up with a good rate. If you're an Independent Software Developer and have any additional advice, I'd love to hear it.

Update: I take back what I said about PayCycle. After having a couple of insufficient funds when re-activating my account in January (they were pulling from the wrong account), they've changed my direct deposit lead-time to 5 days for the next 6 months. This means I forget to create checks on time since their reminder gets sent 2 days before. Time to try Wells Fargo.

Update October 2009: I never left PayCycle and I continue to use them to this day. I'm a satisfied customer, but do believe it still takes too long to make changes to your electronic services setup.

Posted in Java at Feb 10 2009, 12:23:10 PM MST 36 Comments

Portland Tech Meetup Tomorrow Night

If you live in Portland, Oregon - or just happen to be in town - you might want to join us for some beers and tech talk tomorrow (Monday) night. Patrick Lightbody, Howard Lewis Ship and myself will be meeting around 6:30 at the Rogue Distillery & Public House (map). With 36 taps and the delicious beer from Rogue Ales, this is sure to be a fun night.

If you're on Facebook, you can let us know you're coming by RSVP'ing to the Event. Otherwise, please leave a comment or just show up.

Posted in Java at Dec 28 2008, 04:18:35 PM MST 3 Comments