Raible Designs

How to do cross-domain GWT RPC with a ProxyServlet

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPInputStream;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;

/**
 * ProxyServlet from http://edwardstx.net/wiki/attach/HttpProxyServlet/ProxyServlet.java
 * (This seems to be a derivative of Noodle -- http://noodle.tigris.org/)
 * 
 * Patched to skip "Transfer-Encoding: chunked" headers, avoid double slashes
 * in proxied URLs, handle GZip and allow GWT RPC.
 */
public class ProxyServlet extends HttpServlet {

    private static final int FOUR_KB = 4196;

    /**
     * Serialization UID.
     */
    private static final long serialVersionUID = 1L;
    /**
     * Key for redirect location header.
     */
    private static final String STRING_LOCATION_HEADER = "Location";
    /**
     * Key for content type header.
     */
    private static final String STRING_CONTENT_ENGINE_HEADER_NAME = "Content-Type";
    /**
     * Key for content length header.
     */
    private static final String STRING_CONTENT_LENGTH_HEADER_NAME = "Content-Length";
    /**
     * Key for host header
     */
    private static final String STRING_HOST_HEADER_NAME = "Host";
    /**
     * The directory to use to temporarily store uploaded files
     */
    private static final File FILE_UPLOAD_TEMP_DIRECTORY = new File(System.getProperty("java.io.tmpdir"));

    // Proxy host params
    /**
     * The host to which we are proxying requests. Default value is "localhost".
     */
    private String stringProxyHost = "localhost";
    /**
     * The port on the proxy host to wihch we are proxying requests. Default value is 80.
     */
    private int intProxyPort = 80;
    /**
     * The (optional) path on the proxy host to wihch we are proxying requests. Default value is "".
     */
    private String stringProxyPath = "";
    /**
     * Setting that allows removing the initial path from client. Allows specifying /twitter/* as synonym for twitter.com.
     */
    private boolean removePrefix;
    /**
     * The maximum size for uploaded files in bytes. Default value is 5MB.
     */
    private int intMaxFileUploadSize = 5 * 1024 * 1024;
    private boolean isSecure;
    private boolean followRedirects;

    /**
     * Initialize the <code>ProxyServlet</code>
     * @param servletConfig The Servlet configuration passed in by the servlet container
     */
    public void init(ServletConfig servletConfig) {
        // Get the proxy host
        String stringProxyHostNew = servletConfig.getInitParameter("proxyHost");
        if (stringProxyHostNew == null || stringProxyHostNew.length() == 0) {
            throw new IllegalArgumentException("Proxy host not set, please set init-param 'proxyHost' in web.xml");
        }
        this.setProxyHost(stringProxyHostNew);
        // Get the proxy port if specified
        String stringProxyPortNew = servletConfig.getInitParameter("proxyPort");
        if (stringProxyPortNew != null && stringProxyPortNew.length() > 0) {
            this.setProxyPort(Integer.parseInt(stringProxyPortNew));
        }
        // Get the proxy path if specified
        String stringProxyPathNew = servletConfig.getInitParameter("proxyPath");
        if (stringProxyPathNew != null && stringProxyPathNew.length() > 0) {
            this.setProxyPath(stringProxyPathNew);
        }
        // Get the maximum file upload size if specified
        String stringMaxFileUploadSize = servletConfig.getInitParameter("maxFileUploadSize");
        if (stringMaxFileUploadSize != null && stringMaxFileUploadSize.length() > 0) {
            this.setMaxFileUploadSize(Integer.parseInt(stringMaxFileUploadSize));
        }
    }

    /**
     * Performs an HTTP GET request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a GET request
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("GET Request URL: " + httpServletRequest.getRequestURL(),
              "Destination URL: " + destinationUrl);
        GetMethod getMethodProxyRequest = new GetMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, getMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
        // Execute the proxy request
        this.executeProxyRequest(getMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Performs an HTTP POST request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a standard POST request
        String contentType = httpServletRequest.getContentType();
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("POST Request URL: " + httpServletRequest.getRequestURL(),
              "    Content Type: " + contentType,
              " Destination URL: " + destinationUrl);
        PostMethod postMethodProxyRequest = new PostMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, postMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, postMethodProxyRequest);
        // Check if this is a mulitpart (file upload) POST
        if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
            this.handleMultipartPost(postMethodProxyRequest, httpServletRequest);
        } else {
            if (contentType == null || PostMethod.FORM_URL_ENCODED_CONTENT_ENGINE.equals(contentType)) {
                this.handleStandardPost(postMethodProxyRequest, httpServletRequest);
            } else {
                this.handleContentPost(postMethodProxyRequest, httpServletRequest);
            }
        }
        // Execute the proxy request
        this.executeProxyRequest(postMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Sets up the given {@link PostMethod} to send the same multipart POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a multipart POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the mutlipart POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleMultipartPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest)
            throws ServletException {
        // Create a factory for disk-based file items
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        // Set factory constraints
        diskFileItemFactory.setSizeThreshold(this.getMaxFileUploadSize());
        diskFileItemFactory.setRepository(FILE_UPLOAD_TEMP_DIRECTORY);
        // Create a new file upload handler
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        // Parse the request
        try {
            // Get the multipart items as a list
            List<FileItem> listFileItems = (List<FileItem>) servletFileUpload.parseRequest(httpServletRequest);
            // Create a list to hold all of the parts
            List<Part> listParts = new ArrayList<Part>();
            // Iterate the multipart items list
            for (FileItem fileItemCurrent : listFileItems) {
                // If the current item is a form field, then create a string part
                if (fileItemCurrent.isFormField()) {
                    StringPart stringPart = new StringPart(
                            fileItemCurrent.getFieldName(), // The field name
                            fileItemCurrent.getString() // The field value
                            );
                    // Add the part to the list
                    listParts.add(stringPart);
                } else {
                    // The item is a file upload, so we create a FilePart
                    FilePart filePart = new FilePart(
                            fileItemCurrent.getFieldName(), // The field name
                            new ByteArrayPartSource(
                            fileItemCurrent.getName(), // The uploaded file name
                            fileItemCurrent.get() // The uploaded file contents
                            ));
                    // Add the part to the list
                    listParts.add(filePart);
                }
            }
            MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(
                    listParts.toArray(new Part[]{}),
                    postMethodProxyRequest.getParams());
            postMethodProxyRequest.setRequestEntity(multipartRequestEntity);
            // The current content-type header (received from the client) IS of
            // type "multipart/form-data", but the content-type header also
            // contains the chunk boundary string of the chunks. Currently, this
            // header is using the boundary of the client request, since we
            // blindly copied all headers from the client request to the proxy
            // request. However, we are creating a new request with a new chunk
            // boundary string, so it is necessary that we re-set the
            // content-type string to reflect the new chunk boundary string
            postMethodProxyRequest.setRequestHeader(STRING_CONTENT_ENGINE_HEADER_NAME, multipartRequestEntity.getContentType());
        } catch (FileUploadException fileUploadException) {
            throw new ServletException(fileUploadException);
        }
    }

    /**
     * Sets up the given {@link PostMethod} to send the same standard POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleStandardPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) {
        // Get the client POST data as a Map
        Map<String, String[]> mapPostParameters = (Map<String, String[]>) httpServletRequest.getParameterMap();
        // Create a List to hold the NameValuePairs to be passed to the PostMethod
        List<NameValuePair> listNameValuePairs = new ArrayList<NameValuePair>();
        // Iterate the parameter names
        for (String stringParameterName : mapPostParameters.keySet()) {
            // Iterate the values for each parameter name
            String[] stringArrayParameterValues = mapPostParameters.get(stringParameterName);
            for (String stringParamterValue : stringArrayParameterValues) {
                // Create a NameValuePair and store in list
                NameValuePair nameValuePair = new NameValuePair(stringParameterName, stringParamterValue);
                listNameValuePairs.add(nameValuePair);
            }
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestBody(listNameValuePairs.toArray(new NameValuePair[]{}));
    }

    /**
     * Sets up the given {@link PostMethod} to send the same content POST
     * data (JSON, XML, etc.) as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    private void handleContentPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) throws IOException, ServletException {
        StringBuilder content = new StringBuilder();
        BufferedReader reader = httpServletRequest.getReader();
        for (;;) {
            String line = reader.readLine();
            if (line == null) break;
            content.append(line);
        }

        String contentType = httpServletRequest.getContentType();
        String postContent = content.toString();

        if (contentType.startsWith("text/x-gwt-rpc")) {
            String clientHost = httpServletRequest.getLocalName();
            if (clientHost.equals("127.0.0.1")) {
                clientHost = "localhost";
            }

            int clientPort = httpServletRequest.getLocalPort();
            String clientUrl = clientHost + ((clientPort != 80) ? ":" + clientPort : "");
            String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + intProxyPort : "") + httpServletRequest.getServletPath();
            //debug("Replacing client (" + clientUrl + ") with server (" + serverUrl + ")");
            postContent = postContent.replace(clientUrl , serverUrl);
        }

        String encoding = httpServletRequest.getCharacterEncoding();
        debug("POST Content Type: " + contentType + " Encoding: " + encoding,
              "Content: " + postContent);
        StringRequestEntity entity;
        try {
            entity = new StringRequestEntity(postContent, contentType, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new ServletException(e);
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestEntity(entity);
    }

    /**
     * Executes the {@link HttpMethod} passed in and sends the proxy response
     * back to the client via the given {@link HttpServletResponse}
     * @param httpMethodProxyRequest An object representing the proxy request to be made
     * @param httpServletResponse An object by which we can send the proxied
     *                             response back to the client
     * @throws IOException Can be thrown by the {@link HttpClient}.executeMethod
     * @throws ServletException Can be thrown to indicate that another error has occurred
     */
    private void executeProxyRequest(
            HttpMethod httpMethodProxyRequest,
            HttpServletRequest httpServletRequest,
            HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a default HttpClient
        HttpClient httpClient = new HttpClient();
        httpMethodProxyRequest.setFollowRedirects(false);
        // Execute the request
        int intProxyResponseCode = httpClient.executeMethod(httpMethodProxyRequest);
        String response = httpMethodProxyRequest.getResponseBodyAsString();

        // Check if the proxy response is a redirect
        // The following code is adapted from org.tigris.noodle.filters.CheckForRedirect
        // Hooray for open source software
        if (intProxyResponseCode >= HttpServletResponse.SC_MULTIPLE_CHOICES /* 300 */ && intProxyResponseCode < HttpServletResponse.SC_NOT_MODIFIED /* 304 */) {
            String stringStatusCode = Integer.toString(intProxyResponseCode);
            String stringLocation = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
            if (stringLocation == null) {
                throw new ServletException("Received status code: " + stringStatusCode + " but no " + STRING_LOCATION_HEADER + " header was found in the response");
            }
            // Modify the redirect to go to this proxy servlet rather that the proxied host
            String stringMyHostName = httpServletRequest.getServerName();
            if (httpServletRequest.getServerPort() != 80) {
                stringMyHostName += ":" + httpServletRequest.getServerPort();
            }
            stringMyHostName += httpServletRequest.getContextPath();
            if (followRedirects) {
                if (stringLocation.contains("jsessionid")) {
                    Cookie cookie = new Cookie("JSESSIONID", stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
                    cookie.setPath("/");
                    httpServletResponse.addCookie(cookie);
                    //debug("redirecting: set jessionid (" + cookie.getValue() + ") cookie from URL");
                } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
	                Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
                    String[] cookieDetails = header.getValue().split(";");
					String[] nameValue = cookieDetails[0].split("=");

					Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
					cookie.setPath("/");
					//debug("redirecting: setting cookie: " + cookie.getName() + ":" + cookie.getValue() + " on " + cookie.getPath());
					httpServletResponse.addCookie(cookie);
                }
                httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() + this.getProxyPath(), stringMyHostName));
                return;
            }
        } else if (intProxyResponseCode == HttpServletResponse.SC_NOT_MODIFIED) {
            // 304 needs special handling.  See:
            // http://www.ics.uci.edu/pub/ietf/http/rfc1945.html#Code304
            // We get a 304 whenever passed an 'If-Modified-Since'
            // header and the data on disk has not changed; server
            // responds w/ a 304 saying I'm not going to send the
            // body because the file has not changed.
            httpServletResponse.setIntHeader(STRING_CONTENT_LENGTH_HEADER_NAME, 0);
            httpServletResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
            return;
        }

        // Pass the response code back to the client
        httpServletResponse.setStatus(intProxyResponseCode);

        // Pass response headers back to the client
        Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
        for (Header header : headerArrayResponse) {
            if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
                    header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") || // don't copy gzip header
                    header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header so browser doesn't prompt on failed basic auth
                // proxy servlet does not support chunked encoding
            } else {
                httpServletResponse.setHeader(header.getName(), header.getValue());
            }
        }

        List<Header> responseHeaders = Arrays.asList(headerArrayResponse);

        if (isBodyParameterGzipped(responseHeaders)) {
            debug("GZipped: true");
            if (!followRedirects && intProxyResponseCode == HttpServletResponse.SC_MOVED_TEMPORARILY) {
                response = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
                httpServletResponse.setStatus(HttpServletResponse.SC_OK);
                intProxyResponseCode = HttpServletResponse.SC_OK;
                httpServletResponse.setHeader(STRING_LOCATION_HEADER, response);
            } else {
                response = new String(ungzip(httpMethodProxyRequest.getResponseBody()));
            }
            httpServletResponse.setContentLength(response.length());
        }

        // Send the content to the client
        debug("Received status code: " + intProxyResponseCode,
              "Response: " + response);

        httpServletResponse.getWriter().write(response);
    }


    /**
     * The response body will be assumed to be gzipped if the GZIP header has been set.
     *
     * @param responseHeaders of response headers
     * @return true if the body is gzipped
     */
    private boolean isBodyParameterGzipped(List<Header> responseHeaders) {
        for (Header header : responseHeaders) {
            if (header.getValue().equals("gzip")) {
                return true;
            }
        }
        return false;
    }

    /**
     * A highly performant ungzip implementation. Do not refactor this without taking new timings.
     * See ElementTest in ehcache for timings
     *
     * @param gzipped the gzipped content
     * @return an ungzipped byte[]
     * @throws java.io.IOException when something bad happens
     */
    private byte[] ungzip(final byte[] gzipped) throws IOException {
        final GZIPInputStream inputStream = new GZIPInputStream(new ByteArrayInputStream(gzipped));
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(gzipped.length);
        final byte[] buffer = new byte[FOUR_KB];
        int bytesRead = 0;
        while (bytesRead != -1) {
            bytesRead = inputStream.read(buffer, 0, FOUR_KB);
            if (bytesRead != -1) {
                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }
        }
        byte[] ungzipped = byteArrayOutputStream.toByteArray();
        inputStream.close();
        byteArrayOutputStream.close();
        return ungzipped;
    }

    public String getServletInfo() {
        return "GWT Proxy Servlet";
    }

    /**
     * Retrieves all of the headers from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestHeaders(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an Enumeration of all of the header names sent by the client
        Enumeration enumerationOfHeaderNames = httpServletRequest.getHeaderNames();
        while (enumerationOfHeaderNames.hasMoreElements()) {
            String stringHeaderName = (String) enumerationOfHeaderNames.nextElement();
            if (stringHeaderName.equalsIgnoreCase(STRING_CONTENT_LENGTH_HEADER_NAME)) {
                continue;
            }
            // As per the Java Servlet API 2.5 documentation:
            //		Some headers, such as Accept-Language can be sent by clients
            //		as several headers each with a different value rather than
            //		sending the header as a comma separated list.
            // Thus, we get an Enumeration of the header values sent by the client
            Enumeration enumerationOfHeaderValues = httpServletRequest.getHeaders(stringHeaderName);
            while (enumerationOfHeaderValues.hasMoreElements()) {
                String stringHeaderValue = (String) enumerationOfHeaderValues.nextElement();
                // In case the proxy host is running multiple virtual servers,
                // rewrite the Host header to ensure that we get content from
                // the correct virtual server
                if (stringHeaderName.equalsIgnoreCase(STRING_HOST_HEADER_NAME)) {
                    stringHeaderValue = getProxyHostAndPort();
                }
                Header header = new Header(stringHeaderName, stringHeaderValue);
                // Set the same header on the proxy request
                httpMethodProxyRequest.setRequestHeader(header);
            }
        }
    }

    /**
     * Retrieves all of the cookies from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestCookies(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an array of all of all the cookies sent by the client
        Cookie[] cookies = httpServletRequest.getCookies();
        if (cookies == null) {
            return;
        }

        for (Cookie cookie : cookies) {
            cookie.setDomain(stringProxyHost);
            cookie.setPath(httpServletRequest.getServletPath());
            httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() + "=" + cookie.getValue() + "; Path=" + cookie.getPath());
        }
    }

    // Accessors
    private String getProxyURL(HttpServletRequest httpServletRequest) {
        // Set the protocol to HTTP
        String protocol = (isSecure) ? "https://" : "http://";
        String stringProxyURL = protocol + this.getProxyHostAndPort();

        // simply use whatever servlet path that was part of the request as opposed to getting a preset/configurable proxy path
        if (!removePrefix) {
            stringProxyURL += httpServletRequest.getServletPath();
        }
        stringProxyURL += "/";
        
        // Handle the path given to the servlet
        String pathInfo = httpServletRequest.getPathInfo();
        if (pathInfo != null && pathInfo.startsWith("/")) {
            if (stringProxyURL != null && stringProxyURL.endsWith("/")) {
                // avoid double '/'
                stringProxyURL += pathInfo.substring(1);
            }
        } else {
            stringProxyURL += httpServletRequest.getPathInfo();
        }
        // Handle the query string
        if (httpServletRequest.getQueryString() != null) {
            stringProxyURL += "?" + httpServletRequest.getQueryString();
        }
        
        return stringProxyURL;
    }

    private String getProxyHostAndPort() {
        if (this.getProxyPort() == 80) {
            return this.getProxyHost();
        } else {
            return this.getProxyHost() + ":" + this.getProxyPort();
        }
    }

    protected String getProxyHost() {
        return this.stringProxyHost;
    }

    protected void setProxyHost(String stringProxyHostNew) {
        this.stringProxyHost = stringProxyHostNew;
    }

    protected int getProxyPort() {
        return this.intProxyPort;
    }

    protected void setSecure(boolean secure) {
        this.isSecure = secure;
    }
    
    protected void setFollowRedirects(boolean followRedirects) {
        this.followRedirects = followRedirects;
    }

    protected void setProxyPort(int intProxyPortNew) {
        this.intProxyPort = intProxyPortNew;
    }

    protected String getProxyPath() {
        return this.stringProxyPath;
    }

    protected void setProxyPath(String stringProxyPathNew) {
        this.stringProxyPath = stringProxyPathNew;
    }

    protected void setRemovePrefix(boolean removePrefix) {
        this.removePrefix = removePrefix;
    }

    protected int getMaxFileUploadSize() {
        return this.intMaxFileUploadSize;
    }

    protected void setMaxFileUploadSize(int intMaxFileUploadSizeNew) {
        this.intMaxFileUploadSize = intMaxFileUploadSizeNew;
    }

    private void debug(String ... msg) {
        for (String m : msg) {
            System.out.println("[DEBUG] " + m);
        }
    }
}

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

The good ol' Job Hunt

Just over two years ago, I wrote about the good ol' job hunt. Today, I find myself in a very similar situation. My Evite gig ends in a couple hours and I'm heading to The Cabin on Monday for a month of vacation. Similar to last time, there are opportunities out there, but most of them come through recruiters.

I have to admit - it seems like I got lucky the last couple times I tried to find a gig. I simply blogged I was looking and found myself negotiating with companies a few days later. Getting laid off from LinkedIn was awesome in that a few companies contacted me about hiring the whole team the next day. The ability to blog-and-get-a-gig was a great way to cut out the middle-man.

Unfortunately, I think I got spoiled by my blog-and-get-a-gig success. I figured it would happen again this time...

Not so much.

I don't know if it's because of the down economy or because I took the year off from speaking at conferences. Regardless, I'm in an interesting situation since I signed a 1-year lease on an office in downtown Denver. It's easy to market to companies in Denver, but if I land a gig here, there's a good chance the client is going to want me in the office everyday. That leaves me wondering: what's the best way to market to companies outside of Colorado? My ideal contract is one that 1) allows me to work remotely and 2) only requires me to travel once or twice a month.

I believe my ideal gigs are out there, but I think it's difficult to convince companies I'm a good remote worker. In an attempt to convince them otherwise, I'd like to offer recommendations from my last two clients.

"Matt contributed dramatically to the engineering practice at Evite, both directly and indirectly. Directly, he lead the effort to define and prove a successful new web UI architecture for us. Indirectly, he brought senior engineering talent into the organization, and energized the existing team. We are significantly better positioned to deliver on our Product goals as a result of having worked with Matt. I'd love to work with him again someday." David Thomas, VP of Technology, Evite

"Matt has unique abilities in the realm of software engineering. He brings great energy, an amazing breadth of knowledge in UI technologies, along with a can-do attitude that's refreshing to interact with. He is a great leader and communicator. In addition, Matt can apply his deep understanding of the technologies in question pragmatically to the engineering problem at hand, leading his team in execution and delivery. He's an excellent technical visionary to complement any team." Arnold Goldberg, Vice President Platform Engineering, LinkedIn

If you're looking to hire someone with my skills, let me know - there's a good chance you'll be glad you did.

JSON Parsing with JavaScript Overlay Types in GWT

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Implementing OAuth with GWT

I've heard about OAuth for quite some time, but never had an opportunity to implement it on a project. For a good explanation of what OAuth is, see its Introduction. Here's an excerpt:

...it allows you the User to grant access to your private resources on one site (which is called the Service Provider), to another site (called Consumer, not to be confused with you, the User). While OpenID is all about using a single identity to sign into many sites, OAuth is about giving access to your stuff without sharing your identity at all (or its secret parts).

The reason I needed OAuth was to interact with the Google Contacts API. I've always hated how sites make you import all your contacts from Gmail. I wanted to develop a system that'd let you simply read your contacts from Google in real-time.

Since the application I'm working on uses GWT, I chose to implement an OAuth client in GWT. After googling for "gwt oauth", I found two examples. Unfortunately, neither worked out-of-the-box.

The good news is I did manage to create a working solution. The bad news is it only seems to work at random. That's right folks, I created a solution that only works 50% of the time. I'm somewhat embarrassed to post it here, but I also realize the power of open source and community. By sharing, I hope we can find the flaws in my logic and come up with a solution for all GWT applications.

The best project for OAuth libraries seems to be oauth on Google Code. However, you'll notice that there is no JavaScript implementation listed on the homepage. I did look at the Java implementation, but quickly realized it wouldn't be usable in GWT. Therefore, I opted for the JavaScript implementation.

OAuth consists of several steps. The following diagram explains the authentication flow nicely.

In a nutshell, you have to complete the following steps:

1. Get a token from the service provider.
2. Redirect user to service provider to grant access and redirect back to application.
3. Request access token to access protected resources.
4. Access protected resources and pull/push data.

To access a service provider's OAuth service, you'll likely need to start by registering your application. For Google, OAuth Authentication for Web Applications is an excellent resource. Google's OAuth Playground is a great way to with the Google Data APIs after you've registered.

Now that you know how OAuth works, let's look at how I implemented it with GWT. I started by adding the necessary JavaScript references to my *.gwt.xml file.

<script src="//oauth.googlecode.com/svn/code/javascript/oauth.js"/>
<script src="//oauth.googlecode.com/svn/code/javascript/sha1.js"/>

Next, I needed a way to sign the request. I tried to use Sergi Mansilla's OAuth.java for this, but discovered issues with how the parameters were being written with GWT 1.6. I opted for Paul Donnelly's makeSignedRequest function instead. By adding this to my application's HTML page, I'm able to call it using the following JSNI method:

public native static String signRequest(String key, String secret, String tokenSecret, String url) /*-{
    return $wnd.makeSignedRequest(key, secret, tokenSecret, url);
}-*/;

After the URL is signed, it needs to be sent to the provider to get a request token. To do this, I used GWT's RequestBuilder and created a send() method:

protected void send(RequestCallback cb, String URL) {
    RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL);
    builder.setTimeoutMillis(10000);
    builder.setCallback(cb);
    
    Request req = null;
    try {
        req = builder.send();
    } catch (RequestException e) {
        cb.onError(req, e);
    }
}

If you try this with Google's Request Token URL in GWT's hosted mode, nothing will happen. Compile/browse to Safari and you'll still see nothing. Try it in Firefox and you'll see the following.

To workaround browsers' Same Origin Policy, I added a proxy servlet to send the requests. I started with Jason Edwards's ProxyServlet and modified it to fit my needs. I then registered it in both *.gwt.xml and web.xml.

<servlet path="/google/" class="org.appfuse.gwt.servlet.AlternateHostProxyServlet"/>

Now, before calling the send() method, I replace the start of the URL so the request would be routed through the servlet.

public void getToken(RequestCallback cb) {
    String url = signRequest(provider.getConsumerKey(), 
                             provider.getConsumerSecret(), 
                             "", provider.getRequestTokenURL());
    url = url.replace("https://www.google.com/", "/google/");
    send(cb, url);
}

When the request returns, I create two cookies by calling a createOAuthCookies() method with the payload returned:

public static String[] createOAuthCookies(String data) {
    String oauth_token = data.substring(data.indexOf("oauth_token=") + 12);
    oauth_token = oauth_token.substring(0, oauth_token.indexOf("&"));

    String oauth_token_secret = data.substring(data.indexOf("oauth_token_secret=") + 19);

    Cookies.setCookie("oauth_token", URL.decode(oauth_token));
    Cookies.setCookie("oauth_token_secret", URL.decode(oauth_token_secret));
    return new String[]{oauth_token, oauth_token_secret};
}

The next step is to authorize the token. This is where things got tricky with my proxy servlet and I had to add some special logic for GWT. Google was sending back a 302 with a Location header, but it wasn't hitting the onResponseReceived() method in my callback. For this reason, I had to change it to a 200 status code and add the redirect location to the body. I also discovered that sometimes they'd return an HTML page with a <meta http-equiv="refresh" ...> tag. When using Twitter, I discovered the full HTML for the allow/deny page was returned. Below is the callback I'm using. WindowUtils is a class I got from Robert Hanson and the gwt-widget project.

public void onResponseReceived(Request request, Response response) {
    String text = response.getText();
    if (response.getStatusCode() == 200 && response.getText().startsWith("http")) {
        WindowUtils.changeLocation(response.getText());
    } else {
        // look for meta-tag that refreshes and grab its URL
        if (text.contains("";
            String url = text.substring(text.indexOf(tokenToStartWith) + tokenToStartWith.length());
            url = url.substring(0, url.indexOf(tokenToEndWith) + tokenToEndWith.length());
            WindowUtils.changeLocation(url);
        } else {
            // Twitter returns a full HTML page, so redirect to the authorize URL manually
            if (provider instanceof Twitter) {
                String url = provider.getAuthorizeTokenURL();
                url = url.replace("$1", OAuthRequest.getAuthToken());
                url = url.replace("$2", DefaultRequest.getCurrentLocation());
                WindowUtils.changeLocation(url);
            } else {
                onError(request, new RequestException(text));
            }
        }
    }
}

public void onError(Request request, Throwable caught) {
    Window.alert("Calling authorize token failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + caught.getMessage());
}

The 3rd step is to get an access token. The most important thing to remember when you do this is to include the "oauth_token_secret" value when signing the request.

signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
            getAuthTokenSecret(), url);

After this completes with a 200, I create the cookies again (since oauth_token and oauth_token_secret are returned in the body), then call the API to get a list of contacts. The ContactsRequests class is responsible for making the call. The DefaultRequest class contains the send() method as well as utility methods to get the cookie values of the oauth tokens.

public class ContactsRequest extends DefaultRequest {
    private static final String GOOGLE_CONTACTS_URL = 
        "http://www.google.com/m8/feeds/contacts/default/thin?oauth_token=$1";
    private OAuthProvider provider;

    public ContactsRequest(OAuthProvider provider) {
        this.provider = provider;
    }

    public void getContacts(RequestCallback cb) {
        String url = GOOGLE_CONTACTS_URL.replace("$1", getAuthToken());
        url = signRequest(provider.getConsumerKey(), provider.getConsumerSecret(), 
                          getAuthTokenSecret(), url);

        String proxiedURLPrefix = "/contacts/";
        // allow for deploying at /gwt-oauth context
        if (WindowUtils.getLocation().getPath().contains("gwt-oauth")) {
            proxiedURLPrefix = "/gwt-oauth" + proxiedURLPrefix;
        }

        url = url.replace("http://www.google.com/", proxiedURLPrefix);

        send(cb, url);
    }
}

If all goes well, the response contains the data you requested and it's used to populate a textarea (at least in this demo application). Of course, additional processing needs to occur to parse/format this data into something useful.

This all sounds pretty useful for GWT applications, right? I believe it does - but only if it works consistently. I sent a message to the OAuth Google Group explaining the issues I've had.

I'm trying to use the JavaScript API to authenticate with OAuth from a GWT application. I've got it working with both Google and Twitter's OAuth implementations. However, it seems to fail to sign the URL at random. In other words, it works 1 out of 3 times. ... Any idea why this could be happening?

I received a response with a cleaner makeSignedRequest() function. I tried it and, unfortunately, it seems to be equally unreliable. I suspect the problem is with the OAuth JavaScript implementation, GWT's interpretation of it, or that OAuth isn't as mature as it appears to be. I'd like to think one of the first two causes the problem.

To make it easier to create a robust example of GWT and OAuth, I created a gwt-oauth project you can download or view online. Please keep in mind the demo is likely to be flakey. If you're persistent and try enough times, it's likely to work. Firefox seems to succeed moreso than Safari or Chrome. If you have any suggestions for improving this example, please let me know.

Enhancing Evite.com with GWT and Grails

On my LinkedIn Profile, it says my current gig is a SOFEA consultant at a stealth-mode startup.

SOFEA Consultant, Stealth Mode Startup, Los Angeles, CA. December 2008 -- Present.

OK, I lied. It's not a startup, it's a well-known company that helps you plan parties. For the last 5+ months, my UI team from LinkedIn has been working with Evite.com to enhance portions of their site with a SOFEA architecture.

In January, we started evaluating Ajax Frameworks and came to the conclusion that GWT was right for us. After we chose the UI framework, other team members chose Grails and memcached to develop scalable RESTful services. The architecture we implemented involves using GWT's RequestBuilder to talk to Grails' services, which cache almost all their JSON output in memcached.

To see an example of a feature we developed with GWT, see Evite's Design Gallery. I personally worked on this feature and very much enjoyed becoming a GWT aficionado in the process. GWT's zero-turnaround feature made doing pure client-side work a lot of fun. It's definitely something I'd like to continuing doing at my next gig.

Everyone from Evite is very happy with what we've been able to do with GWT and Grails. We have a stateless architecture and are quickly able to develop both client-side and server-side features. We've learned to scale the client by using out-of-the-box GWT components. We've scaled Grails by caching as much as possible. We serve up Ads and Analytics using the same JavaScript mechanisms that traditional server-side frameworks use.

At the end of this month, my gig with Evite comes to an end. I'll be spending a few weeks at my family's cabin in Montana and then it's on to the next big thing. What's the next big thing? I'm not sure yet, hence the reason for writing this. If you're looking to develop a GWT application, introduce a SOFEA architecture at your company, or simply adopt some open source frameworks, I'd love to help out. Drop me a line and let's start a conversation.

Creating a Facebook-style Autocomplete with GWT

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

The End of the Colorado Software Summit

It looks like the Colorado Software Summit has come to an end.

To Our Friends and Supporters,

In these challenging economic times, business has slowed, many companies have had to resort to layoffs and/or closures, and everyone has been tightening their belts. Unfortunately, Colorado Software Summit has not been immune to this downturn. As have so many companies and individuals, we too have experienced a severe decline in our business, and as a result we are not able to continue producing this annual conference.

This year would have been our 18th conference, and we had planned to continue through our 20th in 2011, but instead we must end it now. [Read More]

I first attended this conference in October 2005. I enjoyed it so much, I returned again in 2006, 2007 and 2008. As I mentioned last year, this was my favorite conference.

The reason I like it so much is because it's an annual gathering (this will be my 4th year) with friends and it's somewhat like a vacation, except you get to learn a lot.

While I'm sad to see it go, I completely understand Wayne and Peggy's decision. If you ever attended this conference, I encourage you to join the Colorado Software Summit LinkedIn Group. To see all my posts from the talks I attended at this conference, see my softwaresummit-tagged entries.

To Wayne and Peggy: thanks for all the great memories and for putting together such an excellent conference. Cheers to you both!

My Eye Surgery Experience

On May 7, I visited my local TLC Laser Eye Center for eye surgery. I began looking into eye surgery way back in December. At that time, I wore my glasses for a week, then waltzed into my local TLC and tried to get it done the next day. I quickly found out that 1) it wasn't possible for 2 weeks and 2) I couldn't ski for a couple weeks afterward. Since we were in the midst of a great ski season, I decided to schedule it for May.

I failed the LASIK-eligibility and learned I'd have to have PRK instead. The scars on my right eyeball caused me to fail. I received these scars as a boy when a friend and I blew up a .45-70 bullet with a nail and sledgehammer. I was the one holding the nail and couldn't see for the next 3 days.

In a nutshell: with LASIK they cut a flap, lift it up and shoot the laser under it. It heals quickly and is relatively painless. PRK has been around since before LASIK. With PRK, they seem to shave your eyeball and then shoot a laser into it. PRK takes a lot longer to heal, but the results are often as good or better than LASIK. Wikipedia has a more technical PRK vs. LASIK reference.

The Procedure
My mom flew into town for my surgery and drove me to TLC's office on that Thursday morning. It took 2 hours to prep for surgery, with most of the time spent sitting around and talking to my mom. Finally, they asked her to sit in the lobby and took me back to a waiting area. When I walked in the room, there were 2 other patients with surgery gear (funny hat and booties) and masks. They were leaning back, looking at the ceiling with their masks on and eyes closed. I quickly became the 3rd person who looked like this. Right before they started putting drops in my eyes, I remember being terrified that I might never see daylight again. At the very least, I thought I wouldn't see anything but black for the next 3 days.

Right before they called me in, the previous patient walked out and muttered "Damn, that burns." Believe me, this is not what you want to hear right before it's your turn. I was led into the operating room, sat down and received numbing drops in my eyes. Less than a minute later, I was led over to the operating table.

The rest of the procedure lasted less than 5 minutes. They taped my left eye shut and told me to stare at the red light with my right eye. At this point, they used some contraption to shave my eyeball. It was slightly painful, similar to the mild pain you feel when getting a cavity drilled with Novocaine. After each stroke, the world would ripple like a pebble thrown in a lake. After 10 strokes or so, they shot a laser into my eye for around 20 seconds. You don't actually see the laser (the red light looks the same), but you can smell your eyeball burning. Each eye only took a few minutes. My mom was able to watch the entire surgery on a television on the other side of a glass wall.

The Recovery
I was surprised to discover I was immediately able to open my eyes and see normally. Of course, my eyes felt heavy, so I didn't open them wide nor feel like I could. I was led back to the waiting room where I was given a Valium and sent home. I put my sunglasses on when we left and kept my eyes closed for the 5-minute drive home. After arriving at my house, I immediately downed some Tylenol PM and went to bed. It was hard to fall asleep and my eyes began tearing up. There was a dull pain in my eyes that kept the tears flowing for most of the afternoon. It took me 2 hours to fall asleep and I remember my eyes causing my nose to get stuffed up from all the tears.

When I woke up that evening, everything was blurry, but I was able to open my eyes and see better than I had previously w/o glasses. I didn't expect anything in the form of good vision and was mostly pre-occupied with trying to stop the pain (which wasn't terrible, but definitely present). I was prescribed Vicodin and started taking it on a regular basis. I was completely unable to watch TV at all that night. Viewing the computer screen was unthinkable.

The next morning, my vision was a lot better as evidenced by the tweet from my iPhone. Shortly after, my mom drove me to my eye doctor's office for a 1-day checkup. The results were surprising.

Friday night was one of the most painful. Saturday wasn't very painful, but my eyesight was very blurry. That afternoon, it was hard to keep my eyes open. Every time I tried to open them, I felt like I had to sneeze. Fortunately, I was able to watch the Nuggets game. I couldn't see players' numbers, but I was able to see Melo's last-second 3-pointer to win.

On Monday, I was able to drive to TLC for my 3-day checkup. Things were definitely blurry, but I didn't feel like it was dangerous for me to be behind the wheel. I was able to work on Monday, but I also had to increase my font sizes to 36pt and used a 30" monitor all day. OS X's Universal Access -> Zoom feature came in awful handy. That night, TV was a LOT clearer than the previous night, but it was still fuzzy.

What's it like now?
It's been almost 4 weeks and I'm very glad that I had the surgery done. I haven't felt any pain since the Sunday after surgery and I haven't had any issues with dryness. My vision does fluctuate from day-to-day. Some days I feel like I have super-hero vision and other days there's a halo around objects beyond 10 feet. According to my doctor, fluctuations are expected to continue for 6 months. I don't mind since it never gets bad enough to seem strange.

Now I enjoy not having to worry about glasses or contacts when traveling. I love waking up every morning and not having to do anything to improve my vision. I feel like I have more freedom in my life. Getting eye surgery is definitely one of the best things I've ever done.

Ryan and Breanne's Wedding in Playa del Carmen

Despite the warnings about Swine Flu, two weeks ago I flew to Cancun for my good friends' Ryan and Breanne's wedding. I didn't get much sleep the night before, but managed to get a good nap on the plane. We arrived in Cancun around 2:00pm and begin enjoying the beautiful weather of Mexico. We took a shuttle from CUN to the all-inclusive Paraiso Maya. Thanks to Swine Flu, we got upgraded twice and ended up paying $110/night for a place that's normally $450/night.

My two favorite parts of the trip were 1) the people and 2) the place. There was around 15 of us, many of which have been good friends since college. We stayed at the Paraiso Maya, which was a very nice hotel with beautiful pools, elaborate buffets and awesome beach access. We had a ton of fun at the pool bar, playing water basketball, jet skiing and playing beach volleyball. The dinners at the Steakhouses were great and The Galaxy (Star War themed) club created many good memories. It's great to travel with that many people, especially when the beer is flowing for (what seems like) free and you're partying with old friends.

My third favorite part of the trip was watching the Nuggets vs. Lakers games. We watched 3 games in the resort's "Sports Bar" and had a blast doing it. Since we had a couple Lakers' fans in the mix, it made things interesting.

We spent 3 days at the pool and on the beach before the wedding happened on Saturday. It was a very cool ceremony and we enjoyed a Mariachi band for a good hour afterward. I've definitely become a big fan of beach weddings in the last 6 months.

After 5 days in Playa del Carmen, my buddy (Kevin Navarro) and I spent 2 days at The Westin in Cancun. This was a very nice resort; especially since we had an ocean-front room.

I had so much fun on this trip, I've been inspired to learn Spanish. I don't know when I'll carve out the time to do it, but I know I'll return to Mexico several times, so the sooner the better.

To see all my pictures from this trip, see my Mexico 2009 Collection on Flickr.

Ajax Framework Analysis Results

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

1. Choose a short list of frameworks to prototype with.
2. Create an application prototype with each framework.
3. Document findings and create a matrix with important criteria.
4. Create presentation to summarize document.
5. Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

• GXT: a GWT version of Ext JS
• SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.