JSON Parsing with JavaScript Overlay Types in GWT

A reader recently asked:

I would love to see a snippet of how to eval the JSON coming from RequestBuilder into the OverlayTypes. What is the mapping like? I used OverlayTypes to read in static data that I render into the head section of the hosted page, which is pretty easy and fast, but I don't know how to do this "reading" dynamically at runtime.

If you're not familiar with GWT's Overlay Types (added in 1.5), see Getting to really know GWT, Part 2: JavaScript Overlay Types. In our project, we're using Overlay Types to simplify JSON parsing and make our application lean-and-mean as possible.

First of all, we have a JSOModel class that acts as our overlay type:

import java.util.HashSet;
import java.util.Set;

import com.google.gwt.core.client.JavaScriptObject;
import com.google.gwt.core.client.JsArray;
import com.google.gwt.core.client.JsArrayString;

/**
 * Java overlay of a JavaScriptObject.
 */
public abstract class JSOModel extends JavaScriptObject {

    // Overlay types always have protected, zero-arg constructors
    protected JSOModel() {
    }

    /**
     * Create an empty instance.
     * 
     * @return new Object
     */
    public static native JSOModel create() /*-{
        return new Object();
    }-*/;

    /**
     * Convert a JSON encoded string into a JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '{"foo":"bar","number":123}'
     *
     * @return a populated JSOModel object
     */
    public static native JSOModel fromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    /**
     * Convert a JSON encoded string into an array of JSOModel instance.
     * <p/>
     * Expects a JSON string structured like '[{"foo":"bar","number":123}, {...}]'
     *
     * @return a populated JsArray
     */
    public static native JsArray<JSOModel> arrayFromJson(String jsonString) /*-{
        return eval('(' + jsonString + ')');
    }-*/;

    public final native boolean hasKey(String key) /*-{
        return this[key] != undefined;
    }-*/;

    public final native JsArrayString keys() /*-{
        var a = new Array();
        for (var p in this) { a.push(p); }
        return a;
    }-*/;

    @Deprecated
    public final Set<String> keySet() {
        JsArrayString array = keys();
        Set<String> set = new HashSet<String>();
        for (int i = 0; i < array.length(); i++) {
            set.add(array.get(i));
        }
        return set;
    }

    public final native String get(String key) /*-{
        return "" + this[key];
    }-*/;

    public final native String get(String key, String defaultValue) /*-{
        return this[key] ? ("" + this[key]) : defaultValue;
    }-*/;

    public final native void set(String key, String value) /*-{
        this[key] = value;
    }-*/;

    public final int getInt(String key) {
        return Integer.parseInt(get(key));
    }

    public final boolean getBoolean(String key) {
        return Boolean.parseBoolean(get(key));
    }

    public final native JSOModel getObject(String key) /*-{
        return this[key];
    }-*/;

    public final native JsArray<JSOModel> getArray(String key) /*-{
        return this[key] ? this[key] : new Array();
    }-*/;
}

This class alone allows you to easily parse JSON returned in a callback. For example, here's an example of parsing Twitter's User Timeline in my OAuth with GWT application.

private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<JSOModel> statuses = new ArrayList<JSOModel>();
            for (int i = 0; i < data.length(); i++) {
                statuses.add(data.get(i));
            }

            // populate textarea with returned statuses
            for (JSOModel status : statuses) {
                payload.setValue(payload.getValue() + status.get("text") + "\n\n");
            }
            
            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

To simply things even more, we created a BaseModel class that can be extended.

import java.util.Map;
import java.util.HashMap;

import com.google.gwt.core.client.JsArrayString;
import com.google.gwt.user.client.Element;
import com.google.gwt.user.client.DOM;

public abstract class BaseModel {

    protected JSOModel data;

    public BaseModel(JSOModel data) {
        this.data = data;
    }

    public String get(String field) {
        String val = this.data.get(field);
        if (val != null && "null".equals(val) || "undefined".equals(val)) {
            return null;
        } else {
            return escapeHtml(val);
        }
    }

    public Map<String, String> getFields() {
        Map<String, String> fieldMap = new HashMap<String, String>();

        if (data != null) {
            JsArrayString array = data.keys();

            for (int i = 0; i < array.length(); i++) {
                fieldMap.put(array.get(i), data.get(array.get(i)));
            }
        }
        return fieldMap;
    }

    private static String escapeHtml(String maybeHtml) {
        final Element div = DOM.createDiv();
        DOM.setInnerText(div, maybeHtml);
        return DOM.getInnerHTML(div);
    }
}

You can extend this class and create model objects that represent a more Java-like view of your data. For example, I could create a Status class with the following code:

public class Status extends BaseModel {
    
    public Status(JSOModel data) {
        super(data);
    }

    public String getText() {
        return get("text");
    }
}

Then I could change my JSON parsing in TwitterApiCallback to be:

    private class TwitterApiCallback implements RequestCallback {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() == 200) {
            JsArray<JSOModel> data = JSOModel.arrayFromJson(response.getText());
            List<Status> statuses = new ArrayList<Status>();
            for (int i = 0; i < data.length(); i++) {
                Status s = new Status(data.get(i));
                statuses.add(s);
            }

            // populate textarea with returned statuses
            for (Status status : statuses) {
                payload.setValue(payload.getValue() + status.getText() + "\n\n");
            }

            Label success = new Label("API call successful!");
            success.setStyleName("success");
            form.add(success);
        } else {
            onError(request, new RequestException(response.getText()));
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert("Calling API failed. " + OAuthPage.STANDARD_ERROR + "\n\n" + throwable.getMessage());
    }
}

That's how we're doing lightweight JSON parsing with GWT. I've updated my GWT with OAuth demo with this code. You can also download the source. Please let me know if you have any questions.

Update October 20, 2009: I recently had to enhance the JSOModel and BaseModel classes in my project to handle nested objects and arrays. In my project, I have a Conversation object that has a Channel and a List of Task objects. These objects are available in the JSOModel of my BaseModel, I just needed to grab them a bit differently.

public Channel getChannel() {
    return new Channel(data.getObject("channel"));
}

public List<Task> getTasks() {
    JsArray<JSOModel> array = data.getArray("tasks");
    List<Task> tasks = new ArrayList<Task>(array.length());

    for (int i = 0; i < array.length(); i++) {
        Task task = new Task(array.get(i));
        tasks.add(task);
    }
    
    return tasks;
}

To set a Channel, it's as simple as:

data.set("channel", channel.toJson().toString());

To allow setting Lists, I had to enhance JSOModel by adding the following two methods:

public final void set(String key, List<JSOModel> values) {
    JsArray<JSOModel> array = JavaScriptObject.createArray().cast();
    for (int i=0; i < values.size(); i++) {
        array.set(i, values.get(i));
    }
    setArray(key, array);
}

protected final native void setArray(String key, JsArray<JSOModel> values) /*-{
    this[key] = values;
}-*/;

After making this change, I was able to convert my List to List and set it on the underlying JSOModel.

public void setTasks(List<Task> tasks) {
    List<JSOModel> values = new ArrayList<JSOModel>();
    for (Task task : tasks) {
        values.add(task.getModel());
    }

    data.set("tasks", values);
}

To allow the task.getModel() method to work, I added a getter to BaseModel to allow retrieving the underlying JSOModel. Currently, I'm using a homegrown JSON.java class to produce JSON from my BaseModel objects. It all seems to work great and I'm pumped I can receive and send all my JSON using overlay types.

Posted in Java at Jun 24 2009, 09:52:49 AM MDT 10 Comments
Comments:

Wow! I made it on Matt Raible's blog! I think Time magazine is next ;)

Thanks a lot for the detailed information, I sure can use this and I think a lot of other people, too.

Posted by Sakuraba on June 24, 2009 at 11:01 AM MDT #

Hi,

really nice articel. For future projects I think about a lightweight communication (maybe via JSON)

On a presentation (Google I/O 2009 -> wave related) they mentioned using JSON data to communicate between client and server. (GWT 2.0 JavaScriptObject overlays will be allowed to implement a single interface).

How would this influence your solution? Have you tried to use GWT2.0 trunk in this context?

Regards
Martin

Posted by Martin on June 25, 2009 at 05:09 AM MDT #

If you are concerned about speed, and you are parsing large chunks of json, you should consider an alternative to eval().

This article suggests a 2.7x speedup by using the browser-native JSON.parse().

The performance has probably improved even more since then, but I haven't had a chance to reproduce it. The article also refers to a json parsing library that uses this native option if available, and gracefully falls back to the generic method if necessary.

Posted by Carver on July 01, 2009 at 09:06 AM MDT #

[Trackback] This post was mentioned on Twitter by jcscoobyrs: RT @mraible: Blog post about JSON Parsing with JavaScript Overlay Types in GWT: http://bit.ly/UmweO

Posted by uberVU - social comments on October 21, 2009 at 06:31 AM MDT #

I too would like to know how any changes in GWT 2.x would influence/enhance your integration point with a JSON server. Currently have a stateless RESTful server based on Jersey (producing both xml/json output) for rich java client (swing/fx) but would really like to add GWT also

Posted by aloleary on January 11, 2010 at 09:41 AM MST #

Hi Matt,

I develop with Grails for some time now ...
I am trying to begin with GWT to develop a SOFEA app having a grails app for the server side.
My question is: how do you deploy your GWT and Grails together?

1- The GWT App is in one server/port and the Grails app is in another? If so, how do you deal with Same Origin Policy?
2- Do you compile your GWT app and copy the resulting HTML+JS to your Grails App?
3- any other?

Thanks
Felipe

Posted by Felipe on February 26, 2010 at 03:05 PM MST #

@Felipe - I wrote a couple of posts that might be useful in answering your questions:

Note: I plan on replacing my ProxyServlet implementation with the UrlRewriteFilter as soon as they add support for following redirects when proxying.

Posted by Matt Raible on February 26, 2010 at 08:13 PM MST #

Hi Matt, tks for your reply. After some testing and some experiments with Grails and GWT, I have tried your approach with the ProxyServlet. After successfully trying this ProxyServlet approach, I have seen JSONP, and more specifically the JsonpRequestBuilder from the gwt library. I will try that, and I think that with it we won't need the ProxyServlet approach anymore. Have you ever tried this JSONP approach? I see it has the security implications for Cross-site request forgery, but if we follow its prevention recomendations, I think it should be a nice approach. What do you think about it? Tks again Felipe

Posted by Felipe on March 22, 2010 at 02:18 PM MDT #

great article, awesome1 Just a question. How the JSOModel to deal with JSON string like {"success":"ture","result":{"id":"123", "name":"bob"}}? looks like it dont works. Thank you for your help!

Posted by recoco on May 19, 2010 at 02:21 AM MDT #

"great article, awesome1 Just a question. How the JSOModel to deal with JSON string like {"success":"ture","result":{"id":"123", "name":"bob"}}? looks like it dont works. Thank you for your help!" I have exactly the same issue.

Posted by Cluber on December 07, 2010 at 06:32 PM MST #

Post a Comment:
  • HTML Syntax: Allowed