Raible Designs

Grails OAuth and LinkedIn APIs

Matt Raible — Tue, 22 Dec 2009 15:37:57 -0700

Back in November, I wrote about how to talk to LinkedIn APIs with GWT. A week later, I figured out how to do it with Grails and contributed a patch to the grails-oauth plugin.

Since then, a few folks have asked how I did it. Since code speaks louder than words, I took some time and 1) verified the oauth plugin works as expected and 2) created an example application demonstrating functionality. You can find the results in my fork of grails-oauth on GitHub. You can also view the example online.

Below is a quick tutorial explaining how to integrate LinkedIn into your Grails application.

Download and install Grails 1.1.2.
Run grails create-app to create your application.

Add the following to the bottom of grails-app/conf/Config.groovy:

oauth {
    linkedin {
        requestTokenUrl="https://api.linkedin.com/uas/oauth/requestToken"
        accessTokenUrl="https://api.linkedin.com/uas/oauth/accessToken"
        authUrl="https://api.linkedin.com/uas/oauth/authorize"
        consumer.key="XXX"
        consumer.secret="XXX"
    }
}

You can get your consumer.key and consumer.secret at https://www.linkedin.com/secure/developer. Make sure to set the OAuth Redirect URL to http://localhost:8080/{your.app.name}/oauth/callback for testing.

Download the oauth-plugin, extract it and build it using grails package-plugin. Install it in your project using grails install-plugin path/to/zip.

Add a link to the GSP you want to invoke LinkedIn Authentication from:

<g:oauthLink consumer='linkedin' returnTo="[controller:'profile']">
    Login with LinkedIn
</g:oauthLink>

Create grails-app/controllers/ProfileController.groovy to access your LinkedIn Profile.

class ProfileController {
    def apiUrl = "http://api.linkedin.com/v1/people/~"
    def oauthService
    
    def index = {
 
        if (session.oauthToken == null) {
            redirect(uri:"/")
        }
 
        if (params?.apiUrl) apiUrl = params.apiUrl
        
        def response = oauthService.accessResource(
                apiUrl, 'linkedin', [key:session.oauthToken.key, secret:session.oauthToken.secret], 'GET')
 
        render(view: 'index', model: [profileXML: response, apiUrl: apiUrl])
    }
 
    def change = {
        if (params?.apiUrl) {
            println("Setting api url to " + params.apiUrl)
            apiUrl = params.apiUrl
        }
        
        redirect(action:index,params:params)
    }
}

Create grails-app/views/profile/index.gsp to display the retrieved profile and allow subsequent API calls.

<html>
<head><title>Your Profile</title></head>
<body>
<a class="home" href="${createLinkTo(dir:'')}">Home</a>
<g:hasOauthError>
    <div class="errors">
        <g:renderOauthError/>
    </div>
</g:hasOauthError>

<g:form url="[action:'change',controller:'profile']" method="get">
    Your LinkedIn Profile:
    <textarea id="payload" style="width: 100%; height: 50%; color: red">${profileXML}</textarea>
    <p>
        <g:textField name="apiUrl" value="${apiUrl}" size="100%"/>
        <br/>
        <g:submitButton name="send" value="Send Request"/>
    </p>
</g:form>
</body>
</html>

Start your app using grails run-app and enjoy.

As mentioned earlier, you can download the grails-oauth-example or view it online.

One improvement I'd like to see is to simplify the parsing of XML into a Profile object, much like the linkedin gem does for Rails.

If you're interested in learning more about LinkedIn and OAuth, I encourage you to checkout Taylor Singletary's presentation LinkedIn OAuth: Zero to Hero.

Update: I updated the oauth-plugin so it's backwards-compatible with OAuth 1.0 and added Twitter to the example application to prove it. If you're seeing "Cannot invoke method remove() on null object", it's likely caused by your redirect URL pointing to an application on a different domain.

GWT OAuth and LinkedIn APIs

Matt Raible — Tue, 24 Nov 2009 15:46:05 -0700

When I worked at LinkedIn last year, I received a lot of inquiries from friends and developers about LinkedIn's APIs. After a while, I started sending the following canned response:

For API access to build LinkedIn features into your application, fill out the following form:

http://www.linkedin.com/static?key=developers_apis

For requests to build an application, go to:

http://www.linkedin.com/static?key=developers_opensocial

I talked with the API team and they did say they look at every request that's sent via these forms. They don't respond to all of them b/c they know that many people would be angry if they told them "no", so they'd rather not have that headache.

Yesterday, I was pumped to see that they've finally decided to open up their API to Developers.

Starting today, developers worldwide can integrate LinkedIn into their business applications and Web sites. Developer.linkedin.com is now live and open for business.

First of all, congratulations to the API team on finally making this happen! I know it's no small feat. Secondly, it's great to see them using Jive SBS for their API documentation and developer community. My current client uses this to facilitate development and I love how it integrates a wiki, JIRA, FishEye, Crucible and Bamboo into one central jumping off point.

I've always been a fan of LinkedIn, ever since I joined way back in May 2003. However, I've longed for a way to access my data. LinkedIn Widgets are nice, but there's something to be said for the full power of an API. Last night, I sat down for a couple hours and enhanced my Implementing OAuth with GWT example to support LinkedIn's API.

I'm happy to report my experiment was a success and you can download GWT OAuth 1.2 or view it online. For now, I'm simply authenticating with OAuth and accessing the Profile API.

In the process, I learned a couple things:

LinkedIn's OAuth implementation returns an oauth_verifier parameter after authenticating, whereas Google and Twitter do not. This parameter needs to be included when calling the Access token path.
The Profile API example I implemented gets the current user's profile with http://api.linkedin.com/v1/people/~. This returns a "light" version of your profile. To get a more detailed version, you need to use Field Selectors. For example: http://api.linkedin.com/v1/people/~:(id,first-name,last-name,picture-url,headline,summary,positions,educations)
LinkedIn's API only supports passing OAuth parameters in a header, rather than query parameters. To make this work, I modified my ProxyServlet to convert query parameters to an "Authorization" header at the end of the setProxyRequestHeaders() method.

// For LinkedIn's OAuth API, convert request parameters to an AuthorizationHeader
if (httpServletRequest.getRequestURL().toString().contains("linkedin-api")) {
    String[] parameters = httpServletRequest.getQueryString().split("&");
    StringBuilder sb = new StringBuilder("OAuth realm=\"http://api.linkedin.com/\",");
    for (int i = 0; i < parameters.length; i++) {
        sb.append(parameters[i]);
        if (i < parameters.length - 1) {
            sb.append(",");
        }
    }

    Header authorization = new Header("Authorization", sb.toString());
    httpMethodProxyRequest.setRequestHeader(authorization);
}

You might recall that my previous example had issues authenticating with Google, but worked well with Twitter. LinkedIn's authentication seems to work flawlessly. This leads me to believe that Twitter and LinkedIn have a much more mature OAuth implementation than Google.

Related OAuth News: Apache Roller 5 will be shipping with OAuth support. See Dave Johnson's What's New in Roller 5 presentation for more information.

Update December 6, 2009: I modified the gwt-oauth project to use GWT 1.7.1 and changed to the Maven GWT Plugin from Codehaus. Download GWT OAuth 1.3 or view it online.

The good ol' Job Hunt

Matt Raible — Fri, 26 Jun 2009 15:30:42 -0600

Just over two years ago, I wrote about the good ol' job hunt. Today, I find myself in a very similar situation. My Evite gig ends in a couple hours and I'm heading to The Cabin on Monday for a month of vacation. Similar to last time, there are opportunities out there, but most of them come through recruiters.

I have to admit - it seems like I got lucky the last couple times I tried to find a gig. I simply blogged I was looking and found myself negotiating with companies a few days later. Getting laid off from LinkedIn was awesome in that a few companies contacted me about hiring the whole team the next day. The ability to blog-and-get-a-gig was a great way to cut out the middle-man.

Unfortunately, I think I got spoiled by my blog-and-get-a-gig success. I figured it would happen again this time...

Not so much.

I don't know if it's because of the down economy or because I took the year off from speaking at conferences. Regardless, I'm in an interesting situation since I signed a 1-year lease on an office in downtown Denver. It's easy to market to companies in Denver, but if I land a gig here, there's a good chance the client is going to want me in the office everyday. That leaves me wondering: what's the best way to market to companies outside of Colorado? My ideal contract is one that 1) allows me to work remotely and 2) only requires me to travel once or twice a month.

I believe my ideal gigs are out there, but I think it's difficult to convince companies I'm a good remote worker. In an attempt to convince them otherwise, I'd like to offer recommendations from my last two clients.

"Matt contributed dramatically to the engineering practice at Evite, both directly and indirectly. Directly, he lead the effort to define and prove a successful new web UI architecture for us. Indirectly, he brought senior engineering talent into the organization, and energized the existing team. We are significantly better positioned to deliver on our Product goals as a result of having worked with Matt. I'd love to work with him again someday." David Thomas, VP of Technology, Evite

"Matt has unique abilities in the realm of software engineering. He brings great energy, an amazing breadth of knowledge in UI technologies, along with a can-do attitude that's refreshing to interact with. He is a great leader and communicator. In addition, Matt can apply his deep understanding of the technologies in question pragmatically to the engineering problem at hand, leading his team in execution and delivery. He's an excellent technical visionary to complement any team." Arnold Goldberg, Vice President Platform Engineering, LinkedIn

If you're looking to hire someone with my skills, let me know - there's a good chance you'll be glad you did.

Creating a Facebook-style Autocomplete with GWT

Matt Raible — Fri, 5 Jun 2009 07:05:10 -0600

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

Bye Bye Dream Machine

Matt Raible — Mon, 26 Jan 2009 22:18:33 -0700

This evening, I'm shipping back one of my favorite machines of all time. I received a fully-loaded Mac Pro as part of my employment with LinkedIn last June. It was necessary to run the LinkedIn application locally and I thoroughly enjoyed using it for the last 6 months. With 12GB of RAM and two 23" monitors, it was a great employee perk.

When I became a contractor again, they let me take my dream machine home. I promptly plugged in my 30" monitor and I've been loving my home work environment ever since. I could have bought the machine from LinkedIn, but I discovered I can buy a brand new machine with similar specs for less than their asking price.

The good news is I'm now able to answer the question I asked a couple years ago: One 30" monitor or two 23" monitors? IMO, one 30" monitor is definitely better and two 30" monitors would be awesome.

In addition to the Mac Pro, I'll also be shipping back the 15" MacBook Pro they gave me. This leaves me with my 17" MacBook Pro and an old HP Pavilion with Windows XP. I was hoping to plug my 30" into the HP, but I discovered I don't have a DVI card that will handle it. Over the next few months, I do plan on buying a new MacBook Pro (for work) and a Mac Pro (for home). With my running commute, I need to leave one machine downtown and I like to have one at home for the kids + late night hacking.

I'm currently having a hard time deciding if I should buy a MacBook Pro now or make do with what I have and just buy a new DVI card for my Windows box. I'm leaning towards a new 15" MacBook Pro (17" is too big to travel with). If I could get one with a 256GB SSD, I'd definitely be sold.

What would you do?

LinkedIn Cuts 10% (a.k.a. The Journey is Over)

Matt Raible — Wed, 5 Nov 2008 15:10:06 -0700

This morning, my co-workers and I discovered that LinkedIn decided to trim 10% of its employees. The Denver Office was among those that were laid off. I can't say we didn't see the writing on the wall. In fact, on the evening of October 15, I sent the following e-mail to my co-workers:

LinkedIn's top investor[1] is Sequoia Capital and they recently posted this presentation on the web.

http://www.slideshare.net/eldon/sequoia-capital-on-startups-and-the-economic-downturn-presentation?type=powerpoint

Notice the reduce head count recommendations. ... Oh well, life goes on.

Raible

[1] http://www.linkedin.com/static?key=investors

So, as of today, there is no LinkedIn Denver office. While I had a lot of fun being a UI Architect and managing Engineers, I'm somewhat happy this has happened. After all, now I get to enjoy the best perk about being an employee: the good ol' severance package!

If you're looking for good Engineers, I highly recommend all of the guys who worked for me during this journey. You can read more about the skills they possess and what they're looking for by viewing their LinkedIn Profiles:

Scott Nicholls

Bryan Noll

James Goodwill

As for me, I'm definitely in the market as well. You can view My LinkedIn Profile to see if I might be a good fit for your organization. I'm willing to travel up to 25%, but would prefer not to. After all, ski season is right around the corner.

Lastly, I just wanted to say that I really enjoyed working at LinkedIn. I've never worked with a smarter group of Engineers, nor been so excited about a company's product and vision. I know that LinkedIn will be highly successful and I hope to use their site to find gigs for many years to come.

Building LinkedIn's Next Generation Architecture with OSGi by Yan Pujante

Matt Raible — Mon, 20 Oct 2008 11:20:09 -0600

This week, I'm attending the Colorado Software Summit in Keystone, Colorado. Below are my notes from an OSGi at LinkedIn presentation I attended by Yan Pujante.

LinkedIn was created in March of 2003. Today there's close to 30M members. In the first 6 months, there was 60,000 members that signed up. Now, 1 million sign up every 2-3 weeks. LinkedIn is profitable with 5 revenue lines and there's 400 employees in Mountain View.

Technologies: 2 datacenters (~600 machines). SOA with Java, Tomcat, Spring Framework, Oracle, MySQL, Servlets, JSP, Cloud/Graph, OSGi. Development is done on Mac OS X, production is on Solaris.

The biggest challenges for LinkedIn are:

Growing Engineering Team on a monolithic code base (it's modular, but only one source tree)
Growing Product Team wanting more and more features faster
Growing Operations Team deploying more and more servers

Front-end has many BL services in one webapp (in Tomcat). The backend is many wars in 5 containers (in Jetty) with 1 service per WAR. Production and Development environments are very different. Total services in backend is close to 100, front-end has 30-40.

Container Challenges
1 WAR with N services does not scale for developers (conflicts, monolithic). N wars with 1 service does not scale for containers (no shared JARs). You can add containers, but there's only 12GB of RAM available.

Upgrading back-end service to new version requires downtime (hardware load-balancer does not account for version). Upgrading front-end service to new version requires redeploy. Adding new backend services is also painful because there's lots of configuration (load-balancer, IPs, etc.).

Is there a solution to all these issues? Yan believes that OSGi is a good solution. OSGi stands for Open Services Gateway initiative. Today that term doesn't really mean anything. Today it's a spec with several implementations: Equinox (Eclipse), Knoplerfish and Felix (Apache).

OSGi has some really, really good features. These include smart class loading (multiple versions of JARs is OK), it's highly dynamic (deploy/undeploy built-in), it has a service registry and is highly extensible/configurable. An OSGi bundle is simply a JAR file with an OSGi manifest.

In LinkedIn's current architecture, services are exported with Spring/RPC and services in same WAR can see each other. The problem with this architecture comes to light when you want to move services to a 2nd web container. You cannot share JARs and can't talk directly to the other web app. With OSGi, the bundles (JARs) are shared, services are shared and bundles can be dynamically replaced. OSGi solves the container challenge.

One thing missing from OSGi is allowing services to live on multiple containers. To solve this, LinkedIn has developed a way to have Distributed OSGi. Multicast is used to see what's going on in other containers. Remote servers use the OSGi lifecycle and create dynamic proxies to export services using Spring RPC over HTTP. Then this service is registered in the service registry on the local server.

With Distributed OSGi, there's no more N-1 / 1-N problem. Libraries and services can be shared in one container (memory footprint is much smaller). Services can be shared across containers. The location of the services is transparent to the clients. There's no more configuration to change when adding/removing/moving services. This architecture allows the software to be the load balancer instead of using a hardware load balancer.

Unfortunately, everything is not perfect. OSGi has quite a few problems. OSGi is great, but the tooling is not quite there yet. Not every library is a bundle and many JARs doesn't have OSGi manifests. OSGi was designed for embedded devices and using it for the server-side is very recent (but very active).

OSGi is pretty low-level, but there is some work being done to hide the complexity. Spring DM helps, as do vendor containers. SpringSource has Spring dm Server, Sun has GlassFish, and Paremus has Infiniflow. OSGi is container centric, but next version will add distributed OSGi, but will have no support for load-balancing.

Another big OSGi issue is version management. If you specify version=1.0.0, it means [1.0.0, ∞]. You should at least use version=[1.0.0,2.0.0]. When using OSGi, you have to be careful and follow something similar to Apache's APR Project Versioning Guidelines so that you can easily identify release compatibility.

At LinkedIn, the OSGi implementation is progressing, but there's still a lot of work to do. First of all, a bundle repository needs to be created. Ivy and bnd is used to generate bundles. Containers are being evaluated and Infiniflow is most likely the one that will be used. LinkedIn Spring (an enhanced version of Spring) and SCA will be used to deploy composites. Work on load-balancing and distribution is in progress as is work on tooling and build integration (Sigil from Paremus).

In conclusion, LinkedIn will definitely use OSGi but they'll do their best to hide the complexity from the build system and from developers. For more information on OSGi at LinkedIn, stay tuned to the LinkedIn Engineering Blog. Yan has promised to blog about some of the challenges LinkedIn has faced and how he's fixed them.

Update: Yan has posted his presentation on the LinkedIn Engineering Blog.

Why such a busy week?

Matt Raible — Mon, 25 Aug 2008 07:54:18 -0600

Lance sent me the following e-mail this morning:

Just saw your status message, I think you'd better blog an explanation!

"Matt Raible is getting ready for one of the busiest weeks of his life."

I couldn't think of a good reason not to blog an explanation, so here goes.

I'm currently sitting at Denver's airport ready to hop on a flight to Mountain View. I'll be at LinkedIn's HQ for two days helping tidy up plans for a release in early September that I'm in charge of. It's my first time being a Release Owner, but it should be pretty painless so I'm not too worried. Whenever I travel to Mountain View, I always have a good time, but I'm constantly being pulled into meetings, or arranging meetings myself. I have a presentation to write that I'm delivering on Wednesday. It describes the changes we've made to make the backend of LinkedIn much more stateless and therefore better and faster.

In my hotel room at night, I'll be writing my presentations for the Colorado Software Summit. Having to write 3 presentations in one week always makes me feel super-busy.

On Wednesday night, I fly home late (likely after a game of hoops with co-workers). Thursday is Jack's birthday, so I hope to take the day off and spend the day with him. Thursday night is sleepover night. Being a single parent with two kids is never easy, but always fun. Friday it's back to work, wrapping up things for the week (status reports, bug fixes, etc.) and likely marveling at the traffic from the DNC.

Friday night there's a 9-hour cocktail party with a good friend from Vancouver, BC.

Saturday is the super-busy day. It's time for Jack's birthday party and if last year is any evidence, it's one of those 8-hour cleaning and decorating situations.

I'm sure I'll have a busier week sometime in the future, but this one will surely be one to remember (especially since it's blogged into history now ).

LinkedIn Tech Talk: Kevin Brown on Shindig

Matt Raible — Thu, 17 Jul 2008 07:04:24 -0600

Last Thursday, Kevin Brown visited LinkedIn's Mountain View office to do a presentation on Shindig, an OpenSocial Reference Implementation. Below are my notes from his talk.

In September 2007, Google started thinking about Social APIs. Google Gadgets would be better with access to Social Data ... but that's just Google. It was recognized that this is something that many others would like access to. OpenSocial was announced in November 2007. It's an open standard for developer platforms that has a strong emphasis on "social" data. It's based on gadgets which is now covered by The Open Social Foundation.

In November, many Googlers started working on a Google Code project based on Java and iGoogle. However, there was too much proprietary code. In December, Brian McCallister of Ning created an ASF Proposal for Shindig. It was a rough port of iGoogle but with Ning's PHP code. This turned out to be a great starting point. It immediately got interest from Google, Hi5, MySpace and others. While most committers are still from Google, there are 12 developers that work on it full time and they're adding 2 committers each month. Shindig is a Java/PHP implementation of OpenSocial. Open Campfire is an Apache-licensed .NET implementation that hopes to eventually merge into Shindig.

LinkedIn has the Biggest Rails app in the World

Matt Raible — Tue, 24 Jun 2008 13:25:16 -0600

From the LinkedIn Engineering Blog:

Bumper Sticker started as a small experiment in August, 2007. Facebook had released their development platform while we were hard at work on our own. We were curious to experiment and discover some of the characteristics of an application platform built on a social network and to see what, if any, learning we could apply to our own efforts. After noticing that professional and business-related applications weren't flourishing in the Facebook ecosystem, a few of our Product folks put their heads together while out for a run; one engineer, one week, and a few Joyent accelerators later, Bumper Sticker was born.

We'd be lying if we said that anyone was prepared for the kind of success Bumper Sticker has had since then - though we should have expected it, given the excellent Product team here at LinkedIn. Here's a quick snapshot of Bumper Sticker statistics at this moment: Read More »

The "biggest Rails app in the world" claim comes from this video.

In addition to having a kick-ass RoR team at LinkedIn, we also do a lot with Java and love our Macs. Why wouldn't you want to work here?

If you find a gig you like, or simply have mad programming skills, contact me and I'll see if I can hook you up. And yes, we are hiring at LinkedIn Denver.

Is it possible to replace the syntax parser in Eclipse or IDEA's JSP Plugin?

Matt Raible — Mon, 23 Jun 2008 12:21:36 -0600

At LinkedIn, we have our own JSP Compiler. Our version of JSP is more like FreeMarker than JSP since it solves many of the deficiencies of JSP. Since we allow a different syntax than standard JSP (more powerful EL, new tags for looping, loading from classpath), we (like FreeMarker) don't get much love from IDEs.

We don't get much in the way of syntax-highlighting or code completion. However, since we use JavaCC/JJTree for parsing, I'm wondering if Eclipse or IDEA (or even NetBeans) allows replacing the default syntax definition with a new one.

Has anyone extended one of these IDEs to enhance its JSP syntax highlighting and compilation? If so, I'd love to hear about it. If not, it's likely we'll be doing it in the near future.

Traveled Coast to Coast Last Week

Matt Raible — Mon, 23 Jun 2008 11:40:22 -0600

Last week I traveled from Denver to Mountain View with the UI Frameworks Team. It was the first time all four of us traveled together and we had a great time. On Monday night, I helped LinkedIn Softball beat the only undefeated team in our league. Pitchers were a flowin' at a nearby beer garden afterwards to celebrate.

On Wednesday evening, we returned to Denver and I enjoyed a night with the kids before flying out on Friday to Boston for the American Craft Beer Fest. The flight was no fun as it took me 12 hours door-to-door (on the way home too). I stayed with friends in Concord, Mass. and enjoyed the "country livin'" a whole lot. They had just moved in a few weeks before, and it was awful nice being in an area with huge lots (1 acre +) and neighbors that come over and talk for hours.

On Saturday, we had a great time at the ACBF and even ran into some good friends (Chad and Mike) from Denver. Amazingly enough, both of these guys are going to Oktoberfest at the same time we are. Below are some photos from the festivities (more on Flickr).

On Sunday, we went to the Old North Bridge and learned some fascinating stuff about the beginning of the American Revolution. Did you know that 1/2 of the British Regiment that went to Concord stopped for pints at 9 in the morning? If they hadn't, they would have been able to surprise the Minutemen from behind and the American Revolution may never have happened. I also liked the fact that after the British were fired upon they ran back to town and then had breakfast for a couple hours. Maybe I should modify this Wikipedia page to add these tidbits we learned from our tour guide?

All in all, it was a great week of traveling. The flights to and from Boston were way to long, but the memories I created were worth it. I'll be in Denver all week, enjoying Bike to Work Day on Wednesday and then heading off on vacation for a week. I love summertime.

LinkedIn's Engineering Blog

Matt Raible — Fri, 13 Jun 2008 08:30:19 -0600

Have you been curious about LinkedIn's architecture or how they're using Grails and Rails? If so, you might be interested in LinkedIn's Engineering Blog. Over the past couple of weeks, a few Engineers have starting writing about our architecture, OpenSocial, RailsConf, YUI, Grails and OSGi. Below is a complete listing of Engineering posts.

If there are topics you'd like to see us blog about, please let me know. I've somehow landed in the role of Editor for the Engineering Blog, so I should be able to hook you up if I can find an engineer to blog about what you're interested in.

On a related note, Rob Getzschman's entry LinkedIn discovers the truth about Cannes is quite entertaining. Highly recommended.

Share on LinkedIn

Matt Raible — Thu, 5 Jun 2008 22:13:04 -0600

This is a test to see if I can get the Share on LinkedIn widget working on this site. Click below to invoke.

Share on LinkedIn

Seems to work pretty well. I like how you can select text and it'll automatically populate the summary. You can drag the link above to your toolbar in Safari and Firefox if you want to use it like a favelet.

Now I just need to get one of the designers to create a nifty little "Share on LinkedIn" icon so I can add it to all my entries by default.

Why no more than 500 connections?

Matt Raible — Sun, 1 Jun 2008 18:40:59 -0600

I recently updated my status on LinkedIn to read:

Matt is determined not to have 500+ connections. Will start removing connections soon.

A couple of days later, I received the following message from a connection:

I noticed the other day you mentioned that you are determined to not have over 500 LinkedIn connections. I'm wondering what the reason is? Not just because LinkedIn shows 500+ after that, is it? As you work for LinkedIn, I assume there's some other reason. I'm interested to know what it is...

I joined LinkedIn May 27, 2003, 22 days after it initially launched. For the first few years, I accepted invitations when I received them. Some folks I knew, some I didn't. When I started consulting for LinkedIn last summer, I had somewhere between 200 and 300 connections. Most of them were people who had contacted me, not folks I had contacted.

One day, I used the import webmail contacts feature to pull in my contacts from Gmail. My number of connections quickly jumped by 100 and it's increased quite a bit since then (mostly due to co-workers from LinkedIn). Of the almost 500 connections I have, I believe there's a good 100-200 of them that are folks I don't know, have never had contact with, and will likely never benefit from being "connected" with.

I guess the main reason I'm planning on trimming my connections is to make my network higher quality. I admit I'm somewhat motivated by the 500+ icon, but it's also a genuine feeling that there's quite a few folks I won't benefit from being connected to. I'm not a LION after all. I believe my LinkedIn network should resemble my real-world network.

What's your opinion? Should I have folks in my network that know me, but I don't know them?

LinkedIn Groups

Matt Raible — Tue, 20 May 2008 14:16:18 -0600

This afternoon, I noticed there's a LinkedIn "GlassFish" group now available and it reminded me of a couple things:

LinkedIn currently doesn't have a way to search for groups, but Jason Bailes has setup a LinkedIn Groups Search with Google Custom Search. Thanks Jason!
I created a Apache Software Foundation group on LinkedIn a few months ago. If you're a committer or member, you're more than welcome to join the group.

LinkedIn Groups don't provide a whole lot of functionality at this point, but I've heard there's big things in store for them. Chances are they'll be very valuable in the future.

Happy Cinco de Linko!

Matt Raible — Mon, 5 May 2008 08:40:46 -0600

Today is packed full of celebrations and traveling for me. First of all, it's Cinco de Mayo, which is always a good excuse for a margarita. Secondly, I'm heading to Mountain View to assist my co-workers in celebrating Cinco de Linko. After that, I'm pitching in a LinkedIn softball game trying to beat the #1 team in our league. Then I'll be heading up to the festivities of JavaOne. It's likely I won't be "networking" until late night. I'll be twittering all week if want to know where the good parties are.

Update: I just learned how to find out when you first joined LinkedIn. Click on "Account & Settings" at the top of any LinkedIn page. On that page, in the upper-right corner in grey text, you should see "User since: " and the date you joined. Interestingly enough, I joined May 27, 2003 - just 22 days after the launch.

The LinkedIn Journey Continues

Matt Raible — Thu, 6 Mar 2008 08:00:49 -0700

As you might know, I've spent the last several months working for one of the coolest clients ever: LinkedIn. They hired me back in July 2007 and I was impressed on day one. I was originally hired to help them evaluate open source Java web frameworks and try to determine if moving from their proprietary one to an open source one would help improve developer productivity.

After looking at all the options, I recommended we look at Struts 2 and Spring MVC - primarily because they seemed to be the best frameworks for a LinkedIn-type of application. Another Engineer and I prototyped with Struts 2 for about 6 weeks and came up with a prototype that worked quite well. While our mission was successful, we found a couple issues with Struts 2 and standard JSP that might actually hurt developer productivity more than it helped.

Following this project, I worked on the New Homepage Team, which is now visible to everyone that logs onto LinkedIn. My role was minimal, but it was still a very fun project to work on. You know those widgets in the right panel? I did the initial UI and backend integration for those. All the business logic, Ajax/JavaScript, CSS, and optimization was done by other folks on the team. Shortly after this project went live in November, I started prototyping again with Spring MVC + JSP.

The reason I was asked to prototype with Spring MVC was because they were using Spring on the backend, Spring MVC in a couple other projects, and a new project was being kicked off that used Grails. Rather than add another framework (Struts 2) to the mix, they wanted to see if they could suppress any further framework proliferation.

After a month of prototyping with Spring MVC + JSP, my results weren't as good as Struts 2. With Struts 2, I was able to use OGNL to do all the things their current JSP implementation allows them to do (call methods with arguments, use statics in EL, etc.). With standard JSP, a lot of this wasn't possible. If it was - it required writing lots of tag libraries and made it more cumbersome for developers to do certain things. At the end of that project, I determined that using FreeMarker might solve these problems. I also determined that neither Struts 2 nor Spring MVC would solve the ultimate problem of developer productivity. Neither framework would allow developers to go from make-a-change-and-deploy, wait-3-minutes-to-see-change-in-browser to make-a-change, save and wait-15-seconds-to-see-change-in-browser.

I recommended that this be the ultimate goal - to get rid of the deployment cycle and to allow minimal turnaround when deploying modified classes. After that problem was solved, it's true that moving to an open source web framework would likely provide an easier-to-remember API. However, the problem with moving to a new web framework would be that everything used to construct the existing site would suddenly become legacy code.

In the end, we concluded that the best solution might be to enhance the existing framework to be more like the available open source options. This would allow existing applications to keep using their code -- and if we enhance properly -- new applications can use a simpler, less verbose API and a templating framework that's easier to understand. We can make LinkedIn's version of JSP more like standard JSP while allowing its powerful EL to remain. We can add support for JSP Tag Libraries and Tag Files.

One of the benefits of moving to an open source web framework is there's a community, documentation and books that describe the best (or most common) ways to solve problems with the framework. LinkedIn has this, but it's all in code and no one seems to have a high-level of confidence that the way that they did it is the "best" way. Developers communicate well, but all the knowledge is stuck in their heads and inboxes - there's no way for new developers to search this knowledge and figure it out on their own without asking somebody.

By adopting an open source web framework, it's possible to solve part of this problem, but I think it's still going to exist - where a few engineers know how to use the framework really well (for the specific application) and the rest don't. We determined that regardless of open source vs. proprietary framework, what was needed was a set of developers that acted as authorities on how to develop web applications at LinkedIn. A UI Frameworks Team if you will. This would be their only job and they would never get pulled from this to work on projects or complete tasks related to LinkedIn's products. Some developers mentioned that they'd been asking for this for years, and some folks had even been hired for this. However, the formulation of this group has never happened and it's obvious (now more than ever) that it'd be awesome to have them.

The UI Frameworks Team
At the end of 6 months, it seemed my work was done at LinkedIn. I liked the idea of a UI Frameworks Team and recommended they start it with the authors of the existing web framework. They agreed this was a good idea. A few days later, I was pulled into the CTO's office and he offered me the job. He offered me the challenge of building this team and told me I could do it remotely (from Denver) and hire my own people to help me with it. I gulped as I realized I'd just been offered the opportunity of a lifetime. I knew that while this might not be the best option for LinkedIn, it certainly was an excellent opportunity for me. I said I'd think about it.

In the meantime, I was given a project which you might've read about. They asked me to migrate a Rails application to Grails and try to determine if they really needed both frameworks. I spent 2 weeks coming up to speed on both and flew to Mountain View to deliver my conclusion. Here's an excerpt from an internal blog post I wrote.

As far as I know, Rails has been used at LinkedIn for well over 6 months and Grails has been used for a similar duration. Both projects that've used these technologies have enjoyed extreme success. Both projects have been fun for the developers working on them and both have improved the technologies/frameworks they're using.

Here's an interesting quote about the Rails application:

Another app you might want to look at is BumperSticker, our facebook app. Interestingly we heard through joyent that DHH (the creator of Rails) told them that BumperSticker is the biggest rails app in the world (in terms of page views) - we are closing in on 1 billion monthly page views and we have 1 million unique users per day (about 10 million installs on FB). It's a little trickier to setup in a dev environment since you need to be running on FB, but the code itself is pretty interesting since we've iterated on it a bunch of times and are making extensive use of third party libraries such as memcached.

This quote loosely translates to "We have some Rails Ninjas on staff and we've been quite successful in developing with it and making it scale".

Both platforms have allowed developers to iterate quickly and turbo-charge their productivity.

My Conclusion: Allow Both

Why?

If you have talented developers that can whip out kick-ass code with either platform, pay them and pay them well. Passion is the most important part of any job. If developers are passionate about the application they're developing and the language they're using (notice language is secondary) - they can do great things.

I know this probably isn't the answer you wanted to hear, but it's what I believe. I think both frameworks are very similar. I believe the knowledge you gain from learning one framework is transferable to the other. A lot of the things I learned about Rails worked with Grails. Ruby's syntax is similar to Groovy's.

There's a natural synergy between these two frameworks. The hard part is figuring out when to use which one.

The application that I was asked to port from Rails to Grails? The one that was launched last week - LinkedIn Mobile.

After doing this research, I stepped up to the plate and accepted the offer to start a UI Frameworks Team and recruited some kick-ass Java Developers I know to be the founding members. Last week, I flew out to Mountain View to do some kickoff meetings and start getting the infrastructure in place so we can document, support and release code like a well-oiled open source project. There's nothing saying we won't use an open source web framework as the underlying engine, but I think this should be an excellent chance to see the power of open source governance and development style in a corporate environment.

Director of Engineering, Core Experience
I should mention one last thing. If you're an experienced Java Developer/Architect with a passion and deep knowledge of UI development (JavaScript, CSS, HTML), we've got a Director of Engineering, Core Experience position with your name on it. I might even get to interview you if you apply for this job. Furthermore, whoever gets hired will likely work very closely with my team. What's not to like about that!?

What's the Java Job Market like in Denver?

Matt Raible — Fri, 22 Feb 2008 21:59:22 -0700

I recently received an e-mail from someone asking me a number of questions about Denver's Java Job Market. He's moving from Seattle to Denver and asked me the questions below. Since Denver is one of the best places to live on Earth, I figured some other folks might like to hear my answers.

-------------------------------------

For senior architect types, is the market strong?
I believe it is. I haven't looked for a local gig in quite some time, but when I did back in June - there was lots of opportunities.

Any good employers you could recommend?
Not really, I've done contracting for the most part for the last 11 years. I've always enjoyed smaller companies. The best place to find Java jobs is by subscribing to the Denver JUG Jobs mailing list. There's jobs posted several times per week (both full time and contract).

Any companies to avoid?
Not that I know of.

For senior types, what type of salaries or hourly rates should I expect to find?
I think you'll be lucky to make over $100K as a full-time employee. You can certainly work your way to 110-120K after a couple years, but I think it's tough to hire into that. I'd expect 90+. As a contractor, you can expect $60-70/hour. There's definitely opportunities to get 90-100/hour, but they're hard to find because you have to eliminate the middle-man (recruiters).

Are Colorado Springs or Boulder good options for looking for jobs?
Boulder is definitely hopping. Colorado Springs - not so much.

Are contract positions good in Denver?
I've always liked contract positions.

Any recruiters that would be good talk to?
Lauren Ford is a good resource I've worked with in the past. You can tell her I sent you if you send her an e-mail.

Anything else you'd recommend?
If you can, get a gig downtown. Baseball Season starts in April and downtown has a buzz about it that's very enjoyable. Either that or Golden so you can be close to Mountain Biking.

-------------------------------------

One thing I forgot to mention in my reply is how valuable LinkedIn has become when searching for jobs. I've always believed being well connected is the key to career success and LinkedIn allows you to use the power of network very easily. You may think I'm biased because I work there - but how do you think I got the job there in the first place?

Google Calendar Sync for BlackBerry

Matt Raible — Wed, 12 Dec 2007 13:41:49 -0700

A couple of days ago, I said the Network Updates feature on LinkedIn's New Homepage is kinda boring. I still agree with this, but I think the new LinkedIn News provides some real value. Today my homepage had a link to Google Calendar Sync for BlackBerry. This is something I've been looking for for quite some time.

If you have a BlackBerry, you can install it from http://m.google.com/sync. I use Spanning Sync to allow me to synch my Google Calendar in iCal and it works awesome. Having a sync to my BlackBerry was the missing part that I really wanted - and now I have it. Thanks LinkedIn! (and Google of course)

As far as LinkedIn's New Homepage, I think the biggest improvement would be to add Atom/RSS Feeds so I could get all my homepage updates (news, network updates and widget updates) in NetNewsWire. I asked about this last week and they said this should be coming in Q1 2008.

LinkedIn's New Homepage

Matt Raible — Mon, 10 Dec 2007 11:16:02 -0700

Dion has a post about LinkedIn's New Homepage. In addition to Dion's post, this seems to be a popular topic on Techmeme. I'm proud to say I played a small part in this project and enjoyed working with the fabulous "Homepage Team" that put this together. We celebrated the launch last week while I was out in Mountain View.

To learn more about LinkedIn's New Homepage and the News feature, see the LinkedIn Blog.

Back to Dion's post. He says:

The network connections portion shows me what is wrong with LinkedIn. On Facebook I can see interesting things that my friends have done. On LinkedIn, I see that a connection has added another 6 connections. Who cares?

I agree that Network Updates are kinda boring on LinkedIn. However, I don't find my Facebook News Feed very interesting either. Is your Facebook News Feed interesting? If so, why?

Going to see Fake Steve Jobs Tonight

Matt Raible — Tue, 6 Nov 2007 14:41:15 -0700

One of the perks of working at LinkedIn, and being out in Mountain View this week, is I get to attend a talk tonight where Guy Kawasaki interviews Fake Steve Jobs. It's my first trip to the Computer History Museum, so it should be a fun show. If you're not able to make it tonight, it looks like they'll be live streaming the event from the LinkedIn blog.

My only question is - do you think FSJ will act like Jobs or himself? I'm hoping for the former.

Update: That was an awesome event. Thanks to LinkedIn for hosting it and for Fake Steve for the great stories behind his journey.

Update 2: The LinkedIn Blog has videos from the event.

Want a kick-ass Java/UI Engineering Job in Mountain View?

Matt Raible — Fri, 17 Aug 2007 10:24:00 -0600

The last month working at LinkedIn has been an absolute blast. I'm new to the whole "treating developers like royalty" thing, so that's taken a while to get used to. It's definitely nice, especially when the company gives you ownership of the things you're working on. Sure, there's schedules and priorities, but it seems like each and every engineer has control of their own destiny. As a consultant, I've been very impressed with the way I've been embraced and folded into the team like a regular employee. There's lots of team lunches, a tech meetup every now and then, and I even played hoops with a bunch of guys last night. This is probably the coolest company I've ever worked for.

Wanna have fun like I am? LinkedIn is looking to hire quite aggressively over the next several months. There's new faces almost every week and hopefully I can "hook you up" to be a part of the festivities. Below is a position that we're currently hiring for in the UI Engineering team. Working remotely is not an option at this time, you need to live in (or relocate to) the Bay Area.

LinkedIn is an online network of more than 11 million experienced professionals from around the world, representing 150 industries. We are four years old, profitable and one of the fastest growing pre-IPO Web 2.0 companies in Silicon Valley.

LinkedIn is developing the UI infrastructure for our next generation applications. This is a strategic initiative that will enable LinkedIn to develop highly interactive and intuitive applications leveraging the latest Web UI technologies. We are looking for a world-class software engineer to work on this critical component of our infrastructure, in partnership with one or more technical leads, the engineering and the product team.

POSITION REQUIREMENTS:

EXPERIENCE:
- 3+ years of overall professional work experience
SKILLS & ABILITIES:
- In depth and hands on knowledge of Java, the J2EE platform and experience working with relevant tools (IDEs, ant, junit, etc.)
- A passion for UI frameworks: JSF and Facelets experience preferable.
- In depth knowledge of JSP, JSTL.
- Experience with Ajax.
- Experience with portal technologies.
- I18n experience a plus.
- Solid understanding of design, coding and testing patterns
- Ability to work in a fast paced, test-driven collaborative and iterative programming environment
- Ability to effectively interact with product managers and other organizational units such as QA and CS
- Excellent communication skills
EDUCATION:
- B.S./M.S in Computer Science or equivalent experience.

I don't know if JSF and Facelets experience is still a requirement (now that I'm here ), but a passion for UI frameworks and web development is. You should know at least two leading Java frameworks and have a lot of experiencing with testing web applications out-of-container. We're not looking for Java Developers turned web developers, we're more looking for Web Developers that know Java.

If this sounds interesting to you, shoot me your resume in an e-mail. Don't forget to include a link to your LinkedIn Profile.

First Day at LinkedIn

Matt Raible — Mon, 9 Jul 2007 23:51:18 -0600

Today was my first day onsite at LinkedIn in Mountain View, California. I'm very impressed by two things so far: they gave me a new MacBook Pro and Sushi is on tap for lunch tomorrow. Of course, there's a lot more impressive things going on there, but the new MacBook was today's highlight. The strange thing is I don't need one - I just got a new 17" a few months ago. Nevertheless, I received and configured a new 15" today. It's not the machine that impresses me, but the company's willingness to buy the best machines for its developers.

I was introduced to almost the entire company this morning, and I only saw one Windows machine in a sea of Macs. My favorite quote? "If the MacBook Pro isn't fast enough for you, we can see about getting you a Mac Pro." I like a company that knows what developers like and doesn't have a problem treating them well.

The last time I received a new computer as part of a contract or full-time position? I believe that was way back in 2002. Working at LinkedIn seems like a developer's paradise. Does your company provide new MacBook Pros and Cinema Displays to its developers?

Summer Gigs

Matt Raible — Mon, 18 Jun 2007 13:10:32 -0600

I'm happy to report that after posting The good ol' Job Hunt last week, I had a prosperous week searching for my next gig. I got an offer from a local company in Denver on Tuesday and had 5 interviews on Thursday with various companies. I ended up accepting an offer from LinkedIn for a 3-month contract that starts in a few weeks (thanks for the hookup Brian!).

I also landed a 2-week contract that'll keep me occupied through the end of June. There's a nice week between the two gigs where we're heading to The Cabin for the 4th. I'm currently working on getting an "anchor desk" at the Hive Coorperative, but unfortunately won't be able to use it much in July. I'll be in Mountain View the first couple weeks, followed by Portland (for OSCON) at the end of the month.

Anyone out there interested in a tech meetup in Mountain View in July? My birthday is the 16th - maybe we could meet up around then?

Update: My first day at LinkedIn rocked.