Raible Designs

The good ol' Job Hunt

Just over two years ago, I wrote about the good ol' job hunt. Today, I find myself in a very similar situation. My Evite gig ends in a couple hours and I'm heading to The Cabin on Monday for a month of vacation. Similar to last time, there are opportunities out there, but most of them come through recruiters.

I have to admit - it seems like I got lucky the last couple times I tried to find a gig. I simply blogged I was looking and found myself negotiating with companies a few days later. Getting laid off from LinkedIn was awesome in that a few companies contacted me about hiring the whole team the next day. The ability to blog-and-get-a-gig was a great way to cut out the middle-man.

Unfortunately, I think I got spoiled by my blog-and-get-a-gig success. I figured it would happen again this time...

Not so much.

I don't know if it's because of the down economy or because I took the year off from speaking at conferences. Regardless, I'm in an interesting situation since I signed a 1-year lease on an office in downtown Denver. It's easy to market to companies in Denver, but if I land a gig here, there's a good chance the client is going to want me in the office everyday. That leaves me wondering: what's the best way to market to companies outside of Colorado? My ideal contract is one that 1) allows me to work remotely and 2) only requires me to travel once or twice a month.

I believe my ideal gigs are out there, but I think it's difficult to convince companies I'm a good remote worker. In an attempt to convince them otherwise, I'd like to offer recommendations from my last two clients.

"Matt contributed dramatically to the engineering practice at Evite, both directly and indirectly. Directly, he lead the effort to define and prove a successful new web UI architecture for us. Indirectly, he brought senior engineering talent into the organization, and energized the existing team. We are significantly better positioned to deliver on our Product goals as a result of having worked with Matt. I'd love to work with him again someday." David Thomas, VP of Technology, Evite

"Matt has unique abilities in the realm of software engineering. He brings great energy, an amazing breadth of knowledge in UI technologies, along with a can-do attitude that's refreshing to interact with. He is a great leader and communicator. In addition, Matt can apply his deep understanding of the technologies in question pragmatically to the engineering problem at hand, leading his team in execution and delivery. He's an excellent technical visionary to complement any team." Arnold Goldberg, Vice President Platform Engineering, LinkedIn

If you're looking to hire someone with my skills, let me know - there's a good chance you'll be glad you did.

Creating a Facebook-style Autocomplete with GWT

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

My Eye Surgery Experience

On May 7, I visited my local TLC Laser Eye Center for eye surgery. I began looking into eye surgery way back in December. At that time, I wore my glasses for a week, then waltzed into my local TLC and tried to get it done the next day. I quickly found out that 1) it wasn't possible for 2 weeks and 2) I couldn't ski for a couple weeks afterward. Since we were in the midst of a great ski season, I decided to schedule it for May.

I failed the LASIK-eligibility and learned I'd have to have PRK instead. The scars on my right eyeball caused me to fail. I received these scars as a boy when a friend and I blew up a .45-70 bullet with a nail and sledgehammer. I was the one holding the nail and couldn't see for the next 3 days.

In a nutshell: with LASIK they cut a flap, lift it up and shoot the laser under it. It heals quickly and is relatively painless. PRK has been around since before LASIK. With PRK, they seem to shave your eyeball and then shoot a laser into it. PRK takes a lot longer to heal, but the results are often as good or better than LASIK. Wikipedia has a more technical PRK vs. LASIK reference.

The Procedure
My mom flew into town for my surgery and drove me to TLC's office on that Thursday morning. It took 2 hours to prep for surgery, with most of the time spent sitting around and talking to my mom. Finally, they asked her to sit in the lobby and took me back to a waiting area. When I walked in the room, there were 2 other patients with surgery gear (funny hat and booties) and masks. They were leaning back, looking at the ceiling with their masks on and eyes closed. I quickly became the 3rd person who looked like this. Right before they started putting drops in my eyes, I remember being terrified that I might never see daylight again. At the very least, I thought I wouldn't see anything but black for the next 3 days.

Right before they called me in, the previous patient walked out and muttered "Damn, that burns." Believe me, this is not what you want to hear right before it's your turn. I was led into the operating room, sat down and received numbing drops in my eyes. Less than a minute later, I was led over to the operating table.

The rest of the procedure lasted less than 5 minutes. They taped my left eye shut and told me to stare at the red light with my right eye. At this point, they used some contraption to shave my eyeball. It was slightly painful, similar to the mild pain you feel when getting a cavity drilled with Novocaine. After each stroke, the world would ripple like a pebble thrown in a lake. After 10 strokes or so, they shot a laser into my eye for around 20 seconds. You don't actually see the laser (the red light looks the same), but you can smell your eyeball burning. Each eye only took a few minutes. My mom was able to watch the entire surgery on a television on the other side of a glass wall.

The Recovery
I was surprised to discover I was immediately able to open my eyes and see normally. Of course, my eyes felt heavy, so I didn't open them wide nor feel like I could. I was led back to the waiting room where I was given a Valium and sent home. I put my sunglasses on when we left and kept my eyes closed for the 5-minute drive home. After arriving at my house, I immediately downed some Tylenol PM and went to bed. It was hard to fall asleep and my eyes began tearing up. There was a dull pain in my eyes that kept the tears flowing for most of the afternoon. It took me 2 hours to fall asleep and I remember my eyes causing my nose to get stuffed up from all the tears.

When I woke up that evening, everything was blurry, but I was able to open my eyes and see better than I had previously w/o glasses. I didn't expect anything in the form of good vision and was mostly pre-occupied with trying to stop the pain (which wasn't terrible, but definitely present). I was prescribed Vicodin and started taking it on a regular basis. I was completely unable to watch TV at all that night. Viewing the computer screen was unthinkable.

The next morning, my vision was a lot better as evidenced by the tweet from my iPhone. Shortly after, my mom drove me to my eye doctor's office for a 1-day checkup. The results were surprising.

Friday night was one of the most painful. Saturday wasn't very painful, but my eyesight was very blurry. That afternoon, it was hard to keep my eyes open. Every time I tried to open them, I felt like I had to sneeze. Fortunately, I was able to watch the Nuggets game. I couldn't see players' numbers, but I was able to see Melo's last-second 3-pointer to win.

On Monday, I was able to drive to TLC for my 3-day checkup. Things were definitely blurry, but I didn't feel like it was dangerous for me to be behind the wheel. I was able to work on Monday, but I also had to increase my font sizes to 36pt and used a 30" monitor all day. OS X's Universal Access -> Zoom feature came in awful handy. That night, TV was a LOT clearer than the previous night, but it was still fuzzy.

What's it like now?
It's been almost 4 weeks and I'm very glad that I had the surgery done. I haven't felt any pain since the Sunday after surgery and I haven't had any issues with dryness. My vision does fluctuate from day-to-day. Some days I feel like I have super-hero vision and other days there's a halo around objects beyond 10 feet. According to my doctor, fluctuations are expected to continue for 6 months. I don't mind since it never gets bad enough to seem strange.

Now I enjoy not having to worry about glasses or contacts when traveling. I love waking up every morning and not having to do anything to improve my vision. I feel like I have more freedom in my life. Getting eye surgery is definitely one of the best things I've ever done.

How To Setup Your Own Software Development Company

This post was originally titled "FTE vs. Contract in this Economy", but it didn't seem to capture the essence of this entry. I wanted to write about why I think contracting is better in this down economy, but I also wanted to write about how you you might go about setting up your own company. Starting a company is relatively easy from a legal standpoint, and hopefully I can provide some resources that'll make it even easier.

First of all, I believe that contracting is better in this economy for a very simple reason:

When you're a contractor, you're prepared to be let go.

There's really nothing like being laid off. It sucks. It often shocks you and makes you depressed. The good part is you usually get a good afternoon's worth of drinking out of it, but that's about it. Severance is cool, but let's face it - you'd much rather be employed.

As a contractor, you're always looking for your next gig. You're prepared for the worst. You're more motivated to learn marketable skills. You're constantly thinking about how you can market yourself better. Writing (blogging, articles, books) is an excellent way to do this and I believe it's rare that FTE are as motivated to do these kinds of things.

Being a contractor forces you to better yourself so you're more marketable.

People's biggest fear of contracting is that they'll have a hard time finding their next gig. In my career, I've rarely had an issue with this. There's always contracts available, it's just a matter of how much you're going to get paid. Yes, I've had to suck-it-up and make $55/hour instead of $125/hour, but that was back in 2003 and $55/hour is still more than I would have made as a FTE.

The other thing that makes me believe contracting is better in this economy is I believe companies are hiring more short-term contractors than employees. I don't know if this is because they consider employees liabilities and contractors expenses, but something about it seems to make the books look better.

So you've decided to take my advice and try your hand at contracting. Should you setup your own Corporation or LLC?

Starting a Company
Yes, you should absolutely start your own company. As a Software Developer, chances are you're going to make enough to put you in the highest tax bracket. If you're a Sole Proprietor (no company), you will pay something like 35% of your income to taxes and you can be sued for everything you own by your clients.

Should you create an LLC or Corporation? I started Raible Designs in May 1998. I started out as an LLC and later converted to an S Corp. For the first few years, I made $30-$55/hour and this seemed to work pretty well. I believe this was similar to having a Sole Proprietorship (because I was the only employee), except that I was protected from lawsuits.

In 2001, I got my first high-paying gig at $90/hour and my Accountant suggested I change to an S Corp to save 10K+ on self-employment tax. I'm certainly not an expert on the different types of business entities, but this path seemed to work well for me. It was $50 to convert from an LLC to an S Corp. I'm not sure if you can go from an S Corp to an LLC. The beauty of an S Corp is the corporation typically gets taxed at 15%, so you can run a lot of things through your business and pay less taxes. Date nights can be business meetings, vacations can be Shareholders Meetings, seasons tickets can be client entertainment and you can write off your car and fuel costs.

There's lots of good resources on the web that describe the different business entity options. My favorite is A List Apart's This Web Business IV: Business Entity Options. Another good resource is How to form an LLC.

The hardest part of starting a new business is coming up with a good name. My advice is to make sure the domain name is available and pick something you like. I chose Raible Designs because I designed web sites at the time. Raible is a pretty unique name, so that's worked well having it as part of my business name. Googlability is important - don't choose a generic name that will make you difficult to find. Potential clients should be able to google your business name and find you easily.

Once you've picked a name, the business establishment part is pretty easy. In Colorado, you can File a Document with the Secretary of State. Their site also allows you to reserve a name if you're not quite ready to make the leap.

You'll also need to get a Federal Employer Identification Number (FEIN) from the IRS. The IRS has a good Starting a Business article and also allows you to Apply for an Employer Identification Number (EIN) Online.

Once you've got all the documents setup, you'll want to create a bank account for your business. I'm currently using Wells Fargo and really like how software-friendly they are. Their online banking is clean and easy to use. They also support QuickBooks for the Mac. They have Payroll Services to allow you to pay your quarterly taxes online as well as setup direct deposit, but I'm not using them.

For payroll, I use PayCycle and have nothing but good things to say about them (see update below). I have the Small Business Package at $42.99 per month. This package allows me to pay myself and employees + up to 5 sub-contractors with direct deposit. It also allows me to pay both Federal and State quarterly taxes online. Of course, if you can also get an Accountant to do this for you.

Having a good Accountant and Financial Advisor (for your retirement plan) will likely be an essential part of your business.. LinkedIn's Service Providers is a good way to find recommended professionals in your area. For example, click here to search for Accountants and then click the change location link in the top right corner to specify your zip code.

Finally, you'll need insurance. The Hartford has a good Small Business package that costs around $500/year. It's liability limits have worked for all of my clients and I'm covered if my laptop ever gets stolen. For Health Insurance, I recommend using eHealthInsurance.com to find a good provider for you. I don't get sick or hurt much, so I typically get a disaster prevention plan with a $5K deductible. For dental insurance, brush your teeth. Vision insurance typically sucks, so I wouldn't buy it. Yes, our health care system in the US needs work and I believe if everyone had a small business, it might get more affordable a lot quicker.

Over the next few days, I'll post some additional advice I've received on retirement plans, deducting a home office, drawing up contracts and how to come up with a good rate. If you're an Independent Software Developer and have any additional advice, I'd love to hear it.

Update: I take back what I said about PayCycle. After having a couple of insufficient funds when re-activating my account in January (they were pulling from the wrong account), they've changed my direct deposit lead-time to 5 days for the next 6 months. This means I forget to create checks on time since their reminder gets sent 2 days before. Time to try Wells Fargo.

Update October 2009: I never left PayCycle and I continue to use them to this day. I'm a satisfied customer, but do believe it still takes too long to make changes to your electronic services setup.

Choosing an Ajax Framework

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

1. Choose a short list of frameworks to prototype with.
2. Create an application prototype with each framework.
3. Document findings and create a matrix with important criteria.
4. Create presentation to summarize document.
5. Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Our strategy for filling in this matrix:

• Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
• We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

• Quality of Documentation/Tutorials/Self Help
• Browser support (most important browsers/versions based on web stats)
• Testability (esp. Selenium compatibility)
• Licensing
• Project health/adoption
• Performance
• Scalability
• Flexibility/extensibility
• Productivity (app dev, web dev)
• Richness of widget/component library
• Charting capability
• Ability to create new widgets
• Match to existing Java team skill-set
• Ease of deployment (on Ops, QA, Users)
• Degree of risk generally
• Ability to integrate with existing site (which includes Prototype)
• Easy to style with CSS
• Validation (esp. marking form elements invalid)
• Component Theme-ing/Decoration
• CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.

Dojo/Comet support in Java Web Frameworks

This week I'm doing a research project for a client. The main purpose of the project is to find out which Java-based web framework works best with Dojo and Comet. Here's the key requirement from the client:

It's all about Comet, we want Comet everywhere we can put it, but we want to isolate the icky bits of fiddling with pages with JavaScript. We're kind of wed to the Dojo implementation of the client-side bit, so we may as well use more of the Dojo widgets for a richer UI. For us, "works best with" needs to pay a certain amount of consideration to "fits naturally with", if you understand what I mean. I know that any framework that lets you spit out raw HTML will let you hand code in your Dojo / Comet, but that's certain to become very tiresome very quickly.

The candidate frameworks they asked me to look at are Wicket and Tapestry 5. They're willing to upgrade to Struts 2 since they're already using Struts 1. However, they don't feel that action-based frameworks naturally lead to rich UIs, so they'd prefer a component-based framework. They're currently using Seam for an administration-type application and feel it's too heavy for their customer-facing application.

Here's what I've found so far in my research. Please let me know if anything is incorrect.

• Tapestry 5 doesn't have Dojo or Comet support (Prototype and Scriptaculous are the baked-in Ajax frameworks).
• Struts 2 has old (version 0.4.3) and somewhat deprecated Dojo support. The developers seem to be in favor of removing it and promoting people hand-code Dojo instead. Struts 2 doesn't have support for Comet.
• Wicket has support for Dojo 1.1 that includes Comet support. This was written by Stefan Fußenegger and posted to the mailing list last month. I e-mailed Stefan and asked him about documentation. His response: "I lost my ambition to document it properly since I didn't receive any feedback on the mailing list. :)"

At this point, it seems that if the client really wants to use Dojo, they should use Wicket, and possibly pay Stefan to document it properly. However, they're willing to consider other options, as long as they have Comet support.

One option I thought of is to use DWR and its Reverse Ajax/Comet support. Another option would be to add better Dojo support to Tapestry 5. However, I don't think this is possible since the Prototype/Scriptaculous code is generated by the framework and would likely require a changes to switch it to Dojo.

Are there any other Java-based web frameworks that support easily creating Dojo widgets and working with Comet? Keith Donald tweeted that Spring MVC has Dojo support. However, I believe it's only for widgets and it still requires you to write JavaScript. If your framework doesn't have Dojo/Comet support, how hard would it be to add it?

Update: I also posted this question on LinkedIn. Make sure and check my question for additional thoughts from folks.

What's Next

It's been three weeks since I joined the realm of the unemployed. Fortunately, I didn't stay unemployed for long. In fact, after writing the aforementioned post, I received 5 offers the next day. Of the opportunities I received, the most interesting ones were those from companies interested in hiring the whole team. Not only that, but LinkedIn hired me back as a contractor through the end of the year. The goal of the LinkedIn contract: finish up projects that my team had started in the previous months.

At the end of the first week after the LinkedIn layoffs, we all had individual opportunities, but we also had two team opportunities. The following week (last week), I flew to NYC to meet with one potential client while the other potential client flew to Denver to meet with the rest of the team. After flying to NYC, I traveled to Mountain View to do some on-site work at LinkedIn. At the end of the week, it seemed like most of the remaining tasks at LinkedIn could be done by someone else. I told them I thought it was best that I move onto other things, while staying available for support questions. On the way to the airport, I spoke with both our team opportunities. Following those conversations, I was very pumped about both projects and confident about pending offers. You can imagine my disappointment when my flight was delayed for 5 hours.

After a fun weekend with Abbie, Jack and friends, I woke up Monday morning without a job and it felt great. However, things changed quickly. Monday morning many opportunities landed in my inbox: a 3-day gig this week (helping write open-source training), a 1-week gig in December (evaluating how well Tapestry 5, Wicket and Struts 2 integrate with Dojo/Comet for a client in Europe), a 1-week training gig in Europe next year and a 3-month gig for the whole team. I accepted all these opportunities and am very happy I'll get to work with Jimbo, Country and Scotty again next year. The 3-month gig should be a lot of fun. We're helping build a SOFEA-based architecture that leverages appropriate client technologies (to be determined) to build a kick-ass web application. I look forward to talking about the technologies we use and things we learn along the way.

So the good news is I've entered The Golden Period. The Golden Period is when you don't have a job, but you do have a start date. Unemployment is absolutely blissful during this time. The Golden Period exists a couple times for me over the next 6 weeks.

I'll be traveling to Costa Rica tomorrow for a best friend's wedding. I'm leaving both my laptop and my iPhone at home. Next week, I'll be loving life with my parents in Costa Rica and Panama. The following week, I'll be working on AppFuse all week and hope to make great progress on developing 2.1. Then I have the 1-week Web Framework Analysis gig, followed by 2 weeks of vacation in Oregon. My Golden Period begins this afternoon (for 3 weeks) and happens again over Christmas (for 2 weeks).

Yeah, life is good. Damn good.

RESTful Web Applications with Subbu Allamaraju

Subbu works at Yahoo! developing standards, patterns and practices for HTTP web languages. In the past, he was a web service and Java developer. He was also a standards contributor at BEA and an author of books on Java EE. His current passion is HTTP and REST. Subbu confesses that he's not a web developer, has no interest in the internals of programming models used for web frameworks and he's only interested in the visible aspects of the architecture.

"The Web is Mostly Restful"

Being RESTful in an abstract sense means:

• Resources are named by URIs
• Resources have representations (Atom, HTML, JSON, XML)
• Resources contain contextual links to allow navigation of state
• There's a Uniform Interface

In the web today, some resources and URIs are personalized, but most are not. Some depend on sessions, but most do not. The consequence of a personalized UI with a non-unique URI is you cannot rely on browser caching.

The web is full of different representations (HTML, XML, JS, PDF, CSS, Flash). The problem with HTML is you can't tell links that you want a particular representation based of a link. The links are hard-coded to be a particular content-type. However, some media types on responses are ignored. This is often a problem with browsers and whether the user has plugins installed.

The Uniform Interface for the web is HTML and primarily links and forms (GET and POST). There's still some misconceptions (e.g. POST is secure). However, it's not about security, it's about idempotency and safety. You need to make sure you only use POST when you're changing data. POSTs are not repeatable. GET URIs are not always refreshable, which is quite unfortunate. Users shouldn't have to fight the back button.

Caching is a fundamental aspect of the web. Even in a personalized site, most of the content can be cached. The web is read-only for the most part. However, many enterprise web applications don't take advantage of caching. This is fine when there's not that many users, but it's bad when you want to scale to thousands of users. There's several frameworks that use cache-busting and prefer backend caching over HTTP caching. These frameworks are not using the web like they should.

Backend caching (e.g. Memcached) uses a non-uniform interface and you need to explicitly program to it. Frontend/HTTP caching has a uniform interface that's pluggable. Backend caching is generally more expensive to develop and deploy. There are cases where data should be cached on the backend, but you shouldn't focus all on backend caching w/o doing some frontend caching.

With Ajax, you get more opportunities to be RESTful. XMLHttpRequest is another HTTP client that can be programmed to. It has full support for the uniform interface, which allows content negotiation, optimistic concurrency and caching. Cross-domain hacks can be done with <script> and <iframe> to tunnel requests over GET. The W3C has been working for the last two years on how to do cross-domain Ajax w/o using hacks. The problem with current cross-domain implementations is they often use GET for everything, which isn't very RESTful. Subbu recommends using a proxy on the same domain if you do need to talk to other domains. This will allow your Ajax code to remain RESTful.

Web Frameworks
Web development is hard because of all the moving pieces that exist. Because of this, many web frameworks have been created to solve the various problems. In 1997, there were servlets. They provided basic plumbing and closely reflected HTTP/1.1. Servlets provided a poor programming model, but it allowed a lot of frameworks to be built on top of it. We don't use servlets to write applications, only to write application frameworks. The second era came about in 2001 when Action-oriented frameworks became popular. In 2004, JSF and friends came to play. JSF is a component-based framework with known limitations (complex, slow, uses POST for almost everything, Ajax is difficult). These limitations have resulted in a number of third-party patches being developed to solve these issues.

JSF was designed to use the request to create a component tree that maintains state. Unfortunately, the state is not something the developer has control over. It's not the state of the application, it's the state of the components. The client's knowledge of the state is mentioned with a cookie and the server keeps the state in the session. The problem with JSF is you don't have a choice of state in your application - you can't write stateless applications like you can with servlets.

JSF uses overloaded URIs for its resources. When you have one URI with multiple representations, there's no way to tell how a representation was chosen. JSF's compromise is to allow client-side state saving. However, they do this by putting hidden field in the form and requiring POST for navigation.

JSF vs. REST
Basically, these two are at opposite extremes. JSF is focused heavily on a UI component model. The people that developed it misinterpreted how the web works and made some fundamental questionable choices. You can patch it, but you can not fix it.

Web 2.0 Frameworks
GWT is a cross-compilation based framework. You write Java to generate JavaScript (b/c everyone hates writing JavaScript). It mashes client and server code into a single source. These layers communicate using GWT-RPC. Typical RPC concerns do not apply since code generation handles coupling and the client is downloaded from the same application. GWT-PRC does POSTs to the server and uses HTTP like a transport layer. To be fair, GWT does allow you to use a RequestBuilder to use the web like it should be used. This class allows more control over HTTP requests, it supports GET and POST and it allows so-called RESTful layers (GWT-REST and GET-Restlet). GWT is focused heavily on ease-of-use, which is good. It's modeled after RPC and breaks the uniform interface and focuses on backend caching. Unlike JSF, GWT is fixable, but the community tends to use RPC instead of RequestBuilder.

SOFEA has a central promise of SOA. Business logic is a reusable service that changes less often. The presentation application calls those services and changes more often. The nice thing about this type of architecture is it allows a separation of concerns and loose coupling. However, it doesn't embrace REST like it should. Appcelerator is an implementation of SOFEA that has a Ruby on Rails-like usability. However, it uses a SOAP/HTTP style with messaging and POSTs to a single URI. Appcelerator is interesting, but it introduces a different style of coupling. It breaks URI opacity and client deals with POX instead of links.

Conclusion
Don't fight the architecture of the web. Innovate and enhance instead of breaking. If nothing else, break judiciously. As developers, we should demand more from our frameworks and make sure they use the web and HTTP like it should be used.

Comprehensive Project Intelligence with Jason van Zyl

In this talk, Jason is going to talk about m2eclipse, Nexus, Hudson and Maven. On his Maven bullet-point, it says "The best is yet to come (and we'll fix a bunch of stuff)!"

m2eclipse
The m2eclipse plugin has improved greatly in the last 4 months - there's now 5 full-time developers working on it. If you use the m2eclipse plugin, you never have to leave the IDE for your Maven-related work. m2eclipse has a Configuration Framework that turns Maven's mumbo-jumbo (Jason's words, not mine) into Eclipse talk. The m2eclipse+configuration framework has integration for WTP, JDT, AJDT and they're working on one for Flex. Below is a screenshot of how m2eclipse helps developers stay away from using command-line Maven.

Now Jason is showing a demo of m2eclipse and creating a new Maven project from existing archetypes. It looks like m2eclipse uses "Nexus Indexer" as its Catalog. Presumably this is a Sonatype-hosted service. The Nexus Indexer contains an of Maven Central and is very fast. It's dynamically updated as new things are deployed to Maven Central.

If you use m2eclipse and open a pom.xml, you'll get a visual view rather than an XML view. This UI has tabs for Overview, Dependencies, Repositories, Build, Plugins, Reporting, Profiles, Team, a Dependency Hierarchy and Dependency Graph. You can easily add new dependencies and it finds things quickly because it's using the Nexus index. In addition to visually adding dependencies, you can modify the raw XML and get things like groupId and version code-completion.

Once you have your dependencies listed in a "Maven Dependencies" container to you can "Materialize Project" to create a project from the binary dependency. You'll get the source as a new project in your workspace as well as having your binary dependency turned into a source dependency.

You can easily create a run configuration that runs certain goals, allows you to activate profiles and uses an embedded version of Maven or an external installation. I asked Jason if the Dependency Hierarchy had a right-click -> exclude feature and he said it doesn't exist yet, but it will in the release after next. For now, the pom editor is just eye candy and doesn't have actions.

For Maven Plugins, m2eclipse has workspace resolution so you can develop a plugin and use it in a project at the same time w/o having to install the plugin over and over.

Sonatype has created a Project Materializer Plugin that allows a team lead to create a project for developers. It allows you to create a welcome page that has links, cheat sheets, News and Updates and Tasks for the developer. It also materializes Eclipse projects in the background. Cheat Sheets are a series of tasks that can be run to show developers how to do things.

Another big feature in m2eclipse is nested project support. It only works in Eclipse 3.4 though.

Nexus
Nexus is a repository manager that allows you to keep the cruft from the outside world out of your system. It's primarily for Repository and Configuration Management. It has fine-grained security for authentication and authorization. One nice feature of its security system is you can prevent certain users from seeing source JARs. It also has virtual repositories (a savior for OSGi lovers). UI is written in Ext JS and acts as a simple REST client for Nexus. It has a full REST API using RESTlet.

A repository manager allows you to protect yourself from the Open Source Ghetto. The OS Maven Ghetto has bad POMs, repositories in POMs, mixed snapshot and release dependencies and screwed up metadata. Not only does it offer protection, but it allows you to aggregate repositories and publish your internal artifacts to it. It also allows you to schedule tasks that clean out snapshots so your repositories don't grow out of control.

Typically people deal with OSGi runtimes manually. OSGi can dynamically update dependencies that you drop into your bundle repository. However, many folks maintain their OSGi runtime and bundle repository locally. Some people are trying to get an OSGi runtime to resolve against a P2 repository. P2 is what Eclipse uses for their repository management. Nexus has the ability to lock down the versions that are available to an OSGi runtime. Furthermore, you can use Nexus to manage the versions that get deployed to all your servers. This makes it a lot easier for QA and Production to manage versions of your artifacts. OSGi is great for modularity and solving classpath issues, but it does have issues with versions and how its ranges work.

You can see Nexus in action at http://repository.sonatype.org. It can be configured entirely through the UI, an XML file or through the REST API. RSS feeds exist for configuration and repository updates.

Nexus is free and open source with a GPL license. The next version (1.2) will contain a Plugin API to allow extensions. All of Sonatype's enhancements for its commercial version will be written as plugins. A matrix of what's available in the open source version vs. commercial version should be published sometime next week.

Hudson
Jason believes that Hudson is the future of continuous integration, on-demand results and release management. They're writing all their extension points in Hudson as Maven plugins and Plexus components (with the work they've done, using Spring components should also be possible). Other enhancements they've made to Hudson:

• Integration of JSecurity
• Implementing a similar REST layer as Nexus and creating a UI using Ext JS
• Automatic installation of external Maven installations
• Drools Workflow Integration

They've also enhanced Hudson so it can easily test/publish Maven projects without using the free-form project template. Hudson works well for doing Eclipse headless builds for Eclipse plugins. If you need to test against multiple databases, multiple OS's, it does support a grid-based system that's easy to setup. Hudson does have web services integration that allows you to kick off builds from within Eclipse. Sonatype uses Hudson to run all their nightly builds of Maven.

Maven - the best is yet to come
The three big things coming in the next version of Maven are:

• Refactored Project Builder: includes a spec for building a pom, domain-specific parsers (attribute-based XML, Groovy and Ruby) and mixins.
• Mercury: a new repository and transport layer. Developed by the Jetty people and is super fast (async client with connection pooling and parallelization). Has atomic downloads and deployments (with Nexus), full PGP support and a WebDAV client built-in.
• Maven Embedder: re-written to actually work.

Overall, a good talk with lots of demos. I'm definitely looking forward to Maven improvements in the future.

Building LinkedIn's Next Generation Architecture with OSGi by Yan Pujante

This week, I'm attending the Colorado Software Summit in Keystone, Colorado. Below are my notes from an OSGi at LinkedIn presentation I attended by Yan Pujante.

LinkedIn was created in March of 2003. Today there's close to 30M members. In the first 6 months, there was 60,000 members that signed up. Now, 1 million sign up every 2-3 weeks. LinkedIn is profitable with 5 revenue lines and there's 400 employees in Mountain View.

Technologies: 2 datacenters (~600 machines). SOA with Java, Tomcat, Spring Framework, Oracle, MySQL, Servlets, JSP, Cloud/Graph, OSGi. Development is done on Mac OS X, production is on Solaris.

The biggest challenges for LinkedIn are:

• Growing Engineering Team on a monolithic code base (it's modular, but only one source tree)
• Growing Product Team wanting more and more features faster
• Growing Operations Team deploying more and more servers

Front-end has many BL services in one webapp (in Tomcat). The backend is many wars in 5 containers (in Jetty) with 1 service per WAR. Production and Development environments are very different. Total services in backend is close to 100, front-end has 30-40.

Container Challenges
1 WAR with N services does not scale for developers (conflicts, monolithic). N wars with 1 service does not scale for containers (no shared JARs). You can add containers, but there's only 12GB of RAM available.

Upgrading back-end service to new version requires downtime (hardware load-balancer does not account for version). Upgrading front-end service to new version requires redeploy. Adding new backend services is also painful because there's lots of configuration (load-balancer, IPs, etc.).

Is there a solution to all these issues? Yan believes that OSGi is a good solution. OSGi stands for Open Services Gateway initiative. Today that term doesn't really mean anything. Today it's a spec with several implementations: Equinox (Eclipse), Knoplerfish and Felix (Apache).

OSGi has some really, really good features. These include smart class loading (multiple versions of JARs is OK), it's highly dynamic (deploy/undeploy built-in), it has a service registry and is highly extensible/configurable. An OSGi bundle is simply a JAR file with an OSGi manifest.

In LinkedIn's current architecture, services are exported with Spring/RPC and services in same WAR can see each other. The problem with this architecture comes to light when you want to move services to a 2nd web container. You cannot share JARs and can't talk directly to the other web app. With OSGi, the bundles (JARs) are shared, services are shared and bundles can be dynamically replaced. OSGi solves the container challenge.

One thing missing from OSGi is allowing services to live on multiple containers. To solve this, LinkedIn has developed a way to have Distributed OSGi. Multicast is used to see what's going on in other containers. Remote servers use the OSGi lifecycle and create dynamic proxies to export services using Spring RPC over HTTP. Then this service is registered in the service registry on the local server.

With Distributed OSGi, there's no more N-1 / 1-N problem. Libraries and services can be shared in one container (memory footprint is much smaller). Services can be shared across containers. The location of the services is transparent to the clients. There's no more configuration to change when adding/removing/moving services. This architecture allows the software to be the load balancer instead of using a hardware load balancer.

Unfortunately, everything is not perfect. OSGi has quite a few problems. OSGi is great, but the tooling is not quite there yet. Not every library is a bundle and many JARs doesn't have OSGi manifests. OSGi was designed for embedded devices and using it for the server-side is very recent (but very active).

OSGi is pretty low-level, but there is some work being done to hide the complexity. Spring DM helps, as do vendor containers. SpringSource has Spring dm Server, Sun has GlassFish, and Paremus has Infiniflow. OSGi is container centric, but next version will add distributed OSGi, but will have no support for load-balancing.

Another big OSGi issue is version management. If you specify version=1.0.0, it means [1.0.0, ∞]. You should at least use version=[1.0.0,2.0.0]. When using OSGi, you have to be careful and follow something similar to Apache's APR Project Versioning Guidelines so that you can easily identify release compatibility.

At LinkedIn, the OSGi implementation is progressing, but there's still a lot of work to do. First of all, a bundle repository needs to be created. Ivy and bnd is used to generate bundles. Containers are being evaluated and Infiniflow is most likely the one that will be used. LinkedIn Spring (an enhanced version of Spring) and SCA will be used to deploy composites. Work on load-balancing and distribution is in progress as is work on tooling and build integration (Sigil from Paremus).

In conclusion, LinkedIn will definitely use OSGi but they'll do their best to hide the complexity from the build system and from developers. For more information on OSGi at LinkedIn, stay tuned to the LinkedIn Engineering Blog. Yan has promised to blog about some of the challenges LinkedIn has faced and how he's fixed them.

Update: Yan has posted his presentation on the LinkedIn Engineering Blog.