Raible Designs

GWTTestSuite makes builds faster, but requires JUnit 4.1

Earlier this week, I spent some time implementing GWTTestSuite to speed up my project's build process. In Hudson, the project was taking around 15 minutes to build, locally it was only taking 5 minutes for mvn test. In IDEA, I could run all the tests in under a minute. While 15 minutes isn't a long time for a build to execute, a co-worker expressed some concern:

Does Maven have to run GWT test and individual Java processes? (See target/gwtTest/*.sh) This arrangement and the overhead of JVM launches is another reason why builds take so long. As we add more GWT tests we are going to test that LinkedIn record for the slowest build ever.

After this comment, I started looking into GWTTestSuite using Olivier Modica's blog entry as a guide. It was very easy to get things working in IDEA. However, when I'd run mvn test, I'd get the following error:

Error: java.lang.ClassCastException

No line numbers. No class information. Zilch. After comparing my project's pom.xml with the one from the default gwt-maven archetype, I noticed the default used JUnit 4.1, while I had the latest-and-supposedly-greatest JUnit 4.4. Reverting to JUnit 4.1 fixed the problem. Now Hudson takes 3:15 to execute the build instead of 15 minutes.

The reason for this blog post is this doesn't seem to be documented anywhere. Hopefully other developers will find this entry when googling for this issue.

Related to making GWT faster, I also added the following line to my Application.gwt.xml file:

<set-property name="user.agent" value="safari" />

This dropped the gwt:compile time from 1 minute to 25 seconds. As explained in the documentation, you can use the "user.agent" setting to only generate one JS file for your app instead of 4. The strange thing about adding this setting was I pretty much forgot about it since everything seemed to work fine on both Safari and Firefox. When I started testing things in IE6, I started seeing a lot of JavaScript errors. After debugging for an hour or so, I realized this setting was there, removed it, and everything started working great in all browsers.

Now if I could just figure out how to use safari-only for development, but remove the line when building the WAR. Suggestions welcome.

Enhancing your GWT Application with the UrlRewriteFilter

Last week, I spent some time trying to change the location of my cache/nocache HTML files in my GWT project. I started the project with the gwt-maven-plugin's archetype. The message I posted to the gwt-maven Google Group is below.

Rather than having my application's HTML file in src/main/java/com/mycompany/Application.html, I'd like to move it to src/main/webapp/index.html. I tried copying the HTML and adding the following to my index.html, but no dice:

<meta name="gwt:module" content="com.mycompany.Application"/>

Is this possible with the gwt-maven-plugin? I'd like to have my main HTML and CSS at the root of my application.

The good news is I figured out a solution using the UrlRewriteFilter that 1) allows hosted mode to work as usual and 2) allows your app to be served up from the root URL (/ instead of /com.company.Module/Application.html). Here's the urlrewrite.xml that makes it all possible.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCENGINE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 3.0//EN"
        "http://tuckey.org/res/dtds/urlrewrite3.0.dtd">

<urlrewrite>
    <rule>
        <from>^/$</from>
        <to type="forward" last="true">/com.mycompany.app.Application/Application.html</to>
    </rule>
    <rule>
        <from>/index.html</from>
        <to type="forward" last="true">/com.mycompany.app.Application/Application.html</to>
    </rule>
    <-- This last rule is necessary for JS and CSS files -->
    <rule>
        <from>^/(.*)\.(.*)$</from>
        <to type="forward">/com.mycompany.app.Application/$1.$2</to>
    </rule>
</urlrewrite>

If you're using the gwt-maven plugin, this file goes in src/main/webapp/WEB-INF. In addition, you'll need to add the following to your web.xml.

    <filter>
        <filter-name>rewriteFilter</filter-name>
        <filter-class>org.tuckey.web.filters.urlrewrite.UrlRewriteFilter</filter-class>
    </filter>

    <filter-mapping>
        <filter-name>rewriteFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

Finally, add the UrlRewriteFilter dependency in your pom.xml:

    <dependency>
        <groupId>org.tuckey</groupId>
        <artifactId>urlrewritefilter</artifactId>
        <version>3.1.0</version>
    </dependency>

Please let me know if you have any questions.

Update: Jeff posted an alternative configuration that allows you to eliminate the last rule in urlrewrite.xml, as well as use the beloved mvn jetty:run command. To use cleaner WAR packaging and the Jetty plugin, add the following to your pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
        <webappDirectory>
            ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
        </webappDirectory>
    </configuration>
</plugin>
<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.14</version>
    <configuration>
        <webAppConfig>
            <contextPath>/</contextPath>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
                </resourcesAsCSV>
            </baseResource>
        </webAppConfig>
        <scanIntervalSeconds>3</scanIntervalSeconds>
        <scanTargets>
            <scanTarget>${basedir}/src/main/resources</scanTarget>
            <scanTarget>${basedir}/src/main/webapp/WEB-INF</scanTarget>
            <scanTarget>
                ${project.build.directory}/${project.build.finalName}/com.mycompany.app.Application
            </scanTarget>
        </scanTargets>
    </configuration>
</plugin>

Then you can trim your urlrewrite.xml down to:

<urlrewrite>
    <rule>
        <from>^/$</from>
        <to type="forward" last="true">/Application.html</to>
    </rule>
    <rule>
        <from>/index.html</from>
        <to type="forward" last="true">/Application.html</to>
    </rule>
</urlrewrite>

Of course, you could also change the welcome-file in your web.xml or use index.html and the <meta http-equiv="REFRESH"> option. Personally, I have so much affection for the UrlRewriteFilter that I like having it in my project. I'm sure I'll need it someday.

Thanks Jeff!

Comparing Web Frameworks Book

A publisher recently sent me an e-mail asking some advice. They received a proposal for a book that compares CakePHP, Symfony, Zend, TurboGears, Django, Struts, RoR. Here's a quote from the proposal:

We would like to compare a couple of frameworks and present their advantages and disadvantages in various applications.

Obviously, that kind of manual would be very useful for readers who are starting their 'adventures' with web applications, as it would facilitate their choosing the best framework for their particular application. The manuscript would offer a comparison of the most popular solutions (CakePHP, Symfony, Zend Framework, TurboGears, Django, Struts, Ruby on Rails) and demonstrate the main differences between each.

Therefore, the target audience would mainly be project managers, responsible for deciding on the technologies to be used for in-house projects, as well as less experienced, web application beginners.

Another purpose of the book would be to present 'good practices' in various frameworks, such as code re-factoring, design patterns and application security. From this point of view, it could become a valuable asset for experienced and learner programmers alike.

Since I got a lot of feedback from my tweet on this subject, I figured I'd ask it here.

What do you think of such a book?

Here's my response:

How do PHP books do these days? Of the list of frameworks (CakePHP, Symfony, Zend Framework, TurboGears, Django, Struts, Ruby on Rails), I think there's interest in Django and Rails, but not so much the others. And Struts sucks, so having that as a comparison is obviously going to make it look bad. I wouldn't buy it, but I'm a Java guy that's mostly interested in web frameworks that make developing SOFEA-based applications easier. In my mind, these are Flex and GWT.

The book I'd like to see would cover developing RESTful backends and SOFEA front-ends. RoR, Grails or Django could be used to develop the backend and Flex, GWT and X could be for the front-end. In reality, this is probably a tough book to write b/c things move so fast. If you decide to do it, I'd keep it short and sweet so you can get it to market and update it quickly.

What's the Best Retirement Plan for Independent Consultants?

Before writing How To Setup Your Own Software Development Company, I sent my Financial Planner the following e-mail.

I'm writing up a blog post on how to setup a Software Development Company for consultants and wanted to see what retirement plan I have. I'd like to recommend it (or others, if there's better deals). Do you have the name and a 2-3 sentence description?

Below is his response:

You have a SEP IRA but depending on how much they make and their savings objective they may also want an Individual 401K and/or Defined Benefit Plan.

A SEP IRA allows you to set aside up to 20% of your income after business expenses, up to $49,000 for those with income of $245,000 or more in 2009. An Individual 401K allows you to save a higher percentage of your income depending on your age and income. If you are under age 50 you are able to save $16,500 so long as your income is at least $16,500 (plus FICA, etc) and $22,000 for those over age 55. You are also able to set aside profit sharing and matching contributions in a 401K Plan. Those under age 50 have a maximum of $49,000 while those over age 50 have an increased limit of $54,000. Finally, for those who wish to save more, you could establish a Defined Benefit Plan and make contributions based on your age and income that total potentially more than $200,000 per year. If you establish a Defined Benefit Plan you are still able to have an Individual 401K Plan but the limits are the employee contribution amount ($16,500 or $22,000) plus 6% of your income up to $245,000 (another $14,700) for a combined total that could be well over $200,000 depending on your age and income.

There is always the Roth and Traditional IRA but those are very basic planning tools - they should still be used and considered but everyone should be familiar with them. 2009 allows $5,000 deposit for under age 50 and $6,000 for over age 50. Roth contributions are limited starting at $105,000 if filing single and $166,000 if married filing joint.

Of course, a perk of working for a company with benefits is they sometimes do 401K matching. However, I'd expect many company to be cutting back on that in this economy. If you're an independent consultant, do you have a retirement plan? Do you think you're doing as well as you could if you were a full-time employee?

Writing Off Home Office Space

Before writing yesterday's post on How To Setup Your Own Software Development Company, I sent my Accountant the following e-mail.

I'm writing up a blog post on how to setup a Software Development Company for consultants. I remember talking to you a few years ago about writing off my home office space. At that time, you recommended I didn't because it'd end up on my personal taxes (or something like that). Do you remember that conversation and reasoning? I'd like to post a 2-3 sentence explanation of why this is not a good idea to readers of my blog.

Below is her response:

It's not that it is a bad idea, it just doesn't always result in a big tax savings. There are a lot of factors to consider, but I would not post something that says it's a bad idea. There are types of entities and situations where it is beneficial. Everyone's circumstances are different, so I would not suggest that you make a blanket statement stating that it is not a good idea.

In your particular case, you are an S Corp, which is a pass thru entity...meaning the S Corp does not pay taxes. So, if you want to take a home office deduction you would have your business pay rent to you. In return you would have to claim rental income on that money. So, you could take the deduction on the business side but have to claim it on the personal side...making it a wash. However, where you would get the tax savings, is that you can write off a portion of your utilities against the rental income of the office. It is usually small, like around 10% or so, but it is something.

The only issue is that when you sell your house, you are supposed to treat the sale as two different things...a personal sale and a business sale (for the office). You can often exclude the gain on the personal sale, but can not exclude the gain on the office. So, if the house goes up significantly in value, you could find yourself paying tax on a large gain.

Do you have your own company and write-off your home office space? If so, what kind of company do you have and does it save you a lot of money?

How To Setup Your Own Software Development Company

This post was originally titled "FTE vs. Contract in this Economy", but it didn't seem to capture the essence of this entry. I wanted to write about why I think contracting is better in this down economy, but I also wanted to write about how you you might go about setting up your own company. Starting a company is relatively easy from a legal standpoint, and hopefully I can provide some resources that'll make it even easier.

First of all, I believe that contracting is better in this economy for a very simple reason:

When you're a contractor, you're prepared to be let go.

There's really nothing like being laid off. It sucks. It often shocks you and makes you depressed. The good part is you usually get a good afternoon's worth of drinking out of it, but that's about it. Severance is cool, but let's face it - you'd much rather be employed.

As a contractor, you're always looking for your next gig. You're prepared for the worst. You're more motivated to learn marketable skills. You're constantly thinking about how you can market yourself better. Writing (blogging, articles, books) is an excellent way to do this and I believe it's rare that FTE are as motivated to do these kinds of things.

Being a contractor forces you to better yourself so you're more marketable.

People's biggest fear of contracting is that they'll have a hard time finding their next gig. In my career, I've rarely had an issue with this. There's always contracts available, it's just a matter of how much you're going to get paid. Yes, I've had to suck-it-up and make $55/hour instead of $125/hour, but that was back in 2003 and $55/hour is still more than I would have made as a FTE.

The other thing that makes me believe contracting is better in this economy is I believe companies are hiring more short-term contractors than employees. I don't know if this is because they consider employees liabilities and contractors expenses, but something about it seems to make the books look better.

So you've decided to take my advice and try your hand at contracting. Should you setup your own Corporation or LLC?

Starting a Company
Yes, you should absolutely start your own company. As a Software Developer, chances are you're going to make enough to put you in the highest tax bracket. If you're a Sole Proprietor (no company), you will pay something like 35% of your income to taxes and you can be sued for everything you own by your clients.

Should you create an LLC or Corporation? I started Raible Designs in May 1998. I started out as an LLC and later converted to an S Corp. For the first few years, I made $30-$55/hour and this seemed to work pretty well. I believe this was similar to having a Sole Proprietorship (because I was the only employee), except that I was protected from lawsuits.

In 2001, I got my first high-paying gig at $90/hour and my Accountant suggested I change to an S Corp to save 10K+ on self-employment tax. I'm certainly not an expert on the different types of business entities, but this path seemed to work well for me. It was $50 to convert from an LLC to an S Corp. I'm not sure if you can go from an S Corp to an LLC. The beauty of an S Corp is the corporation typically gets taxed at 15%, so you can run a lot of things through your business and pay less taxes. Date nights can be business meetings, vacations can be Shareholders Meetings, seasons tickets can be client entertainment and you can write off your car and fuel costs.

There's lots of good resources on the web that describe the different business entity options. My favorite is A List Apart's This Web Business IV: Business Entity Options. Another good resource is How to form an LLC.

The hardest part of starting a new business is coming up with a good name. My advice is to make sure the domain name is available and pick something you like. I chose Raible Designs because I designed web sites at the time. Raible is a pretty unique name, so that's worked well having it as part of my business name. Googlability is important - don't choose a generic name that will make you difficult to find. Potential clients should be able to google your business name and find you easily.

Once you've picked a name, the business establishment part is pretty easy. In Colorado, you can File a Document with the Secretary of State. Their site also allows you to reserve a name if you're not quite ready to make the leap.

You'll also need to get a Federal Employer Identification Number (FEIN) from the IRS. The IRS has a good Starting a Business article and also allows you to Apply for an Employer Identification Number (EIN) Online.

Once you've got all the documents setup, you'll want to create a bank account for your business. I'm currently using Wells Fargo and really like how software-friendly they are. Their online banking is clean and easy to use. They also support QuickBooks for the Mac. They have Payroll Services to allow you to pay your quarterly taxes online as well as setup direct deposit, but I'm not using them.

For payroll, I use PayCycle and have nothing but good things to say about them (see update below). I have the Small Business Package at $42.99 per month. This package allows me to pay myself and employees + up to 5 sub-contractors with direct deposit. It also allows me to pay both Federal and State quarterly taxes online. Of course, if you can also get an Accountant to do this for you.

Having a good Accountant and Financial Advisor (for your retirement plan) will likely be an essential part of your business.. LinkedIn's Service Providers is a good way to find recommended professionals in your area. For example, click here to search for Accountants and then click the change location link in the top right corner to specify your zip code.

Finally, you'll need insurance. The Hartford has a good Small Business package that costs around $500/year. It's liability limits have worked for all of my clients and I'm covered if my laptop ever gets stolen. For Health Insurance, I recommend using eHealthInsurance.com to find a good provider for you. I don't get sick or hurt much, so I typically get a disaster prevention plan with a $5K deductible. For dental insurance, brush your teeth. Vision insurance typically sucks, so I wouldn't buy it. Yes, our health care system in the US needs work and I believe if everyone had a small business, it might get more affordable a lot quicker.

Over the next few days, I'll post some additional advice I've received on retirement plans, deducting a home office, drawing up contracts and how to come up with a good rate. If you're an Independent Software Developer and have any additional advice, I'd love to hear it.

Update: I take back what I said about PayCycle. After having a couple of insufficient funds when re-activating my account in January (they were pulling from the wrong account), they've changed my direct deposit lead-time to 5 days for the next 6 months. This means I forget to create checks on time since their reminder gets sent 2 days before. Time to try Wells Fargo.

Update October 2009: I never left PayCycle and I continue to use them to this day. I'm a satisfied customer, but do believe it still takes too long to make changes to your electronic services setup.

Testing GWT Applications

Last week, I did some research on GWT, how to test it and code coverage options for tests that extend GWTTestCase. The reason I did this is because I've found that most of the GWT tests I write have to extend GWTTestCase and I'd like to have code coverage reports. Read below for more information on my findings for testing GWT classes.

There are quite a few articles about testing GWT applications. Here are a few samples:

• Test-First GWT
• Test driven development for GWT UI code
• Test driven development for GWT UI code with asynchronous RPC
• Enabling Test Driven Development in GWT client code
• JUnit tests for widgets - thread about which components to test.

The main gist of these articles is that you should structure your code to make the core functionality of your application testable without having to depend on GWTTestCase.

All of them also advocate using an MVC or MVP (Model View Presenter) pattern. Currently, I'm using GXT and its MVC Framework. Unfortunately, GXT's MVC doesn't have much documentation. The good news is there is a good article that explained enough that I was able to refactor my project to use it.

The unfortunate side of this refactoring was I discovered that classes that extend GXT's MVC Framework have to be tested with GWTTestCase. The downside to extending GWTTestCase is there is it's difficult to create code coverage reports.

GWT's issue 799 has some patches that should make code coverage possible. I tried to implement code coverage with Eclipse and EclEmma using this README, but failed. In the process, I discovered an issue with Eclipse 3.4 and JUnit on OS X. Reverting to Eclipse 3.3 solved this problem, but I was still unable to make EclEmma work with GWT.

After failing with Eclipse, I tried to use the emma-maven-plugin. I was also unable to get this to work, with my findings documented in this thread.

Finally, I did have some luck with getting IDEA's built-in code coverage feature working. However, after getting it to work once, it failed to work for the rest of the day and I haven't had success since.

Code Coverage and GWT
Because of these issues with GWT 1.5 and code coverage, I think I'll wait until GWT 1.6 to worry about it. The good news is 1.6 M1 was released last Friday. If continuing to use GWTTestCase becomes an issue, I may write my own MVC Framework that doesn't use classes that call native JavaScript. Hopefully GXT MVC's framework will provide a good example.

In addition to trying to get code coverage working, I used the internets to figure out how run GWT tests inside of Eclipse and IDEA. I don't remember the resources I used, but hopefully this up-to-date documentation will help others. The nice thing about using an IDE to run these tests is they typically execute much faster.

Running GWT Tests in Eclipse
You should be able to run most of your GWT tests from within Eclipse using the following steps.

1. Right-click on a test that extends GWTTestCase and go to Run As > JUnit Test. It's likely you will see the error message below.

   Invalid launch configuration: -XstartOnFirstThread not specified.
   
   On Mac OS X, GWT requires that the Java virtual machine be invoked with the
   -XstartOnFirstThread VM argument.
   
   Example:
     java -XstartOnFirstThread -cp gwt-dev-mac.jar com.google.gwt.dev.GWTShell
   

2. To fix this error, go to Run > Open Run Dialog. Click on the Arguments tab and add the following values. The 2nd value is to increase the amount of memory available to the test and avoid an OOM error.

   -XstartOnFirstThread -Xmx512M

3. When you re-run the test, you will probably see the following error:

   com.google.gwt.junit.JUnitFatalLaunchException: The test class 'org.richresume.client.home.HomeControllerGwtTest' 
   was not found in module 'org.richresume.client.Application'; no compilation unit for that type was seen
     at com.google.gwt.junit.JUnitShell.checkTestClassInCurrentModule(JUnitShell.java:193)
     at com.google.gwt.junit.JUnitShell.runTestImpl(JUnitShell.java:628)
     at com.google.gwt.junit.JUnitShell.runTest(JUnitShell.java:150)
     at com.google.gwt.junit.client.GWTTestCase.runTest(GWTTestCase.java:219)
   

4. To fix this, open the Run Dialog again, click on the Classpath tab and click on User Entries. Click on the Advanced button and select Add Folders. In the Folder Selection dialog, select your source and test directories (e.g. src/main/java and src/test/java).
5. Run the test again and you should see a green bar in your JUnit tab.
6. To create a JUnit configuration that runs all tests, duplicate the previously mentioned run configuration. Then change the name to "All Tests" and select the 2nd radio button to run all tests in the project.
7. Click Run to execute all the tests in the project.

Running GWT Tests in IDEA
You should be able to run your GWT tests from within IDEA using the following steps.

1. Right-click on a test that extends GWTTestCase and go to Run "TestNameGwtTes...". It's likely you will see the error message below.

   Invalid launch configuration: -XstartOnFirstThread not specified.
   
   On Mac OS X, GWT requires that the Java virtual machine be invoked with the
   -XstartOnFirstThread VM argument.
   
   Example:
     java -XstartOnFirstThread -cp gwt-dev-mac.jar com.google.gwt.dev.GWTShell
   

2. If you get a compiler error instead, you may need to add the GWT Facet to your project. To do this, right-click on your project's top-most folder in the left pane. Select Module Settings > Facets and enable GWT for your module.
3. To fix the -XstartOnFirstThread issue, go to Run > Edit Configurations. Add the following values to the VM Arguments field. The 2nd value is to increase the amount of memory available to the test and avoid an OOM error.

   -XstartOnFirstThread -Xmx512M

   NOTE: If you still get a compiler error, see this page for a possible solution.
4. Run the test again and you should see a green bar in your Run tab.
5. To create a JUnit configuration that runs all tests, duplicate the previously mentioned run configuration. Then change the name to "All Tests" and change the Test configuration to search for tests in the whole project.
6. Run the new configuration to execute all the tests in the project.

Testing GWT applications isn't as straightforward as writing JUnit tests, but I do believe it's getting better. If you have any additional tips and tricks, please let me know.

What's the best Java Hosting Solution?

A friend recently asked me who I'd recommend for a Java hosting provider. Since I get asked this question every-so-often, it seemed appropriate to post my answer here.

1. KGB Internet - I use KGB for this site. I have my own JVM and have full control over what I want to install. I can control Tomcat versions and upgrade as needed. I don't know if I'd recommend him for a business site as he can take up to 12 hours to respond to requests.
2. Kattare - These guys will give you your own Tomcat instance and seem to have reasonable prices. They do seem to take quite some time to respond to requests (24-48 hours). I have a free instance that I use for a non-profit, so that could be the reason.
3. Contegix - These guys are far-and-away the best company for Java-based hosting. They're not cheap though. However, they have the best customer service in the business - often responding to e-mails in less than a minute.

Do you agree with these recommendations? If not, who do you recommend for Java hosting and why?

LA Tech Meetup Tonight

If you live in LA - or just happen to be in town - you should join us for some beers and tech talk this evening. We're meeting at The Village Idiot around 7 and plan on being there until 9. Hope to see you there!

In other Tech Meetup News, it looks like we'll be doing one in Denver next Thursday (January 22nd). Venue TBD.

Choosing an Ajax Framework

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

1. Choose a short list of frameworks to prototype with.
2. Create an application prototype with each framework.
3. Document findings and create a matrix with important criteria.
4. Create presentation to summarize document.
5. Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Our strategy for filling in this matrix:

• Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
• We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

• Quality of Documentation/Tutorials/Self Help
• Browser support (most important browsers/versions based on web stats)
• Testability (esp. Selenium compatibility)
• Licensing
• Project health/adoption
• Performance
• Scalability
• Flexibility/extensibility
• Productivity (app dev, web dev)
• Richness of widget/component library
• Charting capability
• Ability to create new widgets
• Match to existing Java team skill-set
• Ease of deployment (on Ops, QA, Users)
• Degree of risk generally
• Ability to integrate with existing site (which includes Prototype)
• Easy to style with CSS
• Validation (esp. marking form elements invalid)
• Component Theme-ing/Decoration
• CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.