Raible Designs

Play vs. Grails Smackdown at ÜberConf

Play and Grails have been hyped as the most productive JVM Web Frameworks for the last couple of years. That hype has recently grown thanks to both frameworks' 2.0 releases. That's why James Ward and I decided to do a presentation at ÜberConf comparing the two. In April, we proposed the talk to Jay Zimmerman, got accepted and went to work.

How we did it
In the beginning of May, we met at a brewery in LoDo and sketched out the app we wanted to build. We also came up with a schedule for development and a plan for the presentation. We decided to build two different webapps, each with little-to-no Ajax functionality and a few features that we could use to load test and compare the applications.

We started out with the name “Happy Trails” since we both liked trails and happy hours. Later, James found that www.ubertracks.com was available and purchased the domain. We setup the Grails app to be on bike.ubertracks.com and Play/Java to be on hike.ubertracks.com. We managed our source code on GitHub, continuously tested on CloudBees and deployed to Heroku. Two weeks ago, when we were finishing up our apps, we hired a friend (Linsay Shirley) to do QA.

After fixing bugs, I emailed Patrick Lightbody, got some “cloud dollars” for Neustar Web Performance and started running load tests. The Wednesday before last, at 2 in the morning, I recorded a simple browsing regions and routes script and set it to go to 50 users over a 5 minute period and then sustain 50 for another 5 minutes. It was fun to watch the log messages whiz through my console so fast they got blurry. About halfway through testing the Grails app, there was an OOM issue, but it eventually recovered. Limiting db connections to 4 and scaling to 5 Dynos in future tests helped alleviate any issues.

We took our development experience, the load/performance testing data, and a bunch of ecosystem stats and built our smackdown presentation. We used reveal.js, GitHub Files and Google Charts to make things more dynamic.

What we found
We arrived at a number of conclusions after doing our research:

Code

• From a code perspective, Play 2 and Grails 2 are very similar frameworks.
• Code authoring was good in both, but lacking IDE support for Play 2's Scala Templates.
• Grails Plugin Ecosystem is excellent.
• TDD-Style Development is easy with both.
• Type-safety in Play 2 was really useful, especially routes.

Statistical Analysis

• Grails has better support for FEO (YSlow, PageSpeed)
• Grails has less LOC! (6 lines less, but 40% more files)
• 1 Dyno - Grails had 2x transactions!
  • Grails experienced OOM about halfway through.
• Apache Benchmark with 10K requests:
  • Play: ~10% failed requests, Grails: 0
  • Requests per second: {Play: 170, Grails: 198}
  • Requests per second: {Play: 251, Grails: 198}
• Load Test with 100 Real Users:
  • Grails: 10% more transactions, 0 errors

Ecosystem Analysis

• "Play" is difficult to search for.
• Grails is more mature.
• Play has momentum issues.
• LinkedIn: more people know Grails than Spring MVC.
• Play has 3x user mailing list traffic.
• We had similar experiences with documentation and questions.
• Outdated documentation is a problem for both.
• Play has way more hype!

We figured we spent around 100 hours developing the apps, gathering data and creating the presentation. The good news is it's all open source! This means you can clone the project on GitHub (Grails is in the grails2 branch, Play is in the play2_java branch) and help us improve it. The presentation is in the master branch in the preso directory.

All the data we gathered is open for debate and we’d love to tune our apps to handle more requests per second. In fact, we already had a contributor discover an issue and provide a fix for Play that increases its throughput from 170 req/second to 252 req/second!

Regardless of what the stats and pretty graphs say, we both enjoyed our experiences with Play 2 and Grails 2. If you haven't tried them yourself, we encourage you to do so.

Upgrading to Play 2: Anorm and Testing

This time last year, I decided I wanted to learn Scala. I chose the Play Framework as my vehicle for learning and I added CoffeeScript and Jade to the mix. I packaged it all up, learned a bunch and presented it at Devoxx 2011.

In January, I added SecureSocial, JSON Services and worked a bit on the mobile client. I presented my findings at Jfokus shortly after. As part of my aforementioned post, I wrote:

Right before we left for Jfokus, I was able to get everything to work, but didn't spend as much time as I'd like working on the mobile client. If this talk gets accepted for Devoxx France, I plan on spending most of my time enhancing the mobile client.

I had some complications (a.k.a. too much vacation) with Devoxx France and wasn't able to attend. To make up for it, I submitted the talk to ÜberConf. It got accepted and I started working on my app a couple weeks ago. So far, I've spent about 8 hours upgrading it to Play 2 and I hope to start re-writing the mobile client later this week. I plan on using Cordova, jQTouch and releasing it in the App Store sometime this month.

Upgrading to Play 2
When I heard about Play 2, I thought it was a great thing. The developers were re-writing the framework to use Scala at the core and I was already using Scala in my app. Then I learned they were going to throw backwards compatibility out the window and I groaned. "Really? Another web framework (like Tapestry of old) screwing its users and making them learn everything again?!", I thought. "Maybe they should call it Run instead of Play, leaving the old framework that everyone loves intact."

However, after hearing about it at Devoxx and Jfokus, I figured I should at least try to migrate. I downloaded Play 2.0.1, created a new project and went to work.

The first thing I learned about upgrading from Play 1.x to Play 2.x is there's no such thing. It's like saying you upgraded from Struts 1 to Struts 2 or Tapestry 4 to Tapestry 5. It's a migration, with a whole new project.

Evolutions
I started by looking around to see if anyone had documented a similar migration. I found two very useful resources right off the bat:

• Play 2.0 with Scala and Scaml, Part1: Setup of test infrastructure, model and persistence with Anorm by Jan Helwich
• Tutorial: Play Framework 2 with Scala, Anorm, JSON, CoffeeScript, jQuery & Heroku by James Ward

From Jan's Blog, I learned to copy my evolutions from my Play 1.x project into conf/evolutions/default. I changed my application.conf to use PostgreSQL and wrote an EvolutionsTest.scala to verify creating the tables worked.

import org.specs2.mutable._

import play.api.db.DB
import play.api.Play.current

import anorm._

import play.api.test._
import play.api.test.Helpers._

class EvolutionsTest extends Specification {

  "Evolutions" should {
    "be applied without errors" in {
      evolutionFor("default")
      running(FakeApplication()) {
        DB.withConnection {
          implicit connection =>
            SQL("select count(1) from athlete").execute()
            SQL("select count(1) from workout").execute()
            SQL("select count(1) from comment").execute()
        }
      }
      success
    }
  }
}

Then I began looking for how to load seed data with Play 2.x. In Play 1.x, you could use a BootStrap job that would load sample data with YAML.

import play.jobs._
import play.Play

@OnApplicationStart
class BootStrap extends Job {

  override def doJob() {

    import models._
    import play.test._

    // Import initial data if the database is empty
    if (Athlete.count().single() == 0) {
      Yaml[List[Any]]("initial-data.yml").foreach {
        _ match {
          case a: Athlete => Athlete.create(a)
          case w: Workout => Workout.create(w)
          case c: Comment => Comment.create(c)
        }
      }
    }
  }
}

This is no longer a recommended practice in Play 2. Instead, they recommend you turn your YAML into code. 10 minutes later, I had a Global.scala that loaded seed data.

import models._
import play.api._
import play.api.Play.current

import anorm._

object Global extends GlobalSettings {

  override def onStart(app: Application) {
    InitialData.insert()
  }
}

/**
 * Initial set of data to be loaded
 */
object InitialData {

  def date(str: String) = new java.text.SimpleDateFormat("yyyy-MM-dd").parse(str)

  def insert() {

    if (Athlete.count() == 0) {

      Seq(
        Athlete(Id(1), "[email protected]", "beer", "Matt", "Raible"),
        Athlete(Id(2), "[email protected]", "whiskey", "Trish", "McGinity")
      ).foreach(Athlete.create)

      Seq(
        Workout(Id(1), "Chainsaw Trail",
          """
            A beautiful fall ride: cool breezes, awesome views and yellow leaves.

            Would do it again in a heartbeat.
          """, 7, 90, date("2011-10-13"), 1),
        Workout(Id(2), "Monarch Lake Trail",
          "Awesome morning ride through falling yellow leaves and cool fall breezes.",
          4, 90, date("2011-10-15"), 1),
        Workout(Id(3), "Creekside to Flume to Chainsaw",
          "Awesome morning ride through falling yellow leaves and cool fall breezes.",
          12, 150, date("2011-10-16"), 2)
      ).foreach(Workout.create)

      Seq(
        Comment(1, "Jim", "Nice day for it!"),
        Comment(2, "Joe", "Love that trail."),
        Comment(2, "Jack", "Where there any kittens there?")
      ).foreach(Comment.create)
    }
  }
}

Anorm's Missing Magic
Before starting with Play 2, I knew it had lost some of its magic. After all, the developers had mentioned they wanted to get ride of the magic and moving to Scala allowed them to do that. However, I didn't think I'd miss Magic[T] as much as I do. Like Martin Fowler, I like ORMs and having to use SQL again seems painful. It seems like a strange shift for Play to reduce type-safety on the backend, but increase it in its templates. Oh well, to each their own. I may eventually move to Squery, but I wanted to do a side-by-side comparison as part of my migration.

Using the aforementioned tutorial from James and Jan's blog posts, as well as Guillaume's Play 2.0/Anorm, I set about creating new model objects. I wrote a bunch of SQL, typed up some new finders and migrated my tests from ScalaTest to the new default, specs2. The Mosh Pit's Migrating a Play 1.2 website to Play 2.0 was a great help in migrating tests.

That's when I started having issues with Anorm and figuring out how its parser syntax works. After struggling for a few days, I finally found yabe-play20-scala. Since I'd used the yabe tutorial from Play 1.x, it was familiar and helped me get past my problems. Now, things aren't perfect (Workouts aren't ordered by their posted date), but everything compiles and tests pass.

To illustrate how little code was required for Anorm 1.x, checkout Workout.scala in Play 1.x vs. Play 2.x. The Play 1.x version is 66 lines; Play 2.x requires 193 lines. I don't know about you, but I kinda like a little magic in my frameworks to reduce the amount of code I have to maintain.

I was pleasantly surprised by specs2. First of all, it was an easy migration from ScalaTest. Secondly, Play's FakeApplication made it very easy to write unit tests. The line count on my UnitTests.scala in Play 1.x vs. Play 2.x is almost identical.

Summary
The first few hours of developing with Play 2 were frustrating, mostly because I felt like I had to learn everything over again. However, I was pleased to find good references on migrating from Play 1.x. Last night, I migrated all my Controllers, integrated Scalate and got most of my views rendering. I still have issues rendering validation errors, but I hope to figure that out soon. The last 2 hours have been much more fun and I feel like my Scala skills are coming along. I think if the Play Team could eliminate those first few hours of struggling (and provide almost instant joy like Play 1.x) they'd really be onto something.

As soon as I figure out how to validation and how to add a body class based on the URL, I'll write another post on the rest of my migration. A Play 2-compatible version of SecureSocial just came out this evening, so I may integrate that as well. In the meantime, I'll be working on the iPhone app and finishing up a Grails 2 application for James Ward and my Grails vs. Play Smackdown.

Deploying Java and Play Framework Apps to the Cloud with James Ward

Yesterday, I attended James Ward's presentation on Deploying Java & Play Framework Apps to the Cloud at Devoxx. I arrived a bit late, but still managed to get there in time to see a lot of demos and learn more about Heroku. Below are my notes from James's talk.

When I arrived, James was doing a demo using Spring Roo. He was using Roo's Petclinic sample app and showed us how you could use Git to create a local repository of the new project and install Heroku's command line tool. From there, he ran the following command to create a new application on Heroku.

heroku create -s cedar

The Cedar Stack is what supports Java, Scala and Play Framework. It's the 3rd generation stack for Heroku. The command above created two endpoints, one for HTTP and one for Git. It picks from a list of randomly generated names, which all seem to have some humor in them. James ended up with "electric-sword-8877" for this demo.

From there, he ran git push heroku master to deploy the project to Heroku. Unfortunately, this resulted in a login error and there was an akward moment where we all thought the Demo Gods were angry. However, James was able to resolve this by using Heroku's sharing feature with the following command.

heroku sharing:add [email protected]

For Java projects, Heroku looks for a pom.xml file in the root directory and runs a Maven build on project. All the dependencies get downloaded on the cloud rather than put them into a WAR and requiring you to upload a large WAR file. You don't have to upload your source code to Heroku; James did it for the sake of the demo because it was faster.

After the build finishes, it creates a slug file. This file contains everything Heroku needs to run your application.

Next, James showed a demo of the running application and added a new Pet through its UI. Then he scaled it to two servers using the following command:

heroku scale web=2

He proved this was working by running heroku ps, which showed there were two running processes. He showed the app again, but noted that the record he added was missing. This is because when it started up a new dyno, Hibernate created the schema again and deleted all records. To fix, James changed Hibernate to only update the schema instead of create a new one. If you're a Hibernate user, you know this is as simple as changing:

hibernate.hbm2ddl.auto=create

to:

hibernate.hbm2ddl.auto=update

After committing this change, James redeployed using Git.

git push heroku master

The slug file got built again and Heroku deployed the new slug onto both dynos, automatically load balancing the app across two servers. James then ran heroku logs to see the logs of his dynos and prove that a request to his app's HTTP endpoint made requests to both dynos. The logging is powered by Logplex and you can read about how it works in the article Heroku Gets Sweet Logging.

James mentioned that Roo has a Heroku plugin, but after watching his talk and searching a bit on the internet, it seems it's just the jetty-runner setup as described in Getting Started with Spring MVC Hibernate on Heroku/Cedar.

What about autoscaling? There are some 3rd party tools that do this. Heroku's Management infrastructure has APIs that these tools talk too. Heroku hasn't built autoscaling into the platform because they don't know where the bottlenecks are in your application.

Heroku = Polyglot + PaaS + Cloud Components. It supports Ruby, node.js, Java, Clojure, Play and Scala and they're working on native Grails and Gradle support. There's currently 534,374 apps running on Heroku.

Heroku is a cloud application platform and there's 5 different components.

1. Instant deployment
2. HTTP Routing / Load Balancing
3. Elastic Polyglot Runtime
4. Management & Logging
5. Component as a Service Ecosystem

For instant deployment, it's a pretty simple process:

• You add files to a git repo
• You provision the app on Heroku (heroku create)
• You upload the files to Heroku (git push heroku master)
• Heroku runs the build and assembles a "slug" file
• Heroku starts a "dyno"
• Heroku copies the "slug" to the "dyno"
• Heroku starts the web application

Most apps will contain a Procfile that contains information about how to run the web process. For Spring Roo, it has:

web: java $JAVA_OPTS -jar target/dependency/jetty-runner.jar --port $PORT target/*.war

So how does Heroku decide what application server to use? It doesn't, you do. You need to get your application server into the slug file. The easiest way to do this is to specify your application server as a dependency in your pom.xml. In the Roo example, James uses the maven-dependency-plugin to get the jetty-runner dependency and copy it to the target directory. On Heroku, you bring your application server with you.

Heroku gives you 750 free dyno hours per app, per month. For developers, it's very easy to get started and use. Once you extend past one dyno, it's $.05 per dyno hour, which works out to around $30/month. It's only when you want to scale beyond one dyno where you get charged by Heroku, no matter how much data you transfer. Scalatest is running on Heroku. It has one dyno and is doing fine with that. Bill Venners doesn't have to pay anything for it.

java.herokuapp.com is a site James created that allows you to clone example apps and get started quickly with Heroku's Cedar Stack.

For HTTP Routing, Heroku uses an Erlang-based routing system to route all the HTTP requests across your dynos. Heroku doesn't support sticky sessions. Distributed session management does not work well, because it does not scale well. Heroku recommends you use a stateless web architecture or move your state into something like memcached. Jetty has (in the latest version) the ability to automatically serialize your session into a Mongo system. This works fine on Heroku. The problem with this is if you have 2 dynos running, each request can hit a different dyno and get different session state. Hence the recommendation for an external storage mechanism that can synchronize between dynos.

Heroku has native support for the Play framework. To detect Play applications, it look for a conf/application.conf file. You don't need to have a Procfile in your root directory because Heroku knows how to start a Play application.

At this point, James created a new Play application, created a new Heroku app (he got "young-night-7104" this time) and pushed it to Heroku. He created a simple model object, a controller to allow adding new data and then wrote some jQuery to show new records via Ajax and JSON. He also showed how to configure the application to talk to Heroku's PostgreSQL database using the DATABASE_URL environment variable. He explained how you can use the heroku config command to see your environment variables.

The reason they use environment variables is so Heroku can update DATABASE_URL (and other variables) without having to call up all their customers and have them change them in their source code.

Play on Heroku supports Scala if you create your app with Scala. Play 2.0 uses Scala, Akka and SBT. Heroku added support for SBT a couple month ago, so everything will work just fine.

Heroku also supports Scala, detecting it by looking for the build.sbt file in the root directory. Heroku supports SBT 0.11.0 and it builds the 'stage' task. It currently does not support Lift because Lift uses an older version of SBT and because it's a very stateful framework that would require sticky sessions. Use Play, BlueEyes or Scalatra if you want Scala on Heroku.

Heroku has addons for adding functionality to your application, including Custom DNS, HTTPS, Amazon RDS, NoSQL and many more. They're also working on making their add-on and management APIs available via Java, so you'll (hopefully) be able to use them from your IDE in the future.

From there, James showed us how Heroku keeps slug files around so you can do rollbacks with heroku rollback. He also showed how you can use:

heroku run "your bash command"

Summary
I attended James's talk because he's a good friend, but also because I've been using Heroku to host my latest adventures with Play, Scala, CoffeeScript and Jade. I'm glad I attended because I learned some good tips and tricks and more about how Heroku works.

Heroku seems like a great development tool to me. In my experience, it's been really nice to have instant deployments using Git. In fact, I've created a 'push' alias so I can push to my project's repo and heroku at the same time.

alias push='git push origin master && git push heroku master'

I'd like to see more organizations embrace something like Heroku for developers. It'd be great if everyone had their own sandbox that business owners and product managers could see. I can't help but think this would be awesome for demos, prototyping, etc.

There were some other talks I wanted to attend at the same time, particularly Martin Odersky's What's in store for Scala? and WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks. The WWW talk has posted their presentation but I'm sure it'd be more fun to watch.

It's pretty awesome that all the talks from Devoxx 2011 will be up on Parleys.com soon.

Update: James has posted his slides from this talk.

Hyperproductive JSF 2.0 with Ed Burns at Jazoon

This morning, I attended Ed Burn's Talk on Hyperproductive JSF 2.0 at Jazoon. As you might know, I've been a critic of JSF for many years. However, it is one of the most used Java web frameworks, so I was hoping to learn how it's improved in the latest version. Below are my notes from Ed's presentation.

Ed's Plan for our Time Investment:

1. Define a productive development environment
2. JSF for greenfield and brownfield projects
3. List the top 9 productivity killers with JSF projects and solutions

"I am always doing that which I cannot do, in order that I may learn how to do it." -- Pablo Picasso

Software is an executable representation of knowledge, a byproduct of learning how to solve a problem. Knowledge is something that changes as more information comes in. A productive environment makes it as easy as possible to learn how to solve a problem. Learning is an iterative process. Early iterations don't need to be optimal. Later iterations do ... and they need to be maintainable. First is the hardest. Fast iterations are they key. Spring Roo and Play are examples of frameworks that make the first iteration very fast.

You should use a tool to jumpstart JSF development: copy from an old project, use a Maven archetype or use your IDE. With greenfield development, you don't have to bother learning the byproduct of other people's learning. It's a great opportunity to pad your resume with the latest hot technologies. With brownfield development, it's vitally important to understand the existing solution and hidden assumptions. You're also much more constrained in your technology choices. If you want to change, you'll need to come up with a migration strategy, which can be difficult. JSF works well for both because it's not just a runtime framework, it's also a conceptual framework. You need to understand how your framework handles data conversion, validation, page flow, persistence integration, I18N, L10N, A11Y, Web standards and user friendliness.

Top 9 JSF Productivity Killers:

1. Time wasting deployment step
2. The perils of "there's more than one way to do it"
3. Lengthy and painful developer on-boarding process
4. Misused logging and misunderstood error messages
5. Phantoms
6. Under-utilized developer tools
7. Premature optimization
8. Difficulty in doing TDD
9. Lack of an app framework

Time wasting deployment step
ZeroTurnaround solves this problem with JRebel, but there's other ways to do it. Some of the killers of flow state: 1) one large war file, 2) underutilizing dynamic language features, 3) complex server environment and 4) build process is redoing work unnecessarily. To stop the time wasting deployment step, the most important things you can do are as follows:

• Configure your IDE correctly. In Eclipse, compile directly into WEB-INF/classes and use continuous compilation. With NetBeans, use GlassFish.
• Don't do control-flow programming in XML.

Ed then showed a simple demo that showed how you can use Groovy to create a JSF UI Component. He also mentioned that Groovy can be used to author any JSF artifact. The benefit of this is you can simply edit and save a .groovy file without having to recompile or redeploy. Unfortunately, using Groovy didn't eliminate the XML syntax for pages or the XML for defining UI components.

The perils of "there's more than one way to do it"
JSF is very flexible, but flexibility is, more often than not, abused. There's a lack of convention for common things (e.g. master-detail, JSF concepts like converter, validator, etc.). The best way to fix this is to establish the norms for a project and stick with them. For example, Neil Griffin has a good blog entry for the different kind of managed beans you can create. Develop recommendations like Neil's and use them on all your projects.

Lengthy developer on-boarding process
Stick with standards when possible (at least have a common project description and build system across projects). Be committed to periodic cleanup cycles, including documenting for re-use. Pick one JSF component library and stick with it. Support for mixing and switching component libraries has improved with JSF 2, but it's still recommended you use only one.

Misused logging and misunderstood error messages
JSF is notorious for cryptic error messages and very long stack traces. It's still a problem, but the JSF Team is still working on improving them. Good tip: use the <ui:debug> tag. Its recordStateSize="true" attribute can be especially useful. If you're using PrimeFaces, add trace=true to request URLs.

Phantoms
Phantoms is when running code is not the same as the code you are modifying or wrong version of library gets picked up. You should have the capability to hit breakpoints anywhere in your entire software stack, including core Java sources. This is one of the most useful things about open source software. Solutions to phantoms: 1) put a timestamp on every redeploy and have the timestamp appear in the system log 2) write the running library stack to the system log (each library and version being used) and make it easy to compare one developer's runtime stack with another's 3) consider doing all work in tightly controlled VMs (checkout the VM at the beginning of the day, do your work, commit your changes and throw your VM away at the end of the day).

Under-utilizing developer tools
Make sure everyone has the fastest machines available and as much screen real estate as desired. Hardware is much cheaper than developer time. Another tip is to use Hudson as your butler. It's not just the team CI server. In other words, take advantage of automation wherever you can.

Premature Optimization
Keep in mind the trade-offs between readability and performance. When using frameworks such as JSF, don't try to outsmart the implementation. Rather, use the framework as intended and use open-source contributions to treat performance problems. Example, EL expressions got a lot faster between EE5 and EE6. If you spent time trying to optimize EL expressions, you might've been wasting your time.

Difficulty in doing TDD
Try to figure out why TDD is difficult in your company. For JSF, strongly consider JBoss's JSFUnit. Write your testcases to extend from Cactus ServletTestCase and leverage HtmlUnit (JSFUnit does this for you).

Lack of an app framework
Create common components: login panel, CRUD components, etc. If you don't have an app framework, build one over time.

Conclusion
This was an interesting talk by Ed. The dynamics of the room where a bit interesting. Jazoon is held in a movie theater, much like Devoxx. However, it appears there's a spotlight on the speaker that makes it very difficult to see the audience. I don't remember having this problem at Devoxx. Ed asked the audience quite a few questions, but it seemed he had a lot of difficulty in seeing if folks raised their hands. This made for some periods of awkward silence.

Personally, I was hoping to learn some new whizbang tips about JSF that I was not aware of. Unfortunately, I didn't learn anything new and wasn't that impressed with the Groovy demo.

I think Ed's tips about things outside of JSF were good, especially buying developers good hardware. I've seen many companies, including my current client, skimp on developer hardware and cause developer frustration because of it. I think it's great when companies provide developers top-of-the-line hardware and eliminate frustration over CPU and memory resources. LinkedIn and Time Warner Cable both provide their developers with Mac Pros and MacBook Pros as well as huge monitors. IMO, this is one of the best benefits you can provide your engineers.

How I Calculated Ratings for My JVM Web Frameworks Comparison

When I re-wrote my Comparing JVM Web Frameworks presentation from scratch, I decided to add a matrix that allows you to rate a framework based on 20 different criteria. The reason I did this was because I'd used this method when choosing an Ajax framework for Evite last year. The matrix seemed to work well for selecting the top 5 frameworks, but it also inspired a lot of discussion in the community that my ratings were wrong.

I expected this, as I certainly don't know every framework as well as I'd like. The mistake I made was asking for the community to provide feedback on my ratings without describing how I arrived at them. From Peter Thomas's blog:

What you are doing is adjusting ratings based on who in the community shouts the loudest. I can't help saying that this approach comes across as highly arrogant and condescending, you seem to expect framework developers and proponents to rush over and fawn over you to get better ratings, like waiters in a restaurant trying to impress a food-critic for Michelin stars.

I apologize for giving this impression. It certainly wasn't my intent. By having simple numbers (1.0 == framework does well, 0.5 == framework is OK and 0 == framework not good at criteria) with no rationalization, I can see how the matrix can be interpreted as useless (or to put it bluntly, as something you should wipe your ass with). I don't blame folks for getting angry.

For my Rich Web Experience presentation, I documented why I gave each framework the rating I did. Hopefully this will allow folks to critique my ratings more constructively and I can make the numbers more accurate. You can view this document below or on Google Docs.

In the end, what I was hoping to do with this matrix was to simply highlight a technique for choosing a web framework. Furthermore, I think adding a "weight" to each criteria is important because things like books often aren't as important as REST support. To show how this might be done, I added a second sheet to the matrix and made up some weighting numbers. I'd expect anyone that wants to use this to downloaded the matrix, verify the ratings are accurate for your beliefs and weight the criteria accordingly.

Of course, as I and many others have said, the best way to choose a web framework is to try them yourself. I emphasized this at the end of my presentation with the following two slides.

My Comparing JVM Web Frameworks Presentation from Devoxx 2010

This week, I've been having a great time in Antwerp, Belgium at the Devoxx Conference. This morning, I had the pleasure of delivering my Comparing JVM Web Frameworks talk. I thoroughly enjoyed giving this presentation, especially to such a large audience. You can view the presentation below (if you have Flash installed) or download it here.

Unlike previous years, I chose to come up with a spreadsheet matrix that shows why I chose the 5 I did. This spreadsheet and rankings given to each framework are likely to be debated, as I don't know all the frameworks as well as I'd like to. Also, the missing column on this spreadsheet is a "weighting" column where you can prioritize certain criteria like I've done in the past when Comparing Ajax Frameworks. If you believe there are incorrect numbers, please let me know and I'll try to get those fixed before I do this talk again at The Rich Web Experience.

One thing that doesn't come across in this presentation is that I believe anyone can use this matrix, and weightings, to make any of these frameworks come out on top. I also believe web frameworks are like spaghetti sauce in The Ketchup Conundrum. That is, the only way to make more happy spaghetti sauce lovers was to make more types of spaghetti sauce. You can read more about this in my There is no "best" web framework article.

Update: If you disagree with the various ratings I gave to web frameworks in this presentation, please provide your opinions by filling out this survey. Thanks to Sebastien Arbogast for setting this up.

Update: Sebastien has posted his survey results at JVM Web Framework Survey, First Results.

Update 12/6: A video of this presentation is now available on Parleys.com.

P.S. My current gig is ending in mid-December. If you're looking for a UI Architect with a passion for open source frameworks, please let me know.

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

My Incredible Trip to Ireland

If you ever get a chance to travel to Ireland, take it! I don't know when I heard these words, or how they came into my head, but I remembered them clearly when I was first introduced to Barry Alistair by Jeff Genender. Soon after, I was able to negotiate my way into being a speaker at The 2010 Irish Software Show.

The show was last week and I had a blast traveling to Dublin to speak and explore. My sister came me on this trip, but missed a connection in Seattle and had to join me a day late. I left Denver at noon on Monday and arrived at Dublin Airport at 7 am. I was on the same flight as Josh Long and thoroughly enjoyed my iPad as a travel companion. When I got off the plane, my battery life was at 60% and I'd been watching movies and listening to music for 6 hours.

I took a cab through the misty, cool morning to my hotel. I grabbed a coffee, cleaned up, and walked a few blocks to Trinity College for the conference. I made it in time for the opening keynote by Chris Horn. It was an interesting talk, focusing on what needed to happen to make Ireland the Innovation Hub of Europe. After that, I attended Tim Berglund's session on Complexity Theory and Software Development. After lunch and a few more talks, I teamed up with Andres Almiray and Josh Long for a pint at the hotel bar.

That evening, we attended Jeff Genender's talk on Getting into Open Source. The free drinks loosened everyone up and Jeff did a great job with a humorous presentation on how to get Committer Status. After Jeff's talk, about 10 of us headed to a Moroccan restaurant for a late dinner. I was in bed around midnight.

Wednesday morning, my sister arrived in my hotel room at 8 and promptly fell into bed. I set my alarm to sleep an hour and closed the Vegas-style, no-light-allowed curtains. We awoke much later (12:30) than we'd planned (9:00). We quickly got up and headed for some sight-seeing in Dublin. First off, we hit Dublinia and Christ Church Cathedral. Both sites were spectacular and we both learned a lot about the history of Dublin. From there, we skipped across the bridge to The Old Jameson Distillery for a tour and a bit of whiskey.

The picture below was taken on the Ha'penny Bridge as we were heading back from Jameson. The expression of the girl on the left is priceless.

A couple hours later and I was delivering my talk on The Future of Web Frameworks. The crowd was lively; the Guinness I drank while talking was lovely. My session was followed by a Web Framework Experts Panel with Peter Ledbrook (Grails), Jamie van Dyke (Rails), Shay Friedman (ASP.NET MVC), Julian Fitzell (Seaside) and myself (Java Frameworks). The debate was good and there was much discussion about the right apps for each framework and how important statelessness is for scalable applications. After 3 hours of talking, my sister and I headed back to the hotel. I was particularly happy about the evening since it was the first time a family member of mine had seen me speak.

Correction from my Dad: This wasn't the first time a family member saw me speak. He attended my talk at ApacheCon EU 2007.

A block from the hotel, we spotted a nice looking pub (Doyles) and stopped in for a pint. As we bellied up to the end of the bar, we recognized Jamie (from the panel) and got introduced to his friend Rob. We quickly got lost in conversation, stories and laughter and were surprised when we discovered it was 2:30am. Since I had a talk first thing in the morning, we ducked out shortly after.

Thursday started with my talk Comparing Kick-Ass Web Frameworks. Then my sister and I did some more site-seeing, starting at the Guinness Storehouse. We met Josh and John Willis as they were leaving and they advised we go straight to The Gravity Bar at the top. We took there advise and were getting great views of Dublin and savoring sweet pints of Guinness moments later. The tour facility was freakin' awesome and I loved how it was shaped like a pint glass.

We grabbed some gear from the gift shopped and landed (by accident) at The Brazen Head (Ireland's Oldest Pub, Est. 1198) for a pint of cider and Guinness. Since my sister used to be in the cider business, she was particularly happy there was so much on tap in Ireland.

From the pub, we headed to John Willis's session on The Cambrian Cloud Explosion. Following John's session, we headed to the Speaker's Dinner for a very fun evening with the hosts and speakers of the conference.

On Friday, we woke up in the early afternoon and quickly decided the Book of Kells was our best chance of getting some site seeing in. After visiting the Book of Kells, my favorite quote of the conference happened in the courtyard.

Josh looked at Jamie (with his bad hangover) and exclaimed, "My God Man. Your skin is so white it's hurting my eyes!". You probably had to be there (or know Josh) to enjoy the humor, but I wanted to capture the memory in this post so I could laugh whenever I read this in the future. After that, Jamie, Josh, Kalin and I enjoyed a Starbuck's patio talking about living in the South of France for a couple hours. Then we walked 2 blocks to the Porterhouse Brewing Co. to watch the World Cup and enjoy more interesting conversations.

Jamie left the conference that evening and we joined a whole slew of other speakers for dinner at an excellent Lebanese restaurant near Temple Bar. Good times where had afterwards at a nearby Silent Disco.

Saturday, we woke up early to catch a tour bus out to Glendalough with Josh and John. The bus ride was not pleasant, but the destination was spectacular. We hung out there for several hours, exploring the buildings, walking to the lake and humoring each other.

Our last night in Dublin was an early, relaxing one. As you can tell, I really enjoyed this trip, particularly hanging out with my sister and all the cool people we met. I can easily say that this trip registers as one of my favorite conference experiences to date.

To see all the pictures I took on this trip, check out my Irish Software Show 2010 set on Flickr.

My Presentations from The Irish Software Show 2010

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

What's New in Spring 3.0

This morning, I attended Rod Johnson's What's New in Spring 3.0 keynote at TSSJS. Rod ditched his slides for the talk and mentioned that this might be risky. Especially since he was pretty jetlagged (flew in from Paris at 11pm last night). Below are my notes from his talk.

The most important thing for the future of Java is productivity and cloud computing. The focus at SpringSource is heavily on productivity and not just on improving the Spring codebase. If you look at the comparisons out there between Rails and Spring, it's not an apples-to-apples comparison. The philosophy with Spring has always been the developer is always right. However, if you look at something like Rails, you'll see it's far more prescriptive. That layer of opinionated frameworks is important in that it improves your productivity greatly.

SpringSource is putting a lot of emphasis on improving developer productivity with two opinionated frameworks: Grails and Spring Roo. To show how productive developers can be, Rod started to build a web app with Spring Roo. As part of this demo, he mentioned we'd see many of the new features of Spring 3: RestTemplate, @Value and Spring EL.

Rod used STS to write the application and built a Twitter client. After creating a new project using File -> New Roo Project, a Roo Shell tab shows up at the bottom. Typing "hint" tells you what you should do write away. The initial message is "Roo requires the installation of a JPA provider and associated database." The initial command is "persistence setup --provider HIBERNATE --database HYPERSONIC_IN_MEMORY". After running this, a bunch of log messages are shown on the console, most of them indicating that pom.xml has been modified.

The first file that Rod shows is src/main/resources/META-INF/spring/applicationContext.xml. It's the only XML file you'll need in your application and includes a PropertyPlaceHolderConfigurer, a context:component-scan for finding annotations and a transaction manager.

After typing "hint" again, Roo indicates that Rod should create entities. He does this by running "ent --class ~.domain.Term --testAutomatically". A Term class (with a bunch of annotations) is created, as well as a number of *.aj files and an integration test. Most of the files don't have anything in them but annotations. The integration test uses @RooIntegrationTest(entity=Term.class) on its class to fire up a Spring container in the test and do dependency injection (if necessary). From there, Rod demonstrated that he could easily modify the test to verify the database existed.

private SimpleJdbcTemplate jt;

@Autowired
public void init(DataSource ds) {
    this.jt = new SimpleJdbcTemplate(ds);
}

@Test 
public void testDb() {
    jt.queryForInt("SELECT COUNT(0) FROM TERM");
}

Interestingly, after running the test, you could see a whole bunch of tests being run, not just the one that was in the class itself. From there, he modified the Term class to add two new properties: name and searchTerms. He also used JSR 303's @NotNull annotation to make the fields required.

@Entity
@RooJavaBean
@RooToString
@RooEntity
public class Term {

    @NotNull
    private String name;

    @NotNull
    private String searchTerms;
}

Next, Rod added a new test and showed that the setters for these properties were automatically created and he never had to write getters and setters. This is done by aspects that are generated beside your Java files. Roo is smart enough that if you write toString() methods in your Java code, it will delete the aspect that normally generates the toString() method.

To add fields to an entity from the command lie, you can run commands like "field string --fieldName text --notNull" and "field number --type java.lang.Long --fieldName twitterId --notNull". The Roo Shell is also capable of establishing relationships between entities.

After successfully modifying his Entities, Rod started creating code to talk to Twitter's API. He used RestTemplate to do this and spent a good 5 minutes trying to get Eclipse to import the class properly. The best part of this demo was watching him do what most developers do: searching Google for RestTemplate to get the package name to import.

After awkward silence and some fumbling, he opened an existing project (that had the dependencies properly configured) and used Java Config to configure beans for the project. This was done with a @Configuration annotation on the class, @Value annotations on properties (that read from a properties file) and @Bean annotations for the beans to expose. The first time Rod tried to run the test it failed because a twitter.properties file didn't exist. After creating it, he successfully ran the test and successfully searched Twitter's API.

The nice thing about @Configuration is the classes are automatically picked up and you don't need to configure any XML to recognize them. Also, in your Java classes, you don't have to use @Autowired to get @Bean references injected.

After this, Rod attempted to show a web interface of the application. He started the built-in SpringSource tc Server and proceeded to show us Tomcat's 404 page. Unfortunately, Tomcat seemed to startup OK (no errors in the logs), but obviously something didn't work well. For the next few silent moments, we watched him try to delete web.xml from Eclipse. Unfortunately, this didn't work and we weren't able to see the scaffolding the entities that Rod created.

At this point, Rod opened a completed version of the app and was able to show it to us in a browser. You could hear the murmur of the crowd as everyone realized he was about to show the the Twitter search results for #tssjs. Most of the tweets displayed were from folks commenting about how some things didn't work in the demo.

In summary, there's some really cool things in Spring 3: @Configuration, @Value, task scheduling with @Scheduled and one-way methods with @Async.

Final points of SpringSource and VMWare: they're committed to Java and middleware. Their big focus is providing an integrated experience from productivity to cloud. There's other languages that are further along than Java and SpringSource is trying to fix that. One thing they're working on is a private Java cloud that companies can use and leverage as a VMWare appliance.

I think there's a lot of great things in Spring 3 and most users of Roo seem to be happy with it. It's unfortunate that the Demo Gods frowned upon Rod, but it was cool to see him do the "no presentation" approach.