Play vs. Grails Smackdown at ÜberConf
Play and Grails have been hyped as the most productive JVM Web Frameworks for the last couple of years. That hype has recently grown thanks to both frameworks' 2.0 releases. That's why James Ward and I decided to do a presentation at ÜberConf comparing the two. In April, we proposed the talk to Jay Zimmerman, got accepted and went to work.
How we did it
In the beginning of May, we met at a brewery in LoDo and sketched out the app we wanted to build. We also came up with a schedule for development and a plan for the presentation. We decided to build two different webapps, each with little-to-no Ajax functionality and a few features that we could use to load test and compare the applications.
We started out with the name “Happy Trails” since we both liked trails and happy hours. Later, James found that www.ubertracks.com was available and purchased the domain. We setup the Grails app to be on bike.ubertracks.com and Play/Java to be on hike.ubertracks.com. We managed our source code on GitHub, continuously tested on CloudBees and deployed to Heroku. Two weeks ago, when we were finishing up our apps, we hired a friend (Linsay Shirley) to do QA.
After fixing bugs, I emailed Patrick Lightbody, got some “cloud dollars” for Neustar Web Performance and started running load tests. The Wednesday before last, at 2 in the morning, I recorded a simple browsing regions and routes script and set it to go to 50 users over a 5 minute period and then sustain 50 for another 5 minutes. It was fun to watch the log messages whiz through my console so fast they got blurry. About halfway through testing the Grails app, there was an OOM issue, but it eventually recovered. Limiting db connections to 4 and scaling to 5 Dynos in future tests helped alleviate any issues.
We took our development experience, the load/performance testing data, and a bunch of ecosystem stats and built our smackdown presentation. We used reveal.js, GitHub Files and Google Charts to make things more dynamic.
What we found
We arrived at a number of conclusions after doing our research:
Code
- From a code perspective, Play 2 and Grails 2 are very similar frameworks.
- Code authoring was good in both, but lacking IDE support for Play 2's Scala Templates.
- Grails Plugin Ecosystem is excellent.
- TDD-Style Development is easy with both.
- Type-safety in Play 2 was really useful, especially routes.
Statistical Analysis
- Grails has better support for FEO (YSlow, PageSpeed)
- Grails has less LOC! (6 lines less, but 40% more files)
- 1 Dyno - Grails had 2x transactions!
- Grails experienced OOM about halfway through.
- Apache Benchmark with 10K requests:
- Play: ~10% failed requests, Grails: 0
- Requests per second: {Play: 170, Grails: 198}
- Requests per second: {Play: 251, Grails: 198}
- Load Test with 100 Real Users:
- Grails: 10% more transactions, 0 errors
Ecosystem Analysis
- "Play" is difficult to search for.
- Grails is more mature.
- Play has momentum issues.
- LinkedIn: more people know Grails than Spring MVC.
- Play has 3x user mailing list traffic.
- We had similar experiences with documentation and questions.
- Outdated documentation is a problem for both.
- Play has way more hype!
We figured we spent around 100 hours developing the apps, gathering data and creating the presentation. The good news is it's all open source! This means you can clone the project on GitHub (Grails is in the grails2 branch, Play is in the play2_java branch) and help us improve it. The presentation is in the master branch in the preso directory.
All the data we gathered is open for debate and we’d love to tune our apps to handle more requests per second. In fact, we already had a contributor discover an issue and provide a fix for Play that increases its throughput from 170 req/second to 252 req/second!
Regardless of what the stats and pretty graphs say, we both enjoyed our experiences with Play 2 and Grails 2. If you haven't tried them yourself, we encourage you to do so.
Posted by Julien Dubois on June 25, 2012 at 01:53 PM MDT #
Its good to see someone finally evaluated both the so-much-hyped web application development frameworks. However, I saw a line from you Matt, about the Grails plugin eco-system being vast, i would beg to differ on this, though the plugin-eco system is vast for grails but most of the plugins are outdated and close to none involvement of their original authors.
Taken an instance of springws plugin, it doesn't provide some key features like exposing multiple operations in one end-point class and there are so many like this. When you use these new frameworks which are not very mature, I would say instead of shortening the time frame it rather increases the development process in most of the cases.
It always better to use the framework on which you can have finer control. I know lot of companies which once took "Flex" as the most upcoming technology for rapid web development, but now are regretting their decision of using it (after Adobe stopped its support) and resorting back to proven frameworks like Spring etc.
Dont know what future holds for Grails and Play though....I would prefer Spring over them any time.
regards
Akhil
Posted by Akhil Jain on June 25, 2012 at 01:53 PM MDT #
I suppose that Grails could do the same kind of tuning done by Play if we tweaked the server.xml from the tomcat7 you use as we don't really run in our own container :
Regarding the app itself :
Posted by Stephane Maldini on June 25, 2012 at 02:04 PM MDT #
@Julien There was no hack here. If you write an application that's mainly making blocking calls (as the demo app), then you need to run it with thread configuration suitable for this kind of request processing model.
Play's default configuration* is targeting apps that are written with asynchronous, non-blocking request processing model in mind.
Play will scale just fine either way but you have to make sure to run your application with the appropriate configuration profile.
(*the default config therefore uses fewer threads and lower timeouts)
Posted by Peter on June 25, 2012 at 02:35 PM MDT #
@Julien I've written up the details on how to optimize Play 2 for database-driven apps:
http://www.jamesward.com/2012/06/25/optimizing-play-2-for-database-driven-apps
This is based on what Peter and the other Play guys helped Matt and I with.
Posted by James Ward on June 25, 2012 at 05:24 PM MDT #
Posted by Brent Lemons on June 25, 2012 at 08:13 PM MDT #
Posted by Alex on June 25, 2012 at 08:13 PM MDT #
@Brent Honestly I just don't know Scala well enough yet to do it justice in this kind of a a comparison. But we would like to do a version of the app that uses the client-server architecture with JavaScript / HTML5 on the client. Maybe for that one we will write the backend in Scala.
Just curious: Which results do you think would be different if we use Scala?
Posted by James Ward on June 25, 2012 at 09:19 PM MDT #
Posted by Donal on June 26, 2012 at 12:46 AM MDT #
Posted by Romain Sertelon on June 26, 2012 at 12:05 PM MDT #
@James I'm not sure there would be a difference between Play+Java and Play+Scala, but I think it would be an interesting comparison. I think LOC would be smaller with Scala. I'd also be interested in the difference in performance between anorm and ebeans on the persistence side.
One other side note. Any chance you or Matt built, but didn't present on, the same app in Spring MVC? A performance comparison there might be the most helpful for those of us trying to convince people to go to non-Java JVM languages.
Posted by Brent Lemons on June 26, 2012 at 01:18 PM MDT #
GRAILS:
+1 GSP Much easier to use than Scala
+1 Scaffolding Option a Nice Feature
+1 Complete Documentation
+1 Proper Netbeans Support
+1 Deploy straight to WAR file
PLAY:
+1 Documentation Easier to Read
+1 Has a Lighter Feel About it. Less files, etc.
Posted by Mark van Wyk on June 26, 2012 at 01:36 PM MDT #
Posted by James Ward on June 26, 2012 at 03:52 PM MDT #
Very cool you guys. We are trying out Grails at my work for brand new project. It's pretty nice if you need something that works with the tomcat container, can be mavenized, and can step through debug on a production environment. That being said the documentation is definitely lacking we find ourselves hunting around the web for examples of non trivial things. The examples are always so simplistic. The last thing that drove me absolutely nuts was setting up a security manager. The forums basically say just don't do it. :-) We were so close to using Play. It starts up so much faster and definitely feels lighter. But the plugins definitely were not there. And the ones we did use had issues and had fixes as recent as a few weeks ago so we felt they just weren't stable enough for us.
My next task is to do a smackdown like this comparing Grails to Spring MVC. Maybe I'll accept your challenge.
Posted by Robert Beltran on June 27, 2012 at 06:03 AM MDT #
are you doing apples to apples comparison here?
the reason i ask is play2 is using asynchronous operations whereas grails is not. i believe grails2 has async facility built in.
so it would be interesting to see waht the comparsion would be if both play and grails were running in asynchronous mode
Posted by numan on July 21, 2012 at 08:05 PM MDT #
Great job! Simply wonderful!
For an unbelievable boost in Grails performance simply change the GSP to Freemarker Plugin (yes, it has all GSP advantages like taglibs, etc.). Cloud you do that?
Just take a look at this micro-benchmarking on Grails using Freemarker (grails-fm). http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/comment-page-1/#comments
It's amazing! Grails with Freemarker is much more fast and scalable than pure JSP! :-D
PS.: Soon Grails will migrate to Groovy 2.0 and another boost to performance are coming. A great job of Groovy community!
Best regards,
Wanderson Santos
Posted by Wanderson Santos on July 23, 2012 at 09:53 PM MDT #
Posted by indra on August 16, 2012 at 07:55 PM MDT #
Posted by Matt Raible on August 16, 2012 at 09:19 PM MDT #
Hi,
Thanks for putting the effort in, was a great read!
Just wanted to mention that Play! 2.0 Scala templates are now supported in the EAP release of Intellij Idea 12 (Leda). See http://plugins.intellij.net/plugin/?idea&pluginId=7080
Posted by Ali on September 06, 2012 at 01:38 AM MDT #