Matt RaibleMatt Raible is a Web Architecture Consultant specializing in open source frameworks.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Web Framework Comparison Whitepaper

Working at Virtuas in June was really a lot of fun. We worked a fair amount preparing for JavaOne, and also found time to work on a number of whitepapers. These whitepapers are part of an Open Source Landscape Series that has been posted to Virtuas's site. For your convenience, here's a current list:

In addition to the whitepaper, I also wrote an article for JDJ that'll be showing up in the July issue.

Posted in Java at Jun 30 2005, 07:54:10 PM MDT 8 Comments
Comments:

I have one comment about Web Frameworks report from Virtuas. I can understand the logic behind presenting job data for framework comparision purposes. But, if I show a report with this kind of criteria(job numbers) as a corner stone the feedback from upper management would be not so favorable. That is just my opinion. On the same token, great job guys.

Posted by Kishore Dandu on June 30, 2005 at 09:56 PM MDT #

I'm not sure that the "never mind the quality, just count the numbers" approach to the tutorials actually results in a meaningful metric...

Posted by Gwyn Evans on July 01, 2005 at 12:30 PM MDT #

Looks like you've been having too much fun at JavaOne! Anyway, side note, something you might find relevant and worth looking into: http://www.onjava.com/pub/a/onjava/2005/06/29/spring-ejb3.html. Comments? Did they do justice to the comparison? Have you tried EJB 3 yet? At first glance this article makes it look pretty good.

Posted by gerryg on July 01, 2005 at 01:10 PM MDT #

Kishore and Gwyn - I agree that the numbers and graphs I assembled are not "hard evidence", but I do believe they are good indicators. It's one of the problems with open source, there's no real metrics that you can use. For example, I was only able to get number of downloads from a few project leaders, and you can't even track mailing list traffic for Spring since they use forums. If you have any suggestions for gathering more foolproof statistics, please let me know. These whitepapers are living and breathing (much like SourceBeat books), so we plan on refactoring and updating them as necessary.

Gerry - I did read the Spring vs. EJB 3 article yesterday. I don't really know enough about EJB 3 to put together a well-educated response. I agree it does look good, especially if there's an embedded version you can run in Tomcat. I think the biggest argument for Spring today is that it's been proven, many developers are using it, and it works really, really well.

Posted by Matt Raible on July 01, 2005 at 01:52 PM MDT #

Matt, I do understand the problem you have with trying to make hard metrics. It is tough, and it is what a lot of people want to see. Understanding does not mean agreeing though. And I certainly do not believe they are good indicators in their current state.

You provide the metrics of which you know their meaning is questionable, though you bring them as kind of hard evindence. /If/ you want to do the metrics business, you really should do better than just adding some numbers with hardly any context. E.g. don't only count the number of emails on a list, but also count the number of participants. Get an idea of how many times answers actually solved problems. And whether they were answered by core developers. Furthermore, who says having a lists that has a lot of traffic is a good thing? It might just as well be an indication of a too-hard-to-understand framework having insufficient documention. Same goes for the number of tutorials, tools, etc you find by doing a search with google (again, no quality criteria). They might just as well be indicators that there is something wrong with the frameworks in question. The only exception imo is the number of books on a framework, as they are at least an indication on how viable 'the industry'/ publishers think the frameworks is. And get the number of bugreports and how quick they were fixed, but also get an index on the 'quality' of those reports. Get some real software engineering metrics while you are at it (I find code duplication and test- and javadoc coverage pretty good indicators on the quality of the framework) and consider the resumes of the core developers (what have they worked on before, any abandoned frameworks, etc).

And my point is...? I think it is fine to try to include metrics, and most people will understand that these can never be perfect. However, I also think you can do much better than you do currently. The main reason your blog became so popular was that you allways have been targetting the average developer - a good thing! As I told you in that bar on JavaOne, with your recent switch to the consultancy side of things, you're not representing guys like me at all. In your world (representing the large companies I presume), managers and ivory tower architects make the technology decisions. In my world (the small and medium sized companies), the senior developers make the decisions or at least are the major influencers. And they are the ones that want to know /why/ a certain framework suits their needs better than other frameworks. They want to know the ins-en-outs on technical arguments, and are generally not interested in 'metrics' they can collect themselves by using google and doing a simple search on the monster board. And don't forget that the large companies usually hire the smaller ones to do their hard work.

Maybe I should start my own Bile Blog :)

Posted by Eelco on July 01, 2005 at 06:21 PM MDT #

Matt, Your papers represent some excellent efforts!! You have found a good perspective and audience to target. These provide a very good snapshot in time of some of the options and alternatives in each category. I was slightly disappointed that Wicket and Rails were not included in the web frameworks paper, not because they have significant adoption, but rather that they are very capable, recent frameworks for developing web applications. I look forward to your future whitepapers. I am more interested in metrics which represent productivity, perhaps for several types or sizes of web applications. I know these metrics are very difficult to create, as I've been working quite some time on producing a set of meaningful metrics for analyzing enterprise projects and selecting appropriate tools and frameworks. Adoption is one of the concrete aspects of development we can grab onto now. Perhaps we can grow on this in the future to at least partially categorize and quantify enterprise project efforts in these metrics. Cheers, Vince Marco President, Enterprise Frameworks, Inc.

Posted by Vince Marco on July 02, 2005 at 09:51 AM MDT #

Those PDF-links won't work.

Posted by Tuomo Taivalkoski on October 06, 2008 at 12:30 AM MDT #

Yep, Virtuas is no longer in business.

Posted by Matt Raible on October 06, 2008 at 10:31 AM MDT #

Post a Comment:
  • HTML Syntax: Allowed