Raible Designs

Posted by Kishore Dandu on July 01, 2005 at 03:56 AM MDT #

Posted by gerryg on July 01, 2005 at 07:10 PM MDT #

Matt, I do understand the problem you have with trying to make hard metrics. It is tough, and it is what a lot of people want to see. Understanding does not mean agreeing though. And I certainly do not believe they are good indicators in their current state.

You provide the metrics of which you know their meaning is questionable, though you bring them as kind of hard evindence. /If/ you want to do the metrics business, you really should do better than just adding some numbers with hardly any context. E.g. don't only count the number of emails on a list, but also count the number of participants. Get an idea of how many times answers actually solved problems. And whether they were answered by core developers. Furthermore, who says having a lists that has a lot of traffic is a good thing? It might just as well be an indication of a too-hard-to-understand framework having insufficient documention. Same goes for the number of tutorials, tools, etc you find by doing a search with google (again, no quality criteria). They might just as well be indicators that there is something wrong with the frameworks in question. The only exception imo is the number of books on a framework, as they are at least an indication on how viable 'the industry'/ publishers think the frameworks is. And get the number of bugreports and how quick they were fixed, but also get an index on the 'quality' of those reports. Get some real software engineering metrics while you are at it (I find code duplication and test- and javadoc coverage pretty good indicators on the quality of the framework) and consider the resumes of the core developers (what have they worked on before, any abandoned frameworks, etc).

And my point is...? I think it is fine to try to include metrics, and most people will understand that these can never be perfect. However, I also think you can do much better than you do currently. The main reason your blog became so popular was that you allways have been targetting the average developer - a good thing! As I told you in that bar on JavaOne, with your recent switch to the consultancy side of things, you're not representing guys like me at all. In your world (representing the large companies I presume), managers and ivory tower architects make the technology decisions. In my world (the small and medium sized companies), the senior developers make the decisions or at least are the major influencers. And they are the ones that want to know /why/ a certain framework suits their needs better than other frameworks. They want to know the ins-en-outs on technical arguments, and are generally not interested in 'metrics' they can collect themselves by using google and doing a simple search on the monster board. And don't forget that the large companies usually hire the smaller ones to do their hard work.

Maybe I should start my own Bile Blog :)

Posted by Eelco on July 02, 2005 at 12:21 AM MDT #

Posted by Tuomo Taivalkoski on October 06, 2008 at 06:30 AM MDT #