Raible Designs

Short and sweet contracts

This has been a good month for Raible Designs. Not only did I manage to land a new gig, but I've also had a couple of 1-day contracts. The first one was an architectural review (at the beginning of the month), and yesterday I delivered a presentation on TDD and Spring for a client. These short-n-sweet gigs are a lot of fun. It's a good way to get out and meet members of the Java Community and see what tools they're developing with. The first client is using Struts, Spring and Hibernate. The second client wanted to use Struts, but after my talk, they're thinking about Spring MVC. They plan on using Spring to make EJBs easier, and they're using TopLink on the backend.

The only bad part about yesterday's experience is I developed a full-body ache as soon as I left the client's site. I managed to catch a cold from Abbie this weekend, and that turned into a cold+fever last night. I've had the fever ever since and didn't go into work today. It's really shitty timing for getting sick - we have a deadline on Thursday and we're heading to Oregon to see my parents this weekend. Last I checked, my fever was at 102°F. Hopefully, I'll wake up tomorrow and it'll be gone.

Comments are flakey?

I've gotten several e-mails from readers that they haven't been able to leave comments on this blog. I suspect it has something to do with twisty comments, but I just tried them in Firefox 1.0 on Windows XP and they seem to be working fine. In order to help me track down this problem, try posting a comment to this entry and see if it works. Make sure and copy it before you post it, just in case it doesn't work. Clicking on the "Permalink" or title for this post will take you to a backup comment form that seems to work regardless. Thanks for the help!

Update: From the various comments, there doesn't seem to be a common thread. It seems that half the folks that used Firefox + XP it worked for and the other half it didn't. I doubt it's a caching issue, but I have been seeing a lot of the following in my logs:

ERROR 2005-01-22 19:51:01,944 | RequestProcessor:processMapping | Invalid path /comment was requested

Roller recently changed from using /comment to /comments - so I don't know if it's spammers hitting the old URL or what. My guess is yes since it seems to happening in blocks. I'll try to add a preview button tonight so users will get taken to the permalink form for previewing. Either that or I could just change to always using the permalink form. It's up to you guys - let me know what you prefer.

[DJUG] Testing and Handling Exceptions in the Web Tier

I'm attending Denver's JUG tonight, where Scott Davis is talking about Unit Testing the Web Tier. His opening slide says he's going to cover HttpUnit, Canoo WebTest and JMeter. I'm most interested in the JMeter stuff as I've been meaning to integrate it into AppFuse. I've used HttpUnit and it's a little verbose for me. I prefer using Canoo WebTest or jWebUnit over HttpUnit. On my current project, we're considering using jWebUnit or HttpUnit to act as a browser when interacting with a 3rd-party system.

All of these tools run functional tests - which are much different from unit tests. Unit tests usually tests the bricks, whereas functional tests test the building. For unit and functional tests to be truly effective, they must be:

• Scriptable
• Repeatable
• Automated
• Darn close to 100% coverage

The tools Scott is talking about tonight have passed his basic tests:

• Can I learn it in 10 minutes?
• Does it play nicely with my existing test environment?
• Does it play nicely with my existing production environment?

I didn't take any notes about HttpUnit or Canoo WebTest because I didn't really learn anything new. Scott did do a nice job in his HttpUnit examples - he made it look a lot simpler than I've previously seen. I've used it HttpUnit before and it seems a bit verbose. I've always used jWebUnit, which simplifies HttpUnit's API.

JMeter allows you to do the same thing as HttpUnit and Canoo WebTest. It's a standalone GUI, for the complete non-programmer. It does not plug into Ant/JUnit and is mostly used for load testing. I thought it was exclusively used for load testing - and I think it has an Ant task. I could be wrong.

<sidenote>Scott uses Smultron for XML editing on his Mac.</sidenote>

The basic building block of JMeter is a "Thread Group". The Thread Group allows you to control the number of users/threads that run a particular test. You can test gets, posts, change the protocol and even upload files. For load testing, make sure and check "Retrieve all Embedded objects in HTML files". You have to view the "result windows" to view that your tests actually ran - there's no "green bar" feature.

I think JMeter has improved a lot since I last looked at it. Scott's overview and demonstration make it look very straight forward and easy to use. One guy asked if it's possible to see a a global view of all tests run. Scott thinks it's possible by adding a Listener to the Thread Group and creating a graph (one of the options). Scott is now showing a lot of the options in JMeter - there's a ton! It's almost overwhelming.

Next up is "Exceptional" Web Apps by Stephen A. Stelting, a senior Instructor and Author from Sun. His latest book is "Robust Java". Stephen has spent the last year and 1/2 figuring out how to make Java fail.

Objectives:

• Describe the types of errors that occur in the web
• Explain how exceptions and errors can be handled
• Describe the web container response to exceptions
• Present best practices to address web tier exceptions
• Show how web frameworks handle exceptions

Payoff: As a result of this talk, you'll have a better understanding of how to use exceptions in Servlets and JSPs, improving the robustness of your webapps.

I'm a little skeptical at this point. I think most folks don't do exception handling in their webapps. I hope Stephen has some good tips and tricks for those of us who are familiar with handling exceptions. I wonder how he feels about Spring and its runtime exceptions?

There are two types of exceptions in the web tier: HTTP Errors and Java Exceptions. Standard HTTP Errors are handled by the web server. You can also send your own HTTP errors by calling HttpServletResponse.sendError(). If you're using response.sendError(), make sure and call it before you commit the output. The web.xml file allows you to specify errors and exceptions with the <error-page> element.

Servlets and Filters have similar exception behavior. Both declare exceptions in both of their lifecycle methods: init and service (doFilter for Filters). Developers throw exceptions in lifecycle methods to "tell" the container about problems.

• javax.servlet.ServletException
• javax.servlet.UnavailableException
• java.io.IOException

Stephen is now describing the init() method and the exceptions it can throw. Yawn. I think most Java web developers use frameworks these days. Because of this, most developers probably don't use these methods because they don't write plain ol' servlets. One thing I didn't know is that UnavailableException takes a time parameter - if you throw an UnavailableException with this parameter, the container will retry after the specified amount of time.

Result of Exceptions in init(): The destroy() method is never called, since the initialization did not complete. Client calls during component unavailability render a 500 error.

I stopped taking notes at this point because my laptop battery was dying. I didn't really learn much in the rest of the presentation. While I can appreciate Stephen's enthusiasm, it was obvious that he was an instructor and not an in-the-trenches developer. He explained a lot of what and didn't have any code to show how to do stuff. There wasn't a single demo in the entire presentation.

Most of the exception handling stuff Stephen talked about for the rest of the session was common sense (IMO). It also centered around the Servlet and JSP API, which most folks probably don't mess with. The Struts and JSF coverage at the end was cool. If nothing else, it was to nice to hear a Sun employee confirm that JSF is quite deficient in its hooks to allow easy framework-configurable exception handling.

Now that I'm working at home, and working/interacting with friends all day - it seems that the DJUG meetings aren't as exciting. They used to be fun because I could get out of the house and have a few beers with friends. Maybe it was the lack of learning anything new tonight.

How do you become an independent consultant and get contracts?

A friend recently sent me an e-mail looking for advice on becoming an independent consultant and specifically how to get contracts. I thought this advice might be interesting for others. The e-mail below is unmodified for the most part. He's an animator and does a lot of stuff in Flash, so this naturally has that twist to it. For Java Developers, just replace Macromedia/Flash stuff with Java stuff.

The best thing is to subcribe to the job mailing lists in Denver. There's a Rocky Mountain Internet Users Group and Macromedia Users group that would probably help you.

http://www.rmiug.org/html/email_lists.html

http://www.rm3ug.com

Then I would advise you to get involved in with a local user group, like the Macromedia one. Attend meetings, talk to people, see where they're getting their gigs. I go to the Denver Java Users Group on a monthly basis and now I have a lot of friends there. It was nerdy at first, but then I realized they all went out for beers afterwards and it's kinda cool now. If you volunteer to speak at one of the meetings, you'll probably get some leads from that. It's really all about networking.

The best thing you could possibly do for you career and new clients is to start a weblog. On it you can talk about what you do and how you solve problems. Tips and tricks kinda stuff. I put a lot of personal stuff on mine (www.raibledesigns.com) too. Ever since I started my weblog, I haven't had to do much looking. Often, I can just post I'm looking for a new gig and I'll get offers. I get 8000 visitors a day and around 2 million hits a month. It's the main reason I got my last couple of book deals.

Any other advice you might have for people looking to become independent consultants?

In general, I find independent consulting a lot more fun than full-time employment. The main benefits of full-time employment are Health Insurance and 401K Plans. Stock options are not a benefit in my eyes. I've yet to meet anyone who has made money off stock options as a full-time employee.

The main benefits of being an independent consultant are higher pay and freedom. An experience full-time Java Developer (in Denver) makes around $100K year. I've heard of companies hiring employees for $120K, but the most I've known anyone to make is $108K. On other hand, a contractor with only a year or two of experience can easily make $55/hour - or $110K year. Experienced developers get anywhere from $75/hour to $200/hour. That's right folks - $150K year to $400K year! For the higher dollars you often have to travel, which kinda sucks.

People that are full-time employees often like it for the benefits - health care and such. As a consultant, you get to see how screwed up our health care system is and pay for your health insurance out-of-pocket. I've seen folks pay anywhere from $250/month to $1000/month for health insurance. We've done both (the latter thanks to Corba) and I'm happy to say that we're paying $250/month now. United Healthcare, minimal plan - but we're still insured. What about 401K? As an independent, you often will establish your own company, and then you can contribute to an SEP plan. With a 401K, you're limited to contributing something like $15K/year. With an SEP, you can contribute up to $40K! Granted, you won't have any employer matching, but you can still plan for your retirement.

The one downside I've seen personally from being independent is I tend to find a lot of projects where I'm the development team. This is great at first since I can work remotely and don't have to attend any meetings, but I tend to miss the water-cooler talk and synergy that a team provides. With full-time positions, this can happen too, but it's rare. With full-time positions, you're likely part of a development team - and if you're lucky - one full of smart developers. One myth about contractors is that they often don't fit into a team full of full-time employees. Personally, I've found this to be a personality thing and have rarely had issues fitting in with full-time employees. The ideal situation is simply to work with smart people. In my career, working with people smarter than me has always been very lucrative to my knowledge base.

Working from home is not all it's cracked up to be. I've done it off and on for a few years now and I'd much rather go into an office at this point. When I work from home, dinner tends to be a just a "break" in my day and I go back to work after the kids go to bed. When I'm in an office, the work day ends when you leave. So how do you get the team-benefits as a independent consultant? The best way I've found is to work with a group of consultants on a project - where the whole team is contractors. You'll often share the same lifestyle and attitudes about your careers. Also, a lot of independent consultants tend to be smart - so you get that benefit.

The last thing I like about independent consulting is freedom. I had 4 months last year where I attended conferences, went on vacation or worked on Spring Live. Good luck finding that kind of freedom with a full-time gig. The higher rates allow you to take more time off to spend with your family - or just enjoy life in general.

If you're thinking about becoming an independent consultant, now is a better time than ever - especially if you live in Denver. The Denver JUG mailing list received more job postings last year than any previous year. From what I've seen, at least half of these are for contracting positions. Unfortunately, most of them are in the $40-60/hour range, but that's still good money.

Any feedback on why you think being a full-time employee is better is most welcome.

How do I attach a profiler to Ant?

I'll admit, I have very little experience working with a profiler - but I think I need one. Someone e-mailed me about testing all of AppFuse on a super-beafy box, and he got OOM errors after 17 minutes. For a dual CPU, 2Ghz Opterons, with 2GB RAM running RHEL3 workstation, it only made it to the Spring/AppGen test, which is #8 of 21. On my PowerBook, it makes it to #12, but my Linux box only make it to #8 too. My guess is there's some sort of memory leak in one of the testing tools - Tomcat, Cargo, WebTest or AppFuse itself.

Anyone have experience attaching a profiler to their Ant build or testing process to look for memory leaks?

When GMail and Firefox Suck

For the most part, both GMail and Firefox rock. However, when you're typing a long e-mail in GMail and you hit Ctrl+R instead of Ctrl+T, it really sucks. Rather than opening a new tab, I lost my whole e-mail message. Damn. Maybe it's time to go back to the desktop client. I've lost a few e-mails with GMail, but I don't think I've ever lost one with Outlook or Thunderbird. I guess it's time to setup GMail's POP3 access.

Edit Screens with JSF

I'm working with JSF this morning and I'm finding one thing particularly annoying. I'm working on a simple master/detail screen and I'm tweaking the detail screen to fit my needs. It's just a <form> with some form elements. I change something, run "ant deploy-web" and hit "refresh" to see my updated page. Since everything in JSF is a post, I get prompted to re-submit the form. Fine, I agree - then I'm returned to the list screen. Argggh - why can't I just view the form again?! This might just be a MyFaces nuance, I'm not sure. Anyone know of a workaround?

Wanna see the bug/feature in action? Go to http://demo.raibledesigns.com/equinox-jsf/userList.html, click on a row - and after the edit screen displays, hit refresh on your browser. In an ideal world, you'd see the form again, but nope - you get the list instead.

Can you base a web framework on another web framework?

It looks like the Struts guys are going to base Struts 2.0 on JSF (detailed proposal, wiki). It should be interesting to see a web framework developed on top of another web framework. Of course, it's been done before. The two things I really like in this proposal are 1) a built-in IoC container and 2) the security stuff (remember me, user registration). Here's the details from the proposal:

3.5 Service Provisioning APIs

Inversion of Control (IoC) containers (the techniques are also referred to as Dependency Injection) are becoming a popular mechanism for assembling the required services and logic of an application. If Struts included such a framework, it would provide a solid basis for building maintainable apps, as well as allowing the framework to configure itself using the same capabilities.

Rather than building such a container ourselves, we should seek to incorporate an existing one that is license-compatible and which can be integrated into the JSF managed beans facilities (so that value binding and method binding expressions can leverage the facilities of this container transparently). From my research so far, I like Spring's capabilities in this area the best, but am open to other suggestions.

3.6 Authentication and Authorization APIs

In order to support reasonably complete solutions for applications that wish to provide their own authentication and authorization services (as well as interact with container managed security), we need APIs available for performing user registration, implementing "remember me" features, and represent the results via a wrapped request (so that apps depending on getRemoteUser(), getUserPrincipal(), and isUserInRole() will still work). Using JDK 1.4 as a base platform would allow us to integrate mechanisms like JAAS. Other alternatives include plugins like SecurityFilter.

I added the emphsis on Spring. I'm all for this because 1) I'm on the JSF Expert Group (as of last week) and 2) it'd be nice to see JSF stretched, poked and prodded. IMO, JSF needs more "real-world" usage and the popularity of Struts will give it that. By "real-world" usage - I mean it needs to be used by developers that aren't cooped up in some cube with a fancy IDE. I also like the proposal because there's tons of Struts apps out there - and many of them might want to "upgrade". This means more (potential) work for me. Of course, I also plan on spending more time with Tapestry so I can offer client's an "educated" suggestion. BTW, the reason I joined the JSF EG is to be the "developer's voice" among the IDE Vendors. So if you'd like your voice to be heard - send me your suggestions, I'm all ears.

The question is - when will Shale be released? Of course, since it's open-source, the answer is "when it's ready". My guess is 1 year from today. I'm thinking of making AppFuse 2.0 use this new stuff. Of course, it'll have Tapestry and JSF options long before that.

If nothing else, the Shale proposal should motivate other web framework developers to highlight their roadmaps and what's new and cool on their horizons.

Comparing Web Frameworks Presentation

Since Kris let the cat out of the bag, I might as well give you the link to my comparing web frameworks presentation (PDF, 280KB). I also created a page on the Equinox site for this presentation and related materials. In addition to the presentation, this page also has links to the various framework implementation demos. Here they are for your convenience:

• Spring
• JSF
• Struts
• Tapestry
• WebWork

Kris notes that I would still choose Struts. I think it should be noted that I would only choose it in combination with AppFuse (which generates ActionForms). Same goes for Spring and WebWork. I've added interceptors and convenience methods that simply make developing with these frameworks faster and easier. I would've chosen WebWork for my current project, but I'd like to see better client-side validation. Spring needs better tag libraries.

I think the choice of what framework to use is a very personal thing. I think the "best" framework for one person might be very different for someone else. For me, I typically do short-term projects with clients - get them up and running with an application, and then head off to the next project. It makes sense for me to create applications that use a popular framework like Struts that they can easily find developers to maintain it. However, the one thing I'm starting to find is that as long as I use AppFuse - there's good documentation on how to do things. So I've already written the "how to develop and extend this app" for future developers of a client's application. This will (hopefully) open the door for me to use any web framework that AppFuse supports.

I think WebWork rocks, but it's similar to JSF in that it doesn't come with everything your need. The good news is it's easy to write interceptors, but IMO there's a few that should be part of the framework. After working with Tapestry and JSF, I can see how component-based frameworks will be the wave of the future. I think as you develop more and more components, the code you write becomes less and less. It's funny that this is one of the goals of AppFuse - incorporate a bunch of tips and tricks for various frameworks to make development easier. By adding support for Tapestry and JSF, hopefully AppFuse will someday become a repository of useful components. Documentation is good - code is better.

I'd probably be more enthusiastic about Tapestry and JSF if I knew more about them. I still have a lot to learn. I've bought the books (Tapestry in Action and Core JSF), I just haven't had time to read them. I think after incorporating these frameworks in AppFuse (hopefully this year), I'll get a better feel for them and how they make development faster and more efficient. My major problem with JSF is that it's being written for the tools vendors and not for the developers. Make it easy for everyone, not just folks that want to use their hammer-like IDE to develop webapps. The major problem I have with the JSF Tools is 1) none of them are free and 2) most of them are tied to a proprietary app server.

Trails - like Rails, but with Tapestry, Spring and Hibernate

I've been thinking about Rails ever since I wrote a post about it on Monday. The main reason is because of Dion's comment:

Matt - You should follow the lead and do a video of setting up a simple app using AppFuse.

This might sounds like a good idea, but if I did it right now in AppFuse's current state, it'd be a disaster. The reason? Because you have to manually create a whole bunch of classes to do CRUD on a database table. Here's a list of new classes needed for adding a new "person" table.

• model.Person
• dao.PersonDAOTest
• dao.PersonDAO
• dao.hibernate.PersonDAOHibernate
• service.PersonManagerTest
• service.PersonManager
• service.impl.PersonManagerImpl
• webapp.action.PersonActionTest
• webapp.action.PersonAction
• web/pages/personList.jsp
• web/pages/personForm.jsp

The last two JSPs can be generated, but that's still a buttload of classes (9) just to CRUD (and test!) a database table. Not too mention all the files you need to edit for Spring and i18n.

• dao/hibernate/applicationContext-hibernate.xml
• service/applicationContext-service.xml
• test/web/web-tests.xml
• web/WEB-INF/classes/ApplicationResources_en.properties
• web/WEB-INF/menu-config.xml

Result: to CRUD a database table using AppFuse you have to create 11 new files and modify 5 existing files. 16 files. What a beotch, huh? If I made a video of this - it'd be 20 minutes long! While this might make AppFuse look silly, it's really more of a symptom of the patterns we have in J2EE and how we're supposed to architect our apps. 3 tiers, test-driven, loosely-coupled and internationalized. Of course, if I was focused on fast and efficient, I could do this all with 1 JSP and JSTL's SQL tags. Everyone would slap my hand for not following patterns, but I'm willing to bet it'd work just as well and be just as fast. But I digress.

There have been a fair amount of requests (and some patches submitted) to generate and modify all of the files listed above. For the most part, I've frowned upon adding such a feature because I think if folks can run "ant generate -Dmodel=Person" - they'll end up with a whole bunch of code that they know nothing about. Sure there's the tutorials, but folks will quit reading those. Instead, they'll create a whole slew of POJOs (maybe even using Middlegen) and run "ant generate" on all 50 of them. Poof - now they've got 550 new files to maintain. Talk about a maintenance nightmare. Even worse - a support nightmare for me.

Nevertheless, if I wanted to create a cool video for AppFuse, I'd spend a few days writing this code-generation engine. Then I could show how you could create the data, service and web layer (including UI) in a matter of seconds. It'd be cool and folks would dig it. I'm still considering it, but I'm also leary of the resulting support fiasco. Maybe I could just say "use at your own risk".

A while back, I saw Erik Hatcher suggest a better solution than code-generation. I can't remember what he called it, but it was something like "meta-data dynamic rendering". The idea is that your application reads the metadata of a table (or POJO) and renders the appropriate UI for it. I loved the idea as soon as I heard it. I've always wanted a way to dynamically render the UI rather than writing HTML. Of course, I still want the ability to edit the templates and HTML since I fancy that sort of stuff. I don't like writing HTML for each row in a form, but I do like tweaking the HTML and CSS to look good.

Earlier this week, I saw the concept in action with Rails and its demo. IMO, something like Rails would never fly in Java because it appears to be tightly coupled to the database and only MySQL, PostgreSQL, and SQLite are supported. The Java community always seems to pride itself on database abstraction, partly due to JDBC and its ability to connect to anything that has a JDBC Driver. Ruby will probably catch up someday, but right now it appears to be looking for something like JDBC.

Then along comes Trails, which made me smile earlier today when I first read about it on the tapestry-dev mailing list.

I've been working on a project called Trails that uses Tapestry quite heavily and I thought it time to start soliciting feedback. Trails is a domain driven development framework that uses Tapestry, Spring, and Hibernate. Trails is very much in it's infancy, but the current version is functional and should give people a rough idea where I am heading. It's my first real forray into Tapesty and I have really found Tapestry a joy to work with.

Trails is very much like Rails, except that it doesn't talk directly to a database table. Instead, it talks to your domain objects that you mark up with XDoclet/Hibernate tags. To test it out, I dropped a User.java file into the org.trails.demo package, marked it up with XDoclet and deployed. It didn't work at first because the .hbm.xml files are explicitly listed in Spring's applicationContext.xml. I changed the "sessionFactory" bean to use the following and wammo - success! I could list and CRUD the table that my User object was mapped to.

  <property name="mappingDirectoryLocations">
    <list>
      <value>classpath:</value>
    </list>
  </property>

Trails is very cool, and I'd love to incorporate it into AppFuse or Equinox. Does an LGPL license allow me to do this? The one problem I can see with adding it is that it's specific to Tapestry and Hibernate, which doesn't always suite folks. I think developers might be willing to change because this solution will vastly improve their development productivity, but who knows. I think the best solution would be to offer this option in AppFuse/Equinox, but also offer the current manual and code-generation options. The holy grail would be the ability to plug in iBATIS or JDO instead of Hibernate. In addition, using Struts, Spring, WebWork or JSF instead of Tapestry would have folks clammering to use this stuff.

Mad props to David Heinemeier Hansson and Chris Nelson - you guys are developing awesome software.