Matt RaibleMatt Raible is a writer with a passion for software. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.
You searched this site for "framework". 558 entries found.

You can also try this same search on Google.

Wicket is the most widely used Java Web Framework

According to this post, Wicket is the most widely used Java web framework. While I believe his statements are true, IMO it's only true in the context of this guy's post. My guess is that most of the folks that read his post were Wicket users, or somehow the Wicket Team got wind of this and told users to e-mail this guy. The main reason I don't believe that Wicket is the most popular is because it's so new. I do think it's an up and coming framework that may become the most popular, but I don't think it's there yet.

So, to answer this question, I believe that claiming Wicket to be the most widely is Just Good Guerilla PR. To put some numbers behind that, here's some graphs showing mailing list traffic for the various Java web frameworks.

Web Framework Mailing List Traffic - May/July 2005

Granted, these are just indicators of the number of users - but I believe they are a good indicator. One interesting note about these stats is that Wicket's mailing list traffic has increased significantly in the past few months. Ironically, here's what one of the Wicket developers had to say about this statistic about a month ago:

... who says having a lists that has a lot of traffic is a good thing? It might just as well be an indication of a too-hard-to-understand framework having insufficient documention.

In the past couple of months, I've spoke in front of 25+ Java developers on 4 different occasions to talk about web frameworks. I've asked those developers which frameworks they've used, or plan on using. Struts is still the most widely used, with WebWork and Tapestry the least used. Surprisingly, JSF seemed to be getting no traction among the the audiences I spoke to. Even more surprising (to me at least) was that the most popular web framework continues to be the in-house framework. The overwhelming majority of the developers I've talked to aren't even using open-source web frameworks.

Posted in Java at Aug 18 2005, 12:13:11 PM MDT 2 Comments

Integrating Lucene with Hibernate

The code_poet has an interesting post on using a Hibernate interceptor to index objects for Lucene searching. I've been thinking about integrating Lucene into AppFuse (or at least providing a tutorial) for quite some time. Hopefully this post will serve as a starting point when I do. I wonder how much the Lucene code can be simplified by the Spring support in the Spring Modules project?

Posted in Java at Aug 17 2005, 10:19:09 PM MDT 17 Comments

First Magazine Article: Challenges in the J2EE Web Tier

It's official - I've written (and published) my first article in a magazine. If you received the July issue of Java Developer's Journal, you'll see my article titled Challenges in the J2EE Web Tier.

Over the course of its life, the J2EE Web Tier has faced many challenges in easing Web application development. While it's a scalable, enterprise-ready platform, it isn't exactly developer-friendly. Particular challenges to Web developers include the need for a standard Web framework, compatible expression languages, and availability of components. Several Web frameworks have been developed to resolve these issues, each with its own strengths and weaknesses. This article discusses the unique challenges of the J2EE Web Tier and how various technologies have attempted resolve them. By learning from and competing with each other, these Web technologies play an important role in pushing the limits of excellence to produce ever-higher standards of Web application development.

Enjoy!

Posted in Java at Aug 10 2005, 04:30:22 PM MDT 11 Comments

Things I learned from OSCON

While at OSCON last week, I learned quite a few things. Many of these are my own opinions, so feel free to disagree with them.

Ruby is very cool

The main reason (for me) that Ruby is cool is because it's new. I don't have to know any of its history to know what makes it tick. It's easy to learn and has powerful language features that make it easy to use. I think the primary reason people, particularly Java developers, are excited about it is because it releases them from all the shackles they're accustomed to with Java. I have absolutely no plan to ditching Java and jumping to Ruby, but I do want to learn it so I can use it when it's a better fit than Java.

How to learn Ruby and retain that knowledge is the hard part. My current plan is to buy Programming Ruby and Agile Development with Rails. Unfortunately, I realize that I probably won't be able to finish these b/c I'll get bored and both will likely serve as more of a reference than a knowledge creator. I haven't been able to read a technology book cover-to-cover in several years. I couldn't even finish The Pragmatic Programmer for crying out loud. I realize that the best way to learn is to do, but the best way to do is to get paid to do. To facilitate this, I hope to develop some apps we can use at Virtuas. Of course, I'll try to find good open-source solutions first.

Rails has a lot of great ideas

The interesting thing about Rails is many of its good ideas are from Ruby. The built-in Webbrick web server is part of the language. The only way to come close in the Java world is to embed Jetty or something like that. I'll definitely be looking into embedding Jetty into AppFuse in the near future. The other thing I really like about Rails, and that I've been doing in AppFuse a bit is convention over configuration. As part of AppFuse, I'm already making a lot of decisions for users. The next step seems to be making all decisions for users, but allow them to override. It'd be cool to write some code that sweeps through all classes at startup and auto-configures them, w/o the need for any XML. If nothing else, the XML could be generated using reflection.

The biggest thing I learned from Rails is I need to provide 1) an easier upgrade path and 2) better Ajax support. Because I'm supporting so many different web frameworks, solving #2 might be a bit tricky - but could be done by writing tag libraries or components. Hopefully the framework developers will beat me to it and I won't have to do anything. As far as #1, I'm hoping I can move to a single appfuse.jar that contains all the base classes. Hopefully I can use a little JSP pre-compile action to re-use the existing JSPs for user management/etc. If not, I can always use something like FreeMarker to store the default view files in a JAR.

Creating Passionate Users is all about inspiring emotion

Kathy Sierra's talk about inspiring emotions among your users to make them transparently excited about your product/company was a real eye-opener. I can totally see what she's talking about and I'm happy to say I'm already doing some of it. This blog seems to attract lots of readers, some more passionate than others, just by talking about web technologies and Java. This does translate into more business for Raible Designs, whether it's the title image or URL, it doesn't really matter. Folks do figure out that I have a company and I do work with the technologies I talk about.

To apply these concepts to AppFuse and Virtuas is a little more difficult. For AppFuse, I can probably teach people how to better use Hibernate, Spring and Ajax - and then show how AppFuse can simplify things. Building in easy-to-use Ajax support is probably essential to really get this going. For Virtuas, we could re-vamp our site to provide education about open source and its history - providing users with a way to become open source experts. It would also be cool to do real-time reporting of how we helped a company adopt open-source.

Ideas, ideas, I have lots of ideas after last week.

Posted in Java at Aug 08 2005, 03:40:38 PM MDT 4 Comments

Struts Ti

I heard about Struts Ti at OSCON, and after googling a bit today, I've discovered a bit more. Here's a bit about the project from Don Brown's proposal on the Struts Developers Mailing List.

Struts Ti is a simplified Model 2 framework for developing webapps which allows the developer better access to the underlying servlet/portlet environment. It serves a niche of web applications that don't want the additional complexity of server-side components and verbose configuration, yet want the structure and controller features of a modern web framework. Struts Ti builds on the directions of Struts 1.x, yet re-implements the framework to provide a clean slate for the next generation of Struts Ti. It aims to combine the simplicity of Ruby on Rails and NanoWeb, the refinement of WebWork 2, the tool-friendly authoring and Page Flow of Beehive, and the lessons learned from Struts 1.x.

The key word for Struts Ti is simplicity. Ideally, Struts Ti should approach Ruby on Rails levels of easy of use, yet scale up to large applications providing a smooth transition to JSF/Shale if desired.

More information can be found at https://www.twdata.org/projects/struts-ti.

Posted in Java at Aug 08 2005, 01:14:53 PM MDT Add a Comment

[OSCON] Spring MVC vs. WebWork Smackdown

Matthew Porter and I's Spring MVC vs. WebWork Smackdown presentation was a lot of fun this morning. We had a boxing bell (that I got off eBay) and had a good time ragging on the two frameworks. The only surprise was that Matthew actually ran some metrics on the Spring MVC vs. WebWork code in AppFuse and pointed out that the WebWork version required 25% less code than the Spring version. Oh well. The hard part about this presentation for me was trying to defend Spring MVC and saying it's better than WebWork. Matthew obviously felt strongly that WebWork was the better framework, whereas I like them both.

Posted in Java at Aug 03 2005, 05:15:43 PM MDT 7 Comments

[OSCON] Monday Afternoon

Ruby on Rails - Enjoying the ride of programming
Presented by David Heinemeier Hansson, OSCON 2005

About David: started doing Ruby in June 2003. Involuntary programmer of need, served 5 years in PHP. Spent 7 months in a Java shop.

Prerequisites of play: Ruby 1.8.2, dated December 25th. A database, pick one of 6. The RubyGems miner. Some gems called Active and Action.

Directory structure that Rails creates is more for convenience than anything. By picking conventions for you, it makes things easier. It might feel like flexibility is being ripped away from you - but you can change the defaults. However, by following the default settings, things will just work and life will be much easier for you as a developer.

I did a bit of playing on my PowerBook while listening to David's talk. I have Tiger installed, but found that Ruby 1.8.1 was installed on my machine (in /sw/bin/) thanks to Fink. My running "rm -r /sw/bin/ruby" and restarting iTerm, the default changed to /usr/bin/ruby, which is 1.8.2. From there, I downloaded and installed the Rails Installer.

I hate to admit it, but this talk is pretty boring so far. Probably because I've read David's blog for the past 6 months and watched most of the Rails videos. I haven't really learned a whole lot in the first 45 minutes of this talk. To be fair, the content of the talk seems to be properly targeted - there's been a fair amount of questions and everyone seems to be interested. Almost all of the seats are filled in the room; 3-4x as many folks as Dave Thomas's Ruby talk.

One interesting thing I've learned today is many features of Rails (i.e. Webrick) are actually a part of Ruby, not Rails. In addition, Ruby seems to have frequent releases and more features are added to the language each time. I guess that's the advantage of having a language that's not developed by committee.

When creating model objects in Rails, the default is to use a plural form of the object for the database table name. For example, a comment model will map to a comments table. Dave Thomas did mention in this morning's session that Rails isn't smart enough to figure out "sheep" - it gets maps to "sheeps". Apparently, you can easily override this behavior by specifying use_plurals=false somewhere. Another convention built-in to the framework is that the primary key is named "id" and its an auto-incrementing field.

"The database is a data bucket. I don't want any logic in my database, I want it all to be in my data model."

Rails doesn't handle composite primary keys. Rails is mostly designed for green-field development, where you get to control your database and its schema.

There are a number of key properties you can use in your database tables (a.k.a. your model objects) that will automatically get updated if you name them properly. Their names are created_at (datetime), created_on (date), updated_at and updated_on. There are also a number of time-related helpers, i.e. distance_of_time_in_words_to_now(date) » less than a minute ago.

Rails also has the concept of filters, which you can apply to a group of controllers. To use a filter, you define the filter method in controllers/application.rb and then you have to add a before_filter clause in each controller you want it to be applied. While it's cool that Rails has filters, it would be nice if you didn't have to configure the controllers that filters are applied to in the controller. To me, it seems more appropriate to be able to configure the where the filters are applied externally to the controllers. It seems more natural to me that you'd put something like apply_to_controller => { :controller1, :controller2 } in application.rb.

For doing page decoration with Rails (i.e. SiteMesh), you simply create a decorator in views/layouts. If you want a particular decorator to apply to a particular controller, you just name the file the same as the controller's URL. For example, if you have a posts controller (really a PostController.rb file), you'll create a decorator named posts.rhtml to decorate all the HTML rendered from the PostController - regardless of whether you're rendering from a method or from a view template. To have a decorator apply to all controllers, you can simply create a file named view/layouts/application.rhtml. This seems like something that SiteMesh could easily do as well - for example defaulting to /decorators/default.jsp (or something similar).

One thing I like about Rails is it's flash concept and how easy it makes it to display success messages. In my experience with Java web frameworks - many make this more difficult than it should be.

Testing Rails Applications

When running tests, Rails automates the creation of a test database instance that mirrors the schema of your development database. One slick thing in a Rails project's Rakefile is that you can run all the tests that you've touched in the last 10 minutes. I think one of the most unique thing about Rails/Ruby vs. Java is all that almost all of the files (Rakefile, code generation scripts, etc.) are written in Ruby.

Controller tests have a "mini-language" for simulating a browser when testing controllers. For example:

def test_login 
  get :login
  assert_response :success<
  assert_template "login" 
  
  post :login, :password => "secret!"
  assert_response :success
  assert !session[:authorized]
  
  post :login, :password => "secret"
  assert_response :redirect
  assert session[:authorized]
end

In the Controller tests, you can set cookies, parameters and mimic almost everything the browser can do. You can also test that your model objects have been manipulated appropriately. For example:

def test_create_post 
  post :create, :post => { :title => "This is my title", :body => "1" }
  assert_response :redirect
  assert_kind_of Post, Post.find_by_title("This is my title")
  
  post :create, :post => { :title => "", :body => "1" }
  assert_response :success # something was rendered, regardless of error messages
  assert_equal "don't leave me out", assigns(:post).errors.on(:title)
  #or assert_equal 1, assigns(:post).errors.count
end

The find_by_title method is a dynamic finder, where ActiveRecord creates find_by methods for each attribute of the model object. Another cool feature of testing is you can add a line with "breakpoint" in it - and the test will stop executing there - giving you access to all the variables at that point.

Ajax

The main reason for integrating JavaScript into Rails is so developers don't have to write JavaScript. For most developers, writing JavaScript is a pain because of browser incompatibilities and such. Rails ships with 4 JavaScript libraries, including Prototype and Script.aculo.us. It's easy to include the default JavaScript libraries in Rails:

<%= javascript_include_tag :defaults %> 

Both the link_to and form_tag methods have a "remote" equivalent (i.e. link_to_remote) that allows you to hook into Ajax, and by defining a :complete callback, you can call fade effects and the like. You can override many of the lifecycle stages of Ajax, but the most common is the :complete callback. In a Controller, it's easy to distinguish Ajax calls from non-Ajax calls using:

if request.xml_http_request?
  # do logic, for example rendering partials
end

Partials seem to be a pretty cool feature in Rails. They're actually just parts of a page that you include in a parent page with render :partial => "viewName". The slick thing about partials is you can actually populate their model and return them in a controller after an Ajax call.

The Ajax demos that David just showed are pretty cool. He was able to easily show how to delete a comment in his "weblog app", as well as add a new comment - w/o refreshing the page. The slick part of the add was he was easily able to add the new comment id to the Ajax response header, and then grab it in a callback and use the id to reference a <div> and use the yellow fade technique to highlight and fade the new comment.

That's the end of Dave's talk, and the first day at OSCON. Thanks to Dave and David for showing me the cool features of Ruby and Rails.

Posted in Open Source at Aug 01 2005, 05:02:31 PM MDT 5 Comments

[OSCON] Monday Morning

Facets of Ruby
Presented by Dave Thomas, OSCON 2005

I'm sitting in Dave Thomas's session on Intro to Ruby at the Oregon Convention Center. It looks like someone finally figured out the main problem with conferences - lack of power outlets. Kudos to O'Reilly - they've put power strips at the base of every table in this room. With the high-speed wireless and unlimited power, this conference is getting off to a great start.

Is programming still fun?

Round about 2000, programming started getting tedious for Dave - after having fun for the past 25 years. When we program, we combine all the problems of the artistic side of the race with all of the problems of the scientific side of the race. The only way to be successful at it is to enjoy doing it.

Is programming productive?

It has to be to enjoy it. The most satisfying thing about programming is watching it run. That's why scripting languages are so great - because you can see it run now. Myth: a good programmer can be a good programmer in any language. Language and tools make a difference - a good programmer knows which language to choose for a particular problem.

This session is not going to be a syntax session. Damn, sounds like I won't really learn how to program Ruby in this session.

Ruby, the Language

Born in Japan in 1994. Father: Invented by Yukihiro Matsumoto (Matz). Mother: Ada, Smalltalk, CLU, Perl, Lisp. Grew very rapidly in 2000, outpaced Python in 2000. Became international star in 2004.

Dave and Andy are language freaks and downloaded Ruby 1.4 shortly after finishing The Pragmatic Programmer. It passed the 5-minute test, the 1/2-hour test and Dave ended up playing with it all morning. The first Pickax book was 500+ pages long, and they wrote it because there wasn't much English-language documentation on Ruby. Ever since Rails, Ruby's adoption has grown exponentially.

Ruby is a multi-paradigm language: procedural, object-oriented, functional, meta-programming. You can write procedural code, but you'll be using OO concepts at the same time. You can do all of these at the same time.

Ruby code example:

# Generate Fibonacci numbers up to a given limit

def fib_up_to(max)
  f1, f2 = 1, 1
  while f1 <= max
    puts f1
    f1, f2 = f2, f1+f2
  end
end

fib_up_to(1000)

Methods start with def and end with end. The parenthesis around the method arguments are optional.

Now Dave is ragging on Java Programmers and how they discount Ruby because of its duck typing. In a Java program, most things are dynamically typed too. This is because most objects are stored in collections and whenever you pull things out of a collection - you have to cast from Object to the real type. The argument is that you don't have to have static typing. Dave hates Generics because he thinks they should've just done automatic casting.

The basic gist of Dave's argument is that we use dynamic typing (using casting) in Java all the time and you don't see Runtime exceptions all of the place. So the biggest proponents of static typing are actually using dynamic typing all of the place.

Back to the code: in Ruby, you don't need parenthesis around conditionals (for instance in the while statement above). The main reason we have parenthesis is because of Fortran. There's no reason for them. You can put parenthesis and semi-colons in your code, but you don't need to. In this code example, the variables are scoped for the duration of the method. puts (pronounced "put s") just prints the value of a variable to the console.

class Song
  def initialize(a_title)
    @title = a_title
  end
  def title
    @title
  end
end

Instance variables in Ruby start with an @ sign. The first time you use them, they spring into existence. If you access an instance variable and it's value hasn't been set - it's value is nil. Using the return keyword at the end of a method is optional - the default is to return the last line of a method. You can change the "title" method to use

attr_reader :title 

The attr_reader call is actually a method of Class:class. The attr_reader will dynamically add an accessor (that looks like the title method above). To create a setter, you can use attr_accessor and it'll create both a getter and setter.

Ruby is a single Inheritance language.

class KaraokeSong < Song

  attr_reader :lyric

  def initialize(a_title, a_lyric)
    super(a_title)
    @lyric = a_lyric
  end
end

Unlike Java, the super call can happen in any line, or not at all. To solve the single-inheritance problem, you can use mixins and apply them to any class.

Blocks and iterators are pervasive in Ruby, and look to be very easy to use.

3.times { puts "Ho!" }

hash.each { |key, value|
  puts "#{key} -> #{value}"
end

To do method calls with blocks, you use the yield keyword.

You can use blocks to simplify Resource Management and automatically close resources.

File.open("myfile.dat") do |f|
  name = f.gets
  # whatever
end

When the above code hits end, the file is automatically closed.

Duck Typing

Strongly-typed objects, untyped variables and methods. Types are implicitly determined by the things that an object can do. Duck typing is great for testing, refactoring and maintenance. This is very similar to concepts in Smalltalk. There is a strong commitment to unit testing in Ruby - which makes duck typing even easier to use. Duck typing makes things very easy. For example, you can have a method that takes a file as a parameter - and writes data to it. You can test this method by passing in a String (which also supports << for appending) and verify that your method's logic works.

Duck typing is useful in regular code for reducing coupling and increasing flexibility. Ruby community now differentiates the type (what it can do) of an object and the class (what generated it) of an object.

The Road to Metaprogramming

Metaprogramming is really, really powerful in Ruby. Library writers use it, but most developers don't use it. The ActiveRecord framework is an example of metaprogramming. Rather than being an O/R Mapping tool, it's more of a database table wrapper. The belongs_to, has_one, has_many and other validation_presence_of method calls can be written by you. Allowing you to write DSL (domain-specific languages) that appear to be a part of the Ruby language.

4 steps to metaprogramming:

  1. Classes are open: in Ruby, you can always add methods, attributes, etc. to existing classes. For example, you can add an encrypt() method to the String class. Isn't this dangerous? What if someone changes the logic of + for math expressions. No, because if one of your programmers overrides methods that breaks things - you take them out in the parking lot and beat them with a rubber hose! The language shouldn't prohibit us from doing powerful things.
  2. Definitions are active: code executes during what would normally be compilation. As the class if being defined, the code is being executed. For example, you can check if an environment variable is set - and create methods (i.e. log.debug()) accordingly. This can be great for caching.
  3. All method calls have a receiver: Methods are executed on objects. There's always a current object: self. Methods with no explicit receiver are executed on current object.
  4. Classes are objects too: You can easily add methods to classes.

Many more Ruby features: Reflection and ObjectSpace, Marshalling and Distributed Ruby, Tk, Gtk, Fox, networking, databases, etc. Garbage collection, Threads (like Java green threads), Modules and mixins. ObjectSpace - allows you to reflect on all of the objects that exist at runtime. Marshalling allows you to serialize into binary or text formats. No XML - uses YAML instead. Unlike XML, it's readable and looks more like a properties file. Modules (and their methods) can be easily included into a class simply by using "include ModuleName".

Now Dave is going to write a program to extract book sales ranks from Amazon pages, publish them as an RSS Feed, store them in a database, and access via a web application (using Rails). Since this is likely to involve a lot of live coding, I probably won't blog the code Dave writes.

Web Applications in Ruby can be done with Simple CGI, FastCGI, mod_ruby and frameworks (like Rails). Iowa, CGIKit, Nitro and Ruby on Rails are all web frameworks in Ruby. Iowa is a Ruby implementation of Apple's WebObjects. Dave's quote: "Apple really screwed up with WebObjects, they could've owned the market on web frameworks."

Summary

  • Use Ruby because it is Lightweight. A Ruby download is under 10 MB. Ruby Gems allows easy package management for downloading libraries and documentation.
  • Use Ruby is Transparent. It's nice and easy to read - and it only takes a couple of hours to learn its syntax.
  • Portable - runs on PDAs and Mainframes.
  • Open Source - MIT/Artistic style license. 1.8.3's regular expression engine is LGPL, 1.9's engine will be BSD-style license.
  • Easy to Learn - uses the Principle of Least Surprise. Things seem to work as you'd expect. Dave knows people that've downloaded Ruby and put web applications on line in the same morning - w/o any prior knowledge of Ruby.
  • Fun! It's enjoyable to program in.

Ruby Resources

Posted in Open Source at Aug 01 2005, 11:20:05 AM MDT 9 Comments

Taming JSF 1.1

A couple weeks ago, I received an e-mail from Ray Davis of University of California, Berkeley. In the e-mail, he provided me a link to his team's Confluence Wiki - where he describes their experience and frustrations with JSF 1.1. I really like how Ray explains the problems they experienced, as well as how they fixed them. The "request thread" scope they created sounds similar to what Spring Web Flow does.

Our experience left us very happy with Spring, moderately happy with Hibernate, and not at all happy with JSF. We did manage to deliver a Pilot Gradebook that told us what we needed to know, but sacrificed reliability, consistency, and scalability to do so.

In January 2005, when we moved to full-time work on the official Sakai 2.0 Gradebook, JSF was our biggest concern.

It's a good read for those looking to jump into JSF. I think JSF 1.2 will solve a lot of problems, but who knows how long it will take to get a RI and MyFaces version of that.

Posted in Java at Jul 27 2005, 02:51:40 PM MDT 9 Comments

[DJUG] Building an Open Source ESB and Ruby on Rails

Managing Chaos
Building an Open Source Enterprise Service Bus from Scratch

Bruce Tate got rained out in Texas, so Bruce Snyder and Jeff Genender are talking about building an ESB with open source software. Bruce and Jeff created this product in the last project they were on. I was lucky enough to work with them on it the first half of this year - but only after the ESB was created. The final product was very stable and the client loved it.

Bruce is a Geronimo committer and one of the founders of the Castor project. Jeff is also a Geronimo guy and is currently working on a JBoss Live book for SourceBeat. The problem that the ESB was trying to solve was fixing a horrendous data flow. A lot of the data flow was occurring between people's PCs, shared drives, FTP servers, HTTP Servers, web servers - and rarely were things automated. Bruce recalls one of his first days when he heard the trading guys yelling at each other to "close the spreadsheet".

The solution to the problem was building an Enterprise Service Bus (ESB) that performed the following:

  • Centralized Management of Activities
  • Powerful Scheduler
  • Guaranteed Event and Activity Execution
  • Durable Transactions
  • Pluggable ESB Components for Activities
  • Staging Database for Single Common Data Location
  • Logging and Notification of Activities
  • Proactive Response to Failed Activities
  • J2EE Architecture - Provides for True 24/7 Uptime

The pluggable components were called transformers and were standalone JARs that lived on their own, but could be managed by the ESB. Notifications were key so the traders would be notified when something went wrong.

Architecture

Scheduler (Quartz) » Workflow (jBPM) » Persisted Guaranteed Messaging (JMS). JMS talked to Activities (a.k.a. transformers).

The Quartz Java Scheduler is an open source project from OpenSymphony. It's a persisted scheduling engine, so it'll live through app server restarts. It also has millisecond granularity.

For workflow, the Java Business Process Manager (jBPM) was used. It doesn't use BPEL, and was used to track multiple activities and make decisions based on an activity's completion or failure status. Other functionality included tracking the activity state (running, cancelled, completed) and sending/managing notifications. Workflow was very important because the previous system had no way of detecting where things failed in a process. With the new system, downstream dependencies were handled, the escalation path was based on success or failure - and automatic retry occurred on failures if the failure reason was a known and configured expectation.

For messaging, JMS was used - implemented with EJB and MDB. This provided guaranteed and persisted messages. Events were automatically recovered if the server failed.

Activities/Transformers were pluggable components (wrapped with EJBs for transactionality). The nice thing about using EJBs was it was easy to create JARs for each transformer, drop them into JBoss and they'd immediately become available.

The system was all managed with a management console, that was a webapp implemented in Struts, Spring and Hibernate. Most of the Spring and Hibernate classes and configuration was generated with Middlegen and XDoclet. The management console allowed you to kick off activities, monitor their progress, as well as manage users with Active Directory and single sign-on with NTLM and jCIFS.

Solution Facts

Since December 2004, over 200,000 jobs have been run through this system. Of those, 500K activities have been run, with < 10,000 failed activities (2% failure rate). Nearly all failures were due to external issues, such as unavailability of remote systems and databases, or source files not available.

Tools and APIs Used

When Jeff showed up, it was a Microsoft shop using ASP, Visual Studio and some PowerBuilder. They brought Jeff in to help them use and adopt open source. The first thing they did was install and begin to use CVS (previously source control was done on shared drives). They also used Maven to build everything and produce a project site - which the managers and C-types loved. One thing they mentioned is they often got questions from higher-ups like "How much does it cost?" They did have problems with Maven, but it was mainly due to the poor documentation. They found a lot of Maven solutions by cracking open plugins and looking at their Jelly files.

Development Lessons Learned

Configuring EJBs and MDBs as singletons helped solve some problems (a JBoss setting allows you to configure this). As for running Spring in a heavily-managed environment, they found that setting singleton="false" solved a lot of problems. The next problem they had was mixing Hibernate and JTA Transactions. No details, just that they had an interesting time and it took them a few days to get it working. The last problem they encountered was using Hibernate and/or Spring JDBC to manage hundreds of thousands of records. Since these O/R tools create objects for each record, OOM errors occurred with large resultsets.

Business Lessons Learned

All notification messages came from the ESB, leading many to believe the problem was the ESB - rather than the data sources it was talking to. By using open source, they saved the company $500K in licenses fees. The interesting part was the company had a 3rd-generation agreement with IBM, and owned $12 million worth of licenses for WebSphere and WSAD. The reason this group used open source was because there weren't enough licenses.

Bruce and Jeff's presentation was good, but they looked like a couple of goofballs standing up there in their t-shirts and shorts (standard Virtuas gear). I guess that's what happens when you get a 2-hour notice. The worst part about the presentation was the fact that the A/C doesn't work and it's about 85° in here. I told Geary he'd better keep in short or I'm outta here. ;-)

Ruby on Rails
David Geary

David got into Rails by reading an article called "Rolling with Rails". In the article, the author claimed that you can develop webapps in Rails at least 10 times faster than in Java. David responded to the article on his blog with an entry called the Ruby on Rails Koolaid. He experienced quite a butt-whooping from various folks, including Rails' founder - and realized afterwards that the claims might be valid. After working with Rails, David believes that Rails is probably 5-10 times faster.

Ruby

Potent mix of SmallTalk, Perl and Python. Everything is an object. No static type checking. Duck Typing (talks like a duck, walks like a duck, it probably is a duck). Testing usually solves the lack of static types. Blocks - like anonymous inner classes, but retain state. Mixins - a cheap way of doing multiple inheritance. Dynamic classes - can modify at runtime (add methods, renaming methods, etc.). Rails takes great advantage of the dynamic attributes as part of its framework.

David is now showing a ContactsController that has 5 methods for CRUDing a Contact object. 4 of the 5 methods are one-liners. It kinda reminds me of my Hibernate DAOs after integrating Spring. ;-)

Rails

Ruby-based MVC framework. Convention over configuration. Scaffolding - builds pieces of your application for you. ActiveRecord does O/R Mapping. Has a built-in testing framework. Near-zero configuration (no XML). Zero-second deploy time (development environment).

David's first demo is being done by audience member Kirk. David asked for a volunteer from the audience with MySQL experience, and the ability to type in a few commands. David said his 6-year old daughter was able to do this demo last night successfully - so Kirk's gonna look pretty bad if he can't pull it off. ;-)

Using scaffolding, Rails generates 1 controller, a test for it, a helper class, a css file and 5 rhtml templates. Kirk pulled off the demo, even though he had a bit of trouble with the Mac environment. This is a lot like AppFuse's AppGen in a sense - except AppGen has to parse a bunch of XML files and reconfigure them.

David keeps hammering that the most productive feature of Rails is that there is zero deploy time. Save. Refresh. It looks very similar to developing a static HTML site.

Rails has ActiveRecord, ActionPack (MVC), ActionMailer, ActionWebService, Ajax Support, Transactions and Security. Currently at version 0.13.1 - the last version before 1.0.

David is delivering an excellent presentation, but it's too damn hot without A/C - I'm outta here.

Posted in Java at Jul 13 2005, 11:03:46 PM MDT 6 Comments