Raible Designs

Developing Services with Apache Camel - Part I: The Inspiration

In early May, my client asked me to work on a project migrating from IBM Message Broker 6.1 to an open source solution. Their reason was simple, the IBM solution was end of life and outdated. To prove how out of date it was, the Windows version required Windows XP to run. IBM WebSphere Message Broker has been replaced by IBM Integration Bus in recent years, but no upgrade path existed.

At first, I didn't want to do the project. I was hired as a Modern Java/UI Architect and I had enjoyed my first month upgrading libraries, making recommendations and doing a bit of UI performance work. I hadn't done much with ESBs and I enjoy front-end development a lot more than backend. It took me a couple days to realize they were willing to pay me to learn. That's when I decided to clutch up, learn how to do it all, and get the job done. This article is the first in a series on what I learned during this migration project.

My approach for figuring out how everything worked was similar to working on any new application. I get the source code, install the software necessary to run it, and run it locally so I can interact with it.

Installing IBM Message Broker

The hardest part about installing IBM Message Broker 6.1 was getting the bits to do so. I installed Windows XP in a Parallels VM on my Mac, installed Java 7 and started downloading the install files from my client's server. The files consisted of the following, which I used to install the server, WebSphere Message Broker Toolkit (Eclipse-based), and some plugins we were using.

• IBM_Broker_Install_Disks.zip
• IBM_plugins.zip
• IBM_Upgrade_6.1.06.zip
• MessageBrokerInstallFiles.7z

Transferring these files to my laptop took hours over scp. Installing and getting everything to work correctly took days. Much of this time was spent setting up various ODBC data sources, SMTP servers and figuring out how to run a "message flow test" to verify things were working. Before I started working with my client's project, I read Magnus Palmér's article about testing WMB. It was very helpful and I was able to run his project and its tests against my local server.

Choosing an Open Source Solution

When searching for how one might migrate from IBM Message Broker to an open source solution, I stumbled upon this article:

Use an Open Source Broker
Refer to my report for details on Mule ESB and Fuse ESB. If I had to pick right now, I'd use Fuse, as we already use Apache ActiveMQ and Apache CXF, so why not add another Apache-based product? Fuse also has a much higher install base. Mule seems perfectly acceptable, too, though. I would use the simplified routing scheme described in the section above and deploy either Mule or Fuse onto the same server as PFM. Mule/Fuse do not have dependencies on a queue manager like IBM Message Broker does, so there's less overhead. You can also re-use the XSLs from Message Broker.

I think that if you really understood the IBM Message Broker implementation and were comfortable with Active MQ, SOAP, etc... you could have Mule or Fuse implemented in about 2 weeks.

The last sentence inspired me to take a look at Fuse. I knew James Strachan was behind the project, so I shot him an email in early May:

Hey James,

It's been a while - hope you're doing well. I recently started working for a client that has a ton of legacy frameworks and servers. They basically haven't updated anything for 10 years, since the application was originally created by an outsourced company. They're using Acegi 1.0 for crying out loud!

One of the components they have in their system is IBM Message Broker - or maybe it's called WebSphere ESB - I'm not sure. If I succeed in getting it installed, I'm hoping to somehow install the "services" they have configured, which seem to consist of .bar, .esql and .msgflow files. From what I can tell, these seem to be ESB-related, but they might be proprietary as well. I don't have any experience with ESBs, so I'm pretty much fumbling in the dark.

I found the following information online:

https://github.com/cnaphan/osler-mb/blob/master/README.md#alternative-implementations

Under Alternative Implementations:

<quote>
Use an Open Source Broker
...
I think that if you really understood the IBM Message Broker implementation and were comfortable with Active MQ, SOAP, etc... you could have Mule or Fuse implemented in about 2 weeks.
</quote>

Emphasis of the last sentence is mine. So I'm trying to make this happen and migrate from IBM Message Broker to Fuse. I've downloaded JBoss Fuse 6.1, installed it and got it running.

Do you know of any guides or articles about migrating from IBM Message Broker to Fuse?

Thanks,

Matt

James took a couple days to reply, but when he did it was packed with the advice I was hoping for.

I'll ask around to see. TBH the easiest thing really is to just start using Apache Camel for this kinda stuff; its a simple Java library that you can use in any application server. Then make sure you use the hawtio console (which lets you view/edit/trace/debug camel routes in your browser via a nice HTML5 / AngularJS application). http://hawt.io/

JBoss Fuse 6.1 is cool and all though; its based on OSGi which some folks love and some hate. The class loader / application server doesn't really matter to Camel though; use whatever you want (tomcat / jetty / wildfly / karaf / stand alone java applications etc).

Fuse 6.1 is based on a bunch of apache projects plus an upstream open source project called 'fabric8' which deals with provisioning, management, discovery and so forth (i.e. scaling 1 camel route in 1 JVM to many JVMs etc). http://fabric8.io/

The 1.1.x version of fabric8 supports a "Java Container"; i.e. a static flat classpath. For folks who've not done OSGi before (or folks who hate OSGi) I kinda recommend folks not start using OSGi yet - but just start with camel; otherwise it can seem like too much to learn and the OSGi class loading / metadata stuff can kinda get in the way and slow you down. I did a little demo & blog about using 1.1.0.Beta5 of fabric8 (1.1.x of fabric8 will make it into a future JBoss Fuse product release). http://macstrac.blogspot.co.uk/2014/05/micro-services-with-fabric8.html

TL;DR; - play with Apache Camel, it'll solve easily all your integration needs; its got a much bigger & more active community than Mule and unlike Mule - its all open source; there's no locked down, non-open source connectors/magic/tools or proprietary software included. Once you've got your build & tests working fine; ponder which kinda container you wanna use in production (an app server or micro services) - it doesn't matter too much which one. We're working hard on making all our production/testing management/monitoring/diagnostics/debugging tooling work on all containers/app servers anyway (stand alone Java processes via the Java Container, Docker, OSGI (via Karaf like Fuse 6.1), Tomcat, TomEE and Wildfly).

Though if you need 24x7 production support like now, definitely use the JBoss Fuse 6.1 distro If you can wait until later in the year for production support; I'd stick with the fabric8 1.1.x stuff for Java Containers; its simpler & more agile for you go get things sorted.

Here's a little demo of the UI tooling btw https://vimeo.com/album/2635012/video/84674508

It's using the Fuse distro in the demo; but really the hawtio console is doing all the heavy lifting & is available in any JVM thats got a jolokia agent running (and you can use the hawtio Chrome Plugin too). It hopefully will give you a feel for the UI tooling working with camel routes; debugging/tracing them, grokking their metrics and stuff.

After reading James' email, I forwarded it to my client with my recommendation that we start with Apache Camel. He agreed it was a good approach and I went to work.

Development Strategy

On May 21st, I subscribed to the Apache Camel users mailing list and posted my first question the next day. My development strategy was the following:

1. Write an integration test against the existing service.
2. Implement the service with Camel, writing unit tests as I progressed.
3. Copy the logic from step 1's integration test and use it for the new service's integration tests.

I created the project with a Camel Archetype and used Java 7 as the minimum JDK. I decided I wanted no XML in the project (aside from pom.xml) and that I'd use Camel's Java DSL to develop my routes.

Summary

When I first started migrating from IBM Message Broker to an open source solution, I was a bit overwhelmed with the seemingly daunting task. I quickly discovered that Apache Camel was a good replacement and started developing my first route. It took me a couple days to get things working, but I learned a lot in the process - especially around testing. In the next article, I'll talk about how I wrote tests, mocked 3rd party dependencies and configured everything to run in Jenkins. Stay tuned!

Related: I first learned about Apache Camel from Bruce Snyder in 2008 at Colorado Software Summit.