Matt RaibleMatt Raible is a Web Architecture Consultant specializing in open source frameworks.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: AngularJS, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Developing Services with Apache Camel - Part I: The Inspiration

In early May, my client asked me to work on a project migrating from IBM Message Broker 6.1 to an open source solution. Their reason was simple, the IBM solution was end of life and outdated. To prove how out of date it was, the Windows version required Windows XP to run. IBM WebSphere Message Broker has been replaced by IBM Integration Bus in recent years, but no upgrade path existed.

At first, I didn't want to do the project. I was hired as a Modern Java/UI Architect and I had enjoyed my first month upgrading libraries, making recommendations and doing a bit of UI performance work. I hadn't done much with ESBs and I enjoy front-end development a lot more than backend. It took me a couple days to realize they were willing to pay me to learn. That's when I decided to clutch up, learn how to do it all, and get the job done. This article is the first in a series on what I learned during this migration project.

My approach for figuring out how everything worked was similar to working on any new application. I get the source code, install the software necessary to run it, and run it locally so I can interact with it.

Installing IBM Message Broker

The hardest part about installing IBM Message Broker 6.1 was getting the bits to do so. I installed Windows XP in a Parallels VM on my Mac, installed Java 7 and started downloading the install files from my client's server. The files consisted of the following, which I used to install the server, WebSphere Message Broker Toolkit (Eclipse-based), and some plugins we were using.

  • IBM_Broker_Install_Disks.zip
  • IBM_plugins.zip
  • IBM_Upgrade_6.1.06.zip
  • MessageBrokerInstallFiles.7z

Transferring these files to my laptop took hours over scp. Installing and getting everything to work correctly took days. Much of this time was spent setting up various ODBC data sources, SMTP servers and figuring out how to run a "message flow test" to verify things were working. Before I started working with my client's project, I read Magnus Palmér's article about testing WMB. It was very helpful and I was able to run his project and its tests against my local server.

Choosing an Open Source Solution

When searching for how one might migrate from IBM Message Broker to an open source solution, I stumbled upon this article:

Use an Open Source Broker
Refer to my report for details on Mule ESB and Fuse ESB. If I had to pick right now, I'd use Fuse, as we already use Apache ActiveMQ and Apache CXF, so why not add another Apache-based product? Fuse also has a much higher install base. Mule seems perfectly acceptable, too, though. I would use the simplified routing scheme described in the section above and deploy either Mule or Fuse onto the same server as PFM. Mule/Fuse do not have dependencies on a queue manager like IBM Message Broker does, so there's less overhead. You can also re-use the XSLs from Message Broker.

I think that if you really understood the IBM Message Broker implementation and were comfortable with Active MQ, SOAP, etc... you could have Mule or Fuse implemented in about 2 weeks.

The last sentence inspired me to take a look at Fuse. I knew James Strachan was behind the project, so I shot him an email in early May:

Hey James,

It's been a while - hope you're doing well. I recently started working for a client that has a ton of legacy frameworks and servers. They basically haven't updated anything for 10 years, since the application was originally created by an outsourced company. They're using Acegi 1.0 for crying out loud!

One of the components they have in their system is IBM Message Broker - or maybe it's called WebSphere ESB - I'm not sure. If I succeed in getting it installed, I'm hoping to somehow install the "services" they have configured, which seem to consist of .bar, .esql and .msgflow files. From what I can tell, these seem to be ESB-related, but they might be proprietary as well. I don't have any experience with ESBs, so I'm pretty much fumbling in the dark.

I found the following information online:

https://github.com/cnaphan/osler-mb/blob/master/README.md#alternative-implementations

Under Alternative Implementations:

<quote>
Use an Open Source Broker
...
I think that if you really understood the IBM Message Broker implementation and were comfortable with Active MQ, SOAP, etc... you could have Mule or Fuse implemented in about 2 weeks.
</quote>

Emphasis of the last sentence is mine. So I'm trying to make this happen and migrate from IBM Message Broker to Fuse. I've downloaded JBoss Fuse 6.1, installed it and got it running.

Do you know of any guides or articles about migrating from IBM Message Broker to Fuse?

Thanks,

Matt

James took a couple days to reply, but when he did it was packed with the advice I was hoping for.

I'll ask around to see. TBH the easiest thing really is to just start using Apache Camel for this kinda stuff; its a simple Java library that you can use in any application server. Then make sure you use the hawtio console (which lets you view/edit/trace/debug camel routes in your browser via a nice HTML5 / AngularJS application). http://hawt.io/

JBoss Fuse 6.1 is cool and all though; its based on OSGi which some folks love and some hate. The class loader / application server doesn't really matter to Camel though; use whatever you want (tomcat / jetty / wildfly / karaf / stand alone java applications etc).

Fuse 6.1 is based on a bunch of apache projects plus an upstream open source project called 'fabric8' which deals with provisioning, management, discovery and so forth (i.e. scaling 1 camel route in 1 JVM to many JVMs etc). http://fabric8.io/

The 1.1.x version of fabric8 supports a "Java Container"; i.e. a static flat classpath. For folks who've not done OSGi before (or folks who hate OSGi) I kinda recommend folks not start using OSGi yet - but just start with camel; otherwise it can seem like too much to learn and the OSGi class loading / metadata stuff can kinda get in the way and slow you down. I did a little demo & blog about using 1.1.0.Beta5 of fabric8 (1.1.x of fabric8 will make it into a future JBoss Fuse product release). http://macstrac.blogspot.co.uk/2014/05/micro-services-with-fabric8.html

TL;DR; - play with Apache Camel, it'll solve easily all your integration needs; its got a much bigger & more active community than Mule and unlike Mule - its all open source; there's no locked down, non-open source connectors/magic/tools or proprietary software included. Once you've got your build & tests working fine; ponder which kinda container you wanna use in production (an app server or micro services) - it doesn't matter too much which one. We're working hard on making all our production/testing management/monitoring/diagnostics/debugging tooling work on all containers/app servers anyway (stand alone Java processes via the Java Container, Docker, OSGI (via Karaf like Fuse 6.1), Tomcat, TomEE and Wildfly).

Though if you need 24x7 production support like now, definitely use the JBoss Fuse 6.1 distro ;) If you can wait until later in the year for production support; I'd stick with the fabric8 1.1.x stuff for Java Containers; its simpler & more agile for you go get things sorted.

Here's a little demo of the UI tooling btw https://vimeo.com/album/2635012/video/84674508

It's using the Fuse distro in the demo; but really the hawtio console is doing all the heavy lifting & is available in any JVM thats got a jolokia agent running (and you can use the hawtio Chrome Plugin too). It hopefully will give you a feel for the UI tooling working with camel routes; debugging/tracing them, grokking their metrics and stuff.

After reading James' email, I forwarded it to my client with my recommendation that we start with Apache Camel. He agreed it was a good approach and I went to work.

Development Strategy

On May 21st, I subscribed to the Apache Camel users mailing list and posted my first question the next day. My development strategy was the following:

  1. Write an integration test against the existing service.
  2. Implement the service with Camel, writing unit tests as I progressed.
  3. Copy the logic from step 1's integration test and use it for the new service's integration tests.

I created the project with a Camel Archetype and used Java 7 as the minimum JDK. I decided I wanted no XML in the project (aside from pom.xml) and that I'd use Camel's Java DSL to develop my routes.

Summary

When I first started migrating from IBM Message Broker to an open source solution, I was a bit overwhelmed with the seemingly daunting task. I quickly discovered that Apache Camel was a good replacement and started developing my first route. It took me a couple days to get things working, but I learned a lot in the process - especially around testing. In the next article, I'll talk about how I wrote tests, mocked 3rd party dependencies and configured everything to run in Jenkins. Stay tuned!

Related: I first learned about Apache Camel from Bruce Snyder in 2008 at Colorado Software Summit.

Posted in Java at Sep 23 2014, 10:58:25 AM MDT 7 Comments
Comments:

I know it doesn't matter much, but WebSphere Message Broker and WebSphere ESB are (were?) two different products. The ESB product is more like what you'd expect an ESB to be. WMB is really meant to be used with the built in ESQL. Anything else and you are "fighting" the platform (and anyone who consults with you on it - we confused our consultant years ago because we basically didn't use ESQL). The good part about WMB is that it used MQ Series (not called that anymore) as its basis. It is rock solid and gives you some things you cannot do with JMS. But there is the installing it part. :)

FYI, we have been doing the same thing. Moving from WMB to Camel and Camel related products.

Posted by Mark on September 24, 2014 at 10:36 AM MDT #

[Trackback] This article is the second in a series on Apache Camel and how I used it to replace IBM Message Broker for a client. The first article, Developing Services with Apache Camel - Part I: The Inspiration , describes why I chose Camel for this p...

Posted by Raible Designs on September 30, 2014 at 10:08 AM MDT #

Hi Matt!

Welcome to the dark side of backend wiring and plumbing!

As a former consultant on the IBM WMB and ESB products I cannot agree more with your post and congratulate you to the wise choice of switching to Apache Camel. It makes me happy to see more and more developers starting to use this wonderful framework.

Personally Camel was a real eye opener for me and brought back the joy of integration development from the pain that used to be proprietary solutions. Actually to the point of switching my career as an IBM partner consultant to your regular open source tech geek.

Just a quick note.. IBM Integration Bus is actually the same product as WMB 6.1 plus a few major version bumps. So there should be a clear migration part, even though I highly recommend the one you're currently taking :-)

Best of luck, and I look forward to read your followup posts!

Posted by Billy Sjöberg on October 03, 2014 at 08:05 AM MDT #

[Trackback] This article is the third in a series on Apache Camel and how I used it to replace IBM Message Broker for a client. I used Apache Camel for several months this summer to create a number of SOAP services. These services performed various third...

Posted by Raible Designs on October 08, 2014 at 08:38 AM MDT #

[Trackback] Welcome to the final article in a series on my experience developing services with Apache Camel . I learned how to implement CXF endpoints using its Java DSL, made sure everything worked with its testing framework and integrated Spring Boot ...

Posted by Raible Designs on October 16, 2014 at 08:48 AM MDT #

Matt,

Did your team also look into Spring Integration? Were there features/gaps that caused your team to go with Camel?

Were you using Camel as an embedded EAI tool or as an ESB?

-Joshua

Posted by Joshua White on November 15, 2014 at 08:31 AM MST #

Joshua - Nope. We never looked into it. This article explains why we chose Camel. I was merely looking for a solution to replace IBM Message Broker and it worked quite nicely. We used it as an embedded EAI tool with Spring Boot.

Posted by Matt Raible on November 15, 2014 at 08:45 AM MST #

Post a Comment:
  • HTML Syntax: Allowed