Raible Designs

Developing Services with Apache Camel - Part III: Integrating Spring 4 and Spring Boot

This article is the third in a series on Apache Camel and how I used it to replace IBM Message Broker for a client. I used Apache Camel for several months this summer to create a number of SOAP services. These services performed various third-party data lookups for our customers. For previous articles, see Part I: The Inspiration and Part II: Creating and Testing Routes.

In late June, I sent an email to my client's engineering team. Its subject: "External Configuration and Microservices". I recommended we integrate Spring Boot into the Apache Camel project I was working on. I told them my main motivation was its external configuration feature. I also pointed out its container-less WAR feature, where Tomcat (or Jetty) is embedded in the WAR and you can start your app with "java -jar appname.war". I mentioned microservices and that Spring Boot would make it easy to split the project into a project-per-service structure if we wanted to go that route. I then asked two simple questions:

1. Is it OK to integrate Spring Boot?
2. Should I split the project into microservices?

Both of these suggestions were well received, so I went to work.

Spring 4

Before I integrated Spring Boot, I knew I had to upgrade to Spring 4. The version of Camel I was using (2.13.1) did not support Spring 4. I found issue CAMEL-7074 (Support spring 4.x) and added a comment to see when it would be fixed. After fiddling with dependencies and trying Camel 2.14-SNAPSHOT, I was able to upgrade to CXF 3.0. However, this didn't solve my problem. There were some API uncompatible changes between Spring 3.3.x and Spring 4.0.x and the camel-test-spring module wouldn't work with both. I proposed the following:

I think the easiest way forward is to create two modules: camel-test-spring and camel-test-spring3. The former compiles against Spring 4 and the latter against Spring 3. You could switch it so camel-test-spring defaults to Spring 3, but camel-test-spring4 doesn't seem to be forward-looking, as you hopefully won't need a camel-test-spring5.

I've made this change in a fork and it works in my project. I can upgrade to Camel 2.14-SNAPSHOT and CXF 3.0 with Spring 3.2.8 (by using camel-test-spring3). I can also upgrade to Spring 4 if I use the upgraded camel-test-spring.

Here's a pull request that has this change: https://github.com/apache/camel/pull/199

The Camel team integrated my suggested change a couple weeks later. Unfortunately, a similar situation happened with Spring 4.1, so you'll have to wait for Camel 2.15 if you want to use Spring 4.1.

After making a patched 2.14-SNAPSHOT version available to my project, I was able to upgrade to Spring 4 and CXF 3 with a few minor changes to my pom.xml.

    <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
-     <camel.version>2.13.1</camel.version>
-     <cxf.version>2.7.11</cxf.version>
-     <spring.version>3.2.8.RELEASE</spring.version>
+     <camel.version>2.14-SNAPSHOT</camel.version>
+     <cxf.version>3.0.0</cxf.version>
+     <spring.version>4.0.5.RELEASE</spring.version>
    </properties>
...
+      <!-- upgrade camel-spring dependencies -->
+      <dependency>
+         <groupId>org.springframework</groupId>
+         <artifactId>spring-context</artifactId>
+         <version>${spring.version}</version>
+      </dependency>
+      <dependency>
+         <groupId>org.springframework</groupId>
+         <artifactId>spring-aop</artifactId>
+         <version>${spring.version}</version>
+      </dependency>
+      <dependency>
+         <groupId>org.springframework</groupId>
+         <artifactId>spring-tx</artifactId>
+         <version>${spring.version}</version>
+      </dependency>

I also had to change some imports for CXF 3.0 since it includes a new major version of Apache WSS4J (2.0.0).

-import org.apache.ws.security.handler.WSHandlerConstants;
+import org.apache.wss4j.dom.handler.WSHandlerConstants;
...
-import org.apache.ws.security.WSPasswordCallback;
+import org.apache.wss4j.common.ext.WSPasswordCallback;

After getting everything upgraded, I continued developing services for the next couple weeks.

Spring Boot

In late July, I integrated Spring Boot. It was fairly straightforward and mostly consisted of adding/removing dependencies and removing versions already defined in Boot's starter-parent.

+   <parent>
+      <groupId>org.springframework.boot</groupId>
+      <artifactId>spring-boot-starter-parent</artifactId>
+      <version>1.1.4.RELEASE</version>
+   </parent>
...
       <cxf.version>3.0.1</cxf.version>
+      <java.version>1.7</java.version>
+      <servlet-api.version>3.1.0</servlet-api.version>
       <spring.version>4.0.6.RELEASE</spring.version>
...
-            <artifactId>maven-compiler-plugin</artifactId>
-            <version>2.5.1</version>
-            <configuration>
-               <source>1.7</source>
-               <target>1.7</target>
-            </configuration>
-         </plugin>
-         <plugin>
-            <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-resources-plugin</artifactId>
          </plugin>
+         <plugin>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-maven-plugin</artifactId>
+         </plugin>
       </plugins>
    </build>

    <dependencies>
+      <!-- spring boot -->
+      <dependency>
+         <groupId>org.springframework.boot</groupId>
+         <artifactId>spring-boot-starter-actuator</artifactId>
+         <exclusions>
+            <exclusion>
+               <groupId>org.springframework.boot</groupId>
+               <artifactId>spring-boot-starter-logging</artifactId>
+            </exclusion>
+         </exclusions>
+      </dependency>
+      <dependency>
+         <groupId>org.springframework.boot</groupId>
+         <artifactId>spring-boot-starter-log4j</artifactId>
+      </dependency>
+      <dependency>
+         <groupId>org.springframework.boot</groupId>
+         <artifactId>spring-boot-starter-tomcat</artifactId>
+         <scope>provided</scope>
+      </dependency>
+      <dependency>
+         <groupId>org.springframework.boot</groupId>
+         <artifactId>spring-boot-starter-web</artifactId>
+      </dependency>
       <!-- camel -->
...
-      <!-- upgrade camel-spring dependencies -->
-      <dependency>
-         <groupId>org.springframework</groupId>
-         <artifactId>spring-context</artifactId>
-         <version>${spring.version}</version>
-      </dependency>
-      <dependency>
-         <groupId>org.springframework</groupId>
-         <artifactId>spring-aop</artifactId>
-         <version>${spring.version}</version>
-      </dependency>
-      <dependency>
-         <groupId>org.springframework</groupId>
-         <artifactId>spring-tx</artifactId>
-         <version>${spring.version}</version>
-      </dependency>
...
-      <!-- logging -->
-      <dependency>
-         <groupId>org.slf4j</groupId>
-         <artifactId>slf4j-api</artifactId>
-         <version>1.7.6</version>
-      </dependency>
-      <dependency>
-         <groupId>org.slf4j</groupId>
-         <artifactId>slf4j-log4j12</artifactId>
-         <version>1.7.6</version>
-      </dependency>
-      <dependency>
-         <groupId>log4j</groupId>
-         <artifactId>log4j</artifactId>
-         <version>1.2.17</version>
-      </dependency>
-
       <!-- utilities -->
       <dependency>
          <groupId>joda-time</groupId>
          <artifactId>joda-time</artifactId>
-         <version>2.3</version>
       </dependency>
       <dependency>
          <groupId>commons-dbcp</groupId>
          <artifactId>commons-dbcp</artifactId>
-         <version>1.4</version>
...
       <!-- testing -->
       <dependency>
+         <groupId>org.springframework.boot</groupId>
+         <artifactId>spring-boot-starter-test</artifactId>
+         <exclusions>
+            <exclusion>
+               <groupId>org.springframework.boot</groupId>
+               <artifactId>spring-boot-starter-logging</artifactId>
+            </exclusion>
+         </exclusions>
+      </dependency>
+      <dependency>
-      <dependency>
-         <groupId>org.springframework</groupId>
-         <artifactId>spring-test</artifactId>
-         <version>${spring.version}</version>
-         <scope>test</scope>
-      </dependency>
-      <dependency>
-         <groupId>org.mockito</groupId>
-         <artifactId>mockito-core</artifactId>
-         <version>1.9.5</version>
-         <scope>test</scope>
-      </dependency>

Next, I deleted the AppInitializer.java class I mentioned in Part II and added an Application.java class.

import org.apache.cxf.transport.servlet.CXFServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.context.embedded.ConfigurableEmbeddedServletContainer;
import org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer;
import org.springframework.boot.context.embedded.ErrorPage;
import org.springframework.boot.context.embedded.ServletRegistrationBean;
import org.springframework.boot.context.web.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpStatus;

@Configuration
@EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class,
        DataSourceTransactionManagerAutoConfiguration.class})
@ComponentScan
public class Application extends SpringBootServletInitializer {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(Application.class);
    }

    @Bean
    public ServletRegistrationBean servletRegistrationBean() {
        CXFServlet servlet = new CXFServlet();
        return new ServletRegistrationBean(servlet, "/api/*");
    }

    @Bean
    public EmbeddedServletContainerCustomizer containerCustomizer() {
        return new EmbeddedServletContainerCustomizer() {
            @Override
            public void customize(ConfigurableEmbeddedServletContainer container) {
                ErrorPage error401Page = new ErrorPage(HttpStatus.UNAUTHORIZED, "/401.html");
                ErrorPage error404Page = new ErrorPage(HttpStatus.NOT_FOUND, "/404.html");
                ErrorPage error500Page = new ErrorPage(HttpStatus.INTERNAL_SERVER_ERROR, "/500.html");
                container.addErrorPages(error401Page, error404Page, error500Page);
            }
        };
    }
}

The error pages you see configured above were configured and copied from Tim Sporcic's Custom Error Pages with Spring Boot.

Dynamic DataSources

I excluded the DataSource-related AutoConfiguration classes because this application had many datasources. It also had a requirement to allow datasources to be added on-the-fly by simply editing application.properties. I asked how to do this on Stack Overflow and received an excellent answer from Stéphane Nicoll.

Spring Boot Issues

I did encounter a couple issues after integrating Spring Boot. The first was that it was removing the content-* headers for CXF responses. This only happened when running the WAR in Tomcat and I was able to figure out a workaround with a custom ResponseWrapper and Filter. This issue was fixed in Spring Boot 1.1.6.

The other issue was that the property override feature didn't seem to work when setting environment variables. The workaround was to create a setenv.sh script in $CATALINA_HOME/bin and add the environment variables there. See section 3.4 of Tomcat 7's RUNNING.txt for more information.

SOAP Faults

After upgrading to Spring 4 and integrating Spring Boot, I continued migrating IBM Message Broker flows. My goal was to make all new services backward compatible, but I ran into an issue. With the new services, SOAP Faults were sent back to the client instead of error messages in a SOAP Message. I suggested we fix it with one of two ways:

1. Modify the client so it looks for SOAP Faults and handles them appropriately.
2. Modify the new services so messages are returned instead of faults.

For #2, I learned how do to convert from a fault to messages on the Camel user mailing list. However, the team opted to improve the client and we added fault handling there instead.

Microservice Deployment

When I first integrated Spring Boot, I was planning on splitting our project into a project-per-service. This would allow each service to evolve on its own, instead of having a monolithic war that contains all the services. In team discussions, there was some concern about the memory overhead of running multiple instances instead of one.

I pointed out an interesting thread on the Camel mailing list about deploying routes with a route-per-jvm or all in the same JVM. The recommendation from that thread was to bundle similar routes together if you were to split them.

In the end, we decided to allow our Operations team decide how they wanted to manage/deploy everything. I mentioned that Spring Boot can work with Tomcat, Jetty, JBoss and even cloud providers like Heroku and Cloud Foundry. I estimated that splitting the project apart would take less than a day, as would making it back into a monolithic WAR.

Summary

This article explains how we upgraded our Apache Camel application to Spring 4 and integrated Spring Boot. There was a bit of pain getting things to work, but nothing a few pull requests and workarounds couldn't fix. We discovered some issues with setting environment variables for Tomcat and opted not to split our project into small microservices. Hopefully this article will help people trying to Camelize a Spring Boot application .

In the next article, I'll talk about load testing with Gatling, logging with Log4j2 and monitoring with hawtio and New Relic.