Raible Designs

Building a REST API with JAXB, Spring Boot and Spring Data

If someone asked you to develop a REST API on the JVM, which frameworks would you use? I was recently tasked with such a project. My client asked me to implement a REST API to ingest requests from a 3rd party. The project entailed consuming XML requests, storing the data in a database, then exposing the data to internal application with a JSON endpoint. Finally, it would allow taking in a JSON request and turning it into an XML request back to the 3rd party.

With the recent release of Apache Camel 2.14 and my success using it, I started by copying my Apache Camel / CXF / Spring Boot project and trimming it down to the bare essentials. I whipped together a simple Hello World service using Camel and Spring MVC. I also integrated Swagger into both. Both implementations were pretty easy to create (sample code), but I decided to use Spring MVC. My reasons were simple: its REST support was more mature, I knew it well, and Spring MVC Test makes it easy to test APIs.

Camel's Swagger support without web.xml
As part of the aforementioned spike, I learned out how to configure Camel's REST and Swagger support using Spring's JavaConfig and no web.xml. I made this into a sample project and put it on GitHub as camel-rest-swagger.

This article shows how I built a REST API with Java 8, Spring Boot/MVC, JAXB and Spring Data (JPA and REST components). I stumbled a few times while developing this project, but figured out how to get over all the hurdles. I hope this helps the team that's now maintaining this project (my last day was Friday) and those that are trying to do something similar.

XML to Java with JAXB

The data we needed to ingest from a 3rd party was based on the NCPDP Standards. As a member, we were able to download a number of XSD files, put them in our project and generate Java classes to handle the incoming/outgoing requests. I used the maven-jaxb2-plugin to generate the Java classes.

<plugin>
    <groupId>org.jvnet.jaxb2.maven2</groupId>
    <artifactId>maven-jaxb2-plugin</artifactId>
    <version>0.8.3</version>
    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <args>
                    <arg>-XtoString</arg>
                    <arg>-Xequals</arg>
                    <arg>-XhashCode</arg>
                    <arg>-Xcopyable</arg>
                </args>
                <plugins>
                    <plugin>
                        <groupId>org.jvnet.jaxb2_commons</groupId>
                        <artifactId>jaxb2-basics</artifactId>
                        <version>0.6.4</version>
                    </plugin>
                </plugins>
                <schemaDirectory>src/main/resources/schemas/ncpdp</schemaDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

The first error I ran into was about a property already being defined.

[INFO] --- maven-jaxb2-plugin:0.8.3:generate (default) @ spring-app ---
[ERROR] Error while parsing schema(s).Location [ file:/Users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd{1811,48}].
com.sun.istack.SAXParseException2; systemId: file:/Users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd;
    lineNumber: 1811; columnNumber: 48; Property "MultipleTimingModifierAndTimingAndDuration" is already defined.
    Use <jaxb:property> to resolve this conflict.
at com.sun.tools.xjc.ErrorReceiver.error(ErrorReceiver.java:86)

I was able to workaround this by upgrading to maven-jaxb2-plugin version 0.9.1. I created a controller and stubbed out a response with hard-coded data. I confirmed the incoming XML-to-Java marshalling worked by testing with a sample request provided by our 3rd party customer. I started with a curl command, because it was easy to use and could be run by anyone with the file and curl installed.

curl -X POST -H 'Accept: application/xml' -H 'Content-type: application/xml' \
--data-binary @sample-request.xml http://localhost:8080/api/message -v

This is when I ran into another stumbling block: the response wasn't getting marshalled back to XML correctly. After some research, I found out this was caused by the lack of @XmlRootElement annotations on my generated classes. I posted a question to Stack Overflow titled Returning JAXB-generated elements from Spring Boot Controller. After banging my head against the wall for a couple days, I figured out the solution.

I created a bindings.xjb file in the same directory as my schemas. This causes JAXB to generate @XmlRootElement on classes.

<?xml version="1.0"?>
<jxb:bindings version="1.0"
              xmlns:xsd="http://www.w3.org/2001/XMLSchema"
              xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
              xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://java.sun.com/xml/ns/jaxb http://java.sun.com/xml/ns/jaxb/bindingschema_2_0.xsd">

    <jxb:bindings schemaLocation="transport.xsd" node="/xsd:schema">
        <jxb:globalBindings>
            <xjc:simple/>
        </jxb:globalBindings>
    </jxb:bindings>
</jxb:bindings>

To add namespaces prefixes to the returned XML, I had to modify the maven-jaxb2-plugin to add a couple arguments.

<arg>-extension</arg>
<arg>-Xnamespace-prefix</arg>

And add a dependency:

<dependencies>
    <dependency>
        <groupId>org.jvnet.jaxb2_commons</groupId>
        <artifactId>jaxb2-namespace-prefix</artifactId>
        <version>1.1</version>
    </dependency>
</dependencies>

Then I modified bindings.xjb to include the package and prefix settings. I also moved <xjc:simple/> into a global setting. I eventually had to add prefixes for all schemas and their packages.

<?xml version="1.0"?>
<bindings version="2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://java.sun.com/xml/ns/jaxb"
          xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:namespace="http://jaxb2-commons.dev.java.net/namespace-prefix"
          xsi:schemaLocation="http://java.sun.com/xml/ns/jaxb http://java.sun.com/xml/ns/jaxb/bindingschema_2_0.xsd
              http://jaxb2-commons.dev.java.net/namespace-prefix http://java.net/projects/jaxb2-commons/sources/svn/content/namespace-prefix/trunk/src/main/resources/prefix-namespace-schema.xsd">

    <globalBindings>
        <xjc:simple/>
    </globalBindings>

    <bindings schemaLocation="transport.xsd" node="/xsd:schema">
        <schemaBindings>
            <package name="org.ncpdp.schema.transport"/>
        </schemaBindings>
        <bindings>
            <namespace:prefix name="transport"/>
        </bindings>
    </bindings>
</bindings>

I learned how to add prefixes from the namespace-prefix plugins page.

Finally, I customized the code-generation process to generate Joda Time's DateTime instead of the default XMLGregorianCalendar. This involved a couple custom XmlAdapters and a couple additional lines in bindings.xjb. You can see the adapters and bindings.xjb with all necessary prefixes in this gist. Nicolas Fränkel's Customize your JAXB bindings was a great resource for making all this work.

I wrote a test to prove that the ingest API worked as desired.

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
public class InitiateRequestControllerTest {

    @Inject
    private InitiateRequestController controller;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        MockitoAnnotations.initMocks(this);
        this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build();
    }

    @Test
    public void testGetNotAllowedOnMessagesAPI() throws Exception {
        mockMvc.perform(get("/api/initiate")
                .accept(MediaType.APPLICATION_XML))
                .andExpect(status().isMethodNotAllowed());
    }

    @Test
    public void testPostPaInitiationRequest() throws Exception {
        String request = new Scanner(new ClassPathResource("sample-request.xml").getFile()).useDelimiter("\\Z").next();

        mockMvc.perform(post("/api/initiate")
                .accept(MediaType.APPLICATION_XML)
                .contentType(MediaType.APPLICATION_XML)
                .content(request))
                .andExpect(status().isOk())
                .andExpect(content().contentType(MediaType.APPLICATION_XML))
                .andExpect(xpath("/Message/Header/To").string("3rdParty"))
                .andExpect(xpath("/Message/Header/SenderSoftware/SenderSoftwareDeveloper").string("HID"))
                .andExpect(xpath("/Message/Body/Status/Code").string("010"));
    }
}

Spring Data for JPA and REST

With JAXB out of the way, I turned to creating an internal API that could be used by another application. Spring Data was fresh in my mind after reading about it last summer. I created classes for entities I wanted to persist, using Lombok's @Data to reduce boilerplate.

I read the Accessing Data with JPA guide, created a couple repositories and wrote some tests to prove they worked. I ran into an issue trying to persist Joda's DateTime and found Jadira provided a solution.

I added its usertype.core as a dependency to my pom.xml:

<dependency>
    <groupId>org.jadira.usertype</groupId>
    <artifactId>usertype.core</artifactId>
    <version>3.2.0.GA</version>
</dependency>

... and annotated DateTime variables accordingly.

@Column(name = "last_modified", nullable = false)
@Type(type="org.jadira.usertype.dateandtime.joda.PersistentDateTime")
private DateTime lastModified;

With JPA working, I turned to exposing REST endpoints. I used Accessing JPA Data with REST as a guide and was looking at JSON in my browser in a matter of minutes. I was surprised to see a "profile" service listed next to mine, and posted a question to the Spring Boot team. Oliver Gierke provided an excellent answer.

Swagger

Spring MVC's integration for Swagger has greatly improved since I last wrote about it. Now you can enable it with a @EnableSwagger annotation. Below is the SwaggerConfig class I used to configure Swagger and read properties from application.yml.

@Configuration
@EnableSwagger
public class SwaggerConfig implements EnvironmentAware {
	public static final String DEFAULT_INCLUDE_PATTERN = "/api/.*";

	private RelaxedPropertyResolver propertyResolver;

	@Override
	public void setEnvironment(Environment environment) {
		this.propertyResolver = new RelaxedPropertyResolver(environment, "swagger.");
	}

	/**
	 * Swagger Spring MVC configuration
	 */
	@Bean
	public SwaggerSpringMvcPlugin swaggerSpringMvcPlugin(SpringSwaggerConfig springSwaggerConfig) {
		return new SwaggerSpringMvcPlugin(springSwaggerConfig)
				.apiInfo(apiInfo())
				.genericModelSubstitutes(ResponseEntity.class)
				.includePatterns(DEFAULT_INCLUDE_PATTERN);
	}

	/**
	 * API Info as it appears on the swagger-ui page
	 */
	private ApiInfo apiInfo() {
		return new ApiInfo(
				propertyResolver.getProperty("title"),
				propertyResolver.getProperty("description"),
				propertyResolver.getProperty("termsOfServiceUrl"),
				propertyResolver.getProperty("contact"),
				propertyResolver.getProperty("license"),
				propertyResolver.getProperty("licenseUrl"));
	}
}

After getting Swagger to work, I discovered that endpoints published with @RepositoryRestResource aren't picked up by Swagger. There is an open issue for Spring Data support in the swagger-springmvc project.

Liquibase Integration

I configured this project to use H2 in development and PostgreSQL in production. I used Spring profiles to do this and copied XML/YAML (for Maven and application*.yml files) from a previously created JHipster project.

Next, I needed to create a database. I decided to use Liquibase to create tables, rather than Hibernate's schema-export. I chose Liquibase over Flyway based of discussions in the JHipster project. To use Liquibase with Spring Boot is dead simple: add the following dependency to pom.xml, then place changelog files in src/main/resources/db/changelog.

<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
</dependency>

I started by using Hibernate's schema-export and changing hibernate.ddl-auto to "create-drop" in application-dev.yml. I also commented out the liquibase-core dependency. Then I setup a PostgreSQL database and started the app with "mvn spring-boot:run -Pprod".

I generated the liquibase changelog from an existing schema using the following command (after downloading and installing Liquibase).

liquibase --driver=org.postgresql.Driver --classpath="/Users/mraible/.m2/repository/org/postgresql/postgresql/9.3-1102-jdbc41/postgresql-9.3-1102-jdbc41.jar:/Users/mraible/snakeyaml-1.11.jar" --changeLogFile=/Users/mraible/dev/spring-app/src/main/resources/db/changelog/db.changelog-02.yaml --url="jdbc:postgresql://localhost:5432/mydb" --username=user --password=pass generateChangeLog

I did find one bug - the generateChangeLog command generates too many constraints in version 3.2.2. I was able to fix this by manually editing the generated YAML file.

psql -d mydb
drop schema public cascade;
create schema public;

After writing minimal code for Spring Data and configuring Liquibase to create tables/relationships, I relaxed a bit, documented how everything worked and added a LoggingFilter. The LoggingFilter was handy for viewing API requests and responses.

@Bean
public FilterRegistrationBean loggingFilter() {
    LoggingFilter filter = new LoggingFilter();
    FilterRegistrationBean registrationBean = new FilterRegistrationBean();
    registrationBean.setFilter(filter);
    registrationBean.setUrlPatterns(Arrays.asList("/api/*"));
    return registrationBean;
}

Accessing API with RestTemplate

The final step I needed to do was figure out how to access my new and fancy API with RestTemplate. At first, I thought it would be easy. Then I realized that Spring Data produces a HAL-compliant API, so its content is embedded inside an "_embedded" JSON key.

After much trial and error, I discovered I needed to create a RestTemplate with HAL and Joda-Time awareness.

@Bean
public RestTemplate restTemplate() {
	ObjectMapper mapper = new ObjectMapper();
	mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
	mapper.registerModule(new Jackson2HalModule());
	mapper.registerModule(new JodaModule());

	MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
	converter.setSupportedMediaTypes(MediaType.parseMediaTypes("application/hal+json"));
	converter.setObjectMapper(mapper);
	StringHttpMessageConverter stringConverter = new StringHttpMessageConverter();
	stringConverter.setSupportedMediaTypes(MediaType.parseMediaTypes("application/xml"));

	List<HttpMessageConverter<?>> converters = new ArrayList<>();
	converters.add(converter);
	converters.add(stringConverter);

	return new RestTemplate(converters);
}

The JodaModule was provided by the following dependency:

<dependency>
    <groupId>com.fasterxml.jackson.datatype</groupId>
    <artifactId>jackson-datatype-joda</artifactId>
</dependency>

With the configuration complete, I was able to write a MessagesApiITest integration test that posts a request and retrieves it using the API. The API was secured using basic authentication, so it took me a bit to figure out how to make that work with RestTemplate. Willie Wheeler's Basic Authentication With Spring RestTemplate was a big help.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = IntegrationTestConfig.class)
public class MessagesApiITest {
    private final static Log log = LogFactory.getLog(MessagesApiITest.class);
    @Value("http://${app.host}/api/initiate")
    private String initiateAPI;
    @Value("http://${app.host}/api/messages")
    private String messagesAPI;
    @Value("${app.host}")
    private String host;
    @Inject
    private RestTemplate restTemplate;

    @Before
    public void setup() throws Exception {
        String request = new Scanner(new ClassPathResource("sample-request.xml").getFile()).useDelimiter("\\Z").next();

        ResponseEntity<org.ncpdp.schema.transport.Message> response = restTemplate.exchange(getTestUrl(initiateAPI),
                HttpMethod.POST, getBasicAuthHeaders(request), org.ncpdp.schema.transport.Message.class,
                Collections.emptyMap());
        assertEquals(HttpStatus.OK, response.getStatusCode());
    }

    @Test
    public void testGetMessages() {
        HttpEntity<String> request = getBasicAuthHeaders(null);

        ResponseEntity<PagedResources<Message>> result = restTemplate.exchange(getTestUrl(messagesAPI), HttpMethod.GET,
                request, new ParameterizedTypeReference<PagedResources<Message>>() {});
        HttpStatus status = result.getStatusCode();
        Collection<Message> messages = result.getBody().getContent();

        log.debug("messages found: " + messages.size());
        assertEquals(HttpStatus.OK, status);
        for (Message message : messages) {
            log.debug("message.id: " + message.getId());
            log.debug("message.dateCreated: " + message.getDateCreated());
        }
    }

    private HttpEntity<String> getBasicAuthHeaders(String body) {
        String plainCreds = "user:pass";
        byte[] plainCredsBytes = plainCreds.getBytes();
        byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
        String base64Creds = new String(base64CredsBytes);

        HttpHeaders headers = new HttpHeaders();
        headers.add("Authorization", "Basic " + base64Creds);
        headers.add("Content-type", "application/xml");

        if (body == null) {
            return new HttpEntity<>(headers);
        } else {
            return new HttpEntity<>(body, headers);
        }
    }
}

To get Spring Data to populate the message id, I created a custom RestConfig class to expose it. I learned how to do this from Tommy Ziegler.

/**
 * Used to expose ids for resources.
 */
@Configuration
public class RestConfig extends RepositoryRestMvcConfiguration {

    @Override
    protected void configureRepositoryRestConfiguration(RepositoryRestConfiguration config) {
        config.exposeIdsFor(Message.class);
        config.setBaseUri("/api");
    }
}

Summary

This article explains how I built a REST API using JAXB, Spring Boot, Spring Data and Liquibase. It was relatively easy to build, but required some tricks to access it with Spring's RestTemplate. Figuring out how to customize JAXB's code generation was also essential to make things work.

I started developing the project with Spring Boot 1.1.7, but upgraded to 1.2.0.M2 after I found it supported Log4j2 and configuring Spring Data REST's base URI in application.yml. When I handed the project off to my client last week, it was using 1.2.0.BUILD-SNAPSHOT because of a bug when running in Tomcat.

This was an enjoyable project to work on. I especially liked how easy Spring Data makes it to expose JPA entities in an API. Spring Boot made things easy to configure once again and Liquibase seems like a nice tool for database migrations.

If someone asked me to develop a REST API on the JVM, which frameworks would I use? Spring Boot, Spring Data, Jackson, Joda-Time, Lombok and Liquibase. These frameworks worked really well for me on this particular project.