Moving from Spring's XML to Annotations in AppFuse

Last night, I did a spike on AppFuse to change XML to Spring annotations (@Repository, @Service and @Autowired) in its service and data modules. While I was able to accomplish everything in a few hours (including converting tests), I did run into a couple issues.

AbstractTransactionalJUnit4..Tests vs. AbstractTransactionalDataSource..Tests
I've switched from my favorite Spring class to the annotation-happy AbstractTransactionalJUnit4SpringContextTests. However, this has presented an issue: when using ATDSSCT, I was able to call endTransaction() and startNewTransaction(). With ATJ4SCT, this doesn't seem possible. Below is a screenshot of the diff on a test method in the JPA implementation of UserDaoTest:

AbstractTransactionalJUnit4SpringContextTests vs. AbstractTransactionalDataSourceSpringContextTests

On the right, you'll notice that I had to comment out @ExpectedException to get the test to pass. This concerns me since this exception should be thrown. Is there a way to call endTransaction() and startNewTransaction() when subclassing AbstractTransactionalJUnit4SpringContextTests?

Instantiating GenericDao Implementations Programmatically
The second feature I tried to add is the ability to instantiate a GenericDao programatically rather than requiring a XML bean definition. In current versions of AppFuse, you can use the following bean definition to create a GenericDao for a model object.

<bean id="personDao" class="org.appfuse.dao.hibernate.GenericDaoHibernate">
    <constructor-arg value="org.appfuse.tutorial.model.Person"/> 
    <property name="sessionFactory" ref="sessionFactory"/>
</bean> 

When moving to a no-XML required architecture, it'd be nice to allow users to create GenericDao's programmatically. Below is the easiest way I've found to do this in a test:

GenericDao<User, Long> genericDao;
@Autowired
SessionFactory sessionFactory;

@Before
public void setUp() {
    genericDao = new GenericDaoHibernate<User, Long>(User.class);
    genericDao.setSessionFactory(sessionFactory);
}

However, there's a couple problems with this. First of all, mixing constructor injection and setter injection probably isn't a good idea. Changing the constructor to take a SessionFactory solves this problem, but now all subclasses need to have a more verbose constructor:

@Autowired
public UserDaoHibernate(SessionFactory sessionFactory) {
    super(User.class, sessionFactory);
}

Whereas before they had:

public UserDaoHibernate() {
    super(User.class);
}

In an ideal world, I could call new GenericDaoHibernate<User, Long>(User.class) and the SessionFactory would be wired in auto-magically. Is this possible with Spring 2.5?

The 2nd problem this presents is your client code will now be dependent on an implementation rather than the interface. I don't know how to solve that one, but I'd love to figure out a way to create GenericDaos with no XML and no implementation details in the client. Any ideas are most welcome.

If you'd like to see all the changes I made in converting from XML to Annotations, please see this patch.

Posted in Java at Nov 04 2008, 11:39:54 AM MST 14 Comments

Colorado Software Summit 2008 Wrapup

Snowman in Keystone Last week, I attended the Colorado Software Summit in Keystone and had a great time. Not only was the weather beautiful and the food delicious, the sessions I attended were awesome. If you ever get a chance to go to this conference, I highly recommend it. It's like being on vacation and learning with a bunch of old friends.

Yan Pujante also attended this conference and documents his experience, photography and presentations in Colorado Software Summit 2008.

Below is a list of my entries for all the sessions I attended.

For next year, I think the conference should shorten its sessions (from 90 to 60 minutes), invite more speakers and cut the price in half (to $999 per person). How do you think the Software Summit could be improved?

Posted in Java at Oct 28 2008, 11:03:23 PM MDT 2 Comments

Taking Apache Camel for a Ride with Bruce Snyder

Camel is a Java API that allows you to do message routing very easily. It implements many of the patterns found in Enterprise Integration Patterns. It doesn't require a container and can be run in any Java-based environment. Camel has a whole bunch of components - Bruce is showing a 6 x 10 grid with a component name in each grid. In other words, there's 60 components that Camel can use. Examples include: ActiveMQ, SQL, Velocity, File and iBATIS.

Chris Richardson asks "What's left inside of ServiceMix". Why use ServiceMix if you have Camel? ServiceMix is a container that can run standalone or inside an app server. You can run distributed ServiceMix as a federated ESB. Camel is much smaller and lightweight and is really just a Java API. ServiceMix 4 changed from a JBI-based architecture to OSGi (based on Apache Felix). They also expect to create your routes for ServiceMix 4 with Camel instead of XML. To process messages, you can use many different languages: BeanShell, JavaScript, Groovy, Python, PHP, Ruby, JSP EL, OGNL, SQL, XPath and XQuery.

Camel has a CamelContext that's similar to Spring's ApplicationContext. You can initialize it in Java and add your routes to it:

CamelContext context = new DefaultCamelContext();
context.addRoutes(new MyRouterBuilder());
context.start();

Or you can initialize it using XML:

<camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
    <package>com.acme.routes</package>
</camelContext>

Camel's RouteBuilder contains a fluid API that allows you to define to/from and other criteria. At this point, Bruce is showing a number of examples using the Java API. He's showing a Content Based Router, a Message Filter, a Splitter, an Aggregator, a Message Translator, a Resequencer, a Throttler and a Delayer.

Bruce spent the last 10 minutes doing a demo using Eclipse, m2eclipse, the camel-maven-plugin and ActiveMQ. It's funny to see a command-line guy like Bruce say he can't live w/o m2eclipse. I guess Maven's XML isn't so great after all. ;-)

Camel is built on top of Spring and has good integration. Apparently, the Camel developers tried to get it added to Spring, but the SpringSource guys didn't want it. Coincidentally, Spring Integration was released about a year later.

Camel also allows you to use "beans" and bind them to Camel Endpoints with annotations. For example:

public class Foo {

    @MessageDriven (uri="activemq:cheese")
    public void onCheese(String name) {
        ...
    }
}

Other annotations include @XPath, @Header and @EndpointInject.

Camel can also be used for BAM (Business Activity Monitoring). Rather than using RouteBuilder, you can use ActivityBuilder to listen for activities and create event notifications.

Bruce had quite a few folks show up for this presentation. I had trouble finding a seat because I was late. I think he did a good job of showing what Camel is and how you might use it.

Posted in Java at Oct 23 2008, 02:25:30 PM MDT 2 Comments

What's Coming in Spring 3.0

This morning, I delivered my What's Coming in Spring 3.0 talk for the 2nd time at Colorado Software Summit. Since there is no Spring 3.0 source code to speak of, I was unable to do the "Choose Your Own Adventure" demo at the end. :(

The good news is I was able to easily upgrade the Spring Kickstart application from Spring 2.0 to Spring 2.5.5 (using annotations). When 3.0 is released, I hope to update this project to use 3.0 as well as show what I needed to change. If I get ambitious, I might even change the UI to use Flex or Ext JS to show a RESTful client. Below is my presentation - hope you enjoy.

Posted in Java at Oct 22 2008, 11:51:12 AM MDT 12 Comments

Building LinkedIn's Next Generation Architecture with OSGi by Yan Pujante

This week, I'm attending the Colorado Software Summit in Keystone, Colorado. Below are my notes from an OSGi at LinkedIn presentation I attended by Yan Pujante.

LinkedIn was created in March of 2003. Today there's close to 30M members. In the first 6 months, there was 60,000 members that signed up. Now, 1 million sign up every 2-3 weeks. LinkedIn is profitable with 5 revenue lines and there's 400 employees in Mountain View.

Technologies: 2 datacenters (~600 machines). SOA with Java, Tomcat, Spring Framework, Oracle, MySQL, Servlets, JSP, Cloud/Graph, OSGi. Development is done on Mac OS X, production is on Solaris.

The biggest challenges for LinkedIn are:

  • Growing Engineering Team on a monolithic code base (it's modular, but only one source tree)
  • Growing Product Team wanting more and more features faster
  • Growing Operations Team deploying more and more servers

Front-end has many BL services in one webapp (in Tomcat). The backend is many wars in 5 containers (in Jetty) with 1 service per WAR. Production and Development environments are very different. Total services in backend is close to 100, front-end has 30-40.

Container Challenges
1 WAR with N services does not scale for developers (conflicts, monolithic). N wars with 1 service does not scale for containers (no shared JARs). You can add containers, but there's only 12GB of RAM available.

Upgrading back-end service to new version requires downtime (hardware load-balancer does not account for version). Upgrading front-end service to new version requires redeploy. Adding new backend services is also painful because there's lots of configuration (load-balancer, IPs, etc.).

Is there a solution to all these issues? Yan believes that OSGi is a good solution. OSGi stands for Open Services Gateway initiative. Today that term doesn't really mean anything. Today it's a spec with several implementations: Equinox (Eclipse), Knoplerfish and Felix (Apache).

OSGi has some really, really good features. These include smart class loading (multiple versions of JARs is OK), it's highly dynamic (deploy/undeploy built-in), it has a service registry and is highly extensible/configurable. An OSGi bundle is simply a JAR file with an OSGi manifest.

In LinkedIn's current architecture, services are exported with Spring/RPC and services in same WAR can see each other. The problem with this architecture comes to light when you want to move services to a 2nd web container. You cannot share JARs and can't talk directly to the other web app. With OSGi, the bundles (JARs) are shared, services are shared and bundles can be dynamically replaced. OSGi solves the container challenge.

One thing missing from OSGi is allowing services to live on multiple containers. To solve this, LinkedIn has developed a way to have Distributed OSGi. Multicast is used to see what's going on in other containers. Remote servers use the OSGi lifecycle and create dynamic proxies to export services using Spring RPC over HTTP. Then this service is registered in the service registry on the local server.

With Distributed OSGi, there's no more N-1 / 1-N problem. Libraries and services can be shared in one container (memory footprint is much smaller). Services can be shared across containers. The location of the services is transparent to the clients. There's no more configuration to change when adding/removing/moving services. This architecture allows the software to be the load balancer instead of using a hardware load balancer.

Unfortunately, everything is not perfect. OSGi has quite a few problems. OSGi is great, but the tooling is not quite there yet. Not every library is a bundle and many JARs doesn't have OSGi manifests. OSGi was designed for embedded devices and using it for the server-side is very recent (but very active).

OSGi is pretty low-level, but there is some work being done to hide the complexity. Spring DM helps, as do vendor containers. SpringSource has Spring dm Server, Sun has GlassFish, and Paremus has Infiniflow. OSGi is container centric, but next version will add distributed OSGi, but will have no support for load-balancing.

Another big OSGi issue is version management. If you specify version=1.0.0, it means [1.0.0, ∞]. You should at least use version=[1.0.0,2.0.0]. When using OSGi, you have to be careful and follow something similar to Apache's APR Project Versioning Guidelines so that you can easily identify release compatibility.

At LinkedIn, the OSGi implementation is progressing, but there's still a lot of work to do. First of all, a bundle repository needs to be created. Ivy and bnd is used to generate bundles. Containers are being evaluated and Infiniflow is most likely the one that will be used. LinkedIn Spring (an enhanced version of Spring) and SCA will be used to deploy composites. Work on load-balancing and distribution is in progress as is work on tooling and build integration (Sigil from Paremus).

In conclusion, LinkedIn will definitely use OSGi but they'll do their best to hide the complexity from the build system and from developers. For more information on OSGi at LinkedIn, stay tuned to the LinkedIn Engineering Blog. Yan has promised to blog about some of the challenges LinkedIn has faced and how he's fixed them.

Update: Yan has posted his presentation on the LinkedIn Engineering Blog.

Posted in Java at Oct 20 2008, 11:20:09 AM MDT 8 Comments

The Colorado Software Summit and Spring 3.0's SVN

In a little over two weeks, my favorite conference begins in Keystone, Colorado. The reason I like it so much is because it's an annual gathering (this will be my 4th year) with friends and it's somewhat like a vacation, except you get to learn a lot. This year I'll be speaking about Appcelerator and Spring 3.0.

A few weeks ago, I wondered How open source is Spring and expressed my frustration in not being able to find the source code. At almost the same time, SpringSource's controversial maintenance policy was announced. Developers booed, Free Spring was founded and SpringSource changed their tune. In addition, someone sent me the FishEye URL for Spring 3.0's SVN. From that, I was able to figure out how to get Spring 3.0's source code.

svn co https://src.springframework.org/svn/spring-framework/trunk spring-3.0

The only bad news is FishEye shows the last change as "17 September 2008 ... (21 days ago)". If Spring 3.0 M1 is supposed to be released any day now, you'd think there would be more SVN commits. Anyone from SpringSource care to comment?

Posted in Java at Oct 09 2008, 08:23:21 AM MDT 6 Comments

RE: How Open Source is Spring?

Peter Mularien has a very well written post titled How Open Source is Spring?: An Analytical Investigation:

This post is to expand on some of the thoughts I posted on the SpringSource Blog in response to Rod Johnson's excellent description of the SpringSource business model and its commitment to development of open source software.

Now that SpringSource has shown an ability to crank out new product releases on a seemingly weekly basis, I wanted to reflect on where Spring is positioned in the Java open source community, and how open the Spring Core project is to work done by the public.

The hypothesis of my experiment occurred to me when I happened to be reviewing Spring JIRA assignments one day. I was curious whether, following the bug assignments, the majority of development on the "Spring Core" projects (including Spring MVC and what we would consider "classic Spring") is performed solely by SpringSource employees.

Peter's post is near and dear to me because I'm doing a What's coming in Spring 3.0 talk at the Colorado Software Summit in October. The only information I was able to find on Spring 3.0 is from random blog posts and Juergen's presentation. When I give talks about technologies, I prefer to dig in and try them before talking about it. With Spring 3.0's source code nowhere to be found, this is very difficult to do.

I really hope Spring 3.0 becomes available in early October. If it does, I hope to upgrade AppFuse, AppFuse Light and Spring Kickstart. If it doesn't, my talk will most likely be a regurgitation of what Juergen's slides have and that's just not right.

Posted in Java at Sep 19 2008, 09:54:48 AM MDT 14 Comments

Spring MVC vs. JSF and The State of Spring Web

Alternative Adult has only posted a couple times in 2008, but his entries have peaked my interest.

Spring MVC or JSF+?
My business unit is trying to standardize if we can on a single Java-based Web framework going forward to simplify the Web development process, especially as individual developers move from one division to another, or centralized support groups need to maintain multiple applications from multiple divisions.

At the enterprise level within my company, the architecture group says that they will provide support for either Spring MVC or JSF+ (where the + represents the accompanying technologies you would use to provide a more maintainable application and a more rich user experience, e.g. Facelets, Richfaces, etc.).

Now my business unit is trying to decide which of these two frameworks, Spring MVC or JSF+, is the most appropriate to standardize upon for our development community. [Read More]

...and...

State of Spring Web
For those that are interested, the following is a summary of the notes I captured from a conversation with SpringSource on the state of Spring Web. [Read More]

Good stuff Michael - keep it coming.

Posted in Java at May 16 2008, 06:17:52 PM MDT 6 Comments

AppFuse 2.0.2 Released

The AppFuse Team is pleased to announce the release of AppFuse 2.0.2. This release includes upgrades to Spring Security 2.0, jMock 2.4, the ability to customize code generation templates and many bug fixes.

For information on upgrading from 2.0.1, see the Release Notes or changelog. AppFuse 2.0.2 is available as a Maven archetype. For information on creating a new project using AppFuse, please see the QuickStart Guide or the demos and videos.

To learn more about AppFuse, please read Ryan Withers' Igniting your applications with AppFuse.

The 2.0 series of AppFuse has a minimum requirement of the following specification versions:

  • Java Servlet 2.4 and JSP 2.0 (2.1 for JSF)
  • Java 5+

If you've used AppFuse 1.x, but not 2.x, you'll want to read the FAQ. Join the user mailing list if you have any questions.

Thanks to everyone for their help contributing code, writing documentation, posting to the mailing lists, and logging issues.

Please post any issues you have with this release to the mailing list.

Posted in Java at May 11 2008, 11:25:40 PM MDT 4 Comments

Running Spring MVC Web Applications in OSGi

For the past couple of weeks, I've been developing a web application that deploys into an OSGi container (Equinox) and uses Spring DM's Spring MVC support. The first thing I discovered was that Spring MVC's annotations weren't supported in the M1 release. This was apparently caused by a bug in Spring 2.5.3 and not Spring DM. Since Spring DM 1.1.0 M2 was released with Spring 2.5.4 today, I believe this is fixed now.

The story below is about my experience getting a Spring MVC application up and running in Equinox 3.2.2, Jetty 6.1.9 and Spring DM 1.1.0 M2 SNAPSHOT (from last week). If you want to read more about why Spring MVC + OSGi is cool, see Costin Leau's Web Applications and OSGi article.

To get a simple "Hello World" Spring MVC application working in OSGi is pretty easy. The hard part is setting up a container with all the Spring and Jetty bundles installed and started. I imagine SSAP might solve this. Luckily for me, this was done by another member of my team.

After you've done this, it's simply a matter of creating a MANIFEST.MF for your WAR that contains the proper information for OSGi to recognize. Below is the one that I used when I first tried to get my application working.

Manifest-Version: 1
Bundle-ManifestVersion: 2
Spring-DM-Version: 1.1.0-m2-SNAPSHOT
Spring-Version: 2.5.2
Bundle-Name: Simple OSGi War
Bundle-SymbolicName: myapp
Bundle-Classpath: .,WEB-INF/classes,WEB-INF/lib/freemarker-2.3.12.jar,
 WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar,
 WEB-INF/lib/spring-beans-2.5.2.jar,WEB-INF/lib/spring-context-2.5.2.jar,
 WEB-INF/lib/spring-context-support-2.5.2.jar,WEB-INF/lib/spring-core-2.5.2.jar,
 WEB-INF/lib/spring-web-2.5.2.jar,WEB-INF/lib/spring-webmvc-2.5.2.jar 
Import-Package: javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional

Ideally, you could generate this MANIFEST.MF using the maven-bundle-plugin. However, it doesn't support WARs in its 1.4.0 release.

You can see this is an application that uses Spring MVC, FreeMarker, SiteMesh and the URLRewriteFilter. You should be able to download it, unzip it, run "mvn package" and install it into Equinox using "install file://<path to war>".

That's all fine and dandy, but doesn't give you any benefits of OSGi. This setup works great until you try to import OSGi services using a context file with an <osgi:reference> element. After adding such a reference, it's likely you'll get the following error:

SEVERE: Context initialization failed
org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
Configuration problem: Unable to locate Spring NamespaceHandler for
XML schema namespace [http://www.springframework.org/schema/osgi]

To fix this, add the following to your web.xml (if you're using ContextLoaderListener, as an <init-parameter> on DispatcherServlet if you're not):

  <context-param>
    <param-name>contextClass</param-name>
    <param-value>org.springframework.osgi.web.context.support.OsgiBundleXmlWebApplicationContext</param-value>
  </context-param>

After doing this, you might get the following error on startup:

SEVERE: Context initialization failed
org.springframework.context.ApplicationContextException: Custom
context class [org.springframework.osgi.web.context.support.OsgiBundleXmlWebApplicationContext]
is not of type [org.springframework.web.context.ConfigurableWebApplicationContext] 

To fix this, I change from referencing the Spring JARs in WEB-INF/lib to importing the packages for Spring (which were already installed in my Equinox container).

Bundle-Classpath: .,WEB-INF/classes,WEB-INF/lib/freemarker-2.3.12.jar,
 WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar
Import-Package:
javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional,
 org.springframework.osgi.web.context.support,
 org.springframework.context.support,
 org.springframework.web.context,
 org.springframework.web.context.support,
 org.springframework.web.servlet,
 org.springframework.web.servlet.mvc,
 org.springframework.web.servlet.mvc.support,
 org.springframework.web.servlet.view,
 org.springframework.ui,
 org.springframework.web.servlet.view.freemarker 

After rebuilding my WAR and reloading the bundle in Equinox, I was confronted with the following error message:

SEVERE: Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'freemarkerConfig' defined in ServletContext
resource [/WEB-INF/myapp-servlet.xml]: Instantiation of bean failed;
nested exception is java.lang.NoClassDefFoundError:
freemarker/cache/TemplateLoader
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:851) 

As far as I can tell, this is because the version of Spring MVC installed in Equinox cannot resolve the FreeMarker JAR in my WEB-INF/lib directory.

To prove I wasn't going insane, I commented out my "freemarkerConfig" and "viewResolver" beans in myapp-servlet.xml and changed to a regular ol' InternalResourceViewResolver:

<bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/"/>
    <property name="suffix" value=".jsp"/>
</bean>

This worked and I was able to successfully see "Hello World" from a JSP in my browser. FreeMarker/SiteMesh worked too, but FreeMarker didn't work as a View for Spring MVC.

To attempt to solve this, I create a bundle for FreeMarker using "java -jar bnd-0.0.249.jar wrap freemarker-2.3.12.jar" and installed it in Equinox. I then change my MANIFEST.MF to use FreeMarker imports instead of referencing the JAR in WEB-INF/lib.

Bundle-Classpath:
.,WEB-INF/classes,WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar
Import-Package:
javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional,
 org.springframework.osgi.web.context.support,
 org.springframework.context.support,
 org.springframework.web.context,
 org.springframework.web.context.support,
 org.springframework.web.servlet,
 org.springframework.web.servlet.mvc,
 org.springframework.web.servlet.mvc.support,
 org.springframework.web.servlet.view,
 org.springframework.ui,
 org.springframework.web.servlet.view.freemarker,
 freemarker.cache,freemarker.core,freemarker.template,freemarker.ext.servlet 

Unfortunately, this still doesn't work and I still haven't been able to get FreeMarker to work with Spring MVC in OSGi. The crazy thing is I actually solved this at one point a week ago. Shortly after, I rebuilt Equinox from scratch and I'm been banging my head against the wall over this issue ever since. Last week, I entered an issue in Spring's JIRA, but thought I'd fixed it a few hours later.

I've uploaded the final project that's not working to the following URL:

http://static.raibledesigns.com/downloads/myapp-osgi.zip

If you'd like to see this project work with Spring MVC + JSP, simply modify myapp-servlet.xml to remove the FreeMarker references and use the InternalResourceViewResolver instead.

I hope Spring DM + Spring MVC supports more than just JSP as a view technology. I hope I can't get FreeMarker working because of some oversight on my part. If you have a Spring DM + Spring MVC application working with Velocity or FreeMarker, I'd love to hear about it.

Posted in Java at Apr 30 2008, 12:42:34 AM MDT 14 Comments