Search for [sidenote] in weblog rd

Upgrading Grails from 2.0 to 2.2

2013-03-22T09:16:37-06:00

In preparation for my Grails vs. Play Smackdown at Devoxx France next week, I recently upgraded my Grails version of Happy Trails from Grails 2.0.3 to Grails 2.2.1. I ran into a few issues along the way and figured I'd document them here to help others out.

Fixing the source
The first issue I ran into was Spock and Groovy 2 incompatibilities.

| Resolving plugin JAR dependencies
| Error WARNING: Dependencies cannot be resolved for plugin [mail] due to error: startup failed:
Could not instantiate global transform class org.spockframework.compiler.SpockTransform specified at jar:file:/Users/mraible/.grails/ivy-cache/org.spockframework/spock-core/jars/spock-core-0.7-groovy-1.8.jar!/META-INF/services/org.codehaus.groovy.transform.ASTTransformation  because of exception org.spockframework.util.IncompatibleGroovyVersionException: The Spock compiler plugin cannot execute because Spock 0.7.0-groovy-1.8 is not compatible with Groovy 2.0.7. For more information, see http://versioninfo.spockframework.org

I posted the problem to StackOverflow and got a response almost immediately. While this pull request helped me quite a bit, it was ultimately caused by my vision: I had two "geb-spock" dependencies listed in BuildConfig.groovy with different groupIds.

At this point, I also moved all my plugin dependencies from application.properties to BuildConfig.groovy.

The next problem I ran into was a unit test and functional tests failing. The unit testing issue was caused by my Direction model not being in the tests @Mock annotation. After I added it, validation kicked and I recognized my test was invalid. I added @Ignore and continued.

The functional test seemed to be seemed to be caused by Geb and it trying to use the Chrome Driver. One of my tests didn't work with the default HtmlUnitDriver, so I used the ChromeDriver for the one test.

| Running 11 spock tests... 6 of 11
| Failure:  signup as a new user(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.WebDriverException: Unable to either launch or connect to Chrome. Please check that ChromeDriver is up-to-date. Using Chrome binary at: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 45.66 seconds
Build info: version: '2.27.0', revision: '18259', time: '2012-12-05 11:30:53'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.8.2', java.version: '1.7.0_04'
Driver info: org.openqa.selenium.chrome.ChromeDriver
    at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:187)
    at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:145)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:533)
    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:216)
    at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:111)
    at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:115)
    at org.openqa.selenium.chrome.ChromeDriver.(ChromeDriver.java:161)
    at org.openqa.selenium.chrome.ChromeDriver.(ChromeDriver.java:107)
    at happytrails.AuthenticatedUserSpec.signup as a new user(AuthenticatedUserSpec.groovy:25)

Even when running "grails -Dgeb.env=chrome test-app", this still happened. This was caused by the fact that I had GebConfig.groovy in test/functional/happytrails. Move it to test/functional solved the problem. I also discovered that I know longer needed Chrome to get this test to pass. Apparently, the HtmlUnitDriver has issues with Grails 2.2, but it seems to work for me.

After getting the Geb configuration fixed, I ran into a functional test failure:

| Running 11 spock tests... 5 of 11
| Failure:  click signup link(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.ElementNotVisibleException: Element must be displayed to click (WARNING: The server did not provide any stacktrace information)

Even though I could see the "signup" link when I ran "grails run-app", I could see that it didn't show up when running tests in Chrome. This turned out to be caused by an extraneous <div class="nav-collapse"> I had in my main.gsp. Removing it solved the problem. It's strange that this never showed up with Grails 2.0. My only guess is that Geb someone didn't look at the visibility of the element.

The last testing-related issue I ran into was a InvalidElementStateException:

| Running 11 spock tests... 7 of 11
| Failure:  add new route to region(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.InvalidElementStateException: Element must be user-editable in order to clear it. (WARNING: The server did not provide any stacktrace information)

I was able to fix this by changing AddRoutePage.groovy from:

static content = {
    createButton(to: ShowRoutePage) { create() }
    name { value("Name") }
    distance { value("Distance") }
    location { value("Location") }
}

To:

static content = {
    createButton(to: ShowRoutePage) { create() }
    form { $("form") }
}

And then referencing name, distance and location accordingly (form.name, etc.) in AuthenticatedUserSpec.groovy.

CloudBees
After I had everything working locally, I logged into Jenkins on CloudBees. Since I hadn't used it in a while, I had to wait a bit while my Jenkins server was re-commissioned. Once it was up, I tried to select Grails 2.2.1 to build with, but found it wasn't available. After a tweeting this, I learned about Grails Wrapper, found that the latest Grails Jenkins plugin supported it and got everything working. I later discovered that CloudBees does support Grails 2.2.1, I just needed to setup another Grails installation to automatically download and install 2.2.1.

Heroku
The last two issues I ran into were with Heroku. Since I was upgrading everything, I wanted Grails to build/run under Java 7 and use Servlet 3. I changed the appropriate properties in BuildConfig.groovy, configured Heroku and deployed. No dice.

Error Compilation error: startup failed:
Invalid commandline usage for javac.
javac: invalid source release: 1.7
Usage: javac  
use -help for a list of possible options

Sidenote: I tried building with Java 8 on CloudBees, but discovered the searchable plugin doesn't support it.

Compile error during compilation with javac.
/scratch/jenkins/workspace/Happy Trails - Grails 2/work/plugins/searchable-0.6.4/src/java/grails/plugin/searchable/internal/compass/index/DefaultUnindexMethod.java:94: error: reference to delete is ambiguous
                    session.delete(query);
                           ^
  both method delete(CompassQuery) in CompassOperations and method delete(CompassQuery) in CompassIndexSession match

As far as Servlet 3, it was pretty obvious that the Jetty version Heroku uses for Grails doesn't support it. Therefore, I reverted back to Servlet 2.5.

java.lang.NoClassDefFoundError: javax/servlet/AsyncContext
	at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2444)

I sent the Java 7 issue to Heroku Support a few days ago but haven't heard back yet.

Summary
While upgrading Grails from 2.0 to 2.2 wasn't as easy as expected, it is understandable. After all, Grails 2.2 ships with Groovy 2.0, which has a bunch of new features itself. All the issues I ran into were fairly easy to solve, except for Java 7 on Heroku. But hey, what do you expect from a free hosting service?

If you're at Devoxx France next week, I look forward to sharing our research on Grails 2.2.1 vs. Play 2.1.0.

Integration Testing with HTTP, HTTPS and Maven

2011-02-11T15:54:16-07:00

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

Start a container
Deploy the application
Run all integration tests
Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support.

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Choosing an Ajax Framework

2009-01-08T21:36:22-07:00

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Weight	Criteria	Dojo	YUI	GWT	Ext JS	Notes
#	Important Criteria for Customer	0..1	0..1	0..1	0..1	Notes about rankings

Our strategy for filling in this matrix:

Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

Quality of Documentation/Tutorials/Self Help
Browser support (most important browsers/versions based on web stats)
Testability (esp. Selenium compatibility)
Licensing
Project health/adoption
Performance
Scalability
Flexibility/extensibility
Productivity (app dev, web dev)
Richness of widget/component library
Charting capability
Ability to create new widgets
Match to existing Java team skill-set
Ease of deployment (on Ops, QA, Users)
Degree of risk generally
Ability to integrate with existing site (which includes Prototype)
Easy to style with CSS
Validation (esp. marking form elements invalid)
Component Theme-ing/Decoration
CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.

Choosing a JVM Web Framework

2007-08-07T10:10:05-06:00

I plan on rewriting my "Comparing Java Web Frameworks" presentation for this year's Colorado Software Summit. Rather than "Comparing Java Web Frameworks", I'm going to make it into more of a "Choosing a JVM Web Framework" presentation. I think this opens it up to more possibilities such as Grails, JRuby on Rails, Flex and GWT.

One of the things I hope to talk about is choosing the right tool for the job. I think there's 3 types of web applications you can develop:

Consumer-facing, high-traffic, stateless applications
Internal, more desktop-like applications that are stateful
Media-rich applications that require a RIA framework like Flex

Once you've decided on which of these you're developing, it's much easier to narrow down the choices:

Struts 2, Spring MVC, Stripes
JSF, Tapestry, Wicket
GWT, Flex, OpenLaszlo

I'm not sure if GWT fits in the RIA category. I'm not sure where Rails or Grails fit either. They more closely resemble category #1 than any other, yet there's a lot of speculation about their scalability. I think if that perception can be changed, they'll fit into the first category quite well. However, I don't think they compete with component-based or RIA because they don't hold state or offer rich-media capabilities.

Sidenote: I find the scalability debate quite interesting. There's a fair amount of propaganda in Javaland that scalability can be achieved with appservers and clustering tools like Terracotta. If this is true, I've yet to read good solid proof of it. Most of the "how to scale" information out there suggests "share nothing" architectures that shard data and applications across several servers. Of course, there's scalability and then there's massive scalability. Can appservers and clustering solve massive scalability like Google and Amazon require?

The 2nd and 3rd categories have someone of a blurry line, so I'm hoping to figure out how to clarify that. There's also a lot of other factors that will go into choosing a web framework. What if you're simply trying to replace a home-grown framework with an open-source one? If you want to keep your backend and all its logic, does it make sense to use something like Seam, Grails, JRuby on Rails or even AppFuse? Probably not - all their wizbang features and CRUD generation doesn't mean much if all you're using is the web framework. Also, if your application requires support for non-JavaScript browsers (for 508 compliance), then GWT and JSF can be easily eliminated. I know that there are many claims that JSF doesn't require JavaScript, but I've yet to see a real-world application developed with JSF that expects JavaScript to be turned off. Progressive enhancement is a requirement by many of my clients these days.

What's your opinion? How can we make it easier for developers and companies to choose a web framework? Is categorizing application types a good technique?

[TSE] Spring-OSGI with Adrian Colyer

2006-12-09T14:29:58-07:00

Adrian has CTO as his title on his slides. Wasn't he hired as "Chief Scientist" originally? I wonder why his title changed? Maybe having the CTO title is a better way of getting more cash when the buyout happens? What buyout you ask? I don't know, but I can't help but think that Interface21 is on the radar of major organizations like Oracle and BEA. While it might suck for those guys to answer to a larger organization, it would be nice to see Rod, Juergen, et. al. benefit ($$) from all their hard work.

Agenda

What is OSGi?
What problems does it help us to solve?
How does it work?
Where does Spring fit in?
Spring-OSGi

One of the first questions people ask about OSGi is "what the heck is it?"

The 2nd question is "what's with the lowercase i?" Seriously - does anyone know?

Most people don't even know what it is. OSGi stands for Open Services Gateway initiative. From the very beginning, it was designed to be lightweight and dynamic. This is the major difference between it and other containers. It's always been designed to have things added and removed. Now it's tagline is: "The Dynamic Module System for Java".

It's designed to allow you to partition a system into a number of modules (a.k.a. bundles). There's strict visibility rules (similar to protected and private). There's a resolution process (dependencies are satisfied) and it understands versioning.

It's dynamic! Modules can be installed, started, stopped, uninstalled and updated - all at runtime.

It's even service oriented (no wonder IBM loves it). Bundles can publish services (dynamically). The Service Registry allows other bundles to find services and bind to them. Services come and go at runtime, all taken care of for you.

Where does OSGi come from? It's backed by the OSGi Alliance. Unfortunately, this site makes it difficult to figure out exactly what OSGi is. The founders of OSGi understood it needed to be lightweight and dynamic from day one. It started in 1999, and was originally focues on embedded Java nd networked devices. In 2003 it was exteneded to support to mobile devices. In 2004, it had significant ope source community opttion with Eclipse. In 2006 and 2007, it's moving into server-side Java.

Who's doing OSGi containers?

Open-source implementations: Equinox, Felix (Apache), Knopflerfish
Significant Enterprise usage: Eclipse, IBM (WebSphere, Lotus), JOnAS, interest from BEA and Oracle

So what? What problems does OSGi solve?

Visibility By default, a bundle is a black box. It's completely protected and you can't see inside it, not even with reflection or any other classloading trickery. Only exported packages are visible outside of the exporting bundle. This stops unintended coupling between modules, enables independent development, faster development cycles and security.

Operational Control
An OSGi container allows you to see all modules and their status - using the OSGi console or JMX. You can get information on wiring, install new bundles, activate bundles (and publish services), deactivate bundles (and unregister services), refresh bundles, stop bundles and uninstall bundles. The beauty? You can do all of this without stopping or restarting the application!

What is an OSGi bundle?
It's just a jar file. There's no complicated tools or sophisticated packaging required. All you need is a symbolic name in your META-INF/MANIFEST.MF file. For example:

Bundle-SymbolicName: org.swf.beans

There are a number of other attributes you can specify, but the SymbolicName is the only required one.

There's a Maven OSGi Plugin. The major change from a regular JAR project is <packaging>jar</packaging> -> <packaging>osgi-bundle</packaging>.

Sidenote: You have to love how Maven is so Google un-friendly. Searching for Maven OSGi Plugin doesn't seem to provide a definitive answer to the Maven plugin that Adrian is talking about. I'm guessing Maven has OSGi support natively?

Eclipse has fantastic tooling support for OSGi via "plug-ins". You can create a new "plug-in project" and use Eclipse to create/edit your OSGi manifest. Eclipse calls bundles "plug-ins" - so creating a plug-in project is the same thing as creating a new bundle.

Export-Package is a header (a.k.a. manifest entry) used to export types for other bundles to use. If it's not exported, it's not visible. For example:

Export-Package: org.xyz.m1.service, 
                org.xyz.m2.service, 
                org.xyz.m3.service

You can also specify a version by adding a semicolon after the package and specifying "version=#". For example:

Export-Package: org.xyz.m1.service; version=1.0, 
                org.xyz.m2.service; version=1.0, 
                org.xyz.m3.service; version=1.0

In addition to specifying specific versions, there's syntax to control a specific versions. For the examples above, OSGi will actually get any version 1.0 and above.

For the next 10 minutes, Adrian talked about how class-loading and resolution works. He also pointed out how to bootstrap a bundle (using the BundleActivator interface) and the basics of the Service Registry. The BundleContext allows you to get and remove services.

What about JSR 277 (Java Module System)? It's a very crude version of OSGi with its "RequireBundle" mechanism. The key problem is the lack of dynamics (loading/unloading of modules). Peter Kriens (of the OSGi Alliance) shares his thoughts on this in his JSR 277 Review. JSR 291 (Dynamic Component Support for Java SE) may kill 277 because it relies on OSGi. Its goal:

Establish a JCP specification for a dynamic component framework supporting existing Java SE environments based on the OSGi dynamic component model specifications. "

What about Spring?
OSGi offers an excellent foundation for building enterprise applications. How can you exploit the power and sophistication of OSGi w/o adding complexity? How can you split your application into a number of OSGi bundles? Between bundles, you need an easy way to expose bundles as services and wire service references b/w them. What about testing? How can you test bundles w/o running them inside an OSGi container?

Spring-OSGi hopes to allow you to build on OSGi, but deploy in Tomcat, WebSphere, WebLogic or any other application server. Basically, it gives you an "OSGi powered Web Application".

Project Goals: Bring the benefits of OSGi to enterprise applications and use Spring to configure modules/bundles. In addition, make it easy to code your applications w/o depending on OSGi APIs and make it easy to unit and integration test.

Project Collaborators: Adrian is the lead, and there's active committers from BEA and Oracle. Input to the project's specification has been gathered from OSGi Alliance (technical director and CTO), BEA, Oracle, IBM, Eclipse Equinox, Felix and many individuals.

Spring modules (spring-core, spring-beans, spring-aop) will be shipped as OSGi bundles in Spring 2.1. All the necessary import and export package headers will be pre-defined. This will enable an OSGi application to import and use Spring packages and services.

OsgiBundleXmlApplicationContext is a Spring application context based on an OSGi bundle. It uses the bundle context and classloader to load resources and implements Spring's resource abstraction for OSGi. There is a "bundle:" prefix for specifying an explicit path.

To do integration testing with Spring-OSGi, you can extend ConfigurableBundleCreatorTests. This automatically creates an OSGi bundle containing the tests on the fly. This class handles starting an OSGi container, installs all bundles and runs tests inside OSGi. Currently, it supports Equinox, Felix and Knopflerfish. If you're using Maven, it's simple to switch containers with profiles (i.e. -Pfelix). Adrian did a demo in Eclipse. I didn't seem the OSGi container starting (or him specify which one to use) but the test seemed to finish rather quickly.

To interact with OSGi, there are a number of conveniences that Spring-OSGi provides: BundleContextAware, <osgi:property-placeholder>, <osgi:bundle> (starts other bundles), <osgi:virtual-bundle> (bundle created on the fly).

OSGi-based web applications
It's entirely possible to run an OSGi container embedded in another container. Remember it's lightweight enough to run in a phone! Even OSGi in OSGi is supported.

Equinox is the only OSGi container that currently supports web applications. It uses a "bridge servlet" that acts as a front controller and dispatches servlet requests to bundles. To package your application as and deploy it to Equinox, you basically have to create a WAR with the bridge servlet as the only entry in web.xml and the only JAR in WEB-INF/lib is servletbridge.jar. Everything else is packaged under /WEB-INF/eclipse. Here's what that looks like:

WEB-INF 
    web.xml  
    lib/servletbridge.jar 
    eclipse 
        launch.ini 
        configuration/config.ini 
        features 
        plugins 
            <your-bundles>

Adrian did a demo showing a simple "Hello World" example deployed into an OSGi container. The output from the demo in the browser wasn't much. However, he showed how you could use an OSGi Console to interact (start/stop/refresh) all the different bundles. The fact that you can do this all at runtime has some pretty interesting implications.

My Thoughts
The demo makes Servlet/EJB containers look somewhat primitive. Will OSGi containers be the next generation of application servers? It seems somewhat likely, but as Adrian said, most of this stuff is on the bleeding edge (or even "razors edge" to quote Adrian). My guess is OSGi will be a very interesting space in 2007. However, I don't see myself deploying WARs to OSGi containers anytime soon. Maybe in 2008.

Can OSGi power a web application's plugin architecture? It doesn't seem like it. The web application would have to be deployed in an OSGi container to make this possible. Either that, or the web application would have to be an OSGi container itself.

Rails is 8 times slower than Spring+Hibernate

2005-02-28T06:25:27-07:00

Might as well start off this week by getting people's blood boiling. According to a comment on Dion's blog:

Having done extensive performance (scalability means different things to different people) testing on both Rails and a comparable Spring/Hibernate/JSP2 webapp (no one seems have have done any sort of benchmarking on Rails, or they simply don't care, I don't really know, but since benchmarking is what I do... ) I can say that Apache2/FastCGI/Rails is about... 8x slower than the comparable Tomcat/Spring/Hibernate/JSP2 solution. And that is with caching turned on in Rails (using Rails 0.9.5...)

Quite frankly, 62 req/s on a Dual Opteron with 4GB of RAM rendering a simple view with no DB access is too... damn slow.

Personally, I still think Rails looks like a great (and easy) way to develop webapps. I just wonder if there's some truth to the "can't scale" argument. I guess the best way to find out is for me to develop an application like AppFuse with Rails, and then hammer it (and AppFuse) with JMeter to see what kind of results I get.

On a sidenote, I wonder when Rails will hit the illustrious version 1.0? They released 0.1 last week - which is a bad version number for marketing. If it's as mature as folks claim, why not make the next release 1.0? That version number alone will likely allow developers to use it more in big companies.

[DJUG] The Google Guys

2004-08-12T01:01:48-06:00

I'm sitting at the Denver JUG meeting and Joshua Bloch and Neal Gafter just finished a talk on "Java Puzzlers". I didn't show up until halfway through - but it was still a great half hour. They had a bunch of slides with problems that had seemingly easy answers. They'd both have a good dialog about their proposed answers - and then asked the crowd what they thought. The problems were mostly due to dumb (but real world mistakes) - the kind of thing you'd slap your fellow programmer for writing. These guys are definitely fun to listen to - next up is Tiger and what's new in 1.5 (I thought it was 5.0?). Boy, it's a full room tonight - I'd bet there's around 120-150 people here.

Taming the Tiger

Major theme of "JDK 5" is ease of development with features like generics, for-each loop, autoboxing/unboxing, enums, varargs, static imports and annotations. It's designed to make programs clearer, shorter and safer by providing linguistic support for commong idioms. Sidenote: Joshua said that Neal wrote the compiler - and they've basically made it more rigorous so it writes the boilerplate code for you. New features do no sacrifice compatibility or compromise the spirit of the language. Neal has been using these features for a couple of years now and he says he's really enjoyed them.

Goal of this talk is to make it easy for us to understand JDK 5 so we can start using it in our development. Let's look at the different features of 5.0.

Generics, For-Each and Autoboxing/unboxing

Generics allow you to specify the element type of collection. Rather than specifying a List - you specify it's contents - i.e. String. It's basically stronger typing with less typing which enforces the specification at compile time. For example, the following code using the new for-each syntax to iterate through a list of TimerTasks in a collection. Notice the lack of casting and easy-to-read loop syntax.

void cancellAll(Collection<TimerTask> c) {
    for (TimerTask task : c) {
        task.cancel();
    }
}

Bytecode is the same as it is in 1.4 - 5.0 merely converts the code for you. One question that these guys have heard a lot is why ":" rather than "in". The answer is twofold - because "in" is already a keyword (for example, System.in) and they didn't want to introduce a new keyword. Because 'in' is an identifier that is already in widespread use, and thus they could not make it a keyword without serious impact. Only new keyword in JDK 5 is enum.

The Collection Interface has been Generified. All existing code should still work, but you can also use the new stuff if you like. I haven't listened much to what's new in 5.0 - but this is wicked cool. You might say it sucks because now you end up with strongly typed stuff, but at least you won't have any more ClassCastExceptions.

autoboxing: automatic conversion from int to Integer (or from double to Double, etc.)
unboxing: automatic conversion from Integer to int

For example, you can now easily do the following:

Integer i = new Integer(5);
Map map = new HashMap();
map.put("result", i+1);

Notice that the Integer type is converted to an int for the addition, and then back to an Integer when it gets put into the Map. Cool, huh?

JDK 5 also simplifies reflection. Class Class has been generified - Class literal Foo.class is of type Class<Foo>. This enables compile-time type-safe reflection w/o casting. The following used to return an Object and required casting.

Foo foo = Foo.class.newInstance();

This enables strongly typed static factories. I wonder if this can be used with Spring so you don't have to cast a bean when grabbing it from the ApplicationContext?

When should you use Generics? Any time you can - unless you need to run on a pre-5.0 VM. The extra effor in generifying code is worth it - especially b/c of increased clarity and type safety.

When to use for-each loop? Any time you can b/c it really beautifies code and makes it much easier to write. It's probably the smallest new feature in 5.0, but likely to be a favorite. You can't use for-each for these cases:

Removing elements as you traverse a collection (b/c there's no iterator)
Modifying the current slot in an array or list (b/c the index is hidden)
Iterating over multiple collections or arrays

The lack of an index seems to rub the crowd wrong. Joshua and Neal's response is they tried to design something very simple that would capture 80% of usage. If you need an index, just use the old for loop - it ain't that hard; we've been doing it for years!

If you want to use for-each in your APIs - i.e. if you're writing a framework, a class should implement the new Iterable class.

When should you use autoboxing? When there is an impedance mismatch b/w reference types and primitives. Not appropriate for scientific computing. An Integer is not a substitute for an int. It simply hides the distinction between wrappers and primitives. A null unboxes by returning a NullPointerException. They did consider setting it to the primitive's default, but the community voted 50-1 to for NPE.

Enums

JDK 5 includes linguistic support for enumerated types. Advanced OO features include the ability to add methods and fields to enums. Much clearer, safer, more powerful than existing alternatives (i.e. int enums).

enum Season { WINTER, SPRING, SUMMER, FALL }

I just noticed that it's boiling in here - A/C must be out again in the auditorium. It's 8:20 right now, I hope this is over soon, I can feel sweat beading on my forehead.

Enums are Comparable and Serializable. Enum constants should be named similar to constants. Enums are basically a new type of class. As far as I can tell, I have no use for Enums in my code. There's lots of gasps from the crowd as Joshua is describing the features of Enums (i.e. constant-specific methods). Sure it looks cool, but I still don't think I have a use for it. Maybe framework developers will find this useful. BTW, there's two high-performance collection classes: EnumSet (bit-vector) and EnumMap (array). EnumSet replaces traditional bit-flags: i.e. EnumSet.of(Style.BOLD, Style.ITALIC).

When should you use Enums?

Natural enumerated types: days of week, phases of moon, seasons
Other sets where you knkow all possible values: choices on menus, rounding modes, command line flags
As a replacement for flags (EnumSet)

Quote of the night: "It's extraordinarily rare that you'll need to cast when programming with JDK 5".

Varargs

A method that takes an arbitrary number of values requires you to create an array. Varargs automates and hides the process. James Gosling contributed the ... syntax. Varargs always has to be the last parameter. MessageFormat.format has been retrofitted with varargs in JDK 5:

public static String format(String pattern, Object... arguments);

String result = MessageFormat.format("At {1,time} on {1,date}, there was {2} on planet "
                                     + "{0,number,integer}.", 7, new Date(),
                                     "a disturbance in the Force");

Reflection is now much easier with Varargs - so you can call c.getMethod("test").invoke(c.newInstance()) instead of c.getMethod("test", new Object[0]).invoke(c.newInstance(), new Object[0])).

When should you use Varargs?

If you're designing your own APIs - use it sparingly.
Only when the benefit is compelling. Don't overload a varargs method.
In clients, when the API supports them: reflection, message formatting, printf

Static Imports

Clients must qualify static members with class name (Math.PI). To avoid this, some programmers put constants in an interface and implement it. BAD - "Constant Interface Antipattern". They've made this mistake in the JDK - java.util.jar has this pattern. Static import allows unqualified access to static member w/o extending a type. All static fields, methods, etc. will be available for your class using static imports. For example:

import static java.lang.Math.*;
r = cos(PI * theta);

When should you use Static Imports?

Very sparingly - overuse makes programs unreadable.
Only use it when tempted to abuse inheritence.

Metadata

Decorates programs with additional information. Annotations don't directly affect program semantics. They *can* affect treatment by tools and libraries. Can be read from: source, class files, or reflectively. Ad hoc examples: transient, @deprecated. Tiger provides a general purpose metadata facility.

Why Metadata?

Many APIs require a fair amount of boilerplate - i.e. JAX-RPC.
Many APIs require "side files" to be maintained. Examples: BeanInfo class, deployment descriptor.
Many APIs use naming patterns, i.e. JUnit.

Metadata encourages a declarative programming style - tell a computer what to do, now how to do it. Annotation Type Declarations are similar to interface declarations. Special kinds of annotations include Marker annotations and Single-element annotations. The main reason for annotations is for tools providers.

Neal thought that JDK 5 Beta 3 or Release Candidate was available at http://java.sun.com/j2se/1.5.0, but it looks like Beta 2 is the latest release. The fact that he said that implies that a new release should be available shortly. Neal also mentioned that JDK 5 (final) would be shipping soon.

Random fact: Google uses a lot of Java - entire Ads front-end is done in Java.

This was a great talk about all the new features of JDK 5 - I can't wait to start using them. It might be awhile before I can convert AppFuse to JSP 2.0 and JDK 5, but it'll be a good day when I can write my apps using these technologies. Tonight was the best overview of JDK 5 that I've seen so far - in print or person.

Update: Presentations PDFs have been published: Programming Puzzles and Taming the Tiger.

Make your JUnit Tests run faster when using Spring

2004-04-08T01:22:14-06:00

JRoller is down, and has been down for an hour or so - so I've decided to post this Spring Live entry here.

I discovered an interesting thing today about Spring and my JUnit tests. I noticed that the VelocityEngine I was setting on my PositionManager was getting initialized once for each test* method in my Test. This means that since my PositionManagerTest has 10 test methods - it would load the context 10 times.

Loading the context so many times was because the following code was in my Test's parent's constructor:

    ctx = new ClassPathXmlApplicationContext("/applicationContext.xml");

I suppose I expected any constructor-iniatialized variables to be initialized once and only once. So I figured out a solution to make my JUnit tests run faster. By making the ctx variable static, and loading the file in the member variables definition, I greatly reduced the amount of time needed to run tests. Below is the new code I'm using:

    protected static ApplicationContext ctx = 
    	new ClassPathXmlApplicationContext("/applicationContext.xml");

By doing this, the ApplicationContext is only set once, and my tests run much faster. Here's some performance comparisons from Struts Resume:

Average time to run "ant test-dao": 36 seconds
Average time to run "ant test-dao" after this change: 26 seconds

A 10 second improvement - that's crazy talk dontcha think?! I've tried it on single tests, as well as suites - and it seems to improve performance by approximately 30% across the board.

Because of this experience, I have to recommend that when you write JUnit tests that use Spring - you should initialize your ApplicationContext in a static member variable. It seems to be the best performing and logical choice. Of course, if I'm off my rocker - please let me know.

On a sidenote, it would be cool if Roller allowed me to turn off comments for a single post. I like how Simon posts stuff on java.net and then aggregates it to his personal weblog.