Raible Designs

Choosing an Ajax Framework

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

1. Choose a short list of frameworks to prototype with.
2. Create an application prototype with each framework.
3. Document findings and create a matrix with important criteria.
4. Create presentation to summarize document.
5. Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Our strategy for filling in this matrix:

• Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
• We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

• Quality of Documentation/Tutorials/Self Help
• Browser support (most important browsers/versions based on web stats)
• Testability (esp. Selenium compatibility)
• Licensing
• Project health/adoption
• Performance
• Scalability
• Flexibility/extensibility
• Productivity (app dev, web dev)
• Richness of widget/component library
• Charting capability
• Ability to create new widgets
• Match to existing Java team skill-set
• Ease of deployment (on Ops, QA, Users)
• Degree of risk generally
• Ability to integrate with existing site (which includes Prototype)
• Easy to style with CSS
• Validation (esp. marking form elements invalid)
• Component Theme-ing/Decoration
• CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.

AppFuse Light converted to Maven modules, upgraded to Tapestry 5 and Stripes 1.5

This past week, I stayed up a couple of late nights to do some of the AppFuse Light work I wrote about in October. I converted all web frameworks to Maven modules, as well as made them inherit from the appfuse-web project. Below is what the new module structure looks like:

At this point, the project is ready to import into AppFuse's SVN project. Here's a list of other changes I made:

• Modules now depend on AppFuse's backend and allow you to use Hibernate, JPA or iBATIS as the persistence framework. Implementations for Spring JDBC, OJB and JDO have been removed.
• Upgraded to JWebUnit 2.1, which now uses HtmlUnit under the hood and has much better JavaScript support. It also has Selenium support, but I've yet to try it.
• Ajaxified Body integrated into all frameworks. You can easily turn it off by modifying the global.js file.
• Prototype and Scriptaculous loaded from Google's Ajax Libraries CDN.
• Upgraded to Tapestry 5. Mad props to Serge Eby and his tapestry5-appfuse project for showing me how to do this. Serge became a committer on AppFuse recently, so hopefully we'll continue to see great things from the Tapestry 5 support. I really like the clean URLs and minimum configuration required in Tapestry 5. It's testing framework is nice too, but I believe it could be improved.
• Upgraded to Stripes 1.5. This was easy and painless. I'm definitely a fan of Stripes and look forward to reading the Stripes book on my bookshelf.
• Dropped support for: Struts 1.x, WebWork, Spring MVC + Velocity.

If you want to try any of these applications, you can create archetypes using the following commands:

svn co https://appfuse-light.dev.java.net/svn/appfuse-light/trunk appfuse-light
cd appfuse-light/preferred-web-framework
mvn archetype:create-from-project
cd target/generated-sources/archetype
mvn install
cd ~/dev
mvn archetype:generate # The new archetype should show up as an option

Next steps include figuring out a way to flatten the inherited dependencies and plugins so archetype:create-from-project can create truly standalone projects. Please let me know if you have any questions.

AppFuse Light » AppFuse, Maven Archetypes and Shared Web Assets

Last night, I stayed up into the wee hours of the morning working on something I've been wanting to do for a long time. It wasn't until I was trouncing around the woods in Montana that I realized how easy it would be. The something I've wanted to do was to modify AppFuse Light to use AppFuse's core modules (service and dao). It only took me a few hours to make it happen and it inspired additional ideas.

I believe the major mistake we made in AppFuse 2.x was making it easy for user's to upgrade their applications. We currently use the maven-war-plugin and our own maven-warpath-plugin to make it possible to include AppFuse classes and assets in your project. You can easily start a new project w/o having a whole bunch of files in your project. The problem is, you can't easily use "mvn jetty:run" to work on your project. Of course, you can use "mvn appfuse:full-source" to solve this, but I'm starting to think more and more that "full-source" should be the default. This is what we did in 1.x and it seems to be the more natural pattern for folks using AppFuse.

That hard part about moving to "full-source" by default is coming up with a way to share common assets and classes among wars and war projects. Sure, I can copy all the shared images, css and js into each project - but that could become a maintenance nightmare. Subversion 1.5 with relative svn:externals might solve this, but it still seems kinda hacky. I don't want to use the maven-war-plugin because the overlay is kinda hokey and I think it's easier for users to understand when everything is in their project. AppFuse's current directory structure in SVN looks as follows. I've added indicators of what is in each directory.

Rather than using AppFuse's current (manual) archetype-creation process, I'd like to move to a more automated creation process using the maven-archetype-plugin's create-from-project feature. I'd like to figure out a way where I can have the source code and assets from web/common included in each of the other web/* projects (both when using "jetty:run" and "archetype:create-from-project"). One idea I thought of is to make Jetty/Maven aware of multiple src/war directories for "jetty:run" and "package" and then somehow hook into the archetype plugin at creation time to pull in the shared resources. I don't know if something like this is possible. If you know of a good solution to this shared web assets issue, I'd love to hear about it.

Back to AppFuse Light. If I can figure out how to solve shared resources in web modules, I can use this in AppFuse Light to move to a modular SVN structure vs. its current "use Ant to create different combinations" setup. If a modular structure (like appfuse/web/*) is possible for AppFuse Light, I believe it makes sense to move its source into AppFuse's SVN repository. Below is how the directory structure might look after this move.

With this addition and "archetype:create-from-project", we should be able to create all the basic and light archetypes automatically. We'll probably still need a manual archetype-creation process for modular archetypes, but I'm OK with that.

The last thing I'm struggling with is figuring out the best way to create archetypes for something like AppFuse. In the past, we've used dependencies to allow users to inherit dependencies and their versions. This works, but it results in a lot of duplicate XML (in projects and archetypes) for developers. Last night, I tried using a parent project instead of dependencies and it seems to work much better. Not only do you inherit dependencies, but you also inherit plugins, profiles and properties. If you inherit, you can override, which is slick.

If you're an AppFuse user, how would you feel about having an AppFuse module as your project's parent? Would you prefer that, dependencies on AppFuse or full-source with no dependencies on AppFuse? Regardless of parent vs. dependencies, I think running "appfuse:full-source" should allow you to de-couple your project from AppFuse.

Taking Apache Camel for a Ride with Bruce Snyder

Camel is a Java API that allows you to do message routing very easily. It implements many of the patterns found in Enterprise Integration Patterns. It doesn't require a container and can be run in any Java-based environment. Camel has a whole bunch of components - Bruce is showing a 6 x 10 grid with a component name in each grid. In other words, there's 60 components that Camel can use. Examples include: ActiveMQ, SQL, Velocity, File and iBATIS.

Chris Richardson asks "What's left inside of ServiceMix". Why use ServiceMix if you have Camel? ServiceMix is a container that can run standalone or inside an app server. You can run distributed ServiceMix as a federated ESB. Camel is much smaller and lightweight and is really just a Java API. ServiceMix 4 changed from a JBI-based architecture to OSGi (based on Apache Felix). They also expect to create your routes for ServiceMix 4 with Camel instead of XML. To process messages, you can use many different languages: BeanShell, JavaScript, Groovy, Python, PHP, Ruby, JSP EL, OGNL, SQL, XPath and XQuery.

Camel has a CamelContext that's similar to Spring's ApplicationContext. You can initialize it in Java and add your routes to it:

CamelContext context = new DefaultCamelContext();
context.addRoutes(new MyRouterBuilder());
context.start();

Or you can initialize it using XML:

<camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
    <package>com.acme.routes</package>
</camelContext>

Camel's RouteBuilder contains a fluid API that allows you to define to/from and other criteria. At this point, Bruce is showing a number of examples using the Java API. He's showing a Content Based Router, a Message Filter, a Splitter, an Aggregator, a Message Translator, a Resequencer, a Throttler and a Delayer.

Bruce spent the last 10 minutes doing a demo using Eclipse, m2eclipse, the camel-maven-plugin and ActiveMQ. It's funny to see a command-line guy like Bruce say he can't live w/o m2eclipse. I guess Maven's XML isn't so great after all.

Camel is built on top of Spring and has good integration. Apparently, the Camel developers tried to get it added to Spring, but the SpringSource guys didn't want it. Coincidentally, Spring Integration was released about a year later.

Camel also allows you to use "beans" and bind them to Camel Endpoints with annotations. For example:

public class Foo {

    @MessageDriven (uri="activemq:cheese")
    public void onCheese(String name) {
        ...
    }
}

Other annotations include @XPath, @Header and @EndpointInject.

Camel can also be used for BAM (Business Activity Monitoring). Rather than using RouteBuilder, you can use ActivityBuilder to listen for activities and create event notifications.

Bruce had quite a few folks show up for this presentation. I had trouble finding a seat because I was late. I think he did a good job of showing what Camel is and how you might use it.

Building Rich Applications with Appcelerator

This afternoon, I delivered my Building Rich Applications with Appcelerator talk for the 3rd time at Colorado Software Summit. When I first proposed this topic, I hadn't used Appcelerator and saw this as a good opportunity to learn more about it. I'm glad I did.

IMO, Appcelerator is a lot like Dojo in how it parses pages and turns HTML with special attributes into JavaScript widgets. I can't help but think a pre-compilation step would be nice to speed things up. I like Appcelerator's extensive Widget Library, and I especially like that they re-use many widgets rather than re-creating their own. Finally, I really dig the "SOA in a browser" approach where everything is a message and you can easily publish and subscribe to events - on the client and server. Below is my presentation, please let me know if you have any questions.

Applying Flash to Java: Flex and OpenLaszlo with Dustin Marx

If you're going to choose Flex or OpenLaszlo, chances are you're targeting Flash. The Flash Player allows you to abstract the browser idiosyncrasies and give users a better experience. It hides the browser quirks from both developers and users. It's also a highly ubiquitous web browser runtime environment. It provides a user experience way beyond traditional HTTP request-response. Also, its visual effects and "richness" rival of non-browser desktop applications.

Mark's blog stats show that 95% of readers are using Flash 9 and 1% is Flash 10. All others don't have their version exposed. Most other sources claim that Flash 9 has 98% penetration in mature markets. One of the nicest things about Flash is it frees users from limitations of traditional web applications. Request/response is largely a thing of the past and Ajax-like behavior was built-in from the beginning.

Flash Criticisms: has reduced SEO as well as bookmarking and URL history support.

Flash Player Criticisms: not open source, no significant alternative, no 64-bit player, loading performance and it's only useful for games, movies and annoying advertisements.

The good news is Flash is getting better. There's currently a SWF Searchability initiative with Google and Yahoo!. For bookmarking and URL History, you can use "deep linking" with mx.managers.BrowserManager and mx.managers.HistoryManager. Flash Player issues are also being addressed. There's a Flash Open Screen Project, there's a 64-bit player in Adobe Labs and Flash 9/10 is much better than in the past.

To prove that Flash is a compelling technology, all you have to do is look at Microsoft's Silverlight and Sun's JavaFX. These are direct competitors that are fairly new and prove that companies like what Flash has.

Flex
Flex is not Flash. Flex requires Flash (9+), but Flash does not require Flex. Flex is a language and framework and applications are compiled into .swf files. Flash is the runtime environment that executes .swf files. Flex 3 is FREE. The Flex SDK, compiler and debugger are open source. They have no license costs. BlazeDS is also open source and has no license cost.

Flex MXML was formerly called Macromedia XML. It's an XML-based presentation/layout language that's editable with any text editor or IDE. MXML is to ActionScript 3 as JSP is to Java. MXML provides the layout and ActionScript provides the dynamic business logic.

ActionScript
ActionScript is an ECMAScript implementation that's been proposed as the Edition 4 implementation. It's not your older sibling's JavaScript. It uses class-based object-oriented features and static typing. Thanks to the Flash Player, it works the same across multiple browsers. Most of the things you can do with MXML, you can do with ActionScript. However, MXML typically requires less LOC.

ActionScript allows packages, interfaces, inheritance, objects and methods. It includes extensive XML Support, particularly E4X (ECMAScript for XML). It has a large class library and can talk directly to the Flash Player. Finally, it has many Java-like features and it's syntax looks similar.

At this point, Dustin started doing demos of two popular Flex components: RichText Editor (which creates horrible HTML) and Data Grid. More third-party Flex components seem to appear every day. Examples include flexlib, ASDIA as well as all those listed on FlexBox. Additionally, it's not too difficult to create your own Flex components (no proof provided).

Flex's property binding is one of Dustin's favorite features, but he says he has a difficult time conveying how cool it is. Property binding allows you to tie data in one object to data in another object. Updates in one object affect another. To use it in MXML, you can use curly braces or the tag. In ActionScript, you can use the BindingUtils object. You can also use Flex's metadata annotation "Bindable" to denote bindable objects.

To compile Flex applications, you can use mxmlc from the command line. You can use Ant with the <exec> task or using Flex's Ant Tasks. You can also invoke mxmlc from Java as long as you include mxmlc.jar in your classpath. FlexBuilder is an Eclipse-based IDE that's not included with the free Flex SDK. It's not a free product and can be used as a plugin or a standalone IDE.

Flex and Java
There's two predominant out-of-the-box methods for Flex to communicate with Java EE backends.

• HttpService: traditional HTTP request/response.
• WebService: SOAP-based Web Services.

If the above two methods aren't fast enough, you can use BlaseDS and it's additional options.

• Web Messaging: HTTP publish/subscribe with JMS, ColdFusion and/or other Flash/Ajax client.
• Remoting with AMF: access server-side objects from Flash client-tier as if they were hosted there.

Flash applications can access either a client's machine or a remote site, but not both. You can use a crossdomain.xml file on your server to allow remote Flash clients to connect. This file allows access for both Flex and OpenLaszlo applications.

BlazeDS adds proxy server support for HTTPService and WebService. To use, set the useProxy attribute to "true". Features include authentication and logging. It adds a new RPC mechanisms called RemoteObject. This object allows ActionScript and Java EE transparent integration. It also adds publish/subscribe messaging with a JMS Adapter available. Lastly, it adds Ajax support to your Flex application.

GraniteDS is an open-source (LGPL) alternative to Adobe LiveCycle and is similar to BlazeDS. It has COMET-like functionality and supports Spring, Spring Security, EJB 3, Seam and Guice.

Flex Frameworks: Cairngorm (Adobe Consulting), Pure MVC, Mate and many others.

Flex 4 (Gumbo) will have improvements for designers (easier customization, better tool support), improvements for developers (faster compiler, two-way data binding) and will leverage new features of Flash 10.

OpenLazlo
OpenLaszlo 4 is XML-based and uses an XPath syntax for data access. OpenLaszlo was actually created before Flex and is ECMAScript-based. Unlike Flex that requires Flash 9, OpenLaszlo is architected to deploy on different runtime environments, including: Flash 7/8/9 and DHTML. Dustin believes OpenLaszlo would be a lot more appealing if your source code could be compiled into Silverlight or JavaFX.

OpenLaszlo's syntax looks a lot like Flex, except it does not use namespaces. OpenLaszlo's Constraints are similar to Flex's property binding, except the syntax is a bit different. You use ${} in LZX tags or applyConstraint() and LzDelegate in scripts. OpenLaszlo's event handling is similar to Flex and JavaScript event handling. All attributes have an implicit "on" event that is triggered when an attribute's value is changed. Event handlers can be written CSS-style in LZX nodes or using the <handler></handler> tag. Script code can be embedded in LZX XML in many ways:

• Inside event attributes
• Within <script></script> tags
• Within <method></method> tags
• Within <handler></handler> tags
• In a separate file (<script src="someFile.js" />)

Dustin believes the debugger and view-source tools in OpenLaszlo are much better than the ones available for Flex.

For the rest of the presentation, Dustin covered many of Laszlo's feature, how they relate to Java as well as how to integrate SWF and HTML. SWFObject is Dustin's preferred method for adding Flash to HTML. One of its nifty features is it allows SWF-to-SWF communication.

This talk was an excellent and in-depth overview of Flash, Flex and OpenLaszlo. I especially liked all the details on ActionScript and the different methods for remote communication. Nice job Dustin!

Comprehensive Project Intelligence with Jason van Zyl

In this talk, Jason is going to talk about m2eclipse, Nexus, Hudson and Maven. On his Maven bullet-point, it says "The best is yet to come (and we'll fix a bunch of stuff)!"

m2eclipse
The m2eclipse plugin has improved greatly in the last 4 months - there's now 5 full-time developers working on it. If you use the m2eclipse plugin, you never have to leave the IDE for your Maven-related work. m2eclipse has a Configuration Framework that turns Maven's mumbo-jumbo (Jason's words, not mine) into Eclipse talk. The m2eclipse+configuration framework has integration for WTP, JDT, AJDT and they're working on one for Flex. Below is a screenshot of how m2eclipse helps developers stay away from using command-line Maven.

Now Jason is showing a demo of m2eclipse and creating a new Maven project from existing archetypes. It looks like m2eclipse uses "Nexus Indexer" as its Catalog. Presumably this is a Sonatype-hosted service. The Nexus Indexer contains an of Maven Central and is very fast. It's dynamically updated as new things are deployed to Maven Central.

If you use m2eclipse and open a pom.xml, you'll get a visual view rather than an XML view. This UI has tabs for Overview, Dependencies, Repositories, Build, Plugins, Reporting, Profiles, Team, a Dependency Hierarchy and Dependency Graph. You can easily add new dependencies and it finds things quickly because it's using the Nexus index. In addition to visually adding dependencies, you can modify the raw XML and get things like groupId and version code-completion.

Once you have your dependencies listed in a "Maven Dependencies" container to you can "Materialize Project" to create a project from the binary dependency. You'll get the source as a new project in your workspace as well as having your binary dependency turned into a source dependency.

You can easily create a run configuration that runs certain goals, allows you to activate profiles and uses an embedded version of Maven or an external installation. I asked Jason if the Dependency Hierarchy had a right-click -> exclude feature and he said it doesn't exist yet, but it will in the release after next. For now, the pom editor is just eye candy and doesn't have actions.

For Maven Plugins, m2eclipse has workspace resolution so you can develop a plugin and use it in a project at the same time w/o having to install the plugin over and over.

Sonatype has created a Project Materializer Plugin that allows a team lead to create a project for developers. It allows you to create a welcome page that has links, cheat sheets, News and Updates and Tasks for the developer. It also materializes Eclipse projects in the background. Cheat Sheets are a series of tasks that can be run to show developers how to do things.

Another big feature in m2eclipse is nested project support. It only works in Eclipse 3.4 though.

Nexus
Nexus is a repository manager that allows you to keep the cruft from the outside world out of your system. It's primarily for Repository and Configuration Management. It has fine-grained security for authentication and authorization. One nice feature of its security system is you can prevent certain users from seeing source JARs. It also has virtual repositories (a savior for OSGi lovers). UI is written in Ext JS and acts as a simple REST client for Nexus. It has a full REST API using RESTlet.

A repository manager allows you to protect yourself from the Open Source Ghetto. The OS Maven Ghetto has bad POMs, repositories in POMs, mixed snapshot and release dependencies and screwed up metadata. Not only does it offer protection, but it allows you to aggregate repositories and publish your internal artifacts to it. It also allows you to schedule tasks that clean out snapshots so your repositories don't grow out of control.

Typically people deal with OSGi runtimes manually. OSGi can dynamically update dependencies that you drop into your bundle repository. However, many folks maintain their OSGi runtime and bundle repository locally. Some people are trying to get an OSGi runtime to resolve against a P2 repository. P2 is what Eclipse uses for their repository management. Nexus has the ability to lock down the versions that are available to an OSGi runtime. Furthermore, you can use Nexus to manage the versions that get deployed to all your servers. This makes it a lot easier for QA and Production to manage versions of your artifacts. OSGi is great for modularity and solving classpath issues, but it does have issues with versions and how its ranges work.

You can see Nexus in action at http://repository.sonatype.org. It can be configured entirely through the UI, an XML file or through the REST API. RSS feeds exist for configuration and repository updates.

Nexus is free and open source with a GPL license. The next version (1.2) will contain a Plugin API to allow extensions. All of Sonatype's enhancements for its commercial version will be written as plugins. A matrix of what's available in the open source version vs. commercial version should be published sometime next week.

Hudson
Jason believes that Hudson is the future of continuous integration, on-demand results and release management. They're writing all their extension points in Hudson as Maven plugins and Plexus components (with the work they've done, using Spring components should also be possible). Other enhancements they've made to Hudson:

• Integration of JSecurity
• Implementing a similar REST layer as Nexus and creating a UI using Ext JS
• Automatic installation of external Maven installations
• Drools Workflow Integration

They've also enhanced Hudson so it can easily test/publish Maven projects without using the free-form project template. Hudson works well for doing Eclipse headless builds for Eclipse plugins. If you need to test against multiple databases, multiple OS's, it does support a grid-based system that's easy to setup. Hudson does have web services integration that allows you to kick off builds from within Eclipse. Sonatype uses Hudson to run all their nightly builds of Maven.

Maven - the best is yet to come
The three big things coming in the next version of Maven are:

• Refactored Project Builder: includes a spec for building a pom, domain-specific parsers (attribute-based XML, Groovy and Ruby) and mixins.
• Mercury: a new repository and transport layer. Developed by the Jetty people and is super fast (async client with connection pooling and parallelization). Has atomic downloads and deployments (with Nexus), full PGP support and a WebDAV client built-in.
• Maven Embedder: re-written to actually work.

Overall, a good talk with lots of demos. I'm definitely looking forward to Maven improvements in the future.

Building LinkedIn's Next Generation Architecture with OSGi by Yan Pujante

This week, I'm attending the Colorado Software Summit in Keystone, Colorado. Below are my notes from an OSGi at LinkedIn presentation I attended by Yan Pujante.

LinkedIn was created in March of 2003. Today there's close to 30M members. In the first 6 months, there was 60,000 members that signed up. Now, 1 million sign up every 2-3 weeks. LinkedIn is profitable with 5 revenue lines and there's 400 employees in Mountain View.

Technologies: 2 datacenters (~600 machines). SOA with Java, Tomcat, Spring Framework, Oracle, MySQL, Servlets, JSP, Cloud/Graph, OSGi. Development is done on Mac OS X, production is on Solaris.

The biggest challenges for LinkedIn are:

• Growing Engineering Team on a monolithic code base (it's modular, but only one source tree)
• Growing Product Team wanting more and more features faster
• Growing Operations Team deploying more and more servers

Front-end has many BL services in one webapp (in Tomcat). The backend is many wars in 5 containers (in Jetty) with 1 service per WAR. Production and Development environments are very different. Total services in backend is close to 100, front-end has 30-40.

Container Challenges
1 WAR with N services does not scale for developers (conflicts, monolithic). N wars with 1 service does not scale for containers (no shared JARs). You can add containers, but there's only 12GB of RAM available.

Upgrading back-end service to new version requires downtime (hardware load-balancer does not account for version). Upgrading front-end service to new version requires redeploy. Adding new backend services is also painful because there's lots of configuration (load-balancer, IPs, etc.).

Is there a solution to all these issues? Yan believes that OSGi is a good solution. OSGi stands for Open Services Gateway initiative. Today that term doesn't really mean anything. Today it's a spec with several implementations: Equinox (Eclipse), Knoplerfish and Felix (Apache).

OSGi has some really, really good features. These include smart class loading (multiple versions of JARs is OK), it's highly dynamic (deploy/undeploy built-in), it has a service registry and is highly extensible/configurable. An OSGi bundle is simply a JAR file with an OSGi manifest.

In LinkedIn's current architecture, services are exported with Spring/RPC and services in same WAR can see each other. The problem with this architecture comes to light when you want to move services to a 2nd web container. You cannot share JARs and can't talk directly to the other web app. With OSGi, the bundles (JARs) are shared, services are shared and bundles can be dynamically replaced. OSGi solves the container challenge.

One thing missing from OSGi is allowing services to live on multiple containers. To solve this, LinkedIn has developed a way to have Distributed OSGi. Multicast is used to see what's going on in other containers. Remote servers use the OSGi lifecycle and create dynamic proxies to export services using Spring RPC over HTTP. Then this service is registered in the service registry on the local server.

With Distributed OSGi, there's no more N-1 / 1-N problem. Libraries and services can be shared in one container (memory footprint is much smaller). Services can be shared across containers. The location of the services is transparent to the clients. There's no more configuration to change when adding/removing/moving services. This architecture allows the software to be the load balancer instead of using a hardware load balancer.

Unfortunately, everything is not perfect. OSGi has quite a few problems. OSGi is great, but the tooling is not quite there yet. Not every library is a bundle and many JARs doesn't have OSGi manifests. OSGi was designed for embedded devices and using it for the server-side is very recent (but very active).

OSGi is pretty low-level, but there is some work being done to hide the complexity. Spring DM helps, as do vendor containers. SpringSource has Spring dm Server, Sun has GlassFish, and Paremus has Infiniflow. OSGi is container centric, but next version will add distributed OSGi, but will have no support for load-balancing.

Another big OSGi issue is version management. If you specify version=1.0.0, it means [1.0.0, ∞]. You should at least use version=[1.0.0,2.0.0]. When using OSGi, you have to be careful and follow something similar to Apache's APR Project Versioning Guidelines so that you can easily identify release compatibility.

At LinkedIn, the OSGi implementation is progressing, but there's still a lot of work to do. First of all, a bundle repository needs to be created. Ivy and bnd is used to generate bundles. Containers are being evaluated and Infiniflow is most likely the one that will be used. LinkedIn Spring (an enhanced version of Spring) and SCA will be used to deploy composites. Work on load-balancing and distribution is in progress as is work on tooling and build integration (Sigil from Paremus).

In conclusion, LinkedIn will definitely use OSGi but they'll do their best to hide the complexity from the build system and from developers. For more information on OSGi at LinkedIn, stay tuned to the LinkedIn Engineering Blog. Yan has promised to blog about some of the challenges LinkedIn has faced and how he's fixed them.

Update: Yan has posted his presentation on the LinkedIn Engineering Blog.

Appcelerator with Matt Quinlan

This evening I attended the Denver Open Source User Group meeting where the Basic Concepts talk was on Appcelerator. Matt Quinlan (Twitter, Blog, LinkedIn) was the presenter. I arrived 10 minutes late, so I didn't hear any of the intro stuff. Below are my notes from the event.

The Appcelerator developers liked the "onclick" syntax, but found it was too limited to do everything they wanted. Rather than onclick, they use an on attribute. For example:

<div on="click then l:show.box">
Click me
<div>

"DOM Manipulation is JavaScript cruft."

Appcelerator allows you to implement the Observer pattern in the browser. In addition to allowing DOM elements to subscribe to messages, server-side objects (in any language) can subscribe as well. In Appcelerator syntax "l:" means local and "r:" means remote. The messages that are passed to the server are JSON and have payloads. JSON is more popular than XML because you can eval() it and create JavaScript objects from it. Appcelerator allows you to do Declarative Ajax. On the server-side, you can annotation Java and C# classes with @Service to subscribe to messages. In other languages (i.e. PHP), Javadoc-style comments are used.

Tagline: The seamless fusion of RIA and SOA.

Web Expression Language
Goal is to eliminate 90% of the JavaScript you write. Example syntax:

on="[condition] then [action]"

Conditions include DOM events (click, focus, blur, change, mouse events), key events (up, down, press), other (history, drag/drop, selected, resize, iPhone orient, sortXYZ), subscribe to custom message. Actions include Scriptaculous effects (show/hide, fade, move, drop, grow), set element value (static, dynamic, bind), set CSS class or attribute, execute custom JavaScript, publish custom message.

Now Matt is doing a demo on http://try.appcelerator.org. This site consists of a form that allows you to type in Appcelerator code and run it. 3 attributes can be added to any tag: draggable, droppable and resizeable.

Client-Side
Add simple tags to your HTML to inject RIA widgets. Add single property to existing HTML elements for dynamic behavior. Eclipse Plugin built on Aptana, but is generally targeted towards web developers moreso than business analysts (no drag and drop of widgets).

Server-Side
Server-side development done with your IDE of choices. Based on your server-side tehchnology platform. Easily create services using annotations.

App Command
The app command is similar to Rails' GEM command. Allows you to build scaffolding and deploy to cloud (AppEngine, Amazon S3). It also allows you to pulldown components from the main server and auto-updating.

Examples:

• appTunes: widget that wraps Flex Cover Flow widget
• Snapshot: photo editing application
• Radio Javan: Persian music online
• SkyBlox: wireless company

Appcelerator allows you to create prototypes easily by using a JavaScript file with mocks for the server-side objects. In the next version, you can "annotate" the UI and allow end-users to Ctrl+Click on elements and add feedback. For an example of this, see http://dev.appcelerator.com/pearson.

When starting with Appcelerator, you can start by crawling (including appcelerator.js for widgets) then move to walking (decouple server-side and client-side) and finally running (developing working prototypes with mocks for server-side).

"Let's face it, ASP, JSP, PHP and Ruby are just lipstick on CGI."

Maven Plugin for Running Integration Tests against Multiple Containers

Don Brown has created a pretty cool Maven plugin that allows you to run integration tests on multiple containers in one go. I learned about it on the Struts 2 Dev List:

I've started adding functional tests to Struts 2 by adding a few to the REST showcase application, running against Tomcat 5.x, Jetty 6.x, JBoss 4.2.x, and Resin 3.x. The magic happens through a new Maven 2 plugin I developed called maven-itblast-plugin, which enables multiple integration test runs against multiple containers in one go. For more info, see http://github.com/mrdon/maven-itblast-plugin/wikis/home.

You might notice that this plugin is hosted on GitHub. If you want to learn how to use GitHub with Maven, you might want to see Don's entry on Maven-enabled project hosting with GitHub. Thanks Don - good stuff.

Update: Don has posted more information about this plugin on his blog.