Search for [param] in weblog rd

Testing Angular 2.0 RC1 Applications

2016-06-06T09:57:13-06:00

As mentioned on Friday, there's been quite a bit that's changed with Angular 2 between its Beta 9 and RC 1 releases. This article is an update to the Testing Angular 2 Applications I wrote in March. That tutorial was based on Angular 2.0 Beta 9. Rather than simply updating that tutorial and blog post for 2.0 RC1, I decided to create a new version for posterity's sake. The 2.0 Beta 9 version will remain on my blog and I've tagged the source on GitHub.

If you've already read the first version of Testing Angular 2 Applications, checkout the diff of the Asciidoctor version to see what's changed.

What you'll build

You'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. See Angular 2's guide to unit testing if you'd like more information on testing and why it's important.

The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your codebase grows.

What you'll need

About 15-30 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and npm installed. I recommend using nvm.

Get the tutorial project

Clone the angular2-tutorial repository, checkout the testing-start branch, and install its dependencies.

git clone https://github.com/mraible/angular2-tutorial.git
cd angular2-tutorial
git checkout testing-start
npm install

If you haven't completed the Getting Started with Angular 2.0 RC1 tutorial, you should peruse it so you understand how this application works. You can also simply start the app with npm start and view it in your browser at http://localhost:5555/.

Unit test the SearchService

Create src/client/app/shared/search/search.service.spec.ts and setup the test's infrastructure using MockBackend and BaseRequestOptions.

import {
  beforeEachProviders,
  it,
  describe,
  expect,
  inject,
  fakeAsync,
  tick
} from '@angular/core/testing';
import { MockBackend } from '@angular/http/testing';
import { provide } from '@angular/core';
import { Http, ConnectionBackend, BaseRequestOptions, Response, ResponseOptions } from '@angular/http';
import { SearchService } from './search.service';

export function main() {
  describe('Search Service', () => {
    beforeEachProviders(() => {
      return [BaseRequestOptions, MockBackend, SearchService,
        provide(Http, {
          useFactory: (backend:ConnectionBackend, defaultOptions:BaseRequestOptions) => {
            return new Http(backend, defaultOptions);
          }, deps: [MockBackend, BaseRequestOptions]
        }),
      ];
    });
  });
}

If you run npm test, all tests will pass, but you don't see "Search Service" as a listed test. You can fix this by adding the first test of getAll(). This test shows how MockBackend can be used to mock results and set the response.

TIP: When you are testing code that returns either a Promise or an RxJS Observable, you can use the fakeAsync helper to test that code as if it were synchronous. Promises are be fulfilled and Observables are notified immediately after you call tick().

The test below should be on the same level as beforeEachProviders.

it('should retrieve all search results',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res:Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.getAll().subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

Running npm test should result in "12 tests completed". Add a couple more tests for filtering by search term and fetching by id.

it('should filter by search term',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('john').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

it('should fetch by id',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"id": 1, "name": "John Elway"}, {"id": 2, "name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('2').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('Gary Kubiak');
  }))
);

If you want to have tests continually run as you add them, you can run the following commands in separate shell windows.

npm run build.test.watch
npm run karma.start

NOTE: See Running Unit Tests on Karma to learn how to run your tests from IntelliJ IDEA.

Unit test the SearchComponent

To unit test the SearchComponent, create a MockSearchProvider that has spies. These allow you to spy on functions to check if they were called.

Create src/client/app/shared/search/mocks/search.service.ts and populate it with spies for each method, as well as methods to set the response and subscribe to results.

import { provide } from '@angular/core';
import { SpyObject } from './helper';

import { SearchService } from '../search.service';
import Spy = jasmine.Spy;

export class MockSearchService extends SpyObject {
  getAllSpy:Spy;
  getByIdSpy:Spy;
  searchSpy:Spy;
  saveSpy:Spy;
  fakeResponse:any;

  constructor() {
    super(SearchService);

    this.fakeResponse = null;
    this.getAllSpy = this.spy('getAll').andReturn(this);
    this.getByIdSpy = this.spy('get').andReturn(this);
    this.searchSpy = this.spy('search').andReturn(this);
    this.saveSpy = this.spy('save').andReturn(this);
  }

  subscribe(callback:any) {
    callback(this.fakeResponse);
  }

  setResponse(json:any):void {
    this.fakeResponse = json;
  }

  getProviders():Array<any> {
    return [provide(SearchService, {useValue: this})];
  }
}

In this same directory, create a helper.ts class to implement the SpyObject that MockSearchService extends.

import {StringMapWrapper} from '@angular/core/src/facade/collection';

export interface GuinessCompatibleSpy extends jasmine.Spy {
  /** By chaining the spy with and.returnValue, all calls to the function will return a specific
   * value. */
  andReturn(val: any): void;
  /** By chaining the spy with and.callFake, all calls to the spy will delegate to the supplied
   * function. */
  andCallFake(fn: Function): GuinessCompatibleSpy;
  /** removes all recorded calls */
  reset();
}

export class SpyObject {
  static stub(object = null, config = null, overrides = null) {
    if (!(object instanceof SpyObject)) {
      overrides = config;
      config = object;
      object = new SpyObject();
    }

    var m = StringMapWrapper.merge(config, overrides);
    StringMapWrapper.forEach(m, (value, key) => { object.spy(key).andReturn(value); });
    return object;
  }

  constructor(type = null) {
    if (type) {
      for (var prop in type.prototype) {
        var m = null;
        try {
          m = type.prototype[prop];
        } catch (e) {
          // As we are creating spys for abstract classes,
          // these classes might have getters that throw when they are accessed.
          // As we are only auto creating spys for methods, this
          // should not matter.
        }
        if (typeof m === 'function') {
          this.spy(prop);
        }
      }
    }
  }

  spy(name) {
    if (!this[name]) {
      this[name] = this._createGuinnessCompatibleSpy(name);
    }
    return this[name];
  }

  prop(name, value) { this[name] = value; }

  /** @internal */
  _createGuinnessCompatibleSpy(name): GuinessCompatibleSpy {
    var newSpy: GuinessCompatibleSpy = <any>jasmine.createSpy(name);
    newSpy.andCallFake = <any>newSpy.and.callFake;
    newSpy.andReturn = <any>newSpy.and.returnValue;
    newSpy.reset = <any>newSpy.calls.reset;
    // revisit return null here (previously needed for rtts_assert).
    newSpy.and.returnValue(null);
    return newSpy;
  }
}

Alongside, create routes.ts to mock Angular's RouteSegment and passing parameters between components.

import { RouteSegment } from '@angular/router';

export class MockRouteSegment implements RouteSegment {
  urlSegments:any;
  parameters:any;
  outlet:string;
  _type:any;
  _componentFactory:any;
  type:any;
  stringifiedUrlSegments:string;

  constructor(parameters?:{ [key:string]:any; }) {
    this.parameters = parameters;
  }

  getParam(param:string) {
    return this.parameters[param];
  }
}

With mocks in place, you can create a spec for SearchComponent that uses these as providers. Create a file at src/search/components/search.component.spec.ts and populate it with the following code.

import { provide } from '@angular/core';
import { TestComponentBuilder } from '@angular/compiler/testing';
import {
  it,
  describe,
  expect,
  inject,
  beforeEachProviders,
} from '@angular/core/testing';

import { RouteSegment } from '@angular/router';
import { MockRouteSegment } from '../shared/search/mocks/routes';
import { MockSearchService } from '../shared/search/mocks/search.service';

import { SearchComponent } from './search.component';

export function main() {
  describe('Search component', () => {
    var mockSearchService:MockSearchService;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();

      return [
        mockSearchService.getProviders(),
        provide(RouteSegment, { useValue: new MockRouteSegment({ 'term': 'peyton' }) })
      ];
    });
  });
}

Add two tests, one to verify a search term is used when it's set on the component and a second to verify search is called when a term is passed in as a route parameter.

it('should search when a term is set and search() is called', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  return tcb.createAsync(SearchComponent).then((fixture) => {
    let searchComponent = fixture.debugElement.componentInstance;
    searchComponent.query = 'M';
    searchComponent.search();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('M');
  });
}));

it('should search automatically when a term is on the URL', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  return tcb.createAsync(SearchComponent).then((fixture) => {
    fixture.detectChanges();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('peyton');
  });
}));

Add a spec for the EditComponent as well, verifying fetching a single record works. Notice how you can access the component directly with fixture.debugElement.componentInstance, or its rendered version with fixture.debugElement.nativeElement. Create a file at src/search/components/edit.component.spec.ts and populate it with the code below.

import { provide } from '@angular/core';
import { TestComponentBuilder } from '@angular/compiler/testing';
import {
  it,
  describe,
  expect,
  inject,
  beforeEachProviders,
} from '@angular/core/testing';

import { RouteSegment } from '@angular/router';
import { ROUTER_FAKE_PROVIDERS } from '@angular/router/testing';
import { MockRouteSegment } from '../shared/search/mocks/routes';
import { MockSearchService } from '../shared/search/mocks/search.service';

import { EditComponent } from './edit.component';

export function main() {
  describe('Edit component', () => {
    var mockSearchService:MockSearchService;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();

      return [
        mockSearchService.getProviders(),
        ROUTER_FAKE_PROVIDERS,
        provide(RouteSegment, { useValue: new MockRouteSegment({ 'id': '1' }) })
      ];
    });

    it('should fetch a single record', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
      return tcb.createAsync(EditComponent).then((fixture) => {
        let person = {name: 'Emmanuel Sanders', address: {city: 'Denver'}};
        mockSearchService.setResponse(person);

        fixture.detectChanges();
        // verify service was called
        expect(mockSearchService.getByIdSpy).toHaveBeenCalledWith(1);

        // verify data was set on component when initialized
        let editComponent = fixture.debugElement.componentInstance;
        expect(editComponent.editAddress.city).toBe('Denver');

        // verify HTML renders as expected
        var compiled = fixture.debugElement.nativeElement;
        expect(compiled.querySelector('h3')).toHaveText('Emmanuel Sanders');
      });
    }));
  });
}

You should see "✔ 22 tests completed" in the shell window that's running npm run karma.start. If you don't, try cancelling the command and restarting.

Integration test the search UI

To test if the application works end-to-end, you can write tests with Protractor. These are also known as integration tests, since they test the integration between all layers of your application.

To verify end-to-end tests work in the project before you begin, run the following commands in three different console windows.

# npm run webdriver-update <- You will need to run this the first time
npm run webdriver-start
npm run serve.e2e
npm run e2e

You should receive an error stating that the "nav text for About" is incorrect.

This happens because we added a Search link to the navbar and didn't update the test (in app.component.e2e.ts) that looks for the last child.

it('should have correct nav text for About', () => {
    expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('ABOUT');
});

Replace this test with the one below, and add a new one to verify the Search link is last.

it('should have correct nav text for About', () => {
  expect(element(by.css('sd-app sd-navbar nav a:nth-child(2)')).getText()).toEqual('ABOUT');
});

it('should have correct nav text for Search', () => {
  expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('SEARCH');
});

Now when you run npm run e2e, all specs should pass.

Testing the search feature

Create a new search.component.e2e-spec.ts spec in the same directory as your SearchComponent. Add tests to verify elements are rendered correctly and search works. At the time of this writing, Protractor's by.model and by.repeater don't work with Angular 2. For this reason, I used by.css to verify the HTML renders as expected.

describe('Search', () => {

  beforeEach(() => {
    browser.get('/search');
  });

  it('should have an input and search button', () => {
    expect(element(by.css('sd-app sd-search form input')).isPresent()).toEqual(true);
    expect(element(by.css('sd-app sd-search form button')).isPresent()).toEqual(true);
  });

  it('should allow searching', () => {
    let searchButton = element(by.css('button'));
    let searchBox = element(by.css('input'));
    searchBox.sendKeys('M');
    searchButton.click().then(() => {
      // doesn't work as expected - results in 0
      //expect(element.all(by.repeater('person of searchResults')).count()).toEqual(3);
      var list = element.all(by.css('sd-search table tbody tr'));
      expect(list.count()).toBe(3);
    });
  });
});

Testing the edit feature

Create a edit.component.e2e-spec.ts spec to verify the EditComponent renders a person's information and that you can update their information.

describe('Edit', () => {

  beforeEach(() => {
    browser.get('/edit/1');
  });

  let name = element(by.id('name'));
  let street = element(by.id('street'));
  let city = element(by.id('city'));

  it('should allow viewing a person', () => {
    expect(element(by.css('h3')).getText()).toEqual('Peyton Manning');
    expect(name.getAttribute('value')).toEqual('Peyton Manning');
    expect(street.getAttribute('value')).toEqual('1234 Main Street');
    expect(city.getAttribute('value')).toEqual('Greenwood Village');
  });

  it('should allow updating a name', function () {
    let save = element(by.id('save'));
    // send individual characters since sendKeys passes partial values sometimes
    // https://github.com/angular/protractor/issues/698
    ' Won!'.split('').forEach((c) => name.sendKeys(c));
    save.click();
    // verify one element matched this change
    var list = element.all(by.css('sd-search table tbody tr'));
    expect(list.count()).toBe(1);
  });
});

Run npm run e2e to verify all your end-to-end tests pass. You might receive a failure for the "Home" test.

If you do, open src/client/app/+home/home.component.e2e-spec.ts and change line 17 from this:

element(by.css('sd-home form input')).sendKeys('Tim Berners-Lee');

To this:

let input = element(by.css('sd-home form input'));
'Tim Berners-Lee'.split('').forEach((c) => input.sendKeys(c));

Run npm run e2e again. You should see a success message similar to the one below in your terminal window.

If you made it this far and have all 13 specs passing - congratulations! You're well on your way to writing quality code with Angular 2 and verifying it works.

Continuous Integration

The angular2-seed project ships with a .travis.yml that you can use to run continuous integration for this application through Travis CI. To enable builds on Travis CI, login and enable builds for the GitHub repo you created the project in. Then trigger your first build with a git push.

When I first tried this, I received a failure because Protractor on Travis CI is unable to navigate directly to the search and edit components. I was able to workaround this by modifying search.component.e2e-spec.ts to start at the top and navigate to the component.

beforeEach(() => {
  browser.get('/');
  element(by.linkText('SEARCH')).click();
});

I did something similar with edit.component.e2e-spec.ts:

beforeEach(() => {
  browser.get('/');
  element(by.linkText('SEARCH')).click();
  let search = element(by.css('sd-search form input'));
  'Man'.split('').forEach((c) => search.sendKeys(c));
  element(by.css('sd-search form button')).click();
  element(by.linkText('Peyton Manning')).click();
});

After making these changes, all e2e tests passed in Travis CI.

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular2-tutorial. If you have ideas for improvements, please leave a comment or send a pull request.

I wrote this tutorial in Asciidoctor so you can also read this tutorial on GitHub or using DocGist.

Summary

I hope you've enjoyed this quick-and-easy tutorial on testing Angular 2.0 RC1 applications. You can see the test coverage of your project by running npm run serve.coverage. You'll notice that the new components and service could use some additional coverage. I'll leave that as a task for the reader.

I learned a lot about testing from ng-book 2 and its Testing chapter. If you have any Angular 2 testing tips and tricks you'd like to share, I'd love to hear about them.

The Scenic Way to Santa Fe

2014-07-27T11:56:52-06:00

After having a successful run at ski season with our VW Syncro, Trish and I figured we'd see how it performed in the summer. We took it 4x4ing in Moab, rafting in Dinosaur National Monument and camping for Father's Day Weekend. It was a trusty steed for our kids, dogs, skis and raft. Earlier this month, we planned a week-long road trip to Santa Fe to see one of my old college roommates. Because of the holiday weekend, it turned into a 10-day road trip. We left Denver in style on the morning of the 4th.

The motorcycle on the back didn't seem to slow us down much. We drove from Denver to Winter Park, taking the long way over Squaw Pass to avoid traffic.

When we arrived at our Ski Shack, the engine was making a loud knocking sound. We dismissed it as a random occurrence. When I drove to the gas station 45 minutes later, the knocking was loud enough that heads turned when I drove past. After fueling up, I started the van and began driving back to our condo. The engine sputtered, the tires screeched and then the engine died. I pulled to the side, conveniently still in the gas station's parking lot. The engine would no longer turn over.

Below is the email I sent to Mike at Rocky Mountain Westy while walking back to our condo.

Happy Fourth of July! Our van's engine gave up the ghost today. There was a loud knocking, then it felt like is slammed on the back brakes, a screech of the tires and it died. Now it won't even turn over. Sounds like it's seized to me.

This happened in Fraser today. I have AAA so could probably get it towed to you.

Can you help us find and install a new engine? I'd like something with similar HP. Would you recommend a 1.8T, a 3.0 liter or something else?

It was the best breakdown I've ever had. I was able to walk home (one mile) and it didn't even rain on me. From there, we continued to celebrate the 4th and refactor our road-trip plans. With the Syncro, we were planning on camping at The Great Sand Dunes, Pagosa Springs and Mesa Verde. We decided we'd drive our new sedan instead (recently gifted from Trish's Dad). I took the train to Denver, drove it up on July 5th and we departed Winter Park on Sunday, July 6.

Instead of camping in beautiful campgrounds, we changed our plans to stay at B&Bs. In the meantime, Mike replied and said he thought he'd be able to install an H6 3.0 by the end of the month. He recommended it as a good replacement for our SVX engine, particularly because it was more reliable and likely to endure more miles. We agreed to the repairs and had AAA tow it to his shop in Fort Collins.

Our first destination was The Broadmoor. We took the back way through Breckenridge and skipped the holiday traffic. We arrived around 5pm and were warmly welcomed, dogs and all. We celebrated our one-year anniversary early with excellent Italian food, golf on the West course and a fun pub crawl on Monday night.

The Broadmoor was named the #1 Golf Resort in North America by Golf magazine readers. We certainly enjoyed the course, but it's also the first time Trish and I declined beers from the cart lady. Our games were so bad, we weren't in the mood to celebrate anything.

That afternoon, we took our dogs to the excellent Bear Creek Dog Park and sampled some beers at nearby breweries. Our favorite was the Golden Bee and their real yards of Guinness.

On Tuesday afternoon, we left Colorado Springs for Pagosa Springs. There, we enjoyed a country B&B, soaked in the hot springs and had a great hike to Fourmile Falls.

We left on Wednesday afternoon and drove to Taos, New Mexico. We had the best chilli relleno in the world, watched a bluegrass jam, visited Taos Mesa Brewing and hiked to Williams Lake.

On Friday, we left Taos and drove to Santa Fe, our ultimate destination. While arriving, I checked my email and was pleased to see an email from the folks working on my '66 Bus. While flipping through the pictures referenced in the email, I realized there were several videos. I did my best to get us off the road and into a place where I could watch the videos on a proper screen. You can imagine my glee as I watched my bus drive under its own power for the first time.

My "achievement unlocked!" moment was soon complimented by an excellent margarita and tasty New Mexican food. Wow! Those New Mexicans know the right level for the "default" hot sauce. Spicy and delicious!

We spent the weekend with our friends Chris, Mary and Luna and had a great time. The Santa Fe summer weather was beautiful - in the mid-70s most of the time, with nice breezes and sunshine. Trish, Chris, Sagan, Jake and I took a hike on Saturday and thoroughly enjoyed their national forest. On Sunday, we marveled at the art on display at Santa Fe's 11th Annual International Folk Art Market.

More photos on Flickr →

The rest of the time, we hung out in their backyard. Trish played her banjo, we watched the firepit crackle and Dino (their new puppy) kept us entertained with his antics. It was one of the most relaxing weekends I've had in quite some time. Thanks to Chris, Mary and Luna - you guys are awesome and we can't wait to visit again!

A Webapp Makeover with Spring 4 and Spring Boot

2013-12-11T12:47:15-07:00

A typical Maven and Spring web application has a fair amount of XML and verbosity to it. Add in Jersey and Spring Security and you can have hundreds of lines of XML before you even start to write your Java code. As part of a recent project, I was tasked with upgrading a webapp like this to use Spring 4 and Spring Boot. I also figured I'd try to minimize the XML.

This is my story on how I upgraded to Spring 4, Jersey 2, Java 8 and Spring Boot 0.5.0 M6.

When I started, the app was using Spring 3.2.5, Spring Security 3.1.4 and Jersey 1.18. The pom.xml had four Jersey dependencies, three Spring dependencies and three Spring Security dependencies, along with a number of exclusions for "jersey-spring".

Upgrading to Spring 4
Upgrading to Spring 4 was easy, I changed the version property to 4.0.0.RC2 and added the new Spring bill of materials to my pom.xml. I also add the Spring milestone repo since Spring 4 won't be released to Maven central until tomorrow.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-framework-bom</artifactId>
            <version>${spring.framework.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<repositories>
    <repository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Next, I removed all the references to ${spring.framework.version} in dependencies since it'd be controlled by Maven's dependency management feature.

     <dependency>
         <groupId>org.springframework</groupId>
         <artifactId>spring-web</artifactId>
-        <version>${spring.framework.version}</version>
     </dependency>

I also changed to use Maven 3's wildcard syntax to exclude multiple dependencies.

    <dependency>
        <groupId>com.sun.jersey.contribs</groupId>
        <artifactId>jersey-spring</artifactId>
        <exclusions>
             <exclusion>
                 <groupId>org.springframework</groupId>
-                    <artifactId>spring</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-core</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-web</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-beans</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-context</artifactId>
+                    <artifactId>*</artifactId>
             </exclusion>
         </exclusions>
     </dependency>

I confirmed the upgrade worked by running "mvn dependency:tree | grep spring", followed by "mvn jetty:run" and viewing the app in my browser.

Upgrading to Jersey 2
The next item I tackled was upgrading to Jersey 2.4.1. I changed the version number in my pom.xml, then added the Jersey BOM.

<dependency>
    <groupId>org.glassfish.jersey</groupId>
    <artifactId>jersey-bom</artifactId>
    <version>${jersey.version}</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

You might ask "why Jersey?" if we already have Spring MVC and its REST support? You might also ask why not Play or Grails instead of a Java + Spring stack? For this particular project, I recommended technology options, and these were certainly among them. However, the team chose differently and I support their decision. The project is creating an iOS app, as well as a responsive HTML5 mobile/desktop app. We figured we had enough risk with new technologies on the front-end that we should play it a bit safer on the backend. To make the backend work a bit sexier, we've decided to allow Spring 4, Java 8 and possibly some reactive principles.

Next, I changed from the old com.sun.jersey dependencies to org.glassfish.jersey and removed jersey-spring.

<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-json-jackson</artifactId>
</dependency>

The last thing I needed to do was change the servlet-class and param-name in web.xml:

<servlet>
    <servlet-name>jersey-servlet</servlet-name>
    <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
    <init-param>
        <param-name>jersey.config.server.provider.packages</param-name>
        <param-value>com.raibledesigns.boot.service</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>

Requiring Java 8
Requiring Java 8 to compile was easy enough. I added the maven-compiler-plugin to enforce a minimum version.

<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>

I downloaded the latest Java 8 SDK and installed it. Then I set my JAVA_HOME to use it.

export JAVA_HOME=`/usr/libexec/java_home -v 1.8`

Integrating Spring Boot
I learned about Spring Boot a few weeks ago at Devoxx. Josh Long gave me a 3-minute demo at the speaker's dinner and showed me enough to pique my interest. To integrate it into my project, I started with the Quick Start. I added the boot-parent, dependencies for web, security and actuator (logging, metrics, etc.) and the Maven plugin. I removed all the Spring and Spring Security dependencies.

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>0.5.0.M6</version>
</parent>
...
<pluginRepositories>
    <pluginRepository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
    </pluginRepository>
</pluginRepositories>
...
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
...
<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
</plugin>

Upon restarting my app, I got an error about spring-security.xml using a 3.1 XSD. I fixed it by changing to 3.2. Next, I wanted to eliminate web.xml. First of all, I created an ApplicationInitializer so the WAR could be started from the command line.

package com.raibledesigns.boot.config;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class ApplicationInitializer extends SpringBootServletInitializer {

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(ApplicationInitializer.class);
    }

    public static void main(String[] args) {
        SpringApplication.run(ApplicationInitializer.class, args);
    }
}

However, after adding this, I received the following error on startup:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 
'org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor': 
Invocation of init method failed; nested exception is 
java.lang.AbstractMethodError: org.hibernate.validator.internal.engine.ConfigurationImpl
.getDefaultParameterNameProvider()Ljavax/validation/ParameterNameProvider;

Adding hibernate-validator as a dependency solved this problem:

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
</dependency>

To configure Spring Security without web.xml and spring-security.xml, I created WebSecurityConfig.java:

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@EnableWebSecurity
@Order(Ordered.LOWEST_PRECEDENCE - 6)
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .antMatchers("/", "/home").permitAll()
                .antMatchers("/v1.0/**").hasRole("USER")
                .anyRequest().authenticated();
        http.httpBasic().realmName("My API");
    }

    @Override
    protected void configure(AuthenticationManagerBuilder authManagerBuilder) throws Exception {
        authManagerBuilder.inMemoryAuthentication()
                .withUser("test").password("test123").roles("USER");
    }
}

To configure Jersey without web.xml, I created a JerseyConfig class:

package com.raibledesigns.boot.config;

import org.glassfish.jersey.filter.LoggingFilter;
import org.glassfish.jersey.jackson.JacksonFeature;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.ServerProperties;

import javax.ws.rs.ApplicationPath;

@ApplicationPath("/v1.0")
public class JerseyConfig extends ResourceConfig {

    public JerseyConfig() {
        packages("com.raibledesigns.boot.service");
        property(ServerProperties.BV_SEND_ERROR_IN_RESPONSE, true);
        property(ServerProperties.JSON_PROCESSING_FEATURE_DISABLE, false);
        property(ServerProperties.MOXY_JSON_FEATURE_DISABLE, true);
        property(ServerProperties.WADL_FEATURE_DISABLE, true);
        register(LoggingFilter.class);
        register(JacksonFeature.class);
    }
}

Finally, I created MvcConfig.java to set the welcome page.

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.ViewControllerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;

@Configuration
public class MvcConfig extends WebMvcConfigurerAdapter {

    @Override
    public void addViewControllers(ViewControllerRegistry registry) {
        registry.addViewController("/").setViewName("index");
    }
}

To cleanup, I deleted src/main/webapp/WEB-INF and created src/main/resources/logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>
    <logger name="org.springframework.boot" level="INFO"/>
    <logger name="org.springframework.security" level="ERROR"/>
</configuration>

Since Boot doesn't support JSP out-of-the-box, I renamed my index.jsp file to index.html and changed the URL in it to point to "/v1.0/hello". I was pleased to see that everything worked nicely. I learned shortly after that I could remove the Spring BOM since Spring Boot uses a <spring.version> property to control its Spring version.

The only issue I found is when started the app with "mvn package && java -jar target/app.war", it failed to initialize Jersey. I tried adding a @Bean for the servlet:

@Bean
public ServletRegistrationBean jerseyServlet() {
    ServletRegistrationBean registration = new ServletRegistrationBean(new ServletContainer(), "/v1.0/*");
    registration.addInitParameter(ServletProperties.JAXRS_APPLICATION_CLASS, JerseyConfig.class.getName());
    return registration;
}

Unfortunately, when running it using "java -jar", I get the following error:

org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. org.glassfish.jersey.server.internal.scanning.ResourceFinderException: 
java.io.FileNotFoundException: /.../target/app.war!/WEB-INF/classes (No such file or directory)
	at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:869)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:814)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:906)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:898)
	at org.glassfish.jersey.server.ApplicationHandler.createApplication(ApplicationHandler.java:300)
	at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:279)
	at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:302)

This seems strange since there is a WEB-INF/classes in my WAR. Regardless, this is not a Boot problem per se, but more of a Jersey issue. From one of the Boot developers:

The whole idea with Boot is that servlets are just a transport - they are a means to an end, and hopefully not the only one - the "container" is Spring, not the servlet container. We probably could add some form of support for SCI but only by hacking the containers since the spec really doesn't allow for much control of their lifecycle. It hasn't been a priority so far.

Summary
I hope this article is useful to see how you to upgrade your Java webapps to use Spring 4 and Spring Boot. I've created a boot-makeover project on GitHub with all the code mentioned. You can also view the commits for each step.

What have I been working on at Taleo?

2011-12-09T12:57:36-07:00

2011 has been a year of great clients for me. I started working with O.co and very much enjoyed my time there, especially on powder days in Utah. The people were great, the contract was great (no end date), but the work was not my forte. I was on a project to modularize the main shopping site's codebase, which involved mostly refactoring. By refactoring, I mean creating new Maven projects, modifying lots of pom.xml files and literally moving files from one directory to another. IntelliJ made this easy, the hard part was refactoring tests, moving from EasyMock to Mockito and splitting classes into interfaces and implementations where appropriate. As a developer who likes developing UIs and visually seeing my accomplishments, the project wasn't that exciting. However, I knew that it was strategically important to O.co, so I didn't complain much.

In mid-May, I received a LinkedIn message from the Director of Software Engineering at Taleo.

This is OB, I am the Director of Software Engineering at Taleo. We are the 2nd largest Software as a Service company. I am building a new specialist UI team that will take the product to the next level. I am looking for someone to lead this initiative. If you are interested to have a chat about it, please let me know.

At that time, I'd never heard of Taleo and quickly recommended they not hire me.

This probably isn't the best position for me. While I am a good leader, I'm not willing to relocate from Denver. I've found that leaders usually do best when face-to-face with their developers.

This conversation continued back-and-forth where I explained how I wasn't willing to go full-time and I didn't want to leave Overstock. In the end, OB was persistent and explained how the position would entail lots of UI work and wouldn't require me to travel much. Our negotiations trailed off in June and resumed in July after I returned from vacation in Montana. Shortly after, we met each other's expectations, agreed on a start date and I started working at Taleo in early September.

When I started, there were three features they wanted to add to to Taleo Business Edition: Profile Pictures, Talent Cards and Org Charts. They knew the schedule was tight (8 weeks), but I was confident I could make it happen. At first, I groaned at the fact that they were using Ant to build the project. Then I smiled when I learned they'd standardized on IntelliJ and set things up so you could do everything from the IDE. After using Maven for many years, this setup has actually become refreshing and I rarely have to restart or long for something like JRebel. Of course, a new kick-ass laptop and awesome IDE make it so I rarely wait for anything to happen.

To give you a taste of how I implemented each of these new features in 8 weeks, I've broken them into sections below.

Profile Pictures
Adding profile pictures was a pretty simple concept, one you see on my social networking sites today. I needed to give users the ability to upload a JPEG or PNG and crop it so it looked good. The uploading was a pretty straightforward process and I used a lot of internal APIs to grab the file from the request and save it to disk. The more difficult part was scaling the image to certain dimensions on upload (to save space) and allowing users to crop it after.

Most of Taleo Business Edition (TBE) is written in good ol' servlets and JSPs, with lots of scriptlets in their JSPs. When I saw the amount of HTML produced from Java, I laughed out loud and cringed. Soon after, I breathed a sigh of relief when I learned that any new features could be written using FreeMarker templates, which IntelliJ has excellent support for.

For image resizing on upload, I used Chris Campbell's Image.getScaledInstance() tutorial. For creating thumbnails, I used a combination of scaling, getSubimage() and the Java Image I/O API. I made sure to write to BufferedOutputStream for scalability. For cropping images client-side, I used jQuery UI's Dialog and Jcrop, the jQuery image cropping plugin. Below is a screenshot of what the cropping UI looks like:

Talent Cards
Talent Cards were a whole different beast. Not only did they need to display profile pictures, they also needed to contain contact information, work history and a number of other data points. Not just for employees, but for candidates as well. They also needed to be rendered with tabs at the bottom that allowed you to navigate between different data sections.

I'll admit, most of the hard work for this feature was done by the server-side developers (Harish and Vlad) on my team. Vlad built the tabbed interface and Harish built the administrative section that allows you to add/remove/sort fields, as well as show and hide certain tabs. I performed most of my magic with jQuery, its clueTip plugin and good ol' CSS. I was particularly thankful for CSS3 and its border-radius, box-shadow and Justin Maxwell's tutorial on CSS String Truncation with Ellipsis. I used DWR to fetch all the data from the server using Ajax.

Talent Cards are a slick feature in TBE 11.5 and I think they're a great way to see a lot of information about someone very quickly. If you enable them for your company, you'll be able to mouse over any employee or candidate's names and see their information.

Org Chart
The last feature I completed in this 8-week sprint was creating an organization chart. For this, I was given a rough prototype based on Caprica Software's JQuery/CSS Organisation Chart. When I received it, it had all kinds of cool CSS 3 transformations (like this one), but they only worked in Safari and Chrome. I ended up removing the transformations and adding the ability to navigate up and down the org tree with Ajax (we currently only show three levels at a time).

The Org Chart feature also allows you to see how many direct/indirect reports an employee has, as well as access their Talent Card by hovering over their name. It's one of my favorite features because it's so visual and because it builds upon all the other features we've built.

Summary
As you might've guessed by now, I've been having a lot of fun doing UI development over the last few months. While I seem to have a knack for backend Java development, I enjoy developing UIs a lot more. The smile you see on people's faces during demos is priceless. I can't help but think this kind of thing contributes greatly to my developer happiness. All these features will be in next week's release of TBE and I couldn't be happier.

If you'd like to work on my team at Taleo (or even take over my current role as UI Architect), please drop me a line. If you live near their headquarters (Dublin, CA), it'd also be great to see you at the next Silicon Valley Spring User Group meetup. I'll be speaking about What's New in Spring 3.1 on February 1st.

Integrating OAuth with AppFuse and its REST API

2011-07-05T10:56:48-06:00

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
Integrate Spring Security and OAuth
Authenticate and Retrieve Data with Client

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Java Web Application Security - Part V: Penetrating with Zed Attack Proxy

2011-06-21T07:45:41-06:00

Web Application Security is an important part of developing applications. As developers, I think we often forget this, or simply ignore it. In my career, I've learned a lot about web application security. However, I only recently learned and became familiar with the rapidly growing "appsec" industry.

I found a disconnect between what appsec consultants were selling and what I was developing. It seemed like appsec consultants were selling me fear, mostly because I thought my apps were secure. So I set out on a mission to learn more about web application security and penetration testing to see if my apps really were secure. This article is part of that mission, as are the previous articles I've written in this series.

When I first decided I wanted to do a talk on Webapp Security, I knew it would be more interesting if I showed the audience how to hack and fix an application. That's why I wrote it into my original proposal:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

At the time, I hadn't done much webapp pentesting. You can tell this from the fact that I mentioned WebGoat as the pentesting tool. From WebGoat's Project page:

WebGoat is a deliberately insecure J2EE web application maintained by OWASP designed to teach web application security lessons. In each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the WebGoat application. For example, in one of the lessons the user must use SQL injection to steal fake credit card numbers. The application is a realistic teaching environment, providing users with hints and code to further explain the lesson.

What I really meant to say and use was Zed Attack Proxy, also known as OWASP ZAP. ZAP is a Java Desktop application that you setup as a proxy for your browser, then use to find vulnerabilities in your application. This article explains how you can use ZAP to pentest a web applications and fix its vulnerabilities.

The application I'll be using in this article is the Ajax Login application I've been using throughout this series. I think it's great that projects like Damn Vulnerable Web App and WebGoat exist, but I wanted to test one that I think is secure, rather than one I know is not secure. In this particular example, I'll be testing the Spring Security implementation, since that's the framework I most often use in my open source projects.

Zed Attack Proxy Tutorial

Download and Run the Application
Install and Configure ZAP
Perform a Scan
Fix Vulnerabilities
Summary

Download and Run the Application
To begin, download the application and expand it on your hard drive. This app is the completed version of the Ajax Login application referenced in Java Web Application Security - Part II: Spring Security Login Demo. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and you need to login to do anything.

Install and Configure ZAP
The Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. Download the latest version (I used 1.3.0) and install it on your system. After installing, launch the app and change the proxy port to 9000 (Tools > Options > Local Proxy). Next, configure your browser to proxy requests through port 9000 and allow localhost requests to be proxied. I used Firefox 4 (Preferences > Advanced > Network > Connection Settings). When finished, your proxy settings should look like the following screenshot:

Another option (instead of removing localhost) is to add an entry to your hosts file with your production domain name. This is what I've done for this demo.

127.0.0.1       demo.raibledesigns.com

I've also configured Apache to proxy requests to Jetty with the following mod_proxy settings in my httpd.conf:

<IfModule mod_proxy.c>
    ProxyRequests Off 
    ProxyPreserveHost Off 

    <VirtualHost *:80>
       ProxyPass  /  http://localhost:8080/
    </VirtualHost>

    <VirtualHost *:443>
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile "/etc/apache2/ssl.key/server.crt"
        SSLCertificateKeyFile "/etc/apache2/ssl.key/server.key"

        ProxyPass  /  https://localhost:8443/
    </VirtualHost>
</IfModule>

Perform a Scan
Now you need to give ZAP some data to work with. Using Firefox, I navigated to http://demo.raibledesigns.com and browsed around a bit, listing users, added a new one and deleted an existing one. After doing this, I noticed a number of flags in the ZAP UI under Sites. I then right-clicked on each site (one for http and one for https) and selected Attack > Active Scan site. You should be able to do this from the "Active Scan" tab at the bottom of ZAP, but there's a bug when the URLs are the same. After doing this, I received a number of alerts, ranging from high (cross-site scripting) to low (password autocomplete). The screenshot below shows the various issues.

Now let's take a look at how to fix them.

Fix Vulnerabilities
One of the things not mentioned by the scan, but #1 in Seven Security (Mis)Configurations in Java web.xml Files, is Custom Error Pages Not Configured. Custom error pages are configured in this app, but error.jsp contains the following code:

<% if (exception != null) { %>
    <% exception.printStackTrace(new java.io.PrintWriter(out)); %>
<% } else { %>
    Please check your log files for further information.
<% } %>

Stack traces can be really useful to an attacker, so it's important to start by removing the above code from src/main/webapp/error.jsp.

The rest of the issues have to do with XSS, autocomplete, and cookies. Let's start with the easy ones. Fixing autocomplete is easy enough; simply changed the HTML in login.jsp and userform.jsp to have autocomplete="off" as part of the <form> tag.

Then modify web.xml so http-only and secure cookies are used. While you're at it, add session-timeout and tracking-mode as recommended by the aforementioned web.xml misconfigurations article.

<session-config>
    <session-timeout>15</session-timeout>
    <cookie-config>
        <http-only>true</http-only>
        <secure>true</secure>
    </cookie-config>
    <tracking-mode>COOKIE</tracking-mode>
</session-config>

Next, modify Spring Security's Remember Me configuration so it uses secure cookies. To do this, add use-secure-cookies="true" to the <remember-me> element in security.xml.

<remember-me user-service-ref="userService" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"
             use-secure-cookie="true"/>

Unfortunately, Spring Security doesn't support HttpOnly cookies, but will in a future release.

The next issue to solve is disabling directory browsing. You can do this by copying Jetty's webdefault.xml (from the org.eclipse.jetty:jetty-webapp JAR) into src/test/resources and changing its "dirAllowed" <init-param> to false:

<servlet>
  <servlet-name>default</servlet-name>
  <servlet-class>org.mortbay.jetty.servlet.DefaultServlet</servlet-class>
  <init-param>
    <param-name>acceptRanges</param-name>
    <param-value>true</param-value>
  </init-param>
  <init-param>
    <param-name>dirAllowed</param-name>
    <param-value>false</param-value>
  </init-param>
  <init-param>

You'll also need to modify the plugin's configuration to point to this file by adding it to the <webAppConfig> section in pom.xml.

<configuration>
    <webAppConfig>
        <contextPath>/</contextPath>
        <defaultsDescriptor>src/test/resources/webdefault.xml</defaultsDescriptor>
    </webAppConfig>

Of course, if you're running in production you'll want to configure this in your server's settings rather than in your pom.xml file.

Next, I set out to fix secure page browser cache issues. I had the following settings in my SiteMesh decorator:

<meta http-equiv="Cache-Control" content="no-store"/>
<meta http-equiv="Pragma" content="no-cache"/>

However, according to ZAP, the first meta tag should have "no-cache" instead of "no-store", so I changed it to "no-cache".

After making all these changes, I created a new ZAP session and ran an active scan on both sites again. Below are the results:

I believe the first issue (parameter tampering) is because I show the error page when a duplicate user exists. To fix this, I changed UserFormController so it catches a UserExistsException and sends the user back to the form.

try {
    userManager.saveUser(user);
} catch (UserExistsException uex) {
    result.addError(new ObjectError("user", uex.getMessage()));
    return "userform";
}

However, this still doesn't seem to cause the alert to go away. This is likely because I'm not filtering/escaping HTML when it's first submitted. I believe the best solution for this would be to use something like OWASP's ESAPI to filter parameter values. However, I was unable to find integration with Spring MVC's data binding, so I decided not to try and fix this vulnerability.

Finally, I tried to disable jsessionid in URLs using suggestions from Stack Overflow. The previous setting in web.xml (<tracking-mode>COOKIE</tracking-mode>) should do this, but it doesn't seem to work with Jetty 8. The other issues (secure page browser cache, HttpOnly cookies and secure cookies), I was unable to solve. The last two are issues caused by Spring Security as far as I can tell.

Summary
In this article, I've shown you how to pentest a web application using Firefox and OWASP's Zed Attack Proxy (ZAP). I found ZAP to be a nice tool for figuring out vulnerabilities, but it'd be nice if it had a "retest" feature to see if you fixed an issue for a particular URL. It does have a "resend" feature, but running it didn't seem to clear alerts after I'd fixed them.

The issues I wasn't able to solve seemed to be mostly related to frameworks (e.g. Spring Security and HttpOnly cookies) or servers (Jetty not using cookies for tracking). My suspicion is the Jetty issues are because it doesn't support Servlet 3 as well as it advertises. I believe this is fair; I am using a milestone release after all. I tried scanning http://demo.raibledesigns.com/ajax-login (which runs on Tomcat 7 at Contegix) and confirmed that no jsessionid exists.

Hopefully this article has helped you understand how to figure out security vulnerabilities in your web applications. I believe ZAP will continue to get more popular as developers become aware of it. If you feel ambitious and want to try and solve all of the issues in my Ajax Login application, feel free to fork it on GitHub.

If you're interested in talking more about Webapp Security, please leave a comment, meet me at Jazoon later this week or let's talk in July at Über Conf.

Java Web Application Security - Part III: Apache Shiro Login Demo

2011-05-26T16:43:22-06:00

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Store Users in a Database
Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

Java Web Application Security - Part II: Spring Security Login Demo

2011-05-13T09:20:51-06:00

Last week, I wrote a tutorial on how to implement Security in Java EE 6. This week, I'd like to show you how to implement the same features using Spring Security. Before I begin, I'd like to explain my reason for writing this article.

Last month, I presented a talk on Java Web Application Security at the Utah JUG (UJUG). As part of that presentation, I did a number of demos about how to implement security with Java EE 6, Spring Security and Apache Shiro. I told the audience that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense.

Today, I've finished the second screencast showing how to implement security with Spring Security. Below is the presentation (with the screencast embedded on slide 16) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Add Remember Me
Store Users in a Database
Summary

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a Spring context file that contains Spring Security's configuration. Create src/main/webapp/WEB-INF/security.xml and populate it with the contents below:

  <?xml version="1.0" encoding="UTF-8"?>
  <beans:beans xmlns="http://www.springframework.org/schema/security"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:beans="http://www.springframework.org/schema/beans"
               xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">

      <!-- New in Spring Security 3.1 -->
      <!-- <http pattern="/css/**" security="none"/> -->

      <http auto-config="true">
          <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN"/>
          <http-basic/>
      </http>

      <authentication-manager alias="authenticationManager">
          <authentication-provider>
              <password-encoder hash="sha"/>
              <user-service>
                  <user name="user" password="12dea96fec20593566ab75692c9949596833adc9" authorities="ROLE_USER"/>
                  <user name="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" authorities="ROLE_ADMIN"/>
              </user-service>
          </authentication-provider>
      </authentication-manager>

      <!-- Override userSecurityAdvice bean in appfuse-service to allow any role to update a user. -->
      <beans:bean id="userSecurityAdvice" class="org.appfuse.examples.webapp.security.UserSecurityAdvice"/>
  </beans:beans>

The last bean, userSecurityAdvice, is an aspect that's needed to override some behavior in AppFuse. You won't need this normally when implementing Spring Security.

Next, open src/main/webapp/WEB-INF/web.xml and add Spring's DelegatingFilterProxy:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

You don't need to add any dependencies in your pom.xml is because this project depends on AppFuse, which already contains these dependencies.

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Spring Security is a bit easier to configure than Java EE 6 out-of-the-box, mostly because it doesn't require you to configure your container.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Spring Security has a way to configure "logout" to match a URL and get rid of a class like LogoutController. Since it was already in the project, I don't cover that in this tutorial.

Force SSL
Spring Security allows you to switch between secure (https) and non-secure (http) protocols using a simple requires-channel attribute on the <intercept-url> element. Possible values are "http", "https" and "any". Add requires-channel="https" to your security.xml file:

<intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

You can switch between http and https protocols. With Java EE, you can only force https. You have to write a custom filter to switch back to http.
Redirecting to https actually works. With Java EE (on Jetty at least), a 403 is returned instead of redirecting the request.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a <form-login> element in security.xml's <http> element:

<http auto-config="true">
    <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>
    <form-login login-page="/login" authentication-failure-url="/login?error=true"
                login-processing-url="/j_security_check"/>
    <http-basic/>
</http>

You can leave the <http-basic> element since Spring Security is smart enough to serve up the form for browsers and use Basic Authentication for clients such as web services. The login.jsp page (that /login forwards to) already exists in the project, in the src/main/webapp directory. The forwarding is done by the UrlRewriteFilter with the following configuration in src/main/webapp/WEB-INF/urlrewrite.xml.

<rule>
    <from>/login</from>
    <to>/login.jsp</to>
</rule>

This JSP has 3 important elements: 1) a form that submits to "/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Add Remember Me
Remember Me is a feature you see in many web applications today. It's usually a checkbox on the login form that allows you to auto-login the next time you visit a site. This feature doesn't exist in Java EE security, but it does exist in Spring Security. To enable it, add the following just below <form-login> in security.xml:

<remember-me user-service-ref="userDao" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"/>

Next, open src/main/webapp/login.jsp and change the name of the "remember me" checkbox to be _spring_security_remember_me:

<input type="checkbox" name="_spring_security_remember_me" id="rememberMe"/>

After making these changes, you should be able to restart Jetty, go to http://localhost:8080/users, enter admin/adminjdbc, check the Remember Me checkbox and login. Then close your browser, and repeat the process. This time, you won't be prompted to login. For more information on this feature, see Spring Security's Remember Me documentation.

While storing usernames and passwords in a file is convenient for demos, it's not very real-world-ish. The next section shows you how to configure Spring Security to use a database for its user store.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a user-service-ref attribute to the <authentication-provider> element. You can also delete the <user-service> element.

<authentication-manager alias="authenticationManager">
    <authentication-provider user-service-ref="userDao">
        <password-encoder hash="sha"/>
    </authentication-provider>
</authentication-manager>

The "userDao" bean is provided by AppFuse and its UserDaoHibernate.java class. This class implements Spring Security's UserDetailsService interface. With Java EE, I had to configure a database connection and make sure the JDBC Driver was in my container's classpath. With Spring Security, you can talk to the database you already have configured in your application.

Of course, you could do this with Java EE too. One thing I neglected to show in my last tutorial was that 1) the app uses H2 and 2) I had to configure Java EE's database to be MySQL. This was because when I tried to access my H2 instance, I got an error about two threads trying to access it at once.

2011-05-13 08:47:29.081:WARN::UserRealm Java EE Login could not connect to database; will try later
org.h2.jdbc.JdbcSQLException: Database may be already in use: "Locked by another process". 
        Possible solutions: close all other connection(s); use the server mode [90020-154]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:167)
	at org.h2.message.DbException.get(DbException.java:144)
	at org.h2.store.FileLock.getExceptionAlreadyInUse(FileLock.java:443)
	at org.h2.store.FileLock.lockFile(FileLock.java:338)
	at org.h2.store.FileLock.lock(FileLock.java:134)
	at org.h2.engine.Database.open(Database.java:535)
	at org.h2.engine.Database.openDatabase(Database.java:218)

The password for the "admin" user is configured in src/test/resources/sample-data.xml and it's loaded by DbUnit before the application starts. You can view your pom.xml and the dbunit-maven-plugin's configuration if you're interested in learning how this is done. The password is currently configured to "adminjdbc", but you can reset it by generating a new password and modifying sample-data.xml.

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Spring Security 3.0.5. In addition to the basic XML configuration, Spring Security also provides a AOP support and annotations you can use to secure methods. It also has many more features than standard Java EE Security. In my opinion, it's the most mature security framework we have in Java today. Currently, I think its reference documentation is the best place to learn more.

There are a few limitations I found with Spring Security:

The authentication mechanism (file, database, ldap, etc.) is contained in the WAR
Securing methods only works on Spring beans
Remember Me doesn't work in my screencast (because I forgot to rename the checkbox in login.jsp)

Of course, you can configure Spring to load its configuration from outside the WAR (e.g. a file or JNDI), but it's not as easy as including the configuration in your app.

In the next couple weeks, I'll post Part III of this series, where I'll show you how to implement this same set of features using Apache Shiro. In the meantime, please let me know if you have any questions.

I created the screencasts with Camtasia. For small screens, and embedding in the presentation, I created it at 50% and used the SmartFocus feature to zoom in and out during the demo. For larger screens, I published another screencast at 100%, in HD. If you have a preference for which screencast is better, I'd love to hear about it.

Java Web Application Security - Part I: Java EE 6 Login Demo

2011-05-05T16:58:00-06:00

Back in February, I wrote about my upcoming conferences:

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks:

Implement Basic Authentication
Implement Form-based Authentication
Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs)
Force SSL for certain URLs
Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone)
Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone)
Encrypt Passwords
Secure methods with annotations

For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities.

I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml:

<security-constraint>
    <web-resource-collection>
        <web-resource-name>users</web-resource-name>
        <url-pattern>/users</url-pattern>
        <http-method>GET</http-method>
        <http-method>POST</http-method>
    </web-resource-collection>
    <auth-constraint>
        <role-name>ROLE_ADMIN</role-name>
    </auth-constraint>
</security-constraint>

<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Java EE Login</realm-name>
</login-config>

<security-role>
    <role-name>ROLE_ADMIN</role-name>
</security-role>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after </webAppConfig> in the Jetty plugin's configuration.

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.HashLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/realm.properties</config>
    </loginService>
</loginServices>

The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath());
}

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to replace the <login-config> in your web.xml with the following:

<login-config>
    <auth-method>FORM</auth-method>
    <form-login-config>
        <form-login-page>/login.jsp</form-login-page>
        <form-error-page>/login.jsp?error=true</form-error-page>
    </form-login-config>
</login-config>

The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Force SSL
Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same <security-constraint> you already have in web.xml, add the following after </auth-constraint>:

<user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>

To configure Jetty to listen on an SSL port, add the following just after </loginServices> in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you.

Store Users in a Database
Finally, to store your users in a database instead of file, you'll need to change the <loginService> in the Jetty plugin's configuration. Replace the existing <loginService> element with the following:

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.JDBCLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/jdbc-realm.properties</config>
    </loginService>
</loginServices>

The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information.

jdbcdriver = com.mysql.jdbc.Driver
url = jdbc:mysql://localhost/appfuse
username = root
password =
usertable = app_user
usertablekey = id
usertableuserfield = username
usertablepasswordfield = password
roletable = role
roletablekey = id
roletablerolefield = name
userroletable = user_role
userroletableuserkey = user_id
userroletablerolekey = role_id
cachetime = 300

Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands:

mysql -u root -p -e 'create database appfuse'
curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql
mysql -u root -p appfuse < create-appfuse.sql

Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the <configuration> element (before <executions>) in pom.xml:

<dependencies>
    <!-- MySQL for JDBC Realm -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.14</version>
    </dependency>
</dependencies>

Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example:

$ sh jetty-password.sh javaeelogin
javaeelogin
OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur
MD5:53b176e6ce1b5183bc970ef1ebaffd44

The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement.

UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin';

Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0:

authenticate(response)
login(user, pass)
logout()

This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely:

No error messages for failed logins
No Remember Me
No auto-redirect from HTTP to HTTPS
Container has to be configured
Doesn’t support regular expressions for URLs

Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain.

In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions.

Livin' it up in Vegas at TSSJS 2011

2011-03-22T09:04:17-06:00

Last Wednesday, Trish and I traveled to Las Vegas for TheServerSide Java Symposium 2011 conference. We had a free room from TechTarget, but opted to upgrade to a suite with a view over the Bellagio Fountains. Trish won a trip to Vegas as a sales award earlier in the year and cleverly exchanged it for cash, so our upgrade was sort of free.

My first talk was on Online Video and my experience at Time Warner Cable. With my former team's iPad app releasing the day before, it was a fun session. The attendance was kind of sparse, but I had some good competition so wasn't surprised.

After I finished speaking, we headed to happy hour and met up with some friends that happened to be in town. We had dinner at the Todd English Pub and headed to the Penn & Teller show at the Rio. We closed the night after Trish had a 45-minute roll at the craps table at O'Sheas.

We slept in on Thursday and I gave my Comparing JVM Web Frameworks talk that afternoon. I made sure to mention some other methods to choosing web frameworks: doing performance comparisons like Peter Thomas has done or choosing Lift because one of its developers says it's the best. While Vaadin did sneak into the #5 spot, I made sure and mentioned that Wicket and Tapestry seem to belong there moreso (based on stats, mailing list traffic, etc.).

Trish took a bunch of pictures during my talk, which had a great turnout and lots of participation.

That evening, we celebrated St. Patty's Day with some college buddies of mine, ate great sushi at Mizuya and experienced the joys of three card poker. Thanks to TechTarget for inviting me to TSSJS 2011; we had an awesome time. You can find all the pictures we took on Flickr.

P.S. If you can't see the presentations in this post (a.k.a. you don't have Flash), you can view them on on Slideshare or download the PDFs.

Upgrading to JSF 2

2011-03-07T13:24:53-07:00

Last week, I spent a few hours upgrading AppFuse from JSF 1.2 to JSF 2.0. In reality, I upgraded from MyFaces 1.2.7 to 2.0.4, but all JSF implementations should be the same, right? All in all, it was a pretty easy upgrade with a few minor AppFuse-specific things. My goal in upgrading was to do the bare minimum to get things working and to leave integration of JSF 2 features for a later date.

In addition to upgrading MyFaces, I had to upgrade Tomahawk by changing the dependency's artifactId to tomahawk20. I was also able to remove the following listener from my web.xml:

<listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
<listener>

After that, I discovered that MyFaces uses a new URI (/javax.faces.resource/) for serving up some of its resource files. I kindly asked Spring Security to ignore these requests by adding the following to my security.xml file.

<intercept-url pattern="/javax.faces.resource/**" filters="none"/>

Since JSF 2 includes Facelets by default, I tried removing Facelets as a dependency. After doing this, I received the following error:

ERROR [308855416@qtp-120902214-7] ViewHandlerWrapper.fillChain(158) | Error instantiation parent Faces ViewHandler
java.lang.ClassNotFoundException: com.sun.facelets.FaceletViewHandler
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at org.ajax4jsf.framework.ViewHandlerWrapper.fillChain(ViewHandlerWrapper.java:144)
        at org.ajax4jsf.framework.ViewHandlerWrapper.calculateRenderKitId(ViewHandlerWrapper.java:68)
        at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:179)
        at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:113)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:171)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Figuring this was caused by the following element in my web.xml ...

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

... I removed it and tried again. This time I received a NoClassDefFoundError:

java.lang.NoClassDefFoundError: com/sun/facelets/tag/TagHandler
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:392)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.myfaces.shared_impl.util.ClassUtils.classForName(ClassUtils.java:184)
        at org.apache.myfaces.view.facelets.util.ReflectionUtil.forName(ReflectionUtil.java:67)

Since everything seemed to work with Facelets in the classpath, I decided to save this headache for a later date. I entered two issues in AppFuse's JIRA, one for removing Facelets and one for replacing Ajax4JSF with RichFaces.

The next issue I encountered was redirecting from AppFuse's password hint page. The navigation-rule for this page is as follows:

<navigation-rule>
    <from-view-id>/passwordHint.xhtml</from-view-id>
    <navigation-case>
        <from-outcome>success</from-outcome>
        <to-view-id>/login</to-view-id>
        <redirect/>
    </navigation-case>
</navigation-rule>

With JSF 2.0, the rule changes the URL to /login.xhtml when redirecting (where it was left as /login with 1.2) and it was caught by the security setting in my web.xml that prevents users from viewing raw templates.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Protect XHTML Templates</web-resource-name>
        <url-pattern>*.xhtml</url-pattern>
    </web-resource-collection>
    <auth-constraint/>
</security-constraint>

To solve this issue, I had to make a couple of changes:

Comment out the security-constraint in web.xml and move it to Spring Security's security.xml file.
```
<intercept-url pattern="/**/*.xhtml" access="ROLE_NOBODY"/>
```
Add a rule to urlrewrite.xml that redirects back to login (since login.xhtml doesn't exist and I'm using extensionless URLs).
```
<rule match-type="regex">
    <from>^/login.xhtml$</from>
    <to type="redirect">%{context-path}/login</to>
</rule>
```

After getting the Password Hint feature passing in the browser, I tried running the integration tests (powered by Canoo WebTest). The Password Hint test kept failing with the following error:

[ERROR] /Users/mraible/dev/appfuse/web/jsf/src/test/resources/web-tests.xml:51: JavaScript error loading
page http://localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/passwordHint?username=admin: syntax error (http://
localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/javax.faces.resource/oamSubmit.js.jsf?ln=org.apache.myfaces#122)

Figuring this was caused by my hack to submit the form when the page was loaded, I turned to Pretty Faces, which allows you to call a method directly from a URL. After adding the Pretty Faces dependencies to my pom.xml, I created a src/main/webapp/WEB-INF/pretty-config.xml file with the following XML:

<url-mapping>
    <pattern value="/editProfile"/>
    <view-id value="/userForm.jsf"/>
    <action>#{userForm.edit}</action>
</url-mapping>

<url-mapping>
    <pattern value="/passwordHint/#{username}"/>
    <view-id value="/passwordHint.jsf"/>
    <action>#{passwordHint.execute}</action>
</url-mapping>

This allowed me to remove both editProfile.xhtml and passwordHint.xhtml, both of which simply auto-submitted forms.

At this point, I figured I'd be good to go and ran my integration tests again. The first thing I discovered was that ".jsf" was being tacked onto my pretty URL, most likely by the UrlRewriteFilter. Adding the following to my PasswordHint.java class solved this.

if (username.endsWith(".jsf")) {
    username = username.substring(0, username.indexOf(".jsf"));
}

The next thing was a cryptic error that took me a while to figure out.

DEBUG [1152467051@qtp-144702232-0] PasswordHint.execute(38) | Processing Password Hint...
2011-03-05 05:48:52.471:WARN::/passwordHint/admin
com.ocpsoft.pretty.PrettyException: Exception occurred while processing <:#{passwordHint.execute}> null
        at com.ocpsoft.pretty.faces.beans.ActionExecutor.executeActions(ActionExecutor.java:71)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.processEvent(PrettyPhaseListener.java:214)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.afterPhase(PrettyPhaseListener.java:108)
        at org.apache.myfaces.lifecycle.PhaseListenerManager.informPhaseListenersAfter(PhaseListenerManager.java:111)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:185)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Digging into the bowels of MyFaces, I discovered a class was looking for a viewId with an extension and no view-id was being set. Adding the following to the top of my execute() method solved this.

getFacesContext().getViewRoot().setViewId("/passwordHint.xhtml");

After making this change, all AppFuse's integration tests are passing and the upgrade seems complete. The only other issues I encountered were logging-related. The first is an error about Tomahawk that doesn't seem to affect anything.

Mar 5, 2011 6:44:01 AM com.sun.facelets.compiler.TagLibraryConfig loadImplicit
SEVERE: Error Loading Library: jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml
java.io.IOException: Error parsing [jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml]: 
        at com.sun.facelets.compiler.TagLibraryConfig.create(TagLibraryConfig.java:410)
        at com.sun.facelets.compiler.TagLibraryConfig.loadImplicit(TagLibraryConfig.java:431)
        at com.sun.facelets.compiler.Compiler.initialize(Compiler.java:87)
        at com.sun.facelets.compiler.Compiler.compile(Compiler.java:104)

The second is excessive logging from MyFaces. As far as I can tell, this is because MyFaces switched to java.util.logging instead of commons logging. With all the frameworks that AppFuse leverages, I think it has all the logging frameworks in its classpath now. I was hoping to fix this by posting a message to the mailing list, but haven't received a reply yet.

[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider newInstance
[WARNING] [talledLocalContainer] INFO: Creating instance of org.appfuse.webapp.action.BasePage
[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider destroyInstance
[WARNING] [talledLocalContainer] INFO: Destroy instance of org.appfuse.webapp.action.BasePage

After successfully upgrading AppFuse, I turned to AppFuse Light, where things were much easier.

Now that AppFuse uses JSF 2, I hope to start leveraging some of its new features. If you're yearning to get started with them today, I invite you to grab the source and start integrating them yourself.

Fixing XSS in JSP 2

2011-02-28T14:08:46-07:00

Way back in 2007, I wrote about Java Web Frameworks and XSS. My main point was that JSP EL doesn't bother to handle XSS.

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

A couple months later, I proposed a Tomcat enhancement to escape JSP's EL by default. I also entered an enhancement request for this feature and attached a patch. That issue has remained open and unfixed for 3 and 1/2 years.

Yesterday, Chin Huang posted a handy-dandy ELResolver that XML-escapes EL values.

I tried Chin's resolver in AppFuse today and it works as well as advertised. To do this, I copied his EscapeXML*.java files into my project, changed the JSP API's Maven coordinates from javax.servlet:jsp-api:2.0 to javax.servlet.jsp:jsp-api:2.1 and added the listener to web.xml.

With Struts 2 and Spring MVC, I was previously able to have ${param.xss} and pass in ?xss=<script>alert('gotcha')</script> and it would show a JavaScript alert. After using Chin's ELResolver, it prints the string on the page instead of displaying an alert.

Thanks to Chin Huang for this patch! If you're using JSP, I highly recommend you add this to your projects as well.

My Everything You Ever Wanted To Know About Online Video Presentation

2010-12-03T10:16:44-07:00

This week I've had the pleasure of speaking at The Rich Web Experience in Fort Lauderdale. I did two talks, one on Comparing JVM Web Frameworks and one titled Everything You Ever Wanted To Know About Online Video. Both talks had full rooms and very engaged audiences.

In the video talk, there were some audience members that knew way more than me about the topic. This made for a very interactive session and one of the most fun presentations I've ever done. It was also cool to talk about a lot of things I've learned over the last year (for more details on that, check out my team status or team hiring posts). If you don't have Flash installed, you can download a PDF of this presentation.

The first talk about Comparing JVM Web Frameworks was largely an extension of the one I presented at Devoxx two weeks ago. The main differences between this one and the last one is I extended it a bit and took into account some community feedback. However, this seemed to simply inspire anger, so I'll pass on embedding it here. You can view it on Slideshare or download the PDF.

My Comparing Web Frameworks slides often inspire harsh words, but folks really seem to like the presentation. I encourage you to watch my Devoxx presentation on Parleys.com to see for yourself.

This marks the end of 2010 conferences for me. I had a blast speaking at The Rich Web Experience, as well as TheServerSide Java Symposium, The Irish Software Show and Devoxx. Now it's time to sit back, relax, get some powder days in and find my next gig.

Hope y'all have a great holiday season!

Jess and Lili's Legendary Wedding on The Lost Coast

2010-07-29T23:54:00-06:00

If you're a long-time reader of this blog, you'll know I've been to some great weddings in the last couple years. This past weekend, I had the pleasure of experiencing yet another fantastic celebration with two old and close friends, Clint and Jess. You might remember Clint from his wedding in Costa Rica or when we almost slept in a snow cave. I'm happy to report we didn't get in any trouble and everyone survived the weekend without a scratch.

My trip to Jess's wedding (on the Lost Coast of Northern California) started with a flight to Portland, Oregon. After arriving, I drove to Clint and Autumn's house in Eugene where we enjoyed some sweet Oregon micros and reminisced about Costa Rica. The next morning, we headed for the wedding; an 8-hour drive. Our road trip was awesome, especially when we started driving through the Redwood Groves on 101.

We stayed in a sweet beach house for the weekend. While it was foggy most of the time, the sun did come out on Saturday. We quickly became surrounded by beautiful views and headed to the beach to relax with Jess.

The wedding was on Sunday, a mere block from where we were staying. The ceremony was one of the most heartfelt I've ever heard, especially since the Wedding Official was a friend of the bride's since she was born.

The reception afterwards was a truly spectacular party that lasted well into the evening. Clint and I vowed to go to bed early, but we ended up having so much fun we closed the place down. Jess and Lili were an instrumental part in creating a spectacular night, especially with their wedding dance and infectious happiness.

The next day, we woke up on time, embarked on the 10-hour road trip back to Oregon and enjoyed a quick detour through the Avenue of the Giants. I did end up missing my flight home, but it was worth it. Thanks to Lili and Jess (and their families) for showing us such a great time. It was truly spectacular.

For more pictures, see albums on Flickr, Facebook or the slideshow below.

My Summer Vacation in Montana

2010-07-13T08:12:02-06:00

My favorite time of year is summertime. My favorite place to spend it is in Montana, often called "The Last Best Place" by natives. This year was no different and I spent the last two weeks at my family's cabin celebrating the 4th of July. Shortly after returning from our Father's Day Camping Trip, my parents packed up Abbie and Jack and headed on a 3-day road trip through Wyoming and Montana, camping and sight-seeing along the way. I followed them a few days later and made the 950-mile drive in just over 14 hours. With scenes like the one below, the trip was very enjoyable, despite it being so long.

The first week I was there, I worked remotely. It's always fun to tell people The Cabin has no electricity or running water, but it does have DSL. To be fair, it does have electricity, but it's not "on the grid" electricity - it's my Dad's concoction of generators, batteries and inverters. While I worked most of the week, I did manage to get a nice mountain bike ride in along the Foothills Trail to Holland Lake.

My real vacation began on the 4th of July weekend and we did it up right with the Swan Valley Parade and lots of big fireworks I picked up in Wyoming. The kids dressed up as Woody and Jesse (from Toy Story) and walked in the parade all by themselves (first time w/o me). They were especially excited when their pictures appeared in the local paper the following week.

Last week was spent hiking to Glacier Lake in the rain, golfing in Seeley Lake and Columbia Falls and hanging out with my good friend Owen Conley and his family.

The kids and I drove home last Sunday and it only took us 15 minutes longer than it did for me solo. I think they're quickly becoming road-tripping professionals.

My favorite part of this year's trip to The Cabin was seeing it as a home again. My Mom retired in April and my parents moved back to Montana shortly after. Seeing how happy they are there is truly magical. I especially enjoy the thought of visiting them and all the wonderful folks in the Swan Valley many, many times in the future.

To see all the pictures I took on this trip, check out the slideshow below.

P.S. An interesting note about all the pictures I took - they're all from my iPhone 4. I forgot my camera's battery at home and it seemed like a good experiment.

My Presentations from The Irish Software Show 2010

2010-06-10T07:11:35-06:00

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

Abbie and Jack's Field Days

2010-05-26T06:49:39-06:00

Last week, I had the pleasure of attending my kids' field days. For those who aren't familiar with field days, it's basically a sports day for elementary schools. The best part of this year's field days was seeing my kids have so much fun with their classmates. Of course, it didn't hurt that their teachers were also smitten with the thought of the school year coming to a close.

I took a few pictures and shot a bunch of video to remember how much fun they had. I know most readers won't enjoy these as much as I do, but I always like posting fond memories on this blog. Below are a couple videos I compiled and enhanced with appropriate music.

I especially like Jack's friend's dance moves at the end of the video below.

If you have trouble viewing either video here, check them out on my YouTube channel.

My TSSJS 2010 Presentations and Summary

2010-03-19T17:29:08-06:00

This afternoon, I delivered my last talk at TSSJS 2010 on The Future of Web Frameworks. It's true that I made some bold statements, but please remember that this is my personal opinion, based on my experience. For the most part, I've been involved in super high-traffic websites for the last few years and this has influenced my opinion on web frameworks. Just because I don't recommend your favorite framework doesn't mean it won't work for you. In fact, many of the best web applications today were built without an open source (or commercial) web framework. In the end, it's not as much about the web framework you're using as it is about hiring smart people. Below is my slide deck from this talk.

Yesterday, I did a GWT vs. Flex Smackdown with James Ward. While there wasn't as much trash talking as I'd hoped, I enjoyed delivering it and disputing the greatness of Flex. Below is the presentation that James and I delivered.

The show itself was great this year. It had more attendees than I've seen in a long time. There were a lot of really interesting sessions and and an often humorous Twitter back-channel. I attended quite a few talks and jotted down my notes from several of them. Please see the links below if you're interested in the sessions I attended. You can view all of the presentations from TSSJS 2010 on SlideShare.

Thanks to everyone who came to Vegas and to TheServerSide for an excellent conference.

Fantastic Fun in Jackson Hole

2010-03-12T07:04:57-07:00

For the last couple of years, I've done a ski trip with college buddies to an out-of-state destination. Two years ago, we went to Tahoe and had a great time. Last year, we did it again and I (amazingly) flew and rented a car without a driver's license. This year, we decided to switch things up a bit and head for Jackson Hole.

In previous years, only a couple of us went, but this year was organized by my good friend from Boston, Chris Morse. He managed to take it up a notch and invited a great group of guys, 9 of us in all. I knew about half the group, and met everyone else upon arrival.

The only unfortunate part about the trip was that no new snow fell. However, the Spring Skiing was warm and beautiful, somewhat making up for the lack of snow. The thing I enjoyed the most about this trip was how well the group jelled. Kudos to Chris for assembling such an awesome group and putting such a spectacular trip together. Can't wait for next year.

For further action of what the conditions where like, checkout the YouTube video I made. If only I'd recorded it for another 5 seconds to catch the digger that Corey takes at the end.

To see all the pictures we took on this adventure, checkout my Jackson Hole Set on Flickr.

My Future of Web Frameworks Presentation

2010-02-26T08:55:39-07:00

Earlier this week, I tweeted about a history of web frameworks timeline I created for my upcoming Future of Web Frameworks talk at TSSJS Vegas 2010. I immediately received a lot of feedback and requests for adding new frameworks and releases. The image below is the result of that Twitter conversation. Thanks to everyone who contributed.

Back in November, I wrote about my proposals for TSSJS. I've been thinking a lot about web frameworks lately and I can't help but think we live in a very exciting time. As a Java developer, I've been exposed to one of the most vibrant language ecosystems on the planet. As Tim Bray talks about, the Java Platform has 3 legs: the language, the virtual machine and a huge, immense library of APIs (both in the JDK and in open source libraries). The diagram below is something I created based on Tim's podcast.

Tim says, "One of those legs is replaceable and that's the language." And he's right, there's many Java.next languages that run efficiently on the JVM. This is one of the most exciting parts of being a Java web developer today. There's many proven web frameworks and languages that you can pick to build your next web application.

The best part is many of the best web frameworks run on the JVM. Not only that, but the best code editors are the IDEs that you're familiar with and have grown to love. Furthermore, much of the literature for Java.next languages is written for Java developers. As someone who knows Java, you have wealth of web frameworks and languages just waiting for you to learn them.

To create my presentation on the future of web frameworks, I followed the outline I posted previously. I plan on explaining the evolution and history of web frameworks and how we got to where we are today. From there, I'll be speculating on what web applications we'll be developing in the future. Finally, I'll touch on the necessary features of web frameworks that will allow us to develop these applications.

Of course, I haven't actually presented this talk yet, so it's likely to change in the coming weeks before the conference. The good news is this gives you the opportunity to provide constructive criticism on this presentation and help make it better. I realize that a presentation rarely represents the conversation that takes place during a conference. However, I believe it can portray the jist of my thinking and lead to a meaningful conversation in the comments of this post. Below is the presentation I created - thanks in advance for any feedback.

For those who will be joining me at TSSJS ... it's gonna be a great show. St. Patrick's Day in Vegas, what more could you ask for?

Update: This article has been re-posted on Javalobby and contains additional community feedback in the comments.

Upside Down Man Saves the Day

2010-01-31T16:54:31-07:00

Yesterday, Abbie and Jack showed you how to be a superhero. Today they're back with an action-packed video titled Upside Down Man Saves the Day. Watch it if you'd like to see how to take out a tiger, wrestle an alligator and diffuse a bomb. This short film was written, directed and filmed by Abbie, the only thing I did was add the music. Enjoy!

If you have trouble viewing it here, check it out on YouTube.

How to be a Super Hero by Abbie and Jack

2010-01-30T16:42:29-07:00

For some reason, my kids are very interested in Super Heros lately. In fact, they know so much about them that they've decided to start teaching others how to become one. Checkout the videos we shot earlier today with their step-by-step instructions.

I hope Abbie and Jack's tips help you become a super hero!

Comparing Kick-Ass Web Frameworks at The Rich Web Experience

2009-12-04T08:16:48-07:00

Yesterday, I delivered my Comparing Kick-Ass Web Frameworks talk at the Rich Web Experience in Orlando, Florida. Below are the slides I used:

Although it's difficult to convey a presentation in a slide deck, I can offer you my conclusion: there is no "best" web framework. I believe web frameworks are like spaghetti sauce in that everyone has different tastes and having so many choices is necessary to satisfy everyone. You can read more about the plural nature of perfection in Malcolm Gladwell's The Ketchup Conundrum (a written version of What we can learn from spaghetti sauce). Even though there is no "best" web framework, I believe GWT, Flex, Rails and Grails are frameworks that every web developer should try. They really do make it fun to develop web applications.

You can find the slides for my other RWE talk at Building SOFEA Applications with GWT and Grails.

Kudos to Jay Zimmerman for putting on a great show in Orlando this year. I had a great time talking with folks and learning in the sessions I attended. I particularly enjoyed bringing my parents and kids and staying at such a nice resort. Disney World (Magic Kingdom) and Universal Studios was very enjoyable due to the short lines. Also, the weather was perfect - especially considering the freezing cold in Denver this week.

Adding Expires Headers with OSCache's CacheFilter

2009-11-23T11:17:05-07:00

A couple of weeks ago, I wrote about how I improved this site's YSlow grade by concatenating JavaScript and CSS with wro4j. Even though I loved the improvements, there was still work to do:

I'm now sitting at a YSlow (V2) score of 75; 90 if I use the "Small Site or Blog" ruleset. I believe I can improve this by adding expires headers to my images, js and css.

Last Monday, wro4j 1.1.0 was released and I thought it would solve my last remaining issue. Unfortunately, it only adds expires headers (and ETags) to images referenced in included CSS. Of course, this makes sense, but I thought they'd add a filter to explicitly add expires headers.

Since I still wanted this feature, I did some searching around and found what I was looking for: OSCache's CacheFilter. It was surprisingly easy to setup, I downloaded OSCache 2.4.1, added it to my WEB-INF/lib directory, and added the following to my web.xml.

<filter>
    <filter-name>CacheFilter</filter-name>
    <filter-class>com.opensymphony.oscache.web.filter.CacheFilter</filter-class>
    <init-param>
        <param-name>expires</param-name>
        <param-value>time</param-value>
    </init-param>
    <init-param>
        <param-name>time</param-name>
        <param-value>2592000</param-value> <!-- one month -->
    </init-param>
    <init-param>
        <param-name>scope</param-name>
        <param-value>session</param-value>
    </init-param>
</filter>

<filter-mapping>
    <filter-name>CacheFilter</filter-name>
    <url-pattern>*.gif</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>CacheFilter</filter-name>
    <url-pattern>*.jpg</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>CacheFilter</filter-name>
    <url-pattern>*.png</url-pattern>
</filter-mapping>

After restarting Tomcat and clearing out my Firefox cache, I was in business.

I did experience one issue along the way when I tried to remove the oscache.jar from my WEB-INF/lib directory. I'm using the JSPWiki Plugin and it seems to rely on a class in oscache.jar. I'm not sure which version oscache.jar is, but the packages got moved around somewhere along the way. The good news is it seems OK to have both oscache.jar and oscache-2.4.1.jar in Roller's classpath.

After discovering the duplicate JARs issue, I got to thinkin' that EhCache would probably have a solution. Sure enough, it has a SimpleCachingHeadersPageCachingFilter. Since I already had a working solution, I didn't bother trying EhCache (especially since my Roller install uses EhCache 1.1 and the filter is only available in a later version). However, when I implement expires headers in AppFuse, I'll definitely try EhCache's solution.

As for my YSlow score, it didn't improve as much as I'd hoped (low 80s instead of mid 80s). Some of this is due to my embedded presentation from Slideshare. There's also some external images I'm using in my Lightbox JS implementation. So if I can find a better Lightbox implementation (supports rel="lightbox" syntax), there's a good chance I'll switch. In the meantime, I'm lovin' how much faster this site loads.

In case you're wondering, I do plan on adding css/js concatenation and expires headers to both AppFuse 2.1 and Roller 5.

Update: FWIW, I did try to configure expires headers in Apache, but the AJP 1.3 Connector doesn't seem to allow this to work. To quote Keith from KGB Internet:

I added an expires directive and it didn't touch the header for anything served from Tomcat, but does for content served directly by Apache. This might have to be set up in Tomcat.

Building SOFEA Applications with GWT and Grails

2009-11-12T09:30:09-07:00

Last night, I spoke at the Denver Java User Group meeting. The consulting panel with Matthew, Tim and Jim a lot of fun and I enjoyed delivering my Building SOFEA Applications with GWT and Grails presentation for the first time. The talk was mostly a story about how we enhanced Evite.com with GWT and Grails and what we did to make both frameworks scale. I don't believe the presentation reflects the story format that well, but it's not about the presentation, it's about the delivery of it.

If you'd like to hear the story about this successful SOFEA implementation at a high-volume site, I'd recommend attending the Rich Web Experience next month. If you attended last night's meeting and have any feedback on how this talk can be improved, I'd love to hear it.

Developing and Testing GWT Client Services

2009-10-21T06:55:17-06:00

Earlier this week, Hiram Chirino released RestyGWT, a GWT generator for REST services and JSON encoded data transfer objects. You can read more about it in Hiram's post RestyGWT, a Better GWT RPC??. First of all, I'm impressed with RestyGWT because provides something I've always wanted with GWT: the ability to call RESTful services and get a populated POJO in your callback, much like AsyncCallback provides for RPC services.

RestyGWT also allows you to easily create services using only interfaces and JAX-RS annotations. For example:

import javax.ws.rs.POST;
...
public interface PizzaService extends RestService {
    @POST
    public void order(PizzaOrder request, MethodCallback<OrderConfirmation> callback);
}

After taking a brief look at RestyGWT, I thought it'd be interesting to share how I develop and test GWT client services.

Developing GWT Client Services
Writing services in a GWT application can be helpful when you're using MVP, especially since you can EasyMock them in a test. On my GWT projects, I've often used overlay types because they allow me to write less code and they make parsing JSON super simple. I've had issues testing my presenters when using overlay types. The good news is I think I've figured out a reasonable solution, but it does require using GWTTestCase. If RestyGWT supported overlay types, there's a good chance I'd use it, especially since its integration tests seem to require GWTTestCase too.

Rather than using callbacks in my presenters, I try to only use them in my service implementations. That way, my presenters don't have to worry about overlay types and can be tested in a JUnit-only fashion. The callbacks in my services handle JSON parsing/object population and fire events with the populated objects.

GWT's RequestBuilder is one option for communicating with RESTful services. The Development Guide for HTTP Requests explains how to use this class. To simplify REST requests and allow multiple callbacks, I'm using a RestRequest class, and a number of other utility classes that make up a small GWT REST framework (created by a former colleague). RestRequest wraps RequestBuilder and provides a Fluent API for executing HTTP requests. Another class, Deferred, is a GWT implementation of Twisted's Deferred.

As part of my service implementation, I inject an EventBus (with GIN) into the constructor and then proceed to implement callbacks that fire Events to indicate loading, saving and deleting has succeeded. Here's an example service:

public class ConversationServiceImpl implements ConversationService {
    private EventBus eventBus;

    @Inject
    public ConversationServiceImpl(EventBus eventBus) {
        this.eventBus = eventBus;
    }

    public void getConversation(String name) {
        Deferred<Representation> d =
                RestRequest.get(URLs.CONVERSATION + "/" + URL.encode(name)).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceLoadedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void saveConversation(Conversation conversation) {
        Deferred<Representation> d = RestRequest.post(URLs.CONVERSATION)
                .setRequestData(conversation.toJson()).build();
        
        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                Conversation conversation = convertResultToConversation(result);
                eventBus.fireEvent(new ResourceSavedEvent<Conversation>(conversation));
            }
        });

        d.run();
    }

    public void deleteConversation(Long id) {
        Deferred<Representation> d =
                RestRequest.post(URLs.CONVERSATION + "/" + id).build();

        d.addCallback(new Callback<Representation>() {
            public void onSuccess(Representation result) {
                eventBus.fireEvent(new ResourceDeletedEvent());
            }
        });

        d.run();
    }

    /**
     * Convenience method to populate object in one location
     *
     * @param result the result of a resource request.
     * @return the populated object.
     */
    private Conversation convertResultToConversation(Representation result) {
        JSOModel model = JSOModel.fromJson(result.getData());
        return new Conversation(model);
    }
}

In the saveConversation() method you'll notice the conversation.toJson() method call. This method uses a JSON class that loops through an objects properties and constructs a JSON String.

public JSON toJson() {
    return new JSON(getMap());
}

Testing Services
In my experience, the hardest part about using overlay types is writing your objects so they get populated correctly. I've found that writing tests which read JSON from a file can be a great productivity boost. However, because of overlay types, you have to write a test that extends GWTTestCase. When using GWTTestCase, you can't simply read from the filesystem. The good news is there is a workaround where you can subclass GWTShellServlet and overwrite GWT's web.xml to have your own servlet that can read from the filesystem. A detailed explanation of how to do this was written by Alex Moffat in Implementing a -noserver flag for GWTTestCase.

Once this class is in place, I've found you can easily write services using TDD and the server doesn't even have to exist. When constructing services, I've found the following workflow to be the most productive:

Create a file with the expected JSON in src/test/resources/resource.json where resource matches the last part of the URL for your service.
Create a *ServiceGwtTest.java and write tests.
Run tests to make sure they fail.
Implement the service and run tests to ensure JSON is getting consumed/produced properly to/from model objects.

Below is the code for my JsonReaderServlet.java:

public class JsonReaderServlet extends GWTShellServlet {

    public void service(ServletRequest servletRequest, ServletResponse servletResponse)
            throws ServletException, IOException {

        HttpServletRequest req = (HttpServletRequest) servletRequest;
        HttpServletResponse resp = (HttpServletResponse) servletResponse;

        String uri = req.getRequestURI();
        if (req.getQueryString() != null) {
            uri += "?" + req.getQueryString();
        }

        if (uri.contains("/services")) {
            String method = req.getMethod();
            String output;

            if (method.equalsIgnoreCase("get")) {
                // use the part after the last slash as the filename
                String filename = uri.substring(uri.lastIndexOf("/") + 1, uri.length()) + ".json";
                System.out.println("loading: " + filename);
                String json = readFileAsString("/" + filename);
                System.out.println("loaded json: " + json);
                output = json;
            } else {
                // for posts, return the same body content
                output = getBody(req);
            }

            PrintWriter out = resp.getWriter();
            out.write(output);
            out.close();

            resp.setStatus(HttpServletResponse.SC_OK);
        } else {
            super.service(servletRequest, servletResponse);
        }
    }

    private String readFileAsString(String filePath) throws IOException {
        filePath = getClass().getResource(filePath).getFile();
        BufferedReader reader = new BufferedReader(new FileReader(filePath));
        return getStringFromReader(reader);
    }

    private String getBody(ServletRequest request) throws IOException {
        BufferedReader reader = new BufferedReader(new InputStreamReader(request.getInputStream()));
        return getStringFromReader(reader);
    }

    private String getStringFromReader(Reader reader) throws IOException {
        StringBuilder sb = new StringBuilder();
        char[] buf = new char[1024];
        int numRead;
        while ((numRead = reader.read(buf)) != -1) {
            sb.append(buf, 0, numRead);
        }
        reader.close();
        return sb.toString();
    }
}

This servlet is mapped to <url-pattern>/*</url-pattern> in a web.xml file in src/test/resources/com/google/gwt/dev/etc/tomcat/webapps/ROOT/WEB-INF.

My Service Test starts by getting an EventBus from GIN and registering itself to handle the fired events.

public class ConversationServiceGwtTest extends AbstractGwtTestCase
        implements ResourceLoadedEvent.Handler, ResourceSavedEvent.Handler, ResourceDeletedEvent.Handler {
    ConversationService service;
    ResourceLoadedEvent<Conversation> loadedEvent;
    ResourceSavedEvent<Conversation> savedEvent;
    ResourceDeletedEvent deletedEvent;

    @Override
    public void gwtSetUp() throws Exception {
        super.gwtSetUp();
        DesigntimeGinjector injector = GWT.create(MyGinjector.class);
        EventBus eventBus = injector.getEventBus();
        service = new ConversationServiceImpl(eventBus);
        eventBus.addHandler(ResourceLoadedEvent.ENGINE, this);
        eventBus.addHandler(ResourceSavedEvent.ENGINE, this);
        eventBus.addHandler(ResourceDeletedEvent.ENGINE, this);
    }

    @SuppressWarnings("unchecked")
    public void onLoad(ResourceLoadedEvent event) {
        this.loadedEvent = event;
    }

    @SuppressWarnings("unchecked")
    public void onSave(ResourceSavedEvent event) {
        this.savedEvent = event;
    }

    public void onDelete(ResourceDeletedEvent event) {
        this.deletedEvent = event;
    }
}

After this groundwork has been done, a test can be written that loads up the JSON file and verifies the objects are populated correctly.

public void testGetConversation() {

    service.getConversation("test-conversation");

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceLoadedEvent not received", loadedEvent);
            Conversation conversation = loadedEvent.getResource();
            assertEquals("Conversation name is incorrect","Test Conversation", conversation.getName());

            assertNotNull("Conversation has no channel", conversation.getChannel());
            assertEquals("Conversation has incorrect task size", 3, conversation.getTasks().size());

            convertToAndFromJson(conversation);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

private void convertToAndFromJson(Conversation fromJsonModel) {
    Representation json = fromJsonModel.toJson();
    assertNotNull("Cannot convert empty JSON", json.getData());

    // change back into model
    JSOModel data = JSOModel.fromJson(json.getData());
    Conversation toJsonModel = new Conversation(data);
    verifyModelBuiltCorrectly(toJsonModel);
}

private void verifyModelBuiltCorrectly(Conversation model) {
    assertEquals("Conversation name is incorrect", "Test Conversation", model.getString("name"));
    assertEquals("Conversation has incorrect task size", 3, model.getTasks().size());
    assertEquals("Conversation channel is incorrect", "Web", model.getChannel().getString("type"));
}

For more information on the usage of the Timer, finishTest() and delayTestFinish(), see GWTTestCase's javadoc.

The tests for saving and deleting a resource look as follows:

public void testSaveConversation() {
    Conversation conversation = new Conversation().setName("Test").setId("1");

    List<Task> tasks = new ArrayList<Task>();
    for (int i = 1; i < 4; i++) {
        tasks.add(new Task().setName("Task " + i));
    }
    conversation.setTasks(tasks);

    System.out.println("conversation.toJson(): " + conversation.toJson());

    assertTrue(conversation.toJson().toString().contains("Task 1"));

    service.saveConversation(conversation);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceSavedEvent not received", savedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}
  
public void testDeleteConversation() {
    service.deleteConversation(1L);

    Timer t = new Timer() {
        public void run() {
            assertNotNull("ResourceDeletedEvent not received", deletedEvent);
            finishTest();
        }
    };

    delayTestFinish(3000);
    t.schedule(100);
}

Summary
This article has shown you how I develop and test GWT Client Services. If RestyGWT supported overlay types, there's a good chance I could change my service implementation to use it and I wouldn't have to change my test. Robert Cooper, author of GWT in Practice, claims he has a framework that does this. Here's to hoping this article stimulates the GWT ecosystem and we get a GWT REST framework that's as easy to use as GWT RPC.

Update: Today I enhanced this code to use Generics-based classes (inspired by Don't repeat the DAO!) for the boiler-plate CRUD code in a service. In a nutshell, a service interface can now be written as:

public interface FooService extends GenericService<Foo, String> {
 
}

The implementation class is responsible for the URL and converting the JSON result to an object:

public class FooServiceImpl extends GenericServiceImpl<Foo, String> implements FooService {

    @Inject
    public FooServiceImpl(EventBus eventBus) {
        super(eventBus, "/services/foo");
    }

    @Override
    protected Foo convertResultToModel(Representation result) {
        return new Foo(JSOModel.fromJson(result.getData()));
    }
}

I'm sure this can be further enhanced to get rid of the need to create classes altogether, possibly leveraging GIN or some sort of factory. The parent classes referenced in this code can be viewed at the following URLs:

There's also a GenericServiceGwtTest.java that proves it all works as expected.

Integrating GWT with Spring Security

2009-08-06T08:50:15-06:00

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

How to do cross-domain GWT RPC with a ProxyServlet

2009-08-05T16:06:12-06:00

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPInputStream;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;

/**
 * ProxyServlet from http://edwardstx.net/wiki/attach/HttpProxyServlet/ProxyServlet.java
 * (This seems to be a derivative of Noodle -- http://noodle.tigris.org/)
 * 
 * Patched to skip "Transfer-Encoding: chunked" headers, avoid double slashes
 * in proxied URLs, handle GZip and allow GWT RPC.
 */
public class ProxyServlet extends HttpServlet {

    private static final int FOUR_KB = 4196;

    /**
     * Serialization UID.
     */
    private static final long serialVersionUID = 1L;
    /**
     * Key for redirect location header.
     */
    private static final String STRING_LOCATION_HEADER = "Location";
    /**
     * Key for content type header.
     */
    private static final String STRING_CONTENT_ENGINE_HEADER_NAME = "Content-Type";
    /**
     * Key for content length header.
     */
    private static final String STRING_CONTENT_LENGTH_HEADER_NAME = "Content-Length";
    /**
     * Key for host header
     */
    private static final String STRING_HOST_HEADER_NAME = "Host";
    /**
     * The directory to use to temporarily store uploaded files
     */
    private static final File FILE_UPLOAD_TEMP_DIRECTORY = new File(System.getProperty("java.io.tmpdir"));

    // Proxy host params
    /**
     * The host to which we are proxying requests. Default value is "localhost".
     */
    private String stringProxyHost = "localhost";
    /**
     * The port on the proxy host to wihch we are proxying requests. Default value is 80.
     */
    private int intProxyPort = 80;
    /**
     * The (optional) path on the proxy host to wihch we are proxying requests. Default value is "".
     */
    private String stringProxyPath = "";
    /**
     * Setting that allows removing the initial path from client. Allows specifying /twitter/* as synonym for twitter.com.
     */
    private boolean removePrefix;
    /**
     * The maximum size for uploaded files in bytes. Default value is 5MB.
     */
    private int intMaxFileUploadSize = 5 * 1024 * 1024;
    private boolean isSecure;
    private boolean followRedirects;

    /**
     * Initialize the <code>ProxyServlet</code>
     * @param servletConfig The Servlet configuration passed in by the servlet container
     */
    public void init(ServletConfig servletConfig) {
        // Get the proxy host
        String stringProxyHostNew = servletConfig.getInitParameter("proxyHost");
        if (stringProxyHostNew == null || stringProxyHostNew.length() == 0) {
            throw new IllegalArgumentException("Proxy host not set, please set init-param 'proxyHost' in web.xml");
        }
        this.setProxyHost(stringProxyHostNew);
        // Get the proxy port if specified
        String stringProxyPortNew = servletConfig.getInitParameter("proxyPort");
        if (stringProxyPortNew != null && stringProxyPortNew.length() > 0) {
            this.setProxyPort(Integer.parseInt(stringProxyPortNew));
        }
        // Get the proxy path if specified
        String stringProxyPathNew = servletConfig.getInitParameter("proxyPath");
        if (stringProxyPathNew != null && stringProxyPathNew.length() > 0) {
            this.setProxyPath(stringProxyPathNew);
        }
        // Get the maximum file upload size if specified
        String stringMaxFileUploadSize = servletConfig.getInitParameter("maxFileUploadSize");
        if (stringMaxFileUploadSize != null && stringMaxFileUploadSize.length() > 0) {
            this.setMaxFileUploadSize(Integer.parseInt(stringMaxFileUploadSize));
        }
    }

    /**
     * Performs an HTTP GET request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a GET request
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("GET Request URL: " + httpServletRequest.getRequestURL(),
              "Destination URL: " + destinationUrl);
        GetMethod getMethodProxyRequest = new GetMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, getMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
        // Execute the proxy request
        this.executeProxyRequest(getMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Performs an HTTP POST request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a standard POST request
        String contentType = httpServletRequest.getContentType();
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("POST Request URL: " + httpServletRequest.getRequestURL(),
              "    Content Type: " + contentType,
              " Destination URL: " + destinationUrl);
        PostMethod postMethodProxyRequest = new PostMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, postMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, postMethodProxyRequest);
        // Check if this is a mulitpart (file upload) POST
        if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
            this.handleMultipartPost(postMethodProxyRequest, httpServletRequest);
        } else {
            if (contentType == null || PostMethod.FORM_URL_ENCODED_CONTENT_ENGINE.equals(contentType)) {
                this.handleStandardPost(postMethodProxyRequest, httpServletRequest);
            } else {
                this.handleContentPost(postMethodProxyRequest, httpServletRequest);
            }
        }
        // Execute the proxy request
        this.executeProxyRequest(postMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Sets up the given {@link PostMethod} to send the same multipart POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a multipart POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the mutlipart POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleMultipartPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest)
            throws ServletException {
        // Create a factory for disk-based file items
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        // Set factory constraints
        diskFileItemFactory.setSizeThreshold(this.getMaxFileUploadSize());
        diskFileItemFactory.setRepository(FILE_UPLOAD_TEMP_DIRECTORY);
        // Create a new file upload handler
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        // Parse the request
        try {
            // Get the multipart items as a list
            List<FileItem> listFileItems = (List<FileItem>) servletFileUpload.parseRequest(httpServletRequest);
            // Create a list to hold all of the parts
            List<Part> listParts = new ArrayList<Part>();
            // Iterate the multipart items list
            for (FileItem fileItemCurrent : listFileItems) {
                // If the current item is a form field, then create a string part
                if (fileItemCurrent.isFormField()) {
                    StringPart stringPart = new StringPart(
                            fileItemCurrent.getFieldName(), // The field name
                            fileItemCurrent.getString() // The field value
                            );
                    // Add the part to the list
                    listParts.add(stringPart);
                } else {
                    // The item is a file upload, so we create a FilePart
                    FilePart filePart = new FilePart(
                            fileItemCurrent.getFieldName(), // The field name
                            new ByteArrayPartSource(
                            fileItemCurrent.getName(), // The uploaded file name
                            fileItemCurrent.get() // The uploaded file contents
                            ));
                    // Add the part to the list
                    listParts.add(filePart);
                }
            }
            MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(
                    listParts.toArray(new Part[]{}),
                    postMethodProxyRequest.getParams());
            postMethodProxyRequest.setRequestEntity(multipartRequestEntity);
            // The current content-type header (received from the client) IS of
            // type "multipart/form-data", but the content-type header also
            // contains the chunk boundary string of the chunks. Currently, this
            // header is using the boundary of the client request, since we
            // blindly copied all headers from the client request to the proxy
            // request. However, we are creating a new request with a new chunk
            // boundary string, so it is necessary that we re-set the
            // content-type string to reflect the new chunk boundary string
            postMethodProxyRequest.setRequestHeader(STRING_CONTENT_ENGINE_HEADER_NAME, multipartRequestEntity.getContentType());
        } catch (FileUploadException fileUploadException) {
            throw new ServletException(fileUploadException);
        }
    }

    /**
     * Sets up the given {@link PostMethod} to send the same standard POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleStandardPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) {
        // Get the client POST data as a Map
        Map<String, String[]> mapPostParameters = (Map<String, String[]>) httpServletRequest.getParameterMap();
        // Create a List to hold the NameValuePairs to be passed to the PostMethod
        List<NameValuePair> listNameValuePairs = new ArrayList<NameValuePair>();
        // Iterate the parameter names
        for (String stringParameterName : mapPostParameters.keySet()) {
            // Iterate the values for each parameter name
            String[] stringArrayParameterValues = mapPostParameters.get(stringParameterName);
            for (String stringParamterValue : stringArrayParameterValues) {
                // Create a NameValuePair and store in list
                NameValuePair nameValuePair = new NameValuePair(stringParameterName, stringParamterValue);
                listNameValuePairs.add(nameValuePair);
            }
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestBody(listNameValuePairs.toArray(new NameValuePair[]{}));
    }

    /**
     * Sets up the given {@link PostMethod} to send the same content POST
     * data (JSON, XML, etc.) as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    private void handleContentPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) throws IOException, ServletException {
        StringBuilder content = new StringBuilder();
        BufferedReader reader = httpServletRequest.getReader();
        for (;;) {
            String line = reader.readLine();
            if (line == null) break;
            content.append(line);
        }

        String contentType = httpServletRequest.getContentType();
        String postContent = content.toString();

        if (contentType.startsWith("text/x-gwt-rpc")) {
            String clientHost = httpServletRequest.getLocalName();
            if (clientHost.equals("127.0.0.1")) {
                clientHost = "localhost";
            }

            int clientPort = httpServletRequest.getLocalPort();
            String clientUrl = clientHost + ((clientPort != 80) ? ":" + clientPort : "");
            String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + intProxyPort : "") + httpServletRequest.getServletPath();
            //debug("Replacing client (" + clientUrl + ") with server (" + serverUrl + ")");
            postContent = postContent.replace(clientUrl , serverUrl);
        }

        String encoding = httpServletRequest.getCharacterEncoding();
        debug("POST Content Type: " + contentType + " Encoding: " + encoding,
              "Content: " + postContent);
        StringRequestEntity entity;
        try {
            entity = new StringRequestEntity(postContent, contentType, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new ServletException(e);
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestEntity(entity);
    }

    /**
     * Executes the {@link HttpMethod} passed in and sends the proxy response
     * back to the client via the given {@link HttpServletResponse}
     * @param httpMethodProxyRequest An object representing the proxy request to be made
     * @param httpServletResponse An object by which we can send the proxied
     *                             response back to the client
     * @throws IOException Can be thrown by the {@link HttpClient}.executeMethod
     * @throws ServletException Can be thrown to indicate that another error has occurred
     */
    private void executeProxyRequest(
            HttpMethod httpMethodProxyRequest,
            HttpServletRequest httpServletRequest,
            HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a default HttpClient
        HttpClient httpClient = new HttpClient();
        httpMethodProxyRequest.setFollowRedirects(false);
        // Execute the request
        int intProxyResponseCode = httpClient.executeMethod(httpMethodProxyRequest);
        String response = httpMethodProxyRequest.getResponseBodyAsString();

        // Check if the proxy response is a redirect
        // The following code is adapted from org.tigris.noodle.filters.CheckForRedirect
        // Hooray for open source software
        if (intProxyResponseCode >= HttpServletResponse.SC_MULTIPLE_CHOICES /* 300 */ && intProxyResponseCode < HttpServletResponse.SC_NOT_MODIFIED /* 304 */) {
            String stringStatusCode = Integer.toString(intProxyResponseCode);
            String stringLocation = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
            if (stringLocation == null) {
                throw new ServletException("Received status code: " + stringStatusCode + " but no " + STRING_LOCATION_HEADER + " header was found in the response");
            }
            // Modify the redirect to go to this proxy servlet rather that the proxied host
            String stringMyHostName = httpServletRequest.getServerName();
            if (httpServletRequest.getServerPort() != 80) {
                stringMyHostName += ":" + httpServletRequest.getServerPort();
            }
            stringMyHostName += httpServletRequest.getContextPath();
            if (followRedirects) {
                if (stringLocation.contains("jsessionid")) {
                    Cookie cookie = new Cookie("JSESSIONID", stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
                    cookie.setPath("/");
                    httpServletResponse.addCookie(cookie);
                    //debug("redirecting: set jessionid (" + cookie.getValue() + ") cookie from URL");
                } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
	                Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
                    String[] cookieDetails = header.getValue().split(";");
					String[] nameValue = cookieDetails[0].split("=");

					Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
					cookie.setPath("/");
					//debug("redirecting: setting cookie: " + cookie.getName() + ":" + cookie.getValue() + " on " + cookie.getPath());
					httpServletResponse.addCookie(cookie);
                }
                httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() + this.getProxyPath(), stringMyHostName));
                return;
            }
        } else if (intProxyResponseCode == HttpServletResponse.SC_NOT_MODIFIED) {
            // 304 needs special handling.  See:
            // http://www.ics.uci.edu/pub/ietf/http/rfc1945.html#Code304
            // We get a 304 whenever passed an 'If-Modified-Since'
            // header and the data on disk has not changed; server
            // responds w/ a 304 saying I'm not going to send the
            // body because the file has not changed.
            httpServletResponse.setIntHeader(STRING_CONTENT_LENGTH_HEADER_NAME, 0);
            httpServletResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
            return;
        }

        // Pass the response code back to the client
        httpServletResponse.setStatus(intProxyResponseCode);

        // Pass response headers back to the client
        Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
        for (Header header : headerArrayResponse) {
            if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
                    header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") || // don't copy gzip header
                    header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header so browser doesn't prompt on failed basic auth
                // proxy servlet does not support chunked encoding
            } else {
                httpServletResponse.setHeader(header.getName(), header.getValue());
            }
        }

        List<Header> responseHeaders = Arrays.asList(headerArrayResponse);

        if (isBodyParameterGzipped(responseHeaders)) {
            debug("GZipped: true");
            if (!followRedirects && intProxyResponseCode == HttpServletResponse.SC_MOVED_TEMPORARILY) {
                response = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
                httpServletResponse.setStatus(HttpServletResponse.SC_OK);
                intProxyResponseCode = HttpServletResponse.SC_OK;
                httpServletResponse.setHeader(STRING_LOCATION_HEADER, response);
            } else {
                response = new String(ungzip(httpMethodProxyRequest.getResponseBody()));
            }
            httpServletResponse.setContentLength(response.length());
        }

        // Send the content to the client
        debug("Received status code: " + intProxyResponseCode,
              "Response: " + response);

        httpServletResponse.getWriter().write(response);
    }


    /**
     * The response body will be assumed to be gzipped if the GZIP header has been set.
     *
     * @param responseHeaders of response headers
     * @return true if the body is gzipped
     */
    private boolean isBodyParameterGzipped(List<Header> responseHeaders) {
        for (Header header : responseHeaders) {
            if (header.getValue().equals("gzip")) {
                return true;
            }
        }
        return false;
    }

    /**
     * A highly performant ungzip implementation. Do not refactor this without taking new timings.
     * See ElementTest in ehcache for timings
     *
     * @param gzipped the gzipped content
     * @return an ungzipped byte[]
     * @throws java.io.IOException when something bad happens
     */
    private byte[] ungzip(final byte[] gzipped) throws IOException {
        final GZIPInputStream inputStream = new GZIPInputStream(new ByteArrayInputStream(gzipped));
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(gzipped.length);
        final byte[] buffer = new byte[FOUR_KB];
        int bytesRead = 0;
        while (bytesRead != -1) {
            bytesRead = inputStream.read(buffer, 0, FOUR_KB);
            if (bytesRead != -1) {
                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }
        }
        byte[] ungzipped = byteArrayOutputStream.toByteArray();
        inputStream.close();
        byteArrayOutputStream.close();
        return ungzipped;
    }

    public String getServletInfo() {
        return "GWT Proxy Servlet";
    }

    /**
     * Retrieves all of the headers from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestHeaders(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an Enumeration of all of the header names sent by the client
        Enumeration enumerationOfHeaderNames = httpServletRequest.getHeaderNames();
        while (enumerationOfHeaderNames.hasMoreElements()) {
            String stringHeaderName = (String) enumerationOfHeaderNames.nextElement();
            if (stringHeaderName.equalsIgnoreCase(STRING_CONTENT_LENGTH_HEADER_NAME)) {
                continue;
            }
            // As per the Java Servlet API 2.5 documentation:
            //		Some headers, such as Accept-Language can be sent by clients
            //		as several headers each with a different value rather than
            //		sending the header as a comma separated list.
            // Thus, we get an Enumeration of the header values sent by the client
            Enumeration enumerationOfHeaderValues = httpServletRequest.getHeaders(stringHeaderName);
            while (enumerationOfHeaderValues.hasMoreElements()) {
                String stringHeaderValue = (String) enumerationOfHeaderValues.nextElement();
                // In case the proxy host is running multiple virtual servers,
                // rewrite the Host header to ensure that we get content from
                // the correct virtual server
                if (stringHeaderName.equalsIgnoreCase(STRING_HOST_HEADER_NAME)) {
                    stringHeaderValue = getProxyHostAndPort();
                }
                Header header = new Header(stringHeaderName, stringHeaderValue);
                // Set the same header on the proxy request
                httpMethodProxyRequest.setRequestHeader(header);
            }
        }
    }

    /**
     * Retrieves all of the cookies from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestCookies(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an array of all of all the cookies sent by the client
        Cookie[] cookies = httpServletRequest.getCookies();
        if (cookies == null) {
            return;
        }

        for (Cookie cookie : cookies) {
            cookie.setDomain(stringProxyHost);
            cookie.setPath(httpServletRequest.getServletPath());
            httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() + "=" + cookie.getValue() + "; Path=" + cookie.getPath());
        }
    }

    // Accessors
    private String getProxyURL(HttpServletRequest httpServletRequest) {
        // Set the protocol to HTTP
        String protocol = (isSecure) ? "https://" : "http://";
        String stringProxyURL = protocol + this.getProxyHostAndPort();

        // simply use whatever servlet path that was part of the request as opposed to getting a preset/configurable proxy path
        if (!removePrefix) {
            stringProxyURL += httpServletRequest.getServletPath();
        }
        stringProxyURL += "/";
        
        // Handle the path given to the servlet
        String pathInfo = httpServletRequest.getPathInfo();
        if (pathInfo != null && pathInfo.startsWith("/")) {
            if (stringProxyURL != null && stringProxyURL.endsWith("/")) {
                // avoid double '/'
                stringProxyURL += pathInfo.substring(1);
            }
        } else {
            stringProxyURL += httpServletRequest.getPathInfo();
        }
        // Handle the query string
        if (httpServletRequest.getQueryString() != null) {
            stringProxyURL += "?" + httpServletRequest.getQueryString();
        }
        
        return stringProxyURL;
    }

    private String getProxyHostAndPort() {
        if (this.getProxyPort() == 80) {
            return this.getProxyHost();
        } else {
            return this.getProxyHost() + ":" + this.getProxyPort();
        }
    }

    protected String getProxyHost() {
        return this.stringProxyHost;
    }

    protected void setProxyHost(String stringProxyHostNew) {
        this.stringProxyHost = stringProxyHostNew;
    }

    protected int getProxyPort() {
        return this.intProxyPort;
    }

    protected void setSecure(boolean secure) {
        this.isSecure = secure;
    }
    
    protected void setFollowRedirects(boolean followRedirects) {
        this.followRedirects = followRedirects;
    }

    protected void setProxyPort(int intProxyPortNew) {
        this.intProxyPort = intProxyPortNew;
    }

    protected String getProxyPath() {
        return this.stringProxyPath;
    }

    protected void setProxyPath(String stringProxyPathNew) {
        this.stringProxyPath = stringProxyPathNew;
    }

    protected void setRemovePrefix(boolean removePrefix) {
        this.removePrefix = removePrefix;
    }

    protected int getMaxFileUploadSize() {
        return this.intMaxFileUploadSize;
    }

    protected void setMaxFileUploadSize(int intMaxFileUploadSizeNew) {
        this.intMaxFileUploadSize = intMaxFileUploadSizeNew;
    }

    private void debug(String ... msg) {
        for (String m : msg) {
            System.out.println("[DEBUG] " + m);
        }
    }
}

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.

Abbie is a Blue Skier!

2008-12-14T21:55:44-07:00

On Friday afternoon, the kids and I headed up to Winter Park for a night at Zephyr Lodge. The drive up was great (no traffic) and it started snowing as soon as we got off I-70. 45 minutes later and the kids were on the slopes for the last few runs of the day. There was a few inches of snow that turned out to be great - it slowed them down enough that they didn't have to turn or worry about "pizza".

On Saturday, we put Jack in Ski School and Abbie and I had a "Daddy + Daughter" day on the mountain. She didn't want to hit the magic carpet and instead opted for the lift right away. We skied a couple greens and w/in an hour she was ready to try a blue. She did quite well on the blue and even skied a few bumps on the side. Yes, she did fall a few times, but she got up by herself and always had a smile on her face. I was extremely proud. Below are a few pictures and a video from our weekend.

I'm glad I took the kids skiing yesterday. It's been dumping with cold temperatures ever since we left. It's currently -15°F in Denver (a new record) and it's snowed 2 feet at some resorts. Now I just need to figure out a way to make it up a couple of times this week.

Building Rich Applications with Appcelerator

2008-10-22T16:18:42-06:00

This afternoon, I delivered my Building Rich Applications with Appcelerator talk for the 3rd time at Colorado Software Summit. When I first proposed this topic, I hadn't used Appcelerator and saw this as a good opportunity to learn more about it. I'm glad I did.

IMO, Appcelerator is a lot like Dojo in how it parses pages and turns HTML with special attributes into JavaScript widgets. I can't help but think a pre-compilation step would be nice to speed things up. I like Appcelerator's extensive Widget Library, and I especially like that they re-use many widgets rather than re-creating their own. Finally, I really dig the "SOA in a browser" approach where everything is a message and you can easily publish and subscribe to events - on the client and server. Below is my presentation, please let me know if you have any questions.

View on Slideshare

What's Coming in Spring 3.0

2008-10-22T11:51:12-06:00

This morning, I delivered my What's Coming in Spring 3.0 talk for the 2nd time at Colorado Software Summit. Since there is no Spring 3.0 source code to speak of, I was unable to do the "Choose Your Own Adventure" demo at the end.

The good news is I was able to easily upgrade the Spring Kickstart application from Spring 2.0 to Spring 2.5.5 (using annotations). When 3.0 is released, I hope to update this project to use 3.0 as well as show what I needed to change. If I get ambitious, I might even change the UI to use Flex or Ext JS to show a RESTful client. Below is my presentation - hope you enjoy.

View on Slideshare

Oktoberfest: Best. Vacation. Ever.

2008-10-01T22:37:49-06:00

As mentioned earlier, a good friend (Miller) and I took a vacation to Oktoberfest last week. The flight over was brutal: Denver -> Newark (4 hour layover) -> Paris (2 hour layover) -> Munich. I was pretty disappointed to see "No Service" on my iPhone when we landed in France. This continued in Munich and I quickly decided connectivity wasn't necessary. After a 80 € cab ride from the airport, we arrived at our hotel around 1PM on Monday. We both woke up at 4AM on Sunday to start the journey, so we were pretty beat.

It didn't take us long to decide we had to suck it up and head to "The Tents" at Oktoberfest. We walked the 4 blocks in the rain and quickly ended up in the Armbrustschutzenzelt tent. The crowd was small and we found a table and enjoyed our first Liter.

I won't go into details about how much fun we had during the week, except for these simple stats:

Made it into The Tents 5 days in a row
Mostly slept in until 2 or 4PM each day
No Hangovers (for me at least)

I've uploaded most of the pictures I took to an Oktoberfest 2008 set on Flickr. I would like to thank Peter and friends for being fabulous hosts and showing us a great time on Thursday night. Below is a video I took of what the festivities were like (yes, that is Bon Jovi):

If you ever get a chance to attend Oktoberfest, I highly recommend it.

[OSCON 2008] Web Frameworks of the Future: Flex, GWT, Grails and Rails

2008-07-23T16:25:23-06:00

Below is the presentation I'm delivering at OSCON today. Unfortunately, I had to remove slides on GWT and Flex to fit w/in the 45 minute time limit. I hope to expand this presentation in the future, as well as continue to develop the side project I'm working on using these technologies.

view on slideshare | download

Running Spring MVC Web Applications in OSGi

2008-04-30T00:42:34-06:00

For the past couple of weeks, I've been developing a web application that deploys into an OSGi container (Equinox) and uses Spring DM's Spring MVC support. The first thing I discovered was that Spring MVC's annotations weren't supported in the M1 release. This was apparently caused by a bug in Spring 2.5.3 and not Spring DM. Since Spring DM 1.1.0 M2 was released with Spring 2.5.4 today, I believe this is fixed now.

The story below is about my experience getting a Spring MVC application up and running in Equinox 3.2.2, Jetty 6.1.9 and Spring DM 1.1.0 M2 SNAPSHOT (from last week). If you want to read more about why Spring MVC + OSGi is cool, see Costin Leau's Web Applications and OSGi article.

To get a simple "Hello World" Spring MVC application working in OSGi is pretty easy. The hard part is setting up a container with all the Spring and Jetty bundles installed and started. I imagine SSAP might solve this. Luckily for me, this was done by another member of my team.

After you've done this, it's simply a matter of creating a MANIFEST.MF for your WAR that contains the proper information for OSGi to recognize. Below is the one that I used when I first tried to get my application working.

Manifest-Version: 1
Bundle-ManifestVersion: 2
Spring-DM-Version: 1.1.0-m2-SNAPSHOT
Spring-Version: 2.5.2
Bundle-Name: Simple OSGi War
Bundle-SymbolicName: myapp
Bundle-Classpath: .,WEB-INF/classes,WEB-INF/lib/freemarker-2.3.12.jar,
 WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar,
 WEB-INF/lib/spring-beans-2.5.2.jar,WEB-INF/lib/spring-context-2.5.2.jar,
 WEB-INF/lib/spring-context-support-2.5.2.jar,WEB-INF/lib/spring-core-2.5.2.jar,
 WEB-INF/lib/spring-web-2.5.2.jar,WEB-INF/lib/spring-webmvc-2.5.2.jar 
Import-Package: javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional

Ideally, you could generate this MANIFEST.MF using the maven-bundle-plugin. However, it doesn't support WARs in its 1.4.0 release.

You can see this is an application that uses Spring MVC, FreeMarker, SiteMesh and the URLRewriteFilter. You should be able to download it, unzip it, run "mvn package" and install it into Equinox using "install file://<path to war>".

That's all fine and dandy, but doesn't give you any benefits of OSGi. This setup works great until you try to import OSGi services using a context file with an <osgi:reference> element. After adding such a reference, it's likely you'll get the following error:

SEVERE: Context initialization failed
org.springframework.beans.factory.parsing.BeanDefinitionParsingException:
Configuration problem: Unable to locate Spring NamespaceHandler for
XML schema namespace [http://www.springframework.org/schema/osgi]

To fix this, add the following to your web.xml (if you're using ContextLoaderListener, as an <init-parameter> on DispatcherServlet if you're not):

  <context-param>
    <param-name>contextClass</param-name>
    <param-value>org.springframework.osgi.web.context.support.OsgiBundleXmlWebApplicationContext</param-value>
  </context-param>

After doing this, you might get the following error on startup:

SEVERE: Context initialization failed
org.springframework.context.ApplicationContextException: Custom
context class [org.springframework.osgi.web.context.support.OsgiBundleXmlWebApplicationContext]
is not of type [org.springframework.web.context.ConfigurableWebApplicationContext]

To fix this, I change from referencing the Spring JARs in WEB-INF/lib to importing the packages for Spring (which were already installed in my Equinox container).

Bundle-Classpath: .,WEB-INF/classes,WEB-INF/lib/freemarker-2.3.12.jar,
 WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar
Import-Package:
javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional,
 org.springframework.osgi.web.context.support,
 org.springframework.context.support,
 org.springframework.web.context,
 org.springframework.web.context.support,
 org.springframework.web.servlet,
 org.springframework.web.servlet.mvc,
 org.springframework.web.servlet.mvc.support,
 org.springframework.web.servlet.view,
 org.springframework.ui,
 org.springframework.web.servlet.view.freemarker

After rebuilding my WAR and reloading the bundle in Equinox, I was confronted with the following error message:

SEVERE: Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'freemarkerConfig' defined in ServletContext
resource [/WEB-INF/myapp-servlet.xml]: Instantiation of bean failed;
nested exception is java.lang.NoClassDefFoundError:
freemarker/cache/TemplateLoader
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:851)

As far as I can tell, this is because the version of Spring MVC installed in Equinox cannot resolve the FreeMarker JAR in my WEB-INF/lib directory.

To prove I wasn't going insane, I commented out my "freemarkerConfig" and "viewResolver" beans in myapp-servlet.xml and changed to a regular ol' InternalResourceViewResolver:

<bean id="viewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/"/>
    <property name="suffix" value=".jsp"/>
</bean>

This worked and I was able to successfully see "Hello World" from a JSP in my browser. FreeMarker/SiteMesh worked too, but FreeMarker didn't work as a View for Spring MVC.

To attempt to solve this, I create a bundle for FreeMarker using "java -jar bnd-0.0.249.jar wrap freemarker-2.3.12.jar" and installed it in Equinox. I then change my MANIFEST.MF to use FreeMarker imports instead of referencing the JAR in WEB-INF/lib.

Bundle-Classpath:
.,WEB-INF/classes,WEB-INF/lib/sitemesh-2.3.jar,WEB-INF/lib/urlrewritefilter-3.0.4.jar
Import-Package:
javax.servlet,javax.servlet.http,javax.servlet.resources,javax.swing.tree,
 javax.naming,org.w3c.dom,org.apache.commons.logging,javax.xml.parsers;resolution:=optional,
 org.xml.sax;resolution:=optional,org.xml.sax.helpers;resolution:=optional,
 org.springframework.osgi.web.context.support,
 org.springframework.context.support,
 org.springframework.web.context,
 org.springframework.web.context.support,
 org.springframework.web.servlet,
 org.springframework.web.servlet.mvc,
 org.springframework.web.servlet.mvc.support,
 org.springframework.web.servlet.view,
 org.springframework.ui,
 org.springframework.web.servlet.view.freemarker,
 freemarker.cache,freemarker.core,freemarker.template,freemarker.ext.servlet

Unfortunately, this still doesn't work and I still haven't been able to get FreeMarker to work with Spring MVC in OSGi. The crazy thing is I actually solved this at one point a week ago. Shortly after, I rebuilt Equinox from scratch and I'm been banging my head against the wall over this issue ever since. Last week, I entered an issue in Spring's JIRA, but thought I'd fixed it a few hours later.

I've uploaded the final project that's not working to the following URL:

http://static.raibledesigns.com/downloads/myapp-osgi.zip

If you'd like to see this project work with Spring MVC + JSP, simply modify myapp-servlet.xml to remove the FreeMarker references and use the InternalResourceViewResolver instead.

I hope Spring DM + Spring MVC supports more than just JSP as a view technology. I hope I can't get FreeMarker working because of some oversight on my part. If you have a Spring DM + Spring MVC application working with Velocity or FreeMarker, I'd love to hear about it.

Upgrading to Spring Security 2.0

2008-04-17T02:45:47-06:00

This evening I spent a few hours and upgraded AppFuse to use Acegi Spring Security 2.0. The upgrade was fairly straightforward:

%s/org.acegisecurity/org.springframework.security/g

Upgraded dependencies (exclusions are necessary if you're using Spring 2.5.x and don't want 2.0.x dependencies pulled in):

<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-core-tiger</artifactId>
    <version>${spring.security.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.security.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Changed taglib prefix from "authz" to "security" and change the associated taglib declaration to:
```
<%@ taglib uri="http://www.springframework.org/security/tags" 
    prefix="security" %>
```
In web.xml, I changed <filter-class> to org.springframework.web.filter.DelegatingFilterProxy. Since I didn't name my filter springSecurityFilterChain, I also had to add the following <init-param>:
```
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
```
Lastly, I modified security.xml to use the new syntax. AppFuse's security.xml went from 175 lines to 33 with the new security namespace configuration!

It's hard to believe I first looked at Acegi almost 4 years ago. At that time, I said it contained too much XML for my needs. Ben's reaction:

Seriously, the "whole lotta XML" gives you exponentially more power and flexibility than a method such as this could ever hope to provide you.

It's nice to see that Spring Security 2.0 gives you exponentially more power and flexibility without all the XML. Thanks guys!

P.S. You can also view the full changelog for this upgrade.

Update: If you're using <authz:authentication property="fullName"/> in your JSPs, you'll need to change it to <security:authentication property="principal.fullName"/>.

Selenium User Meetup Videos Posted

2008-03-05T21:19:23-07:00

Videos of last week's Selenium User Meetup have been posted on YouTube. You can watch them below or click on the video to watch it on YouTube. Enjoy!

Lightning Talks

Q and A Session

The National Western Stock Show

2008-01-24T12:24:23-07:00

Last night, a friend and I went to the Rodeo at the National Western Stock Show. We had really nice seats (thanks to Cletus from Nebraska) and had a great time watching the Bareback Ridin', Bull Fightin', Bull Ridin', Mutton Bustin' and Barrel Racin'. The highlights were the bull fighting (the bulls almost always won) and the Mutton Bustin'. This is where they plop little kids on the back of a sheep and they hang on for deer life. I shot a video for your entertainment.

Click here for more pictures from the event. Sorry about the picture blurriness, the high action and bad results from my camera made me realize I might need to upgrade.

Go Light with Apache Struts 2 and REST by Don Brown

2007-11-15T18:12:58-07:00

After attending Dan's talk on REST, I stayed in the same room and listened to Don Brown talk about Struts 2's support for building RESTful applications. Below are my notes from the event.

What is wrong with today's web applications? You're using a modern web framework and you've cleanly separated your presentation and business logic. The biggest problem in modern web applications is Confusing URLs.

A URL should be a resource indicator - not a method invocation. Often, web applications have little or no caching. People use GET to perform data manipulation and POST may or may not change state (especially with JSF). Another big issue with modern web frameworks is there's too many abstraction layers that hide HTTP headers and it's difficult to manipulate them.

Today's applications are "information silos". There's a lot of information in your applications, but it's all buried in HTML, JavaScript and CSS. There's no way to get this information out of your application unless you explicitly expose it.

The answer to many of these problem is REST. It's the Way of the Web. To solve the information silo problem, you can create a single interface that has multiple representation of the same resource. There's one URI for all types of resources - be it XML, JSON or HTML. How does this work w/o modifying the URL? You modify the URL's extension.

Struts 2 has a couple of plugins that make developing RESTful services easier. The first is the Codebehind plugin and the 2nd is the REST plugin. Don is doing a demo with the REST plugin and shows that there's no Struts configuration files needed (no struts.xml and no struts.properties). The only thing that's necessary is to specify an "actionPackages" init-param on the DispatcherFilter in web.xml. This activates the Codebehind plugin that uses conventions to determine the view template's path.

In Don's demo, he's creating an "OrdersController" that implements ModelDriven. After implementing a setId() method (to set the id from the request parameters), a getModel() method (to return the Order object) and implementing a show() method that returns HttpHeaders, Don starts up his server and shows that http://localhost:8080/order/5 returns an HTML page. Changing the URL to end in /5.json returns JSON, /5.xml returns XML.

public HttpHeaders create() {
    service.save(order);
    return new DefaultHttpHeaders("success").setLocationId(order.getId());
}

The Poster Plugin for Firefox is great when you're working with REST services. Don used this plugin to show us that it's possible to post to JSON and get back JSON results. His demo was impressive, especially the fact that there was no XML configuration required for Struts. I also like how the DefaultHttpHeaders class allows you to manipulate headers in a type-safe manner.

To use the REST plugin, you'll want to use Struts 2.1. If you're using Maven, all you need to depend on is struts-rest-plugin. The struts-codebehind-plugin (as well as struts-core) will be pulled in by transitive dependencies.

One disadvantage of REST vs. WS-* is you can't generate client code from a WSDL. You'll have to write your client by hand. However, one advantage of REST is there's already lots of clients - your browser, curl, etc.

The Struts REST Plugin hasn't been officially released, but hopefully will be in Struts 2.1.1. You can checkout the code from SVN using the URL below. The documentation is located here.

http://svn.apache.org/repos/asf/struts/struts2/trunk/plugins/rest

Great talk Don - and excellent work on the REST plugin for Struts. I can't wait to try it out.

Integrating Struts 2 + JSF + Facelets

2007-07-26T12:13:59-06:00

What if you didn't have to choose between a request-based framework and a component-based framework? What if you could use them together and use request-based for some pages and component-based for others? This is the functionality that the Struts 2 JSF Plugin provides.

To be fair, the JSF-Spring project says it does the same thing for Spring MVC + JSF, but there doesn't appear to be any documentation.

I did some prototyping of Struts 2 + JSF and discovered that it does indeed work. I also discovered that there's no documentation on integrating it with Facelets. Luckily, it's pretty easy to do - hence my reason for writing this entry. You might ask why I want to use Facelets when JSF 1.2 supports JSP fairly well? My reason is because JSP 2.1 hijacks #{}, which Struts 2's OGNL uses for some expressions. Because of this, I want to be able to run on a JSP 2.0 container until a workaround comes along. Sun's JSF 1.2 RI can run on a JSP 2.0 container, while MyFaces 2.1 cannot (at least in my experience).

There's two ways to get Struts 2 + JSF + Facelets working:

Create a WEB-INF/faces-config.xml file and override the default view-handler:

<?xml version="1.0" encoding="UTF-8"?>
<faces-config xmlns="http://java.sun.com/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee 
        http://java.sun.com/xml/ns/javaee/web-facesconfig_1_2.xsd"
    version="1.2">

    <application>
        <view-handler>com.sun.facelets.FaceletViewHandler</view-handler>
    </application>
</faces-config>

The 2nd way is to use Ajax4JSF and declare the view-handler in your web.xml (allowing you to get rid of faces-config.xml):

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

NOTE: You have to use 2nd method if you want to use Ajax4JSF. It won't read the view-handler from faces-config.xml.

If you're using SiteMesh, you may have to add another <parser> element to your sitemesh.xml to get Facelets pages decorated:

<parser content-type="application/xhtml+xml"
    class="com.opensymphony.module.sitemesh.parser.HTMLPageParser"/>

Thanks to Laurie Harper for his assistance figuring this stuff out.

Now you might ask - why would you want to do this? For one, Struts 2 has a better navigation model (IMO) than JSF. Also, if developers want to use JSF and think it's a better way for a certain module - let them go to it!

Java Web Frameworks and XSS

2007-07-19T10:16:15-06:00

In preparation for my talk at OSCON next week, I've been doing some research on cross-site scripting and how good Java web frameworks handle it. I'm disappointed to report that the handling of XSS in Java web frameworks is abysmal. First of all, the JSP EL doesn't bother to handle XSS:

With JSP 2.0 you can use the following to emit the description of a "todo" item:
${todo.description}
That's pretty nice. What happens when someone has entered a description like this?
<script type="text/javascript">alert('F#$@ you!');</script>
Well, it executes the JavaScript and pops up a nice little message to you.
...
My question is this: Why in the world did the expert group on the JSP 2.0 JSR decide to make not escaping XML content the default for EL expressions, when they made the opposite decision for c:out?

(Emphasis mine) If a company/developer wants to make sure their JSP-based code is not susceptible to XSS, they have two choices (as I see it):

Do lots of code review to make sure <c:out> is used instead of ${}.
Hack the jsp-compiler/el-engine to escape XML by default.

The good news is #2 doesn't seem to be that hard. I pulled down commons-el yesterday, added a hack to escape XML, re-jarred and put it in Tomcat 5.0.25's classpath. This actually worked and I was impressed it was so easy. However, when I looked at Tomcat 6, commons-el is no longer used and now there's a "jasper-el.jar" in the lib directory. I don't mind modifying another library, but what's the difference between jasper-el and commons-el?

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

On a related note, there's a couple of web frameworks that I've found to be susceptible to XSS: namely Spring MVC and Struts 2. For Spring MVC, its <form:input> and <form:errors> tags are vulnerable. For Struts 2, OGNL expressions are evaluated, which is way worse than XSS and actually allows you to shutdown the JVM by putting %{@java.lang.System@exit(0)}" in a text field.

Even though it was surprising for me to see the issues with Struts 2 and Spring MVC, I'm somewhat glad they exist. If I hadn't discovered them, I might blissfully think that Java web frameworks aren't susceptible to XSS. However, it appears they're not only susceptible, but no one is really thinking about XSS when developing these framework. To further prove that theory, the Spring MVC and Struts 2 teams are aware of these issues, have been for quite some time - yet they've done nothing in the form of releasing upgrades or patches.

Seems kinda strange doesn't it?

Zero Configuration in Struts 2

2007-03-07T17:19:18-07:00

Struts 2 has a nifty zero configuration feature. However, it's only useful for registering actions, not for automatically registering results. In other words, you still have to use an @Result annotation to tell your action what page to dispatch to. To use default view names instead of requiring @Result, you can use the Codebehind Plugin. Also, did you know Struts 2 will autowire your Spring dependencies? It's pretty slick.

What does this all mean? It means you can write your Struts 2 application without writing any XML. Of course, you can still use XML to tweak behavior, but with these plugins enabled, you won't have to.

IMO, these plugins should be combined into a single zero configuration feature.

Here's how you can enable Struts 2's Zero Configuration feature in AppFuse 2.0:

Add a packageNames parameter to the "struts" filter in your web.xml:

<filter>
    <filter-name>struts</filter-name>
    <filter-class>org.apache.struts2.dispatcher.FilterDispatcher</filter-class>
    <init-param>
        <param-name>actionPackages</param-name>
        <param-value>com.company.newapp.webapp.action</param-value>
    </init-param>
</filter>

Add the Codebehind Plugin as a dependency in your pom.xml:

<dependency>
    <groupId>org.apache.struts</groupId>
    <artifactId>struts2-codebehind-plugin</artifactId>
    <version>2.0.6</version>
</dependency>

Add a struts.codebehind.pathPrefix constant in struts.xml for your default pages directory:
```
<constant name="struts.codebehind.pathPrefix" value="/WEB-INF/pages/"/>
```

That's it - now you can code away without configuring anything!

How does this compare to other web frameworks in AppFuse? Tapestry has a similar feature, but Spring MVC and JSF don't. Spring MVC still requires you create a bean definition for Controllers and JSF requires you write a chunk of XML for each managed bean. Of course, if you know how to do something similar with Spring MVC or JSF, please let me know.

Integrating Facelets and Ajax4JSF with MyFaces

2006-10-12T02:41:24-06:00

I spent a few hours tonight integrating Facelets into an AppFuse-based application. The integrating was fairly easy thanks to the work Thomas Gaudin did back in January. For the most part, it was just a matter of 1) replacing <fmt:message> tags with #{text['key']} tags, 2) replacing the <%@ include file="/common/taglibs.jsp"%> at the top of each page with Facelet's namespace tags and 3) renaming the pages from *.jsp to *.xhtml. The only thing that tripped me up was I thought the "c" namespace was the same as JTLS's URI, but it's actually a whole new URI. Thanks to the Facelets developers for a much-needed fix for JSF.

After I got Facelets integrated and working, I dove into integrating Ajax4JSF. Two hours later and I have the simple repeater demo working. What took so long? I spent an hour staring at (and googling for) the solution to the following error:

PhaseListenerManager.informPhaseListenersBefore(74) | Exception in PhaseListener RENDER_RESPONSE(6) beforePhase.
java.lang.NullPointerException
        at com.sun.facelets.FaceletViewHandler.writeState(FaceletViewHandler.java:759)
        at org.ajax4jsf.framework.renderer.AjaxRendererUtils.writeState(AjaxRendererUtils.java:850)
        at org.ajax4jsf.framework.renderer.AjaxRendererUtils.encodeAreas(AjaxRendererUtils.java:740)
        at org.ajax4jsf.framework.renderer.AjaxContainerRenderer.encodeAjax(AjaxContainerRenderer.java:128)
        at org.ajax4jsf.ajax.UIAjaxRegion.encodeAjax(UIAjaxRegion.java:210)

The solution turned out to be removing the FaceletsViewHandler from faces-config.xml:

<view-handler>com.sun.facelets.FaceletViewHandler</view-handler>

Also, it seems the following is required in web.xml:

    <context-param>
        <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
        <param-value>com.sun.facelets.FaceletViewHandler</param-value>
    </context-param>

Hopefully this helps others googling for the exception above.

Dependency Injection with SiteMesh

2006-02-16T09:57:23-07:00

Let me start off by saying I think that both SiteMesh and Tiles are great frameworks. I was a long time user and fan of Tiles, and I think it's appropriate for certain situations. However, I've been a heavy user of SiteMesh since it passed the 10 minute test. While most heavy users of SiteMesh (the Atlassian guys come to mind) say that it can do everything that Tiles can do, these features are largely undocumented. This is my attempt to document a cool feature.

In a site I recently helped develop, we needed a couple of features:

A tabbed menu that highlighted the current tab based on which page you were on.
A bunch of "panels" on the right sidebar that changed according to the page.

To make this work, we used the meta tag functionality that SiteMesh provides.

Funny side/related note, I just googled for this tag and found this howto, which is similar to this one.

In our pages, we added the meta tags to set the active menu, as well as which panels to show in the sidebar:

<head>
    <title><fmt:message key="authorList.title"/></title>
    <meta name="menu" content="Authors"/>
    <meta name="panels" content="administration,blogs,events"/>
</head>

Then, in our decorator, we interpret these separately. First, we used Struts Menu (with Velocity) for the navigation system:

<c:set var="currentMenu" scope="request">
    <decorator:getProperty property="meta.menu"/>
</c:set>
<c:import url="/WEB-INF/pages/menu.jsp">
    <c:param name="template" value="/template/menu/tabs.html"/>
</c:import>

The menu.jsp page takes "template" as a parameter so we display the same menu links using a different Velocity template (for example, links at the bottom of the page).

<menu:useMenuDisplayer name="Velocity" config="${param.template}" permissions="rolesAdapter">

Then our tabs.html Velocity template uses the "currentMenu" attribute to determine which menu to highlight.

## displayMenu is defined in WEB-INF/classes/globalMacros.vm
#macro( menuItem $menu $level )
  #set ($title = $displayer.getMessage($menu.title))
  #if ($menu.url)
    #if ($menu.name == $currentMenu)
      <span class="current">
    #end
      <a href="$!menu.url" title="$title"><span>$title</span></a>
    #if ($menu.name == $request.getAttribute('currentMenu'))
      </span>
    #end
  #end
#end

#if ($displayer.isAllowed($menu))
    #displayMenu($menu 0)
#end

As far as the panel injection goes, that's processed using the following logic in our decorator:

<c:set var="panels"><decorator:getProperty property="meta.panels"/></c:set>
<!-- No panels set, use default set of panels -->
<c:if test="${empty panels}"><c:set var="panels" value="different,partners"/></c:if>
<c:forEach var="panel" items="${panels}">
    <c:import url="/WEB-INF/pages/panels/${panel}.jsp"/>
</c:forEach>

Since this site used WebWork, the <ww:action> tag made it easy to give each panel independence. That is, each panel could load on its own, supply its own data, and not worry about the data being prepared beforehand. Here's an example:

<%@ include file="/common/taglibs.jsp"%>

<h2>Author Blogs</h2>

<ww:action name="'authors'" id="authors" namespace="default"/>

<div class="item">
    <ww:iterator value="#authors.authors" status="index">
        <a href="<ww:property value="blog.feedUrl"/>">
            <img src="${ctxPath}/images/icons/xml.gif" alt="XML Feed" 
                style="margin-right: 5px; vertical-align: middle"/></a>
        <a href="<ww:property value="blog.url"/>"><ww:property value="name"/></a>
        <br />
    </ww:iterator>
</div>

Of course, now that you can use Tiles with WebWork, Struts, Spring MVC and JSF - you could use Tiles for the injection and SiteMesh for the decoration.

Now if we could just get someone to write a JSF Decorator for SiteMesh, like Erik Hatcher did for Tapestry.

How to use Tiles with WebWork

2006-02-16T01:08:42-07:00

This evening, I created a TilesResult for WebWork that allows you to use Tiles with WebWork. For the following to work in your application, you'll need a nightly build of Tiles, commons-digester (which Tiles requires) and this patch for WebWork. For your convenience, I've posted a patched webwork-2.2.2.jar (with TilesResult).

I also posted a webwork-tiles.war that you can try and download yourself. It's based on Equinox, so you will need to setup PostgreSQL and an "equinox" database - or you can just change the database settings in WEB-INF/lib/jdbc.properties.

On to the instructions:

1. In your web.xml file, you need to add a servlet entry for TilesServlet to load the tiles definitions into the ServletContext.

    <servlet>
        <servlet-name>tiles</servlet-name>
        <servlet-class>org.apache.tiles.servlets.TilesServlet</servlet-class>
        <init-param>
            <param-name>definitions-config</param-name>
            <param-value>/WEB-INF/tiles-config.xml</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

2. In xwork.xml, use type="tiles" on your <result>.

    <action name="editUser" class="userAction" method="edit">
        <result name="success" type="tiles">userForm</result>
        <result name="input" type="tiles">userList</result>
    </action>

I'm sure WebWork has a way of making this result type the default, I just haven't found it yet.

Hat tip to Spring's TilesView (source) for showing how to make this work.

Update: While I'm a happy SiteMesh user, I've recently had some clients who were more interested in Tiles. This largely inspired me to see if WebWork + Tiles was possible.

Update 2: It looks like TilesResult will be included in WebWork 2.2.2. Now if we could just get the Tiles team to cut a release.

Using DWR with Spring and Hibernate

2005-04-28T14:10:26-06:00

For the past few weeks, I've been developing an application using Struts, Spring, Hibernate and the DWR project for my XmlHttpRequest framework. As you might remember, I used JSON-RPC for Ajax stuff on my last project. I found DWR to be much more full-featured and easier to use. This post is meant to capture some issues I encountered so others won't have to jump the hurdles that I did. For those of you that get bored quickly, here's a movie (QuickTime) of the app's Ajax features.

I've been using version 0.4 of DWR, and I haven't had a chance to try out version 0.5. When I first started using it, I ran into a ThreadDeath problem that was easily resolved by changing a log.debug message to System.out.println. I tried to reproduce this issue yesterday and couldn't, so who knows what that was all about. As far as configuring DWR in your webapp, that's pretty easy to do, and well documented. See the project's documentation or this Spring MVC HowTo.

Here are a few things I remember from my development experience.

The examples are great, especially how to dynamically edit a table.
When developing, make sure to set the "debug" init-param to "true". This allows you to go to http://location:8080/yourapp/dwr and see a screen that allows you to call methods on your exposed classes.
In WEB-INF/dwr.xml, you need to specify a converter for each POJO you want to expose to your UI via JavaScript. I started out by converting a whole package, but found this to be *extremely* slow (we have a package of around 50 DTOs). So I changed it to be only the DTOs I was using. This turned out to take about 30 seconds to do the conversion, and was again unacceptable. The problem turned out to be that the converter was invoking all the lazy-loaded children for each DTO. My final solution was to create a NameValue object and only convert that. Then in my Spring bean, I populate it from DAOs and DTOs. I'm using Spring's OSIVF for Hibernate to ensure that DWR doesn't invoke lazy-loading.
I had to override a few of DWR's JavaScript functions in util.js b/c they didn't work for me. I changed showById() and toggleDisplay() to use style.display='' instead of style.display='block' b/c this is what I've always used and block doesn't work that well. I also changed useLoadingMessage() to have a cleaner-looking load message.
I used the Fade Anything Technique in this project and found that IE likes to have full 6-digit hex values for colors in CSS rules. The shorter 3-digit hex values simply don't work in IE.
Using "test" buttons that only showed up for my username proved to be a great way to test the UI and the Ajax stuff. These buttons called a number of JavaScript functions to drive the UI and wait between invoking different functions using window.setTimeout.

All in all, using DWR was a great experience and I definitely plan to use it more in my projects. The client loves the app - especially since it's wicked fast and seems to work like a desktop app.

Use SiteMesh to decorate multiple webapps

2005-04-12T10:05:19-06:00

Did you know it's possible to decorate multiple webapps with the same SiteMesh decorator? I learned how to do this from the SiteMesh mailing list:

Put sitemesh.jar in your container's classpath. You could put it into each webapp's WEB-INF/lib, but it looks like there's problems with that.
Create and deploy a webapp that contains the common decorator.

In each webapp's decorators.xml, add a "webapp" attribute to point to the webapp you deployed in Step 2.

<decorators>
    <!-- load decorator from a different web-app deployed in the server -->
    <decorator name="main" webapp="some-other-webapp" 
        page="/decorators/main.jsp">
        <pattern>/*</pattern>
    </decorator>
</decorators>

Try to do *that* with Tiles.

Trim Spaces in your JSP's HTML

2005-03-23T22:24:58-07:00

One of the annoying things about JSPs is all of the dynamic (non-rendered) parts of the page still produce line breaks. This means that if you do a view-source, you'll likely see large blocks of whitespace.

The good news is you can get rid of this whitespace if you're using Tomcat 5.5.x. Just locate the "jsp" servlet in $CATALINA_HOME/conf/web.xml and add the following <init-param>:

    <init-param>
        <param-name>trimSpaces</param-name>
        <param-value>true</param-value>
    </init-param>

I tested it and it works great. This begs the question - why isn't this on by default? Source: Struts Mailing List.

Update: JSP 2.1 adds the ability to trim whitespaces.

Ajax webapps are cool, but non-javascript versions still needed

2005-03-07T07:42:18-07:00

I think we can all probably learn a lesson from Google. I've heard that GMail is the "gold standard" for Ajax applications. If that's the case, then you should note that they've recently added a "basic HTML" link to the bottom of their pages. With this link, you can view your e-mail using the old way: Yahoo-style, no-JavaScript-needed. My guess is they added it because of demand, or simply to compete with other providers who have this feature. I think it's a good lesson though: use Ajax features in webapps where appropriate, but don't make JavaScript necessary to use your app.

A couple of Ajax features I've been thinking of developing:

Saving forms with XMLHttpRequest: just display a success message at the top, and switch the "Cancel" button to "Done". Since the form's content doesn't change, this seems like a reasonable use of the technology.
Switching out entire "content" <div> elements. Most of my apps have a <div id="content">, so it'd probably pretty easy to just replace that in response to button and link clicks. Of course, the hard part is having the requested server-side object load the view template, process it, and send back the content. This is probably more trouble than it's worth.

Simplifying XmlHttpRequest with JSON-RPC

2005-03-03T08:44:40-07:00

In my day job, we decided to use a little XMLHttpRequest lovin' to populate one drop-down from another. This is my review of JSON-RPC, an open source JavaScript library and servlet for simplifying XMLHttpRequest. I considered integrate Direct Web Remoting (DWR) as well, but its java.net site was down the day I needed it. I started out with JSON-RPC 0.7, which caused some conflicts with Commons Validator client-side validation. This was fixed in the 0.8 release. JSON-RPC takes a little more setup than I care for, but it's pretty easy nonetheless:

Download the 0.8 release from http://oss.metaparadigm.com/jsonrpc-dist/json-rpc-java-0.8.tar.gz.
Add the JAR to your project and the webapps/jsonrpc/jsonrpc.js to your projects' "scripts" folder. Include this file in your SiteMesh decorator or Tiles layout. If you're not using SiteMesh or Tiles, it's high time you started.

JSON-RPC currently requires that you register each class you want call methods on. In our project, I registered a Spring bean (LookupHelper) that's a singleton with references to Maps in the ServletContext. Then we used JavaScript functions to call JSON-PRC and look up units for a plant, and vice versa. I'm not going to put the LookupHelper class here - you'll have to trust its methods return a single String or a comma-separated list of Strings. To register this bean with JSON-RPC, I created an HttpSessionListener and configured it in web.xml.


/**

 * UserListener class used to add/remove session attributes when

 * a user first logs in.  Mainly for JavaScript Remote Scripting stuff.

 *

 * @author Matt Raible

 */

public class UserListener implements HttpSessionListener, HttpSessionAttributeListener { 

    private final Log log = LogFactory.getLog(UserListener.class);

    public final static String BRIDGE_KEY = "JSONRPCBridge";



    /**

     * Initializes LookupHelper singleton with values needed for lookup

     *

     * @param event the HttpSessionEvent to grab session information from

     */

    public void sessionCreated(HttpSessionEvent event) {

        // Find the JSONRPCBridge for this session or create one

        // if it doesn't exist. Note the bridge must be named BRIDGE_KEY

        // in the HttpSession for the JSONRPCServlet to find it.

        HttpSession session = event.getSession();

        JSONRPCBridge jsonBridge = new JSONRPCBridge();

        jsonBridge.setDebug(true);

        session.setAttribute(BRIDGE_KEY, jsonBridge);

    }



    /**

     * Destroys LookupHelper

     *

     * @param event the HttpSessionEvent to grab session information from

     */

    public void sessionDestroyed(HttpSessionEvent event) {

        if (event.getSession() != null) {

            event.getSession().removeAttribute(BRIDGE_KEY);

        }

    }



    public void attributeAdded(HttpSessionBindingEvent event) {

        if (event.getName().equals(BRIDGE_KEY)) {

            HttpSession session = event.getSession();

            // register LookupHelper so we can call methods on it

            ApplicationContext ctx =

                    WebApplicationContextUtils

                    .getWebApplicationContext(session.getServletContext());



            // check for null so we don't have to initialize Spring in tests

            if (ctx != null) {

                log.debug("Registering lookupHelper for XmlHttpRequest...");

                JSONRPCBridge jsonBridge =

                    (JSONRPCBridge) session.getAttribute(BRIDGE_KEY);

                jsonBridge.registerObject("lookupHelper",

                                          ctx.getBean("lookupHelper"));

            }

        }

    }



    public void attributeRemoved(HttpSessionBindingEvent event) {

        // don't care

    }



    public void attributeReplaced(HttpSessionBindingEvent event) {

        // same as attribute added

        attributeAdded(event);

    }

}

After this setup was complete, I was able to add the following JavaScript to the bottom of my JSP. These are functions that our drop-downs call to populate each other, and keep their options in synch.


<script type="text/javascript">

var jsonurl = "${ctx}/jsonrpc";

var jsonrpc = null;

var unitDropDown = document.getElementById("equipmentName");



function filterUnits(plantDropDown) {

    var plantName = plantDropDown.options[plantDropDown.selectedIndex].value;



    if (plantName == "") {

        reloadUnits("");

        return;

    }



    try {

        jsonrpc = new JSONRpcClient(jsonurl);

    } catch(e) {

        alert(e);

    }



    // Call a Java method on the server

    var units = jsonrpc.lookupHelper.getUnitsForPlant(plantName);

    setUnits(units);

}



function reloadUnits(value) {

    if (value == "") {

        try {

            jsonrpc = new JSONRpcClient(jsonurl);

        } catch(e) {

            alert(e);

        }



        // Call a Java method on the server

        var units = jsonrpc.lookupHelper.getAllUnits();

        setUnits(units);

    }

}



function setUnits(units) {

    var unitArray = units.split(",");



    unitDropDown.options.length = 1;  // keep "All" option

    for (i=0; i < unitArray.length; i++) {

        unitDropDown.options[unitDropDown.options.length] =

            new Option(unitArray[i], unitArray[i]);

    }

}

</script>

The hardest part of using JSON-RPC is setting it up. We only experienced minor issues with Commons Validator, but since the JSON-RPC 0.8 release - everything has worked great, on all browsers we need to support. The only thing I don't like about this library is that you have to register objects for each user's session. I briefly looked at DWR and it looks a little cleaner - especially b/c of its Spring integration. The next time we need XMLHttpRequest, we'll probably use DWR just to compare the two.

Edit Screens with JSF

2004-11-28T10:02:25-07:00

I'm working with JSF this morning and I'm finding one thing particularly annoying. I'm working on a simple master/detail screen and I'm tweaking the detail screen to fit my needs. It's just a <form> with some form elements. I change something, run "ant deploy-web" and hit "refresh" to see my updated page. Since everything in JSF is a post, I get prompted to re-submit the form. Fine, I agree - then I'm returned to the list screen. Argggh - why can't I just view the form again?! This might just be a MyFaces nuance, I'm not sure. Anyone know of a workaround?

Wanna see the bug/feature in action? Go to http://demo.raibledesigns.com/equinox-jsf/userList.html, click on a row - and after the edit screen displays, hit refresh on your browser. In an ideal world, you'd see the form again, but nope - you get the list instead.