Search for [form] in weblog rd

Getting Started + Testing with Angular CLI and Angular 2 (RC5)

2016-08-23T17:18:41-06:00

I started creating Angular 2 applications when it was in beta (back in March). To keep up with Angular 2's changes, I wrote a tutorial about developing with RC1 in June. Earlier this month, RC5 was released and many things changed once again. I think Scott Davis sums it up nicely in a tweet.

They keep saying "Release Candidate", but I don't think it means what they think it means...

/cc #angular2 #rc5 https://t.co/WmNalTYgTN
— Scott Davis (@scottdavis99) August 10, 2016

To keep up with the rapid pace of change in Angular 2, I decided to write another tutorial, this time using Angular CLI. The biggest change I found since writing the last tutorial is testing infrastructure changes. Since Angular's Testing documentation hasn't been updated recently, hopefully this tutorial will help.

Below is a table of contents in case you want to skip right to a particular section.

What you'll build

You'll build a simple web application with Angular CLI, a new tool for Angular 2 development. You'll create an application with search and edit features.

What you'll need

About 30-40 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA and its Angular 2 TypeScript Live Templates plugin.
Node.js and npm installed. I recommend using nvm.
Angular CLI installed. If you don't have Angular CLI installed, install it using npm install -g angular-cli@latest.

The latest release of Angular CLI (beta 10) uses Angular 2 RC4. Because of this, I used the master branch of Angular CLI to create this tutorial. To do this, clone angular-cli and run npm link in the directory you cloned it into. If you have issues, see #1733.

Angular Augury is a Google Chrome Dev Tools extension for debugging Angular 2 applications. I haven't needed it much myself, but I can see how it might come in handy.

Create your project

Create a new project using the ng new command:

ng new ng2-demo

This will create a ng2-demo project and run npm install in it. It takes about a minute to complete, but will vary based on your internet connection speed.

[mraible:~/dev] 45s $ ng new ng2-demo
installing ng2
  create .editorconfig
  create README.md
  create src/app/app.component.css
  create src/app/app.component.html
  create src/app/app.component.spec.ts
  create src/app/app.component.ts
  create src/app/environment.ts
  create src/app/index.ts
  create src/app/shared/index.ts
  create src/favicon.ico
  create src/index.html
  create src/main.ts
  create src/system-config.ts
  create src/tsconfig.json
  create src/typings.d.ts
  create angular-cli-build.js
  create angular-cli.json
  create config/environment.dev.ts
  create config/environment.js
  create config/environment.prod.ts
  create config/karma-test-shim.js
  create config/karma.conf.js
  create config/protractor.conf.js
  create e2e/app.e2e-spec.ts
  create e2e/app.po.ts
  create e2e/tsconfig.json
  create e2e/typings.d.ts
  create .gitignore
  create package.json
  create public/.npmignore
  create tslint.json
  create typings.json
Successfully initialized git.
- Installing packages for tooling via npm
  -- es6-shim (global)
  -- angular-protractor (global dev)
  -- jasmine (global dev)
  -- selenium-webdriver (global dev)

Installed packages for tooling via npm.
[mraible:~/dev] 1m5s $

You can see the what version of Angular CLI you're using with ng --version.

$ ng --version
angular-cli: local (v1.0.0-beta.11-webpack.2, branch: master)
node: 4.4.7
os: darwin x64

Run the application

The project is configured with a simple web server for development. To start it, run:

ng serve

You should see a screen like the one below at http://localhost:4200.

You can make sure your new project's tests pass, run ng test:

$ ng test
Built project successfully. Stored in "dist/".
...
Chrome 52.0.2743 (Mac OS X 10.11.6): Executed 2 of 2 SUCCESS (0.039 secs / 0.012 secs)

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the ng2-demo directory.

The Basics

In a terminal window, cd into your project's directory and run the following command. This will create a search component.

$ ng g component search
installing component
  create src/app/search/search.component.css
  create src/app/search/search.component.html
  create src/app/search/search.component.spec.ts
  create src/app/search/search.component.ts
  create src/app/search/index.ts

Adding a Search Route
In previous versions of CLI, you could generate a route and a component. However, since beta 8, route generation has been disabled. This will likely be re-enabled in a future release.

The Router documentation for Angular 2 RC5 provides the information you need to setup a route to the SearchComponent you just generated. Here's a quick summary:

Create src/app/app.routing.ts to define your routes.

import { Routes, RouterModule } from '@angular/router';
import { SearchComponent } from './search/index';

const appRoutes: Routes = [
  { path: 'search', component: SearchComponent },
  { path: '', redirectTo: '/search', pathMatch: 'full' }
];

export const appRoutingProviders: any[] = [];

export const routing = RouterModule.forRoot(appRoutes);

Without the last path to redirect, there's a Cannot match any routes: '' console error.

In src/app/app.module.ts, import the two constants you exported and configure them in @NgModule:

import { routing, appRoutingProviders } from './app.routing';

import { SearchComponent } from './search/search.component';

@NgModule({
  ...
  imports: [
    ...
    routing
  ],
  providers: [appRoutingProviders],
  ...
})
export class AppModule { }

In src/app/app.component.html, add a RouterOutlet to display routes.

<!-- Routed views go here -->
<router-outlet></router-outlet>

Now that you have routing setup, you can continue writing the search feature.

To allow navigation to the SearchComponent, you can add a link in src/app/app.component.html.

<nav>
  <a routerLink="/search" routerLinkActive="active">Search</a>
</nav>

Open src/app/search/search.component.html and replace its default HTML with the following:

<h2>Search</h2>
<form>
  <input type="search" name="query" [(ngModel)]="query" (keyup.enter)="search()">
  <button type="button" (click)="search()">Search</button>
</form>
<pre>{{searchResults | json}}</pre>

If you still have ng serve running, your browser should refresh automatically. If not, navigate to http://localhost:4200, and you should see the search form.

If you want to add CSS for this components, open src/app/search/search.component.css and add some CSS. For example:

:host {
  display: block;
  padding: 0 20px;
}

This section has shown you how to generate a new component to a basic Angular 2 application with Angular CLI. The next section shows you how to create a use a JSON file and localStorage to create a fake API.

The Backend

To get search results, create a SearchService that makes HTTP requests to a JSON file. Start by generating a new service.

ng g service search

Move the generated search.service.ts and its test to app/shared/search. You will likely need to create this directory.

Then, create src/app/shared/search/data/people.json to hold your data.

[
  {
    "id": 1,
    "name": "Peyton Manning",
    "phone": "(303) 567-8910",
    "address": {
      "street": "1234 Main Street",
      "city": "Greenwood Village",
      "state": "CO",
      "zip": "80111"
    }
  },
  {
    "id": 2,
    "name": "Demaryius Thomas",
    "phone": "(720) 213-9876",
    "address": {
      "street": "5555 Marion Street",
      "city": "Denver",
      "state": "CO",
      "zip": "80202"
    }
  },
  {
    "id": 3,
    "name": "Von Miller",
    "phone": "(917) 323-2333",
    "address": {
      "street": "14 Mountain Way",
      "city": "Vail",
      "state": "CO",
      "zip": "81657"
    }
  }
]

Modify src/app/shared/search/search.service.ts and provide Http as a dependency in its constructor. In this same file, create a getAll() method to gather all the people. Also, define the Address and Person classes that JSON will be marshalled to.

import { Injectable } from '@angular/core';
import { Http, Response } from '@angular/http';

@Injectable()
export class SearchService {
  constructor(private http: Http) {}

  getAll() {
    return this.http.get('app/shared/search/data/people.json').map((res: Response) => res.json());
  }
}

export class Address {
  street: string;
  city: string;
  state: string;
  zip: string;

  constructor(obj?: any) {
    this.street = obj && obj.street || null;
    this.city = obj && obj.city || null;
    this.state = obj && obj.state || null;
    this.zip = obj && obj.zip || null;
  }
}

export class Person {
  id: number;
  name: string;
  phone: string;
  address: Address;

  constructor(obj?: any) {
    this.id = obj && Number(obj.id) || null;
    this.name = obj && obj.name || null;
    this.phone = obj && obj.phone || null;
    this.address = obj && obj.address || null;
  }
}

To make these classes available for consumption by your components, edit src/app/shared/index.ts and add the following:

export * from './search/search.service';

In search.component.ts, add imports for these classes.

import { Person, SearchService } from '../shared/index';

You can now add query and searchResults variables. While you're there, modify the constructor to inject the SearchService.

export class SearchComponent implements OnInit {
  query: string;
  searchResults: Array<Person>;

  constructor(private searchService: SearchService) {}

Then implement the search() method to call the service's getAll() method.

search(): void {
  this.searchService.getAll().subscribe(
    data => { this.searchResults = data; },
    error => console.log(error)
  );
}

At this point, you'll likely see the following message in your browser's console.

ORIGINAL EXCEPTION: No provider for SearchService!

To fix the "No provider" error from above, update app.component.ts to import the SearchService and add the service to the list of providers.

import { SearchService } from './shared/index';

@Component({
  ...
  styleUrls: ['app.component.css'],
  viewProviders: [SearchService]
})

Now clicking the search button should work. To make the results look better, remove the <pre> tag and replace it with a <table>.

<table *ngIf="searchResults">
  <thead>
  <tr>
    <th>Name</th>
    <th>Phone</th>
    <th>Address</th>
  </tr>
  </thead>
  <tbody>
  <tr *ngFor="let person of searchResults; let i=index">
    <td>{{person.name}}</td>
    <td>{{person.phone}}</td>
    <td>{{person.address.street}}<br/>
      {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
    </td>
  </tr>
  </tbody>
</table>

Then add some additional CSS to improve its table layout.

table {
  margin-top: 10px;
  border-collapse: collapse;
}

th {
  text-align: left;
  border-bottom: 2px solid #ddd;
  padding: 8px;
}

td {
  border-top: 1px solid #ddd;
  padding: 8px;
}

Now the search results look better.

But wait, we still don't have search functionality! To add a search feature, add a search() method to SearchService.

search(q: string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results: any = [];
    data.map(item => {
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

Then refactor SearchComponent to call this method with its query variable.

search(): void {
  this.searchService.search(this.query).subscribe(
    data => { this.searchResults = data; },
    error => console.log(error)
  );
}

Now search results will be filtered by the query value you type in.

This section showed you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an edit feature

Modify search.component.html to add a click handler for editing a person.

<td><a (click)="onSelect(person)">{{person.name}}</a></td>

In previous versions of Angular 2, you could embed a link with parameters directly into the HTML. For example:

<a [routerLink]="['/edit', person.id]">

Unfortunately, this doesn't work with RC5. Another issue is adding href="" causes the page to refresh. Without href, the link doesn't look like a link. If you know of a solution to this problem, please send me a pull request.

Then add onSelect(person) to search.component.ts. You'll need to import Router and set it as a local variable to make this work.

import { Router } from '@angular/router';
...
export class SearchComponent implements OnInit {
  ...

  constructor(private searchService: SearchService, private router: Router) { }

  ...

  onSelect(person: Person) {
    this.router.navigate(['/edit', person.id]);
  }
}

Run the following command to generate an EditComponent.

$ ng g component edit
installing component
  create src/app/edit/edit.component.css
  create src/app/edit/edit.component.html
  create src/app/edit/edit.component.spec.ts
  create src/app/edit/edit.component.ts
  create src/app/edit/index.ts

Add a route for this component in app.routing.ts:

import { EditComponent } from './edit/index';

const appRoutes: Routes = [
  { path: 'search', component: SearchComponent },
  { path: 'edit/:id', component: EditComponent },
  { path: '', redirectTo: '/search', pathMatch: 'full' }
];

Update src/app/edit/edit.component.html to display an editable form. You might notice I've added id attributes to most elements. This is to make things easier when writing integration tests with Protractor.

<div *ngIf="person">
  <h3>{{editName}}</h3>
  <div>
    <label>Id:</label>
    {{person.id}}
  </div>
  <div>
    <label>Name:</label>
    <input [(ngModel)]="editName" name="name" id="name" placeholder="name"/>
  </div>
  <div>
    <label>Phone:</label>
    <input [(ngModel)]="editPhone" name="phone" id="phone" placeholder="Phone"/>
  </div>
  <fieldset>
    <legend>Address:</legend>
    <address>
      <input [(ngModel)]="editAddress.street" id="street"><br/>
      <input [(ngModel)]="editAddress.city" id="city">,
      <input [(ngModel)]="editAddress.state" id="state" size="2">
      <input [(ngModel)]="editAddress.zip" id="zip" size="5">
    </address>
  </fieldset>
  <button (click)="save()" id="save">Save</button>
  <button (click)="cancel()" id="cancel">Cancel</button>
</div>

Modify EditComponent to import model and service classes and to use the SearchService to get data.

import { Component, OnInit, OnDestroy } from '@angular/core';
import { Address, Person, SearchService } from '../shared/index';
import { Subscription } from 'rxjs';
import { ActivatedRoute, Router } from '@angular/router';

@Component({
  selector: 'app-edit',
  templateUrl: 'edit.component.html',
  styleUrls: ['edit.component.css']
})
export class EditComponent implements OnInit, OnDestroy {
  person: Person;
  editName: string;
  editPhone: string;
  editAddress: Address;

  sub: Subscription;

  constructor(private route: ActivatedRoute,
              private router: Router,
              private service: SearchService) {
  }

  ngOnInit() {
    this.sub = this.route.params.subscribe(params => {
      let id = + params['id']; // (+) converts string 'id' to a number
      this.service.get(id).subscribe(person => {
        if (person) {
          this.editName = person.name;
          this.editPhone = person.phone;
          this.editAddress = person.address;
          this.person = person;
        } else {
          this.gotoList();
        }
      });
    });
  }

  ngOnDestroy() {
    this.sub.unsubscribe();
  }

  cancel() {
    this.router.navigate(['/search']);
  }

  save() {
    this.person.name = this.editName;
    this.person.phone = this.editPhone;
    this.person.address = this.editAddress;
    this.service.save(this.person);
    this.gotoList();
  }

  gotoList() {
    if (this.person) {
      this.router.navigate(['/search', {term: this.person.name} ]);
    } else {
      this.router.navigate(['/search']);
    }
  }
}

Modify SearchService to contain functions for finding a person by their id, and saving them. While you're in there, modify the search() method to be aware of updated objects in localStorage.

search(q: string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results: any = [];
    data.map(item => {
      // check for item in localStorage
      if (localStorage['person' + item.id]) {
        item = JSON.parse(localStorage['person' + item.id]);
      }
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

get(id: number) {
  return this.getAll().map(all => {
    if (localStorage['person' + id]) {
      return JSON.parse(localStorage['person' + id]);
    }
    return all.find(e => e.id === id);
  });
}

save(person: Person) {
  localStorage['person' + person.id] = JSON.stringify(person);
}

You can add CSS to src/app/edit/edit.component.css if you want to make the form look a bit better.

:host {
  display: block;
  padding: 0 20px;
}

button {
  margin-top: 10px;
}

At this point, you should be able to search for a person and update their information.

The <form> in src/app/edit/edit.component.html calls a save() function to update a person's data. You already implemented this above. The function calls a gotoList() function that appends the person's name to the URL when sending the user back to the search screen.

gotoList() {
  if (this.person) {
    this.router.navigate(['/search', {term: this.person.name} ]);
  } else {
    this.router.navigate(['/search']);
  }
}

Since the SearchComponent doesn't execute a search automatically when you execute this URL, add the following logic to do so in its constructor.

import { Router, ActivatedRoute } from '@angular/router';
import { Subscription } from 'rxjs';
...
  sub: Subscription;

  constructor(private searchService: SearchService, private router: Router, private route: ActivatedRoute) {
    this.sub = this.route.params.subscribe(params => {
      if (params['term']) {
        this.query = decodeURIComponent(params['term']);
        this.search();
      }
    });
  }

You'll want to implement OnDestroy and define the ngOnDestroy method to clean up this subscription.

import { Component, OnInit, OnDestroy } from '@angular/core';

export class SearchComponent implements OnInit, OnDestroy {
...
  ngOnDestroy() {
    this.sub.unsubscribe();
  }
}

After making all these changes, you should be able to search/edit/update a person's information. If it works - nice job!

Testing

Now that you've built an application, it's important to test it to ensure it works. The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your application grows.

In this section, you'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. Angular's testing documentation lists good reasons to test, but doesn't currently have many examples.

Unit test the SearchService

Modify src/app/shared/search/search.service.spec.ts and setup the test's infrastructure using MockBackend and BaseRequestOptions.

import { MockBackend } from '@angular/http/testing';
import { Http, ConnectionBackend, BaseRequestOptions, Response, ResponseOptions } from '@angular/http';
import { SearchService } from './search.service';
import { tick, fakeAsync } from '@angular/core/testing/fake_async';
import { inject, TestBed } from '@angular/core/testing/test_bed';

describe('SearchService', () => {
  beforeEach(() => {

    TestBed.configureTestingModule({
      providers: [
        {
          provide: Http, useFactory: (backend: ConnectionBackend, defaultOptions: BaseRequestOptions) => {
          return new Http(backend, defaultOptions);
        }, deps: [MockBackend, BaseRequestOptions]
        },
        {provide: SearchService, useClass: SearchService},
        {provide: MockBackend, useClass: MockBackend},
        {provide: BaseRequestOptions, useClass: BaseRequestOptions}
      ]
    });
  });
});

If you run ng test, you will likely see some errors about the test stubs that Angular CLI created for you. You can ignore these for now.

ERROR in [default] /Users/mraible/ng2-demo/src/app/edit/edit.component.spec.ts:10:20
Supplied parameters do not match any signature of call target.

ERROR in [default] /Users/mraible/ng2-demo/src/app/search/search.component.spec.ts:10:20
Supplied parameters do not match any signature of call target.

Add the first test of getAll() to search.service.spec.ts. This test shows how MockBackend can be used to mock results and set the response.

TIP: When you are testing code that returns either a Promise or an RxJS Observable, you can use the fakeAsync helper to test that code as if it were synchronous. Promises are be fulfilled and Observables are notified immediately after you call tick().

The test below should be on the same level as beforeEach.

it('should retrieve all search results',
  inject([SearchService, MockBackend], fakeAsync((searchService: SearchService, mockBackend: MockBackend) => {
    let res: Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.getAll().subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

Notice that tests continually run as you add them when using ng test. You can run tests once by using ng test --watch=false. You will likely see "Executed 5 of 5 (1 FAILED)" in your terminal. Add a couple more tests for filtering by search term and fetching by id.

it('should filter by search term',
  inject([SearchService, MockBackend], fakeAsync((searchService: SearchService, mockBackend: MockBackend) => {
    let res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('john').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

it('should fetch by id',
  inject([SearchService, MockBackend], fakeAsync((searchService: SearchService, mockBackend: MockBackend) => {
    let res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"id": 1, "name": "John Elway"}, {"id": 2, "name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('2').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('Gary Kubiak');
  }))
);

Unit test the SearchComponent

To unit test the SearchComponent, create a MockSearchProvider that has spies. These allow you to spy on functions to check if they were called.

Create src/app/shared/search/mocks/search.service.ts and populate it with spies for each method, as well as methods to set the response and subscribe to results.

import { SpyObject } from './helper';
import { SearchService } from '../search.service';
import Spy = jasmine.Spy;

export class MockSearchService extends SpyObject {
  getAllSpy: Spy;
  getByIdSpy: Spy;
  searchSpy: Spy;
  saveSpy: Spy;
  fakeResponse: any;

  constructor() {
    super( SearchService );

    this.fakeResponse = null;
    this.getAllSpy = this.spy('getAll').andReturn(this);
    this.getByIdSpy = this.spy('get').andReturn(this);
    this.searchSpy = this.spy('search').andReturn(this);
    this.saveSpy = this.spy('save').andReturn(this);
  }

  subscribe(callback: any) {
    callback(this.fakeResponse);
  }

  setResponse(json: any): void {
    this.fakeResponse = json;
  }
}

In this same directory, create a helper.ts class to implement the SpyObject that MockSearchService extends.

import {StringMapWrapper} from '@angular/core/src/facade/collection';

export interface GuinessCompatibleSpy extends jasmine.Spy {
  /** By chaining the spy with and.returnValue, all calls to the function will return a specific
   * value. */
  andReturn(val: any): void;
  /** By chaining the spy with and.callFake, all calls to the spy will delegate to the supplied
   * function. */
  andCallFake(fn: Function): GuinessCompatibleSpy;
  /** removes all recorded calls */
  reset();
}

export class SpyObject {
  static stub(object = null, config = null, overrides = null) {
    if (!(object instanceof SpyObject)) {
      overrides = config;
      config = object;
      object = new SpyObject();
    }

    let m = StringMapWrapper.merge(config, overrides);
    StringMapWrapper.forEach(m, (value, key) => { object.spy(key).andReturn(value); });
    return object;
  }

  constructor(type = null) {
    if (type) {
      for (let prop in type.prototype) {
        let m = null;
        try {
          m = type.prototype[prop];
        } catch (e) {
          // As we are creating spys for abstract classes,
          // these classes might have getters that throw when they are accessed.
          // As we are only auto creating spys for methods, this
          // should not matter.
        }
        if (typeof m === 'function') {
          this.spy(prop);
        }
      }
    }
  }

  spy(name) {
    if (!this[name]) {
      this[name] = this._createGuinnessCompatibleSpy(name);
    }
    return this[name];
  }

  prop(name, value) { this[name] = value; }

  /** @internal */
  _createGuinnessCompatibleSpy(name): GuinessCompatibleSpy {
    let newSpy: GuinessCompatibleSpy = <any>jasmine.createSpy(name);
    newSpy.andCallFake = <any>newSpy.and.callFake;
    newSpy.andReturn = <any>newSpy.and.returnValue;
    newSpy.reset = <any>newSpy.calls.reset;
    // revisit return null here (previously needed for rtts_assert).
    newSpy.and.returnValue(null);
    return newSpy;
  }
}

Alongside, create routes.ts to mock Angular's Router and ActivatedRoute.

import { ActivatedRoute, Params } from '@angular/router';
import { Observable } from 'rxjs';

export class MockActivatedRoute extends ActivatedRoute {
  params: Observable<Params>

  constructor(parameters?: { [key: string]: any; }) {
    super();
    this.params = Observable.of(parameters);
  }
}

export class MockRouter {
  navigate = jasmine.createSpy('navigate');
}

With mocks in place, you can TestBed.configureTestingModule() to setup SearchComponent to use these as providers.

import { ActivatedRoute, Router } from '@angular/router';
import { MockActivatedRoute, MockRouter } from '../shared/search/mocks/routes';
import { MockSearchService } from '../shared/search/mocks/search.service';
import { SearchComponent } from './search.component';
import { TestBed } from '@angular/core/testing/test_bed';
import { FormsModule } from '@angular/forms';
import { SearchService } from '../shared/search/search.service';

describe('Component: Search', () => {
  let mockSearchService: MockSearchService;
  let mockActivatedRoute: MockActivatedRoute;
  let mockRouter: MockRouter;

  beforeEach(() => {
    mockSearchService = new MockSearchService();
    mockActivatedRoute = new MockActivatedRoute({'term': 'peyton'});
    mockRouter = new MockRouter();

    TestBed.configureTestingModule({
      declarations: [SearchComponent],
      providers: [
        {provide: SearchService, useValue: mockSearchService},
        {provide: ActivatedRoute, useValue: mockActivatedRoute},
        {provide: Router, useValue: mockRouter}
      ],
      imports: [FormsModule]
    });
  });
});

Add two tests, one to verify a search term is used when it's set on the component, and a second to verify search is called when a term is passed in as a route parameter.

it('should search when a term is set and search() is called', () => {
  let fixture = TestBed.createComponent(SearchComponent);
  let searchComponent = fixture.debugElement.componentInstance;
  searchComponent.query = 'M';
  searchComponent.search();
  expect(mockSearchService.searchSpy).toHaveBeenCalledWith('M');
});

it('should search automatically when a term is on the URL', () => {
  let fixture = TestBed.createComponent(SearchComponent);
  fixture.detectChanges();
  expect(mockSearchService.searchSpy).toHaveBeenCalledWith('peyton');
});

After adding these tests, you should see the first instance of all tests passing (Executed 8 of 8 SUCCESS).

Update the test for EditComponent, verifying fetching a single record works. Notice how you can access the component directly with fixture.debugElement.componentInstance, or its rendered version with fixture.debugElement.nativeElement.

import { MockSearchService } from '../shared/search/mocks/search.service';
import { EditComponent } from './edit.component';
import { TestBed } from '@angular/core/testing/test_bed';
import { SearchService } from '../shared/search/search.service';
import { MockRouter, MockActivatedRoute } from '../shared/search/mocks/routes';
import { ActivatedRoute, Router } from '@angular/router';
import { FormsModule } from '@angular/forms';

describe('Component: Edit', () => {
  let mockSearchService: MockSearchService;
  let mockActivatedRoute: MockActivatedRoute;
  let mockRouter: MockRouter;

  beforeEach(() => {
    mockSearchService = new MockSearchService();
    mockActivatedRoute = new MockActivatedRoute({'id': 1});
    mockRouter = new MockRouter();

    TestBed.configureTestingModule({
      declarations: [EditComponent],
      providers: [
        {provide: SearchService, useValue: mockSearchService},
        {provide: ActivatedRoute, useValue: mockActivatedRoute},
        {provide: Router, useValue: mockRouter}
      ],
      imports: [FormsModule]
    });
  });

  it('should fetch a single record', () => {
    const fixture = TestBed.createComponent(EditComponent);

    let person = {name: 'Emmanuel Sanders', address: {city: 'Denver'}};
    mockSearchService.setResponse(person);

    fixture.detectChanges();
    // verify service was called
    expect(mockSearchService.getByIdSpy).toHaveBeenCalledWith(1);

    // verify data was set on component when initialized
    let editComponent = fixture.debugElement.componentInstance;
    expect(editComponent.editAddress.city).toBe('Denver');

    // verify HTML renders as expected
    let compiled = fixture.debugElement.nativeElement;
    expect(compiled.querySelector('h3').innerHTML).toBe('Emmanuel Sanders');
  });
});

You should see "Executed 8 of 8 SUCCESS (0.238 secs / 0.259 secs)" in the shell window that's running ng test. If you don't, try cancelling the command and restarting.

Integration test the search UI

To test if the application works end-to-end, you can write tests with Protractor. These are also known as integration tests, since they test the integration between all layers of your application.

To verify end-to-end tests work in the project before you begin, run the following commands in three different console windows.

ng serve
ng e2e

All tests should pass.

$ ng e2e

> ng2-demo@0.0.0 pree2e /Users/mraible/dev/ng2-demo
> webdriver-manager update

Updating selenium standalone to version 2.52.0
downloading https://selenium-release.storage.googleapis.com/2.52/selenium-server-standalone-2.52.0.jar...
Updating chromedriver to version 2.21
downloading https://chromedriver.storage.googleapis.com/2.21/chromedriver_mac32.zip...
chromedriver_2.21mac32.zip downloaded to /Users/mraible/dev/ng2-demo/node_modules/protractor/selenium/chromedriver_2.21mac32.zip
selenium-server-standalone-2.52.0.jar downloaded to /Users/mraible/dev/ng2-demo/node_modules/protractor/selenium/selenium-server-standalone-2.52.0.jar

> ng2-demo@0.0.0 e2e /Users/mraible/dev/ng2-demo
> protractor "config/protractor.conf.js"

[00:01:07] I/direct - Using ChromeDriver directly...
[00:01:07] I/launcher - Running 1 instances of WebDriver
Spec started

  ng2-demo App
    ✔ should display message saying app works

Executed 1 of 1 spec SUCCESS in 0.684 sec.
[00:01:09] I/launcher - 0 instance(s) of WebDriver still running
[00:01:09] I/launcher - chrome #01 passed

All end-to-end tests pass.

Testing the search feature

Create end-to-end tests in e2e/search.e2e-spec.ts to verify the search feature works. Populate it with the following code:

describe('Search', () => {

  beforeEach(() => {
    browser.get('/search');
    element(by.linkText('Search')).click();
  });

  it('should have an input and search button', () => {
    expect(element(by.css('app-root app-search form input')).isPresent()).toEqual(true);
    expect(element(by.css('app-root app-search form button')).isPresent()).toEqual(true);
  });

  it('should allow searching', () => {
    let searchButton = element(by.css('button'));
    let searchBox = element(by.css('input'));
    searchBox.sendKeys('M');
    searchButton.click().then(() => {
      var list = element.all(by.css('app-search table tbody tr'));
      expect(list.count()).toBe(3);
    });
  });
});

Testing the edit feature

Create a e2e/edit.e2e-spec.ts test to verify the EditComponent renders a person's information and that their information can be updated.

describe('Edit', () => {

  beforeEach(() => {
    browser.get('/edit/1');
  });

  let name = element(by.id('name'));
  let street = element(by.id('street'));
  let city = element(by.id('city'));

  it('should allow viewing a person', () => {
    expect(element(by.css('h3')).getText()).toEqual('Peyton Manning');
    expect(name.getAttribute('value')).toEqual('Peyton Manning');
    expect(street.getAttribute('value')).toEqual('1234 Main Street');
    expect(city.getAttribute('value')).toEqual('Greenwood Village');
  });

  it('should allow updating a name', function () {
    let save = element(by.id('save'));
    // send individual characters since sendKeys passes partial values sometimes
    // https://github.com/angular/protractor/issues/698
    ' Won!'.split('').forEach((c) => name.sendKeys(c));
    save.click();
    // verify one element matched this change
    var list = element.all(by.css('app-search table tbody tr'));
    expect(list.count()).toBe(1);
  });
});

Run ng e2e to verify all your end-to-end tests pass. You should see a success message similar to the one below in your terminal window.

If you made it this far and have all your specs passing - congratulations! You're well on your way to writing quality code with Angular 2 and verifying it works.

You can see the test coverage of your project by opening coverage/index.html in your browser. You might notice that the new components and service could use some additional coverage. If you feel the need to improve this coverage, please send me a pull request!

Continuous Integration

At the time of this writing, Angular CLI did not have any continuous integration support. However, it's easy to add with Travis CI. If you've checked in your project to GitHub, you can easily use Travis CI. Simply login and enable builds for the GitHub repo you created the project in. Then add the following .travis.yml in your root directory and git push. This will trigger the first build.

language: node_js
sudo: true

cache:
  directories:
    - node
    - node_modules

dist: trusty

node_js:
  - '5.6.0'

branches:
  only:
  - master

before_install:
 - npm install -g angular-cli
 - export CHROME_BIN=/usr/bin/google-chrome
 - export DISPLAY=:99.0
 - sh -e /etc/init.d/xvfb start
 - sudo apt-get update
 - sudo apt-get install -y libappindicator1 fonts-liberation
 - wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
 - sudo dpkg -i google-chrome*.deb

script:
 - ng test --watch false # http://mseemann.de/frontend/2016/05/31/setup-angular-2-app-part-1.html
 - ng serve &
 - ng e2e

notifications:
  webhooks:
    on_success: change  # options: [always|never|change] default: always
    on_failure: always  # options: [always|never|change] default: always
    on_start: false     # default: false

Here is a build showing all unit and integration tests passing.

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/ng2-demo. If you have ideas for improvements, please leave a comment or send a pull request.

This tutorial was originally written using Asciidoctor. This means you can read it using DocGist if you like.

Summary

I hope you've enjoyed this in-depth tutorial on how to get started with Angular 2 and Angular CLI. Angular CLI takes much of the pain out of setting up an Angular 2 project and using Typescript. I expect great things from Angular CLI, mostly because the Angular 2 setup process can be tedious and CLI greatly simplifies things.

Testing Angular 2.0 RC1 Applications

2016-06-06T09:57:13-06:00

As mentioned on Friday, there's been quite a bit that's changed with Angular 2 between its Beta 9 and RC 1 releases. This article is an update to the Testing Angular 2 Applications I wrote in March. That tutorial was based on Angular 2.0 Beta 9. Rather than simply updating that tutorial and blog post for 2.0 RC1, I decided to create a new version for posterity's sake. The 2.0 Beta 9 version will remain on my blog and I've tagged the source on GitHub.

If you've already read the first version of Testing Angular 2 Applications, checkout the diff of the Asciidoctor version to see what's changed.

What you'll build

You'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. See Angular 2's guide to unit testing if you'd like more information on testing and why it's important.

The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your codebase grows.

What you'll need

About 15-30 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and npm installed. I recommend using nvm.

Get the tutorial project

Clone the angular2-tutorial repository, checkout the testing-start branch, and install its dependencies.

git clone https://github.com/mraible/angular2-tutorial.git
cd angular2-tutorial
git checkout testing-start
npm install

If you haven't completed the Getting Started with Angular 2.0 RC1 tutorial, you should peruse it so you understand how this application works. You can also simply start the app with npm start and view it in your browser at http://localhost:5555/.

Unit test the SearchService

Create src/client/app/shared/search/search.service.spec.ts and setup the test's infrastructure using MockBackend and BaseRequestOptions.

import {
  beforeEachProviders,
  it,
  describe,
  expect,
  inject,
  fakeAsync,
  tick
} from '@angular/core/testing';
import { MockBackend } from '@angular/http/testing';
import { provide } from '@angular/core';
import { Http, ConnectionBackend, BaseRequestOptions, Response, ResponseOptions } from '@angular/http';
import { SearchService } from './search.service';

export function main() {
  describe('Search Service', () => {
    beforeEachProviders(() => {
      return [BaseRequestOptions, MockBackend, SearchService,
        provide(Http, {
          useFactory: (backend:ConnectionBackend, defaultOptions:BaseRequestOptions) => {
            return new Http(backend, defaultOptions);
          }, deps: [MockBackend, BaseRequestOptions]
        }),
      ];
    });
  });
}

If you run npm test, all tests will pass, but you don't see "Search Service" as a listed test. You can fix this by adding the first test of getAll(). This test shows how MockBackend can be used to mock results and set the response.

TIP: When you are testing code that returns either a Promise or an RxJS Observable, you can use the fakeAsync helper to test that code as if it were synchronous. Promises are be fulfilled and Observables are notified immediately after you call tick().

The test below should be on the same level as beforeEachProviders.

it('should retrieve all search results',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res:Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.getAll().subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

Running npm test should result in "12 tests completed". Add a couple more tests for filtering by search term and fetching by id.

it('should filter by search term',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('john').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

it('should fetch by id',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('app/shared/search/data/people.json');
      let response = new ResponseOptions({body: '[{"id": 1, "name": "John Elway"}, {"id": 2, "name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('2').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('Gary Kubiak');
  }))
);

If you want to have tests continually run as you add them, you can run the following commands in separate shell windows.

npm run build.test.watch
npm run karma.start

NOTE: See Running Unit Tests on Karma to learn how to run your tests from IntelliJ IDEA.

Unit test the SearchComponent

To unit test the SearchComponent, create a MockSearchProvider that has spies. These allow you to spy on functions to check if they were called.

Create src/client/app/shared/search/mocks/search.service.ts and populate it with spies for each method, as well as methods to set the response and subscribe to results.

import { provide } from '@angular/core';
import { SpyObject } from './helper';

import { SearchService } from '../search.service';
import Spy = jasmine.Spy;

export class MockSearchService extends SpyObject {
  getAllSpy:Spy;
  getByIdSpy:Spy;
  searchSpy:Spy;
  saveSpy:Spy;
  fakeResponse:any;

  constructor() {
    super(SearchService);

    this.fakeResponse = null;
    this.getAllSpy = this.spy('getAll').andReturn(this);
    this.getByIdSpy = this.spy('get').andReturn(this);
    this.searchSpy = this.spy('search').andReturn(this);
    this.saveSpy = this.spy('save').andReturn(this);
  }

  subscribe(callback:any) {
    callback(this.fakeResponse);
  }

  setResponse(json:any):void {
    this.fakeResponse = json;
  }

  getProviders():Array<any> {
    return [provide(SearchService, {useValue: this})];
  }
}

In this same directory, create a helper.ts class to implement the SpyObject that MockSearchService extends.

import {StringMapWrapper} from '@angular/core/src/facade/collection';

export interface GuinessCompatibleSpy extends jasmine.Spy {
  /** By chaining the spy with and.returnValue, all calls to the function will return a specific
   * value. */
  andReturn(val: any): void;
  /** By chaining the spy with and.callFake, all calls to the spy will delegate to the supplied
   * function. */
  andCallFake(fn: Function): GuinessCompatibleSpy;
  /** removes all recorded calls */
  reset();
}

export class SpyObject {
  static stub(object = null, config = null, overrides = null) {
    if (!(object instanceof SpyObject)) {
      overrides = config;
      config = object;
      object = new SpyObject();
    }

    var m = StringMapWrapper.merge(config, overrides);
    StringMapWrapper.forEach(m, (value, key) => { object.spy(key).andReturn(value); });
    return object;
  }

  constructor(type = null) {
    if (type) {
      for (var prop in type.prototype) {
        var m = null;
        try {
          m = type.prototype[prop];
        } catch (e) {
          // As we are creating spys for abstract classes,
          // these classes might have getters that throw when they are accessed.
          // As we are only auto creating spys for methods, this
          // should not matter.
        }
        if (typeof m === 'function') {
          this.spy(prop);
        }
      }
    }
  }

  spy(name) {
    if (!this[name]) {
      this[name] = this._createGuinnessCompatibleSpy(name);
    }
    return this[name];
  }

  prop(name, value) { this[name] = value; }

  /** @internal */
  _createGuinnessCompatibleSpy(name): GuinessCompatibleSpy {
    var newSpy: GuinessCompatibleSpy = <any>jasmine.createSpy(name);
    newSpy.andCallFake = <any>newSpy.and.callFake;
    newSpy.andReturn = <any>newSpy.and.returnValue;
    newSpy.reset = <any>newSpy.calls.reset;
    // revisit return null here (previously needed for rtts_assert).
    newSpy.and.returnValue(null);
    return newSpy;
  }
}

Alongside, create routes.ts to mock Angular's RouteSegment and passing parameters between components.

import { RouteSegment } from '@angular/router';

export class MockRouteSegment implements RouteSegment {
  urlSegments:any;
  parameters:any;
  outlet:string;
  _type:any;
  _componentFactory:any;
  type:any;
  stringifiedUrlSegments:string;

  constructor(parameters?:{ [key:string]:any; }) {
    this.parameters = parameters;
  }

  getParam(param:string) {
    return this.parameters[param];
  }
}

With mocks in place, you can create a spec for SearchComponent that uses these as providers. Create a file at src/search/components/search.component.spec.ts and populate it with the following code.

import { provide } from '@angular/core';
import { TestComponentBuilder } from '@angular/compiler/testing';
import {
  it,
  describe,
  expect,
  inject,
  beforeEachProviders,
} from '@angular/core/testing';

import { RouteSegment } from '@angular/router';
import { MockRouteSegment } from '../shared/search/mocks/routes';
import { MockSearchService } from '../shared/search/mocks/search.service';

import { SearchComponent } from './search.component';

export function main() {
  describe('Search component', () => {
    var mockSearchService:MockSearchService;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();

      return [
        mockSearchService.getProviders(),
        provide(RouteSegment, { useValue: new MockRouteSegment({ 'term': 'peyton' }) })
      ];
    });
  });
}

Add two tests, one to verify a search term is used when it's set on the component and a second to verify search is called when a term is passed in as a route parameter.

it('should search when a term is set and search() is called', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  return tcb.createAsync(SearchComponent).then((fixture) => {
    let searchComponent = fixture.debugElement.componentInstance;
    searchComponent.query = 'M';
    searchComponent.search();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('M');
  });
}));

it('should search automatically when a term is on the URL', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  return tcb.createAsync(SearchComponent).then((fixture) => {
    fixture.detectChanges();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('peyton');
  });
}));

Add a spec for the EditComponent as well, verifying fetching a single record works. Notice how you can access the component directly with fixture.debugElement.componentInstance, or its rendered version with fixture.debugElement.nativeElement. Create a file at src/search/components/edit.component.spec.ts and populate it with the code below.

import { provide } from '@angular/core';
import { TestComponentBuilder } from '@angular/compiler/testing';
import {
  it,
  describe,
  expect,
  inject,
  beforeEachProviders,
} from '@angular/core/testing';

import { RouteSegment } from '@angular/router';
import { ROUTER_FAKE_PROVIDERS } from '@angular/router/testing';
import { MockRouteSegment } from '../shared/search/mocks/routes';
import { MockSearchService } from '../shared/search/mocks/search.service';

import { EditComponent } from './edit.component';

export function main() {
  describe('Edit component', () => {
    var mockSearchService:MockSearchService;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();

      return [
        mockSearchService.getProviders(),
        ROUTER_FAKE_PROVIDERS,
        provide(RouteSegment, { useValue: new MockRouteSegment({ 'id': '1' }) })
      ];
    });

    it('should fetch a single record', inject([TestComponentBuilder], (tcb:TestComponentBuilder) => {
      return tcb.createAsync(EditComponent).then((fixture) => {
        let person = {name: 'Emmanuel Sanders', address: {city: 'Denver'}};
        mockSearchService.setResponse(person);

        fixture.detectChanges();
        // verify service was called
        expect(mockSearchService.getByIdSpy).toHaveBeenCalledWith(1);

        // verify data was set on component when initialized
        let editComponent = fixture.debugElement.componentInstance;
        expect(editComponent.editAddress.city).toBe('Denver');

        // verify HTML renders as expected
        var compiled = fixture.debugElement.nativeElement;
        expect(compiled.querySelector('h3')).toHaveText('Emmanuel Sanders');
      });
    }));
  });
}

You should see "✔ 22 tests completed" in the shell window that's running npm run karma.start. If you don't, try cancelling the command and restarting.

Integration test the search UI

To test if the application works end-to-end, you can write tests with Protractor. These are also known as integration tests, since they test the integration between all layers of your application.

To verify end-to-end tests work in the project before you begin, run the following commands in three different console windows.

# npm run webdriver-update <- You will need to run this the first time
npm run webdriver-start
npm run serve.e2e
npm run e2e

You should receive an error stating that the "nav text for About" is incorrect.

This happens because we added a Search link to the navbar and didn't update the test (in app.component.e2e.ts) that looks for the last child.

it('should have correct nav text for About', () => {
    expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('ABOUT');
});

Replace this test with the one below, and add a new one to verify the Search link is last.

it('should have correct nav text for About', () => {
  expect(element(by.css('sd-app sd-navbar nav a:nth-child(2)')).getText()).toEqual('ABOUT');
});

it('should have correct nav text for Search', () => {
  expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('SEARCH');
});

Now when you run npm run e2e, all specs should pass.

Testing the search feature

Create a new search.component.e2e-spec.ts spec in the same directory as your SearchComponent. Add tests to verify elements are rendered correctly and search works. At the time of this writing, Protractor's by.model and by.repeater don't work with Angular 2. For this reason, I used by.css to verify the HTML renders as expected.

describe('Search', () => {

  beforeEach(() => {
    browser.get('/search');
  });

  it('should have an input and search button', () => {
    expect(element(by.css('sd-app sd-search form input')).isPresent()).toEqual(true);
    expect(element(by.css('sd-app sd-search form button')).isPresent()).toEqual(true);
  });

  it('should allow searching', () => {
    let searchButton = element(by.css('button'));
    let searchBox = element(by.css('input'));
    searchBox.sendKeys('M');
    searchButton.click().then(() => {
      // doesn't work as expected - results in 0
      //expect(element.all(by.repeater('person of searchResults')).count()).toEqual(3);
      var list = element.all(by.css('sd-search table tbody tr'));
      expect(list.count()).toBe(3);
    });
  });
});

Testing the edit feature

Create a edit.component.e2e-spec.ts spec to verify the EditComponent renders a person's information and that you can update their information.

describe('Edit', () => {

  beforeEach(() => {
    browser.get('/edit/1');
  });

  let name = element(by.id('name'));
  let street = element(by.id('street'));
  let city = element(by.id('city'));

  it('should allow viewing a person', () => {
    expect(element(by.css('h3')).getText()).toEqual('Peyton Manning');
    expect(name.getAttribute('value')).toEqual('Peyton Manning');
    expect(street.getAttribute('value')).toEqual('1234 Main Street');
    expect(city.getAttribute('value')).toEqual('Greenwood Village');
  });

  it('should allow updating a name', function () {
    let save = element(by.id('save'));
    // send individual characters since sendKeys passes partial values sometimes
    // https://github.com/angular/protractor/issues/698
    ' Won!'.split('').forEach((c) => name.sendKeys(c));
    save.click();
    // verify one element matched this change
    var list = element.all(by.css('sd-search table tbody tr'));
    expect(list.count()).toBe(1);
  });
});

Run npm run e2e to verify all your end-to-end tests pass. You might receive a failure for the "Home" test.

If you do, open src/client/app/+home/home.component.e2e-spec.ts and change line 17 from this:

element(by.css('sd-home form input')).sendKeys('Tim Berners-Lee');

To this:

let input = element(by.css('sd-home form input'));
'Tim Berners-Lee'.split('').forEach((c) => input.sendKeys(c));

Run npm run e2e again. You should see a success message similar to the one below in your terminal window.

If you made it this far and have all 13 specs passing - congratulations! You're well on your way to writing quality code with Angular 2 and verifying it works.

Continuous Integration

The angular2-seed project ships with a .travis.yml that you can use to run continuous integration for this application through Travis CI. To enable builds on Travis CI, login and enable builds for the GitHub repo you created the project in. Then trigger your first build with a git push.

When I first tried this, I received a failure because Protractor on Travis CI is unable to navigate directly to the search and edit components. I was able to workaround this by modifying search.component.e2e-spec.ts to start at the top and navigate to the component.

beforeEach(() => {
  browser.get('/');
  element(by.linkText('SEARCH')).click();
});

I did something similar with edit.component.e2e-spec.ts:

beforeEach(() => {
  browser.get('/');
  element(by.linkText('SEARCH')).click();
  let search = element(by.css('sd-search form input'));
  'Man'.split('').forEach((c) => search.sendKeys(c));
  element(by.css('sd-search form button')).click();
  element(by.linkText('Peyton Manning')).click();
});

After making these changes, all e2e tests passed in Travis CI.

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular2-tutorial. If you have ideas for improvements, please leave a comment or send a pull request.

I wrote this tutorial in Asciidoctor so you can also read this tutorial on GitHub or using DocGist.

Summary

I hope you've enjoyed this quick-and-easy tutorial on testing Angular 2.0 RC1 applications. You can see the test coverage of your project by running npm run serve.coverage. You'll notice that the new components and service could use some additional coverage. I'll leave that as a task for the reader.

I learned a lot about testing from ng-book 2 and its Testing chapter. If you have any Angular 2 testing tips and tricks you'd like to share, I'd love to hear about them.

Getting Started with Angular 2.0 RC1

2016-06-03T07:16:18-06:00

A few months ago, I wrote a tutorial on Getting Started with Angular 2. That tutorial was based on Angular 2.0.0 Beta 9. Rather than simply updating that tutorial and blog post for 2.0.0 RC1, I decided to create a new version for posterity's sake. The 2.0 Beta 9 version will remain on my blog and I've tagged the source on GitHub. This is an updated version of Getting Started with Angular 2, complete with the largely undocumented component router, and lazy-loaded components.

If you'd just like to see what's changed since the last release of this tutorial, you can view the pull request on GitHub. Note that I did sync my angular2-tutorial project with angular2-seed. This made it fairly easy to upgrade, believe it or not. My upgrade notes are in a gist. The best diff to read to see what changed is likely the diff of this tutorial.

What you'll build

You'll build a simple web application with Angular 2 and TypeScript. You'll add search and edit features with mock data.

What you'll need

About 15-30 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and npm installed. I recommend using nvm.

Create your project

Clone the angular2-seed repository using git:

git clone https://github.com/mgechev/angular2-seed.git angular2-tutorial
cd angular2-tutorial

NOTE: The angular2-seed project requires node v4.x.x or higher and npm 2.14.7. I used node v4.2.6 and npm 3.9.3.

Install the project's dependencies:

npm install

Run the application

The project is configured with a simple web server for development. To start it, run:

npm start

You should see a screen like the one below at http://localhost:5555.

You can see your new project's test coverage by running npm test:

=============================== Coverage summary ===============================
Statements   : 96.36% ( 159/165 )
Branches     : 62.5% ( 120/192 )
Functions    : 100% ( 35/35 )
Lines        : 100% ( 129/129 )
================================================================================

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the directory you cloned angular2-seed to.

The Basics

Create a src/client/app/+search folder and a search.component.html file in it. The + prefix is an indicator to Angular that you want to lazy-load the components in this directory. Populate the HTML file you created with the following:

<h2>Search</h2>
<form>
  <input type="search" [(ngModel)]="query" (keyup.enter)="search()">
  <button type="button" (click)="search()">Search</button>
</form>
<div *ngIf="loading">loading...</div>
<pre>{{searchResults | json}}</pre>

Create src/client/app/+search/search.component.ts to define the SearchComponent and point to this template.

import { Component } from '@angular/core';
import { CORE_DIRECTIVES, FORM_DIRECTIVES } from '@angular/common';
import { ROUTER_DIRECTIVES } from '@angular/router';

@Component({
  selector: 'sd-search',
  moduleId: module.id,
  templateUrl: 'search.component.html',
  directives: [FORM_DIRECTIVES, CORE_DIRECTIVES, ROUTER_DIRECTIVES]
})
export class SearchComponent {
  loading: boolean;
  query: string;
  searchResults: any;

  constructor() {
    console.log('initialized search component');
  }
}

Create src/client/app/+search/index.ts to export SearchComponent.

/**
* This barrel file provides the export for the lazy loaded SearchComponent.
*/
export * from './search.component';

Update src/client/app/app.component.ts to import this component and include its route.

import { SearchComponent } from './+search/index';

@Routes([
  { path: '/', component: HomeComponent },
  { path: '/about', component: AboutComponent },
  { path: '/search', component: SearchComponent }
])

Your browser should refresh automatically, thanks to Browsersync. Navigate to http://localhost:5555/search and you should see the search component.

You can see it needs a bit of styling. Angular 2 allows you to provide styles specific for your component using a styleUrls property on your component. Add this property to search.component.ts like you see below.

templateUrl: 'search.component.html',
styleUrls: ['search.component.css'],
directives: [FORM_DIRECTIVES, CORE_DIRECTIVES, ROUTER_DIRECTIVES]

Create src/client/app/+search/search.component.css and add some CSS.

:host {
  display: block;
  padding: 0 16px;
}

There, that looks better!

Finally, update src/client/app/shared/navbar/navbar.component.html to include a link to the search route.

<nav>
  <a [routerLink]="['/']">HOME</a>
  <a [routerLink]="['/about']">ABOUT</a>
  <a [routerLink]="['/search']">SEARCH</a>
</nav>

This section has shown you how to add a new component to a basic Angular 2 application. The next section shows you how to create a use a JSON file and localStorage to create a fake API.

The Backend

To get search results, create a SearchService that makes HTTP requests to a JSON file. Start by creating src/client/app/shared/search/data/people.json to hold your data.

[
  {
    "id": 1,
    "name": "Peyton Manning",
    "phone": "(303) 567-8910",
    "address": {
      "street": "1234 Main Street",
      "city": "Greenwood Village",
      "state": "CO",
      "zip": "80111"
    }
  },
  {
    "id": 2,
    "name": "Demaryius Thomas",
    "phone": "(720) 213-9876",
    "address": {
      "street": "5555 Marion Street",
      "city": "Denver",
      "state": "CO",
      "zip": "80202"
    }
  },
  {
    "id": 3,
    "name": "Von Miller",
    "phone": "(917) 323-2333",
    "address": {
      "street": "14 Mountain Way",
      "city": "Vail",
      "state": "CO",
      "zip": "81657"
    }
  }
]

Create src/client/app/shared/search/search.service.ts and provide Http as a dependency in its constructor. In this same file, define the Address and Person classes that JSON will be marshalled to.

import { Injectable } from '@angular/core';
import { Http, Response } from '@angular/http';

@Injectable()
export class SearchService {
  constructor(private http:Http) {}

  getAll() {
    return this.http.get('app/shared/search/data/people.json').map((res:Response) => res.json());
  }
}

export class Address {
  street:string;
  city:string;
  state:string;
  zip:string;

  constructor(obj?:any) {
    this.street = obj && obj.street || null;
    this.city = obj && obj.city || null;
    this.state = obj && obj.state || null;
    this.zip = obj && obj.zip || null;
  }
}

export class Person {
  id:number;
  name:string;
  phone:string;
  address:Address;

  constructor(obj?:any) {
    this.id = obj && Number(obj.id) || null;
    this.name = obj && obj.name || null;
    this.phone = obj && obj.phone || null;
    this.address = obj && obj.address || null;
  }
}

In search.component.ts, add imports for these classes.

import { Person, SearchService } from '../shared/index';

You can now add a type to the searchResults variable. While you're there, modify the constructor to inject the SearchService.

searchResults: Array<Person>;

constructor(public searchService: SearchService) {}

Then implement the search() method to call the service's getAll() method.

search(): void {
  this.searchService.getAll().subscribe(
    data => {this.searchResults = data;},
    error => console.log(error)
  );
}

At this point, you'll likely see the following message in your browser's console.

ORIGINAL EXCEPTION: No provider for SearchService!

This happens because the app hasn't provided this service to components. To fix this, create src/client/app/shared/search/index.ts and populate it with the following.

/**
 * This barrel file provides the export for the shared SearchService.
 */
export * from './search.service';

Then add a reference to this file in src/client/app/shared/index.ts so its included in the shared export.

/**
 * This barrel file provides the exports for the shared resources (services, components).
 */
export * from './name-list/index';
export * from './navbar/index';
export * from './toolbar/index';
export * from './search/index';

To fix the "No provider" error from above, update app.component.ts to import the SearchService and add the service to the list of provider.

import { NameListService, NavbarComponent, ToolbarComponent, SearchService } from './shared/index';

@Component({
  selector: 'sd-app',
  viewProviders: [NameListService, SearchService, HTTP_PROVIDERS]
  moduleId: module.id,

Now clicking the search button should work. To make the results look better, remove the <pre> tag in search.component.html and replace it with a <table>.

<table *ngIf="searchResults">
  <thead>
  <tr>
    <th>Name</th>
    <th>Phone</th>
    <th>Address</th>
  </tr>
  </thead>
  <tbody>
  <tr *ngFor="let person of searchResults; let i=index">
    <td>{{person.name}}</td>
    <td>{{person.phone}}</td>
    <td>{{person.address.street}}<br/>
      {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
    </td>
  </tr>
  </tbody>
</table>

Then add some additional CSS for this component in search.component.css.

table {
  margin-top: 10px;
  border-collapse: collapse;
}

th {
  text-align: left;
  border-bottom: 2px solid #ddd;
  padding: 8px;
}

td {
  border-top: 1px solid #ddd;
  padding: 8px;
}

Now the search results look better.

But wait, we still don't have search functionality! To add a search feature, add a search() method to search.service.ts.

search(q:string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results:any = [];
    data.map(item => {
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

Then refactor SearchComponent to call this method with its query variable.

search(): void {
  this.searchService.search(this.query).subscribe(
    data => {this.searchResults = data;},
    error => console.log(error)
  );
}

Now search results will be filtered by the query value you type in.

This section showed you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an edit feature

Modify search.component.html to add a link for editing a person.

<td><a [routerLink]="['/edit', person.id]">{{person.name}}</a></td>

Create src/client/app/+edit/edit.component.html to display an editable form. You might notice I've added id attributes to most elements. This is to make things easier when writing integration tests with Protractor.

<div *ngIf="person">
  <h3>{{editName}}</h3>
  <div>
    <label>Id:</label>
    {{person.id}}
  </div>
  <div>
    <label>Name:</label>
    <input [(ngModel)]="editName" id="name" placeholder="name"/>
  </div>
  <div>
    <label>Phone:</label>
    <input [(ngModel)]="editPhone" id="phone" placeholder="Phone"/>
  </div>
  <fieldset>
    <legend>Address:</legend>
    <address>
      <input [(ngModel)]="editAddress.street" id="street"><br/>
      <input [(ngModel)]="editAddress.city" id="city">,
      <input [(ngModel)]="editAddress.state" id="state" size="2">
      <input [(ngModel)]="editAddress.zip" id="zip" size="5">
    </address>
  </fieldset>
  <button (click)="save()" id="save">Save</button>
  <button (click)="cancel()" id="cancel">Cancel</button>
</div>

Create an EditComponent in src/search/components/edit.component.ts that references this template and handles communication with the SearchService.

import { Component, OnInit } from '@angular/core';
import { Person, Address, SearchService } from '../shared/index';
import { RouteSegment, Router } from '@angular/router';

@Component({
  selector: 'sd-edit',
  moduleId: module.id,
  templateUrl: 'edit.component.html',
  styleUrls: ['edit.component.css']
})
export class EditComponent implements OnInit {

  person: Person;
  editName: string;
  editPhone: string;
  editAddress: Address;

  constructor(
    private _service: SearchService,
    private _router: Router,
    private _routeSegment: RouteSegment
  ) { }

  ngOnInit() {
    let id = +this._routeSegment.getParam('id');
    this._service.get(id).subscribe(person => {
      if (person) {
        this.editName = person.name;
        this.editPhone = person.phone;
        this.editAddress = person.address;
        this.person = person;
      } else {
        this.gotoList();
      }
    });
  }

  cancel() {
    this._router.navigate(['/search']);
  }

  save() {
    this.person.name = this.editName;
    this.person.phone = this.editPhone;
    this.person.address = this.editAddress;
    this._service.save(this.person);
    this.gotoList();
  }

  gotoList() {
    if (this.person) {
      this._router.navigate(['/search', {term: this.person.name} ]);
    } else {
      this._router.navigate(['/search']);
    }
  }
}

Create src/client/app/+edit/index.ts to export EditComponent.

/**
 * This barrel file provides the export for the lazy loaded EditComponent.
 */
export * from './edit.component';

Modify SearchService to contain functions for finding a person by their id, and saving them. While you're in there, modify the search() method to be aware of updated objects in localStorage.

search(q:string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results = [];
    data.map(item => {
      // check for item in localStorage
      if (localStorage['person' + item.id]) {
        item = JSON.parse(localStorage['person' + item.id]);
      }
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

get(id: number) {
  return this.getAll().map(all => {
    if (localStorage['person' + id]) {
      return JSON.parse(localStorage['person' + id]);
    }
    return all.find(e => e.id === id);
  });
}

save(person: Person) {
  localStorage['person' + person.id] = JSON.stringify(person);
}

To make the app aware of this new component, add an import and route configuration in app.component.ts.

import { EditComponent } from '../+edit/index';

@Routes([
  { path: '/', component: HomeComponent },
  { path: '/about', component: AboutComponent },
  { path: '/search', component: SearchComponent },
  { path: '/edit/:id', component: EditComponent }
])

Then create src/client/app/+edit/edit.component.css to make the form look a bit better.

:host {
  display: block;
  padding: 0 16px;
}

button {
  margin-top: 10px;
}

At this point, you should be able to search for a person and update their information.

The <form> in src/client/app/+edit/edit.component.html calls a save() function to update a person's data. You already implemented this above. The function calls a gotoList() function that appends the person's name to the URL when sending the user back to the search screen.

gotoList() {
  if (this.person) {
    this._router.navigate(['/search', {term: this.person.name} ]);
  } else {
    this._router.navigate(['/search']);
  }
}

Since the SearchComponent doesn't execute a search automatically when you execute this URL, add the following logic to do so in its constructor.

constructor(public searchService: SearchService, routeSegment: RouteSegment) {
  if (routeSegment.getParam('term')) {
    this.query = decodeURIComponent(routeSegment.getParam('term'));
    this.search();
  }
}

You'll need to import RouteSegment in order for everything to compile.

import { ROUTER_DIRECTIVES, RouteSegment } from '@angular/router';

After making all these changes, you should be able to search/edit/update a person's information. If it works - nice job!

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular2-tutorial. If you have ideas for improvements, please leave a comment or send a pull request.

I originally wrote this tutorial in Asciidoctor because it has a slick feature where you can include the source code from files rather than copying and pasting. Since GitHub doesn't support includes, I changed things so the code is now embedded in the document. You can now read this tutorial on GitHub or using DocGist.

Summary

I hope you've enjoyed this quick-and-easy tutorial on how to get started with Angular 2.0 RC1. In a future tutorial, I'll show you how to write unit tests and integration tests for this application.

Resources

I used a number of resources while creating this application. ng-book 2 was an invaluable resource and I highly recommend it if you're learning Angular 2. I found Chariot Solution's article on Angular2 Observables, Http, and separating services and components to be quite helpful. Finally, the angular-cli project was a big help, especially its ng generate route <object> feature.

Kudos to all the pioneers in Angular 2 land that've been using it and writing about it on blogs and Stack Overflow. Getting started with Angular 2 would've been a real pain without your trailblazing.

Testing Angular 2 Applications

2016-03-29T08:08:58-06:00

Click here to see an updated version of this tutorial that's been upgraded for Angular 2.0 RC1.

This article is the second in a series about learning Angular 2. It describes how to test a simple Angular 2 application. In a previous article, Getting Started with Angular 2, I showed how to develop a simple search and edit feature. In this tutorial, I did my best to keep the tests similar to last year's Testing AngularJS Applications so you can compare the code between AngularJS and Angular 2.

What you'll build

You'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. Angular's documentation has a good guide to unit testing if you'd like more information on testing and why it's important.

The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your codebase grows.

What you'll need

About 15-30 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and npm installed. I recommend using nvm.

Get the tutorial project

Clone the angular2-tutorial repository using git and install its dependencies.

git clone https://github.com/mraible/angular2-tutorial.git
cd angular2-tutorial
npm install

If you haven't completed the Getting Started with Angular 2 tutorial, you should peruse it so you understand how this application works. You can also simply start the app with npm start and view it in your browser at http://localhost:5555/.

Unit test the SearchService

Create src/shared/services/search.service.spec.ts and setup the test's infrastructure using MockBackend and BaseRequestOptions.

import {it, describe, expect, inject, fakeAsync, beforeEachProviders, tick} from 'angular2/testing';
import {MockBackend} from 'angular2/http/testing';
import {provide} from 'angular2/core';
import 'rxjs/add/operator/map';
import {Http, ConnectionBackend, BaseRequestOptions, Response, ResponseOptions} from 'angular2/http';

import {SearchService} from './search.service';

export function main() {
  describe('Search Service', () => {
    beforeEachProviders(() => {
      return [BaseRequestOptions, MockBackend, SearchService,
        provide(Http, {
          useFactory: (backend:ConnectionBackend, defaultOptions:BaseRequestOptions) => {
            return new Http(backend, defaultOptions);
          }, deps: [MockBackend, BaseRequestOptions]
        }),
      ];
    });
  });
}

If you run npm test, you'll get a failed build from a number of unused imports. You can fix those by adding the first test of getAll(). This test shows how MockBackend can be used to mock results and set the response.

When you are testing code that returns either a Promise or an RxJS Observable, you can use the fakeAsync helper to test that code as if it were synchronous. Promises are be fulfilled and Observables are notified immediately after you call tick().

The test below should be on the same level as beforeEachProviders.

it('should retrieve all search results',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res:Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('shared/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.getAll().subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

Running npm test should result in "10 tests completed". Add a couple more tests for filtering by search term and fetching by id.

it('should filter by search term',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res:Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('shared/data/people.json');
      let response = new ResponseOptions({body: '[{"name": "John Elway"}, {"name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('john').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('John Elway');
  }))
);

it('should fetch by id',
  inject([SearchService, MockBackend], fakeAsync((searchService:SearchService, mockBackend:MockBackend) => {
    var res:Response;
    mockBackend.connections.subscribe(c => {
      expect(c.request.url).toBe('shared/data/people.json');
      let response = new ResponseOptions({body: '[{"id": 1, "name": "John Elway"}, {"id": 2, "name": "Gary Kubiak"}]'});
      c.mockRespond(new Response(response));
    });
    searchService.search('2').subscribe((response) => {
      res = response;
    });
    tick();
    expect(res[0].name).toBe('Gary Kubiak');
  }))
);

If you want to have tests continually run as you add them, you can run the following commands in separate shell windows.

npm run build.test.watch
npm run karma.start

NOTE: See Running Unit Tests on Karma to learn how to run your tests from IntelliJ IDEA.

Unit test the SearchComponent

To unit test the SearchComponent, create a MockSearchProvider that has spies. These allow you to spy on functions to check if they were called.

Create src/shared/services/mocks/search.service.ts and populate it with spies for each method, as well as methods to set the response and subscribe to results.

import {provide} from 'angular2/core';
import {SpyObject} from 'angular2/testing_internal';

import {SearchService} from '../search.service';
import Spy = jasmine.Spy;

export class MockSearchService extends SpyObject {
  getAllSpy:Spy;
  getByIdSpy:Spy;
  searchSpy:Spy;
  saveSpy:Spy;
  fakeResponse;

  constructor() {
    super(SearchService);

    this.fakeResponse = null;
    this.getAllSpy = this.spy('getAll').andReturn(this);
    this.getByIdSpy = this.spy('get').andReturn(this);
    this.searchSpy = this.spy('search').andReturn(this);
    this.saveSpy = this.spy('save').andReturn(this);
  }

  subscribe(callback) {
    callback(this.fakeResponse);
  }

  setResponse(json: any): void {
    this.fakeResponse = json;
  }

  getProviders(): Array<any> {
    return [provide(SearchService, {useValue: this})];
  }
}

In this same directory, create routes.ts to mock Angular's Router, RouteParams and RouterProvider.

import {provide} from 'angular2/core';
import {
  ComponentInstruction,
  Router,
  RouteParams
} from 'angular2/router';
import {ResolvedInstruction} from 'angular2/src/router/instruction';
import {SpyObject} from 'angular2/testing_internal';

export class MockRouteParams extends SpyObject {
  private ROUTE_PARAMS = {};

  constructor() { super(RouteParams); }

  set(key: string, value: string) {
    this.ROUTE_PARAMS[key] = value;
  }

  get(key: string) {
    return this.ROUTE_PARAMS[key];
  }
}

export class MockRouter extends SpyObject {
  constructor() { super(Router); }
  isRouteActive(s) { return true; }
  generate(s) {
    return new ResolvedInstruction(new ComponentInstruction('detail', [], null, null, true, '0'), null, {});
  }
}

export class MockRouterProvider {
  mockRouter: MockRouter = new MockRouter();
  mockRouteParams: MockRouteParams = new MockRouteParams();

  setRouteParam(key: string, value: any) {
    this.mockRouteParams.set(key, value);
  }

  getProviders(): Array<any> {
    return [
      provide(Router, {useValue: this.mockRouter}),
      provide(RouteParams, {useValue: this.mockRouteParams}),
    ];
  }
}

import {
  it,
  describe,
  expect,
  injectAsync,
  beforeEachProviders,
  TestComponentBuilder,
} from 'angular2/testing';

import {MockRouterProvider} from '../../shared/services/mocks/routes';
import {MockSearchService} from '../../shared/services/mocks/search.service';

import {SearchComponent} from './search.component';

export function main() {
  describe('Search component', () => {
    var mockSearchService:MockSearchService;
    var mockRouterProvider:MockRouterProvider;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();
      mockRouterProvider = new MockRouterProvider();

      return [
        mockSearchService.getProviders(), mockRouterProvider.getProviders()
      ];
    });
  });
}

Add two tests, one to verify a search term is used when it's set on the component and a second to verify search is called when a term is passed in as a route parameter.

it('should search when a term is set and search() is called', injectAsync([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  return tcb.createAsync(SearchComponent).then((fixture) => {
    let searchComponent = fixture.debugElement.componentInstance;
    searchComponent.query = 'M';
    searchComponent.search();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('M');
  });
}));

it('should search automatically when a term is on the URL', injectAsync([TestComponentBuilder], (tcb:TestComponentBuilder) => {
  mockRouterProvider.setRouteParam('term', 'peyton');
  return tcb.createAsync(SearchComponent).then((fixture) => {
    fixture.detectChanges();
    expect(mockSearchService.searchSpy).toHaveBeenCalledWith('peyton');
  });
}));

import {
  it,
  describe,
  expect,
  injectAsync,
  beforeEachProviders,
  TestComponentBuilder,
} from 'angular2/testing';

import {MockRouterProvider} from '../../shared/services/mocks/routes';
import {MockSearchService} from '../../shared/services/mocks/search.service';

import {EditComponent} from './edit.component';

export function main() {
  describe('Edit component', () => {
    var mockSearchService:MockSearchService;
    var mockRouterProvider:MockRouterProvider;

    beforeEachProviders(() => {
      mockSearchService = new MockSearchService();
      mockRouterProvider = new MockRouterProvider();

      return [
        mockSearchService.getProviders(), mockRouterProvider.getProviders()
      ];
    });

    it('should fetch a single record', injectAsync([TestComponentBuilder], (tcb:TestComponentBuilder) => {
      mockRouterProvider.setRouteParam('id', '1');
      return tcb.createAsync(EditComponent).then((fixture) => {
        let person = {name: 'Emmanuel Sanders', address: {city: 'Denver'}};
        mockSearchService.setResponse(person);

        fixture.detectChanges();
        // verify service was called
        expect(mockSearchService.getByIdSpy).toHaveBeenCalledWith(1);

        // verify data was set on component when initialized
        let editComponent = fixture.debugElement.componentInstance;
        expect(editComponent.editAddress.city).toBe('Denver');

        // verify HTML renders as expected
        var compiled = fixture.debugElement.nativeElement;
        expect(compiled.querySelector('h3')).toHaveText('Emmanuel Sanders');
      });
    }));
  });
}

You should see "? 20 tests completed" in the shell window that's running npm run karma.start. If you don't, try cancelling the command and restarting.

Integration test the search UI

To test if the application works end-to-end, you can write tests with Protractor. These are also known as integration tests, since they test the integration between all layers of your application.

To verify end-to-end tests work in the project before you begin, run the following commands in three different console windows.

# npm run webdriver-update <- You will need to run this the first time
npm run webdriver-start
npm run serve.e2e
npm run e2e

You should receive an error stating that the "nav text for About" is incorrect.

This happens because we added a Search link to the navbar and didn't update the test (in app.component.e2e.ts) that looks for the last child.

it('should have correct nav text for About', () => {
    expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('ABOUT');
});

Replace this test with the one below, and add a new one to verify the Search link is last.

it('should have correct nav text for About', () => {
  expect(element(by.css('sd-app sd-navbar nav a:nth-child(2)')).getText()).toEqual('ABOUT');
});

it('should have correct nav text for Search', () => {
  expect(element(by.css('sd-app sd-navbar nav a:last-child')).getText()).toEqual('SEARCH');
});

Now when you run npm run e2e, all specs should pass.

Testing the search feature

Create a new search.component.e2e.ts spec in the same directory as your SearchComponent. Add tests to verify elements are rendered correctly and search works. At the time of this writing, Protractor's by.model and by.repeater don't work with Angular 2. For this reason, I used by.css to verify the HTML renders as expected.

describe('Search', () => {

  beforeEach(() => {
    browser.get('/search');
  });

  it('should have an input and search button', () => {
    expect(element(by.css('sd-app sd-search form input')).isPresent()).toEqual(true);
    expect(element(by.css('sd-app sd-search form button')).isPresent()).toEqual(true);
  });

  it('should allow searching', () => {
    let searchButton = element(by.css('button'));
    let searchBox = element(by.css('input'));
    searchBox.sendKeys('M');
    searchButton.click().then(() => {
      // doesn't work as expected - results in 0
      //expect(element.all(by.repeater('person of searchResults')).count()).toEqual(3);
      var list = element.all(by.css('sd-search table tbody tr'));
      expect(list.count()).toBe(3);
    });
  });
});

Testing the edit feature

Create a edit.component.e2e.ts spec to verify the EditComponent renders a person's information and that you can update their information.

describe('Edit', () => {
 
  beforeEach(() => {
    browser.get('/edit/1');
  });
 
  let name = element(by.id('name'));
  let street = element(by.id('street'));
  let city = element(by.id('city'));
 
  it('should allow viewing a person', () => {
    expect(element(by.css('h3')).getText()).toEqual('Peyton Manning');
    expect(name.getAttribute('value')).toEqual('Peyton Manning');
    expect(street.getAttribute('value')).toEqual('1234 Main Street');
    expect(city.getAttribute('value')).toEqual('Greenwood Village');
  });
 
  it('should allow updating a name', function () {
    let save = element(by.id('save'));
    // send individual characters since sendKeys passes partial values sometimes
    // https://github.com/angular/protractor/issues/698
    ' Won!'.split('').forEach((c) => name.sendKeys(c));
    save.click();
    // verify one element matched this change
    var list = element.all(by.css('sd-search table tbody tr'));
    expect(list.count()).toBe(1);
  });
});

Run npm run e2e to verify all your end-to-end tests pass. You should see a success message similar to the one below in your terminal window.

If you made it this far and have all 13 specs passing - congratulations! You're well on your way to writing quality code with Angular 2 and verifying it works.

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular2-tutorial. If you have ideas for improvements, please leave a comment or send a pull request.

I originally wrote this tutorial in Asciidoctor because it has a slick feature where you can include the source code from files rather than copying and pasting. Unfortunately, GitHub doesn't support includes. You can use DocGist to view this tutorial, but includes don't work there either.

If you'd like to see the Asciidoctor-generated version of this tutorial, you can install the gem, checkout the project from GitHub, and then run asciidoctor TESTING.adoc to produce a TESTING.html file.

Summary

I hope you've enjoyed this quick-and-easy tutorial on testing Angular 2 applications. You can see the test coverage of your project by running npm run serve.coverage. You'll notice that the new components and service could use some additional coverage. I'll leave that as a task for the reader.

I learned a lot about testing from ng-book 2 and its Testing chapter. If you have any Angular 2 testing tips and tricks you'd like to share, I'd love to hear about them.

Getting Started with Angular 2

2016-03-23T09:23:53-06:00

Click here to see an updated version of this tutorial that's been upgraded for Angular 2.0 RC1.

I was hired by one of my current clients in November to help them develop a project management application with AngularJS. I'm proud to say we've built the application, it looks great, and it's scheduled to be released next month. The team had lots of experience with ExtJS, but was new to AngularJS. While using AngularJS worked, they're keen on moving to Angular 2 shortly after it's released.

To help them learn Angular 2, I decided to write a couple tutorials similar to the AngularJS tutorials I wrote last year. In this tutorial, I did my best to keep the functionality and features similar to Getting Started with AngularJS so you can compare the code between the two.

What you'll build

You'll build a simple web application with Angular 2 and TypeScript. You'll add search and edit features with mock data.

What you'll need

About 15-30 minutes.
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and npm installed. I recommend using nvm.

Create your project

Clone the angular2-seed repository using git:

git clone https://github.com/mgechev/angular2-seed.git angular2-tutorial
cd angular2-tutorial

NOTE: The angular2-seed project requires node v4.x.x or higher and npm 2.14.7. I used node v4.2.6 and npm 3.6.0.

Install ts-node for TypeScript:

npm install -g ts-node

Install the project's dependencies:

npm install

Run the application

The project is configured with a simple web server for development. To start it, run:

npm start

You should see a screen like the one below at http://localhost:5555.

You can see your new project's test coverage by running npm test:

=============================== Coverage summary ===============================
Statements : 86.11% ( 93/108 )
Branches : 48.28% ( 70/145 )
Functions : 100% ( 25/25 )
Lines : 94.32% ( 83/88 )
================================================================================

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the directory you cloned angular2-seed to.

The Basics

Create a file at src/search/components/search.component.html with the following HTML:

<h2>Search</h2>
<form>
  <input type="search" [(ngModel)]="query" (keyup.enter)="search()">
  <button type="button" (click)="search()">Search</button>
</form>
<div *ngIf="loading">loading...</div>
<pre>{{searchResults | json}}</pre>

Create src/search/components/search.component.ts to define the SearchComponent and point to this template.

import {Component} from 'angular2/core';
import {CORE_DIRECTIVES, FORM_DIRECTIVES} from 'angular2/common';
import {ROUTER_DIRECTIVES} from 'angular2/router';

@Component({
  selector: 'sd-search',
  moduleId: module.id,
  templateUrl: './search.component.html',
  directives: [FORM_DIRECTIVES, CORE_DIRECTIVES, ROUTER_DIRECTIVES]
})
export class SearchComponent {
  loading: boolean;
  query: string;
  searchResults: any;

  constructor() {
    console.log('initialized search component');
  }
}

Update src/app/components/app.component.ts to import this component and include its route.

import {SearchComponent} from '../../search/components/search.component';

@RouteConfig([
  { path: '/',      name: 'Home',  component: HomeComponent  },
  { path: '/about', name: 'About', component: AboutComponent },
  { path: '/search', name: 'Search', component: SearchComponent }

Your browser should refresh automatically, thanks to Browsersync. Navigate to http://localhost:5555/search and you should see the search component.

templateUrl: './search.component.html',
styleUrls: ['./search.component.css'],
directives: [FORM_DIRECTIVES, CORE_DIRECTIVES, ROUTER_DIRECTIVES]

Create src/search/components/search.component.css and add some CSS.

:host {
  display: block;
  padding: 0 16px;
}

There, that looks better!

Finally, update src/app/components/navbar.component.html to include a link to the search route.

<nav>
  <a [routerLink]="['Home']">HOME</a>
  <a [routerLink]="['About']">ABOUT</a>
  <a [routerLink]="['Search']">SEARCH</a>
</nav>

This section has shown you how to add a new component to a basic Angular 2 application. The next section shows you how to create a use a JSON file and localStorage to create a fake API.

The Backend

To get search results, create a SearchService that makes HTTP requests to a JSON file. Start by creating src/shared/data/people.json to hold your data.

[
  {
    "id": 1,
    "name": "Peyton Manning",
    "phone": "(303) 567-8910",
    "address": {
      "street": "1234 Main Street",
      "city": "Greenwood Village",
      "state": "CO",
      "zip": "80111"
    }
  },
  {
    "id": 2,
    "name": "Demaryius Thomas",
    "phone": "(720) 213-9876",
    "address": {
      "street": "5555 Marion Street",
      "city": "Denver",
      "state": "CO",
      "zip": "80202"
    }
  },
  {
    "id": 3,
    "name": "Von Miller",
    "phone": "(917) 323-2333",
    "address": {
      "street": "14 Mountain Way",
      "city": "Vail",
      "state": "CO",
      "zip": "81657"
    }
  }
]

Create src/shared/services/search.service.ts and provide Http as a dependency in its constructor. In this same file, define the Address and Person classes that JSON will be marshalled to.

import {Injectable} from 'angular2/core';
import {Http, Response} from 'angular2/http';

@Injectable()
export class SearchService {
  constructor(private http:Http) {}

  getAll() {
    return this.http.get('shared/data/people.json').map((res:Response) => res.json());
  }
}

export class Address {
  street:string;
  city:string;
  state:string;
  zip:string;

  constructor(obj?:any) {
    this.street = obj && obj.street || null;
    this.city = obj && obj.city || null;
    this.state = obj && obj.state || null;
    this.zip = obj && obj.zip || null;
  }
}

export class Person {
  id:number;
  name:string;
  phone:string;
  address:Address;

  constructor(obj?:any) {
    this.id = obj && Number(obj.id) || null;
    this.name = obj && obj.name || null;
    this.phone = obj && obj.phone || null;
    this.address = obj && obj.address || null;
  }
}

In search.component.ts, add imports for these classes.

import {Person, SearchService} from '../../shared/services/search.service';

You can now add a type to the searchResults variable. While you're there, modify the constructor to inject the SearchService.

searchResults: Array<Person>;

constructor(public searchService: SearchService) {}

Then implement the search() method to call the service's getAll() method.

search(): void {
  this.searchService.getAll().subscribe(
    data => {this.searchResults = data;},
    error => console.log(error)
  );
}

At this point, you'll likely see the following message in your browser's console.

EXCEPTION: No provider for SearchService! (SearchComponent -> SearchService)

This happens because the app hasn't provided this service to components. To fix this, modify app.component.ts to import this component and add the service to the list of providers.

import {NameListService} from '../../shared/services/name-list.service';
import {SearchService} from '../../shared/services/search.service';

@Component({
  selector: 'sd-app',
  viewProviders: [NameListService, SearchService],
  moduleId: module.id,

Next, you'll likely get an error about the Http dependency in SearchService.

EXCEPTION: No provider for Http! (SearchComponent -> SearchService -> Http)

To solve this problem, modify src/main.ts to import the Http service and make it available to the app.

import {HTTP_PROVIDERS} from 'angular2/http';

bootstrap(AppComponent, [
  HTTP_PROVIDERS, ROUTER_PROVIDERS,
  provide(APP_BASE_HREF, { useValue: '<%= APP_BASE %>' })
]);

Now the page will load without errors. However, when you click on the button, you'll see the following error.

ORIGINAL EXCEPTION: TypeError: this.http.get(...).map is not a function

I was stuck here for quite some time when I first encountered this issue. I was able to solve it with a simple import in main.ts.

import 'rxjs/add/operator/map';

Now clicking the search button should work. To make the results look better, remove the <pre> tag in search.component.html and replace it with a <table>.

<table *ngIf="searchResults">
  <thead>
  <tr>
    <th>Name</th>
    <th>Phone</th>
    <th>Address</th>
  </tr>
  </thead>
  <tbody>
  <tr *ngFor="#person of searchResults; #i=index">
    <td>{{person.name}}</td>
    <td>{{person.phone}}</td>
    <td>{{person.address.street}}<br/>
      {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
    </td>
  </tr>
  </tbody>
</table>

Then add some additional CSS for this component in search.component.css.

table {
  margin-top: 10px;
  border-collapse: collapse;
}

th {
  text-align: left;
  border-bottom: 2px solid #ddd;
  padding: 8px;
}

td {
  border-top: 1px solid #ddd;
  padding: 8px;
}

Now the search results look better.

But wait, we still don't have search functionality! To add a search feature, add a search() method to search.service.ts.

search(q:string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results = [];
    data.map(item => {
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

Then refactor SearchComponent to call this method with its query variable.

search(): void {
  this.searchService.search(this.query).subscribe(
    data => {this.searchResults = data;},
    error => console.log(error)
  );
}

Now search results will be filtered by the query value you type in.

This section showed you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an edit feature

Modify search.component.html to add a link for editing a person.

<td><a [routerLink]="['Edit', { id: person.id }]">{{person.name}}</a></td>

Create src/search/components/edit.component.html to display an editable form. You might notice I've added id attributes to most elements. This is to make things easier when writing integration tests with Protractor.

<div *ngIf="person">
  <h3>{{editName}}</h3>
  <div>
    <label>Id:</label>
    {{person.id}}
  </div>
  <div>
    <label>Name:</label>
    <input [(ngModel)]="editName" id="name" placeholder="name"/>
  </div>
  <div>
    <label>Phone:</label>
    <input [(ngModel)]="editPhone" id="phone" placeholder="Phone"/>
  </div>
  <fieldset>
    <legend>Address:</legend>
    <address>
      <input [(ngModel)]="editAddress.street" id="street"><br/>
      <input [(ngModel)]="editAddress.city" id="city">,
      <input [(ngModel)]="editAddress.state" id="state" size="2">
      <input [(ngModel)]="editAddress.zip" id="zip" size="5">
    </address>
  </fieldset>
  <button (click)="save()" id="save">Save</button>
  <button (click)="cancel()" id="cancel">Cancel</button>
</div>

Create an EditComponent in src/search/components/edit.component.ts that references this template and handles communication with the SearchService.

import {Component, OnInit} from 'angular2/core';
import {Person, Address, SearchService} from '../../shared/services/search.service';
import {RouteParams, Router} from 'angular2/router';
import {CanDeactivate, ComponentInstruction} from 'angular2/router';

@Component({
  selector: 'sd-edit',
  moduleId: module.id,
  templateUrl: './edit.component.html',
  styleUrls: ['./edit.component.css']
})
export class EditComponent implements OnInit, CanDeactivate {

  person: Person;
  editName: string;
  editPhone: string;
  editAddress: Address;

  constructor(
    private _service: SearchService,
    private _router: Router,
    private _routeParams: RouteParams
  ) { }

  ngOnInit() {
    let id = +this._routeParams.get('id');
    this._service.get(id).subscribe(person => {
      if (person) {
        this.editName = person.name;
        this.editPhone = person.phone;
        this.editAddress = person.address;
        this.person = person;
      } else {
        this.gotoList();
      }
    });
  }

  routerCanDeactivate(next: ComponentInstruction, prev: ComponentInstruction): any {
    if (!this.person || this.person.name === this.editName || this.person.phone === this.editPhone
      || this.person.address === this.editAddress) {
      return true;
    }

    return new Promise<boolean>((resolve, reject) => resolve(window.confirm('Discard changes?')));
  }

  cancel() {
    this._router.navigate(['Search']);
  }

  save() {
    this.person.name = this.editName;
    this.person.phone = this.editPhone;
    this.person.address = this.editAddress;
    this._service.save(this.person);
    this.gotoList();
  }

  gotoList() {
    if (this.person) {
      this._router.navigate(['Search', { term: this.person.name }]);
    } else {
      this._router.navigate(['Search']);
    }
  }
}

Modify SearchService to contain functions for finding a person by their id, and saving them. While you're in there, modify the search() method to be aware of updated objects in localStorage.

search(q:string) {
  if (!q || q === '*') {
    q = '';
  } else {
    q = q.toLowerCase();
  }
  return this.getAll().map(data => {
    let results = [];
    data.map(item => {
      // check for item in localStorage
      if (localStorage['person' + item.id]) {
        item = JSON.parse(localStorage['person' + item.id]);
      }
      if (JSON.stringify(item).toLowerCase().includes(q)) {
        results.push(item);
      }
    });
    return results;
  });
}

get(id: number) {
  return this.getAll().map(all => {
    if (localStorage['person' + id]) {
      return JSON.parse(localStorage['person' + id]);
    }
    return all.find(e => e.id === id);
  });
}

save(person: Person) {
  localStorage['person' + person.id] = JSON.stringify(person);
}

To make the app are of this new component, add an import and route configuration in app.component.ts.

import {EditComponent} from '../../search/components/edit.component';

@RouteConfig([
  { path: '/',      name: 'Home',  component: HomeComponent  },
  { path: '/about', name: 'About', component: AboutComponent },
  { path: '/search', name: 'Search', component: SearchComponent },
  { path: '/edit/:id', name: 'Edit', component: EditComponent }
])

Then create src/search/components/edit.component.css to make the form look a bit better.

:host {
  display: block;
  padding: 0 16px;
}

button {
  margin-top: 10px;
}

At this point, you should be able to search for a person and update their information.

The <form> in src/search/components/edit.component.html calls a save() function to update a person's data. You already implemented this above. The function calls a gotoList() function that appends the person's name to the URL when sending the user back to the search screen.

gotoList() {
  if (this.person) {
    this._router.navigate(['Search', { term: this.person.name }]);
  } else {
    this._router.navigate(['Search']);
  }
}

Since the SearchComponent doesn't execute a search automatically when you execute this URL, add the following logic to do so in its constructor.

constructor(public searchService: SearchService, params: RouteParams) {
  if (params.get('term')) {
    this.query = decodeURIComponent(params.get('term'));
    this.search();
  }
}

You'll need to import RouteParams in order for everything to compile.

import {ROUTER_DIRECTIVES, RouteParams} from 'angular2/router';

After making all these changes, you should be able to search/edit/update a person's information. If it works - nice job!

Source code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular2-tutorial. If you have ideas for improvements, please leave a comment or send a pull request.

If you'd like to see the Asciidoctor-generated version of this tutorial, you can install the gem, checkout the project from GitHub, and then run asciidoctor README.adoc to produce a README.html file.

Summary

I hope you've enjoyed this quick-and-easy tutorial on how to get started with Angular 2. In a future tutorial, I'll show you how to write unit tests and integration tests for this application. I've also started looking into creating an ES6 version of this tutorial using Soós Gábor's angular2-es6-starter. If you know of a better starter for Angular 2 and ES6, please let me know.

Resources

Kudos to all the pioneers in Angular 2 land that've been using it and writing about it on blogs and Stack Overflow. Getting started with Angular 2 would've been a real pain without your trailblazing.

How do I become a programmer?

2015-08-13T08:32:43-06:00

Yesterday, I received a message from a friend, asking about how to become a programmer. It's not the first time I've been asked this. In fact, this summer I've been asked by several friends how to get into the field. It seems that as people grow older, they see the lifestyle of working remotely and enjoying their job as an attractive thing to do. In yesterday's case, this friend is a mom that now has her days free because all her kids are in school. Here's what she wrote:

Now that my girls are both in school full day, I've been thinking about taking some programming classes. It's something I started to do while I was working at [ABC Company], but obviously didn't pursue once I quit to have kids. I'm thinking of getting my MIS in web development or specializing in designing apps if that's even a thing? Anyway, what languages would you recommend I concentrate on? JavaScript, Python? Lastly, is there a particular school you would recommend? I can't afford DU on my stay-at-home-mom salary, or even Regis which is where I started when I was getting tuition reimbursement. I was hoping I could do most of my education online while the kids are in school? Any advice or words of wisdom would be greatly appreciated!

Since this is a common question I see, I figured I'd publish my answers here, and get some advice from y'all too. Here's my response:

Python would definitely be good, as would JavaScript. JavaScript can be done on the client and server these days, so you could do that and be able to do front-end and backend development.

For programming specifically, I've heard these guys have a good JavaScript course: https://www.codecademy.com. Here's how to get started with Python in eight weeks: http://lifehacker.com/how-i-taught-myself-to-code-in-eight-weeks-511615189. And one of my favorites: http://programming-motherfucker.com/become.html.

I've taken a Scala course from Coursera, it was hard and intense, but I learned a lot. They have lots of courses and give you certifications you can put on your LinkedIn profile: https://www.coursera.org.

I've also recommended https://teamtreehouse.com to folks and https://www.khanacademy.org has always been good, even for kids.

Ultimately, the best way to learn to code is by doing. It's definitely good to study, learn and practice, but it'll probably won't sink in and become real knowledge until you're getting paid to do it. With the plethora of high-priced programmers out there, you can likely find a junior position, show a willingness to learn and come up to speed quickly. If you can couple that with a remote position, I think you'll really enjoy yourself.

Her response was interesting, as she thought she might need a CS degree to even get a programming job.

Coincidentally I looked over many of these coding sites yesterday but wasn't sure if I needed an accredited diploma. It sounds like it's more important that I just get some experience.

From my experience, a college degree matters, but not a CS degree. I told her people skills make programmers stand out and she's a witty person that certainly has those. What's your advice as a programmer? What would you tell people to do if they want to break into the field?

More importantly, if you're on the hiring side, what would it take for you to hire a 40-something person with no programming background? If they've been studying for six months and have really good people skills, would you hire them for a junior position?

UberConf 2015: My Presentations on Apache Camel and Java Webapp Security

2015-07-27T08:08:48-06:00

Last week I had the pleasure of speaking at UberConf 2015. My first talk was on Developing, Testing and Scaling with Apache Camel. This presentation contained an intro to Apache Camel and a recap of my experience using it at a client last year. You can click through the presentation below, download it from my presentations page, or view it on SlideShare.

My second presentation was about implementing Java Web Application Security with Java EE, Spring Security and Apache Shiro. I updated this presentation to use Java EE 7 and Jersey, as well as Spring Boot. I used Spring Boot to manage dependencies in all three projects, then showed the slick out-of-the-box security Spring Boot has (when you include the Spring Security on the classpath). For Apache Shiro, I configured its filter and required dependencies using Spring's JavaConfig. You can click through my security presentation below, download it from my presentations page, or view it on SlideShare.

One thing that didn't make it into the presentation was the super-helpful pull request from Rob Winch, Spring Security Lead. He showed me how you can use basic and form-based authentication in the same app, as well how to write tests with MockMvc and Spring Security's Testing support.

The next time I do this presentation (at the Rich Web Experience), I'd like to see if it's possible to use all-Java to configure the Java EE 7 example. I used web.xml in this example and the Servlet 3.0 Security Annotations might offer enough to get rid of it.

All the demos I did during the security presentation can be seen in my java-webapp-security-examples project on GitHub. There's branches for where I started (javaee-start, springsecurity-start and apacheshiro-start) as well as "complete" branches for where I finished. The complete examples should also be in-sync with the master branch.

If you have any questions about either presentation, please let me know.

How To Setup Your Own Software Development Company, 6 Years Later

2015-03-02T09:26:01-07:00

Just over six years ago, I wrote a popular post titled How To Setup Your Own Software Development Company. I'd just left LinkedIn a few months earlier and was enjoying consulting life again, working with a group of friends at Evite. In the article, I wrote about how I liked consulting because it forces you to keep your skills up-to-date and it pays a lot better. I also talked about the type of legal entity you should form (I have an S Corp), what business insurance you should buy, what I had for health insurance and how I automated payroll and tax payments.

I recently received an email from a reader, asking me if I had any updated thoughts.

It's been nearly six years since you wrote the article about starting your own business ... and thanks, by the way.

I am starting my venture into independent contract work as a software engineer (Java technology) in California and most likely will setup an S corp entity.

Seeing that you wrote this six years ago and things have considerably changed in the U.S. (economy, health care, etc.), I was wondering if you had some updated thoughts to share, perhaps some learned lessons even.

And also, I have questions about business insurance: what type of insurance should I opt for? Is there really an umbrella insurance out there? Or does each (or many) clients out there dictate the insurance you need?

Yes, a lot has changed in the last six years. The economy has improved and health care costs have risen. Through this time, I've been able to continue to operate as an independent software developer and keep the contracts flowing. Personally, the biggest changes in my life have been outside of work. I met an exceptional woman, traveled to conferences all around the world with her, got married, traveled some more, then bought a VW Westfalia so we could have lots of fun traveling in our own backyard. All the while, I've worked for some great clients. I built a team of hot shots at Time Warner Cable (many of them still work there), I skied the awesome powder of Utah while working at Overstock and I enjoyed a long-term contract at Oracle. After Oracle, I got into the healthcare industry and I've been working in it ever since.

In fact, I just finished working for a healthcare company last week and I'm on the hunt for my next gig in April. Check out my LinkedIn profile if you'd like to see my résumé.

I've learned quite a few lessons over the last several years. As an independent developer, the biggest thing I've learned is marketing is key. I've always known this, but I've been reminded of its importance a few times. When I worked at Taleo (after Overstock), I was on a 3-month contract that turned into a 9-month contract that got a 1-year extension when Oracle bought them. The work was challenging, but the application was outdated. Getting them to adopt new technologies like Bootstrap and AngularJS was difficult. When Oracle took over, they offered me a 1-year contract at a great rate. I accepted, never thinking it would be difficult to get paid from someone like Oracle. It took them over three months to pay my first invoice and it took me another three months to get payments flowing regularly. I felt like I was trapped. I felt like I could quit, but that wouldn't speed up the process of getting my invoices paid. From this experience, I'm hesitant to start with any contract that's longer than three months.

During my time at Oracle, I didn't blog as much as I had previously (because the day-to-day work wasn't that exciting), but I did still speak at conferences. Last year, I took the year off from speaking at conferences altogether. Speaking is an excellent marketing tool. Because of my lack of speaking, I saw a downturn in contract opportunities in Q4 last year.

As far as health insurance is concerned, I continued to have a disaster prevention plan, with a $5K per year deductible. I paid around $300/month for this, and rarely used it. By riding my bike to my office in downtown Denver, and skiing a bunch in the winter, I felt like I was pretty healthy. After I stopped eating sugar last fall, I became much healthier. So much healthier that I've stopped taking high blood pressure medication. Today, I don't pay for health insurance. Trish went back to IT Security Sales in November and she was able to get me on her company's plan for $100 cheaper than what I was paying. I didn't have dental insurance for the last five years and I did have to shell out $5K for a tooth implant at one point.

For business insurance, I have the Business Owner's Policy from The Hartford. I pay around $600/year and I've gotten that back when I've had laptops stolen or accidentally killed my iPhone. I've got automated backups going all the time, so I haven't lost any data in several years. This insurance policy and its liability coverage has been "good enough" for all my clients, including the big ones.

I think the biggest lesson I've learned in the last several years is that the best way to be rich is to be rich in time. I've always dreamed of making $500/hour and working 20 hours per week. While $500/hour sounds crazy, you know there's consultants out there that are making that kinda cash. They're probably not in software, maybe they're political consultants, or former professional athletes, but those consulting rates do exist. In software, there's certainly companies that bill those kinda rates. My rates for the last several years haven't been that good, but they've been pretty awesome.

Earlier this year, I had the opportunity to work 20 hours per week instead of 40. It was one of the greatest work-life experiences I've had to date. I was still able to pay all my bills, and I had time during each-and-every-day to do something fun. When working 40 hours per week, exercising and cooking dinner were somewhat of a chore. When I flipped to working less, work became the chore and exercise and cooking became the fun parts of my day. I read somewhere recently that if Americans valued health over wealth, we'd be a lot better off. I felt like I did this when working less and that I was rich in time.

Related to feeling better over making more, I've started to target employment opportunities that offer a good team to work with. For the last year, most of my contracts have been with remote clients, where they haven't required me to travel onsite. While this sounds great in theory, I do miss the comradery that exists when working with a team. Working with someone over a Skype/HipChat call is nothing like sitting next to each other and cracking jokes while writing code. Don't get me wrong, I love remote work, but I do think it's important to be onsite and collaborating face-to-face at least once per month.

To those individuals looking to start their own Solopreneurship, I hope this advice helps. It's been a great experience for me.

Testing AngularJS Applications

2015-02-02T10:11:56-07:00

This article is the second in a series about learning AngularJS. It describes how to test a simple AngularJS application. In a previous article, Getting Started with AngularJS, I showed how to develop a simple search and edit feature.

What you'll learn

You'll learn to use Jasmine for unit testing controllers and Protractor for integration testing. Angular's documentation has a good developer's guide to unit testing if you'd like more information on testing and why it's important.

The best reason for writing tests is to automate your testing. Without tests, you'll likely be testing manually. This manual testing will take longer and longer as your codebase grows.

What you'll need

About 15-30 minutes
A favorite text editor or IDE. We recommend IntelliJ IDEA.
Git installed.
Node.js and NPM installed.

Get the tutorial project

Clone the angular-tutorial repository using git and install the dependencies.

git clone https://github.com/mraible/angular-tutorial.git
cd angular-tutorial
npm install

If you haven't completed the Getting Started with AngularJS tutorial, you should peruse it so you understand how this application works. You can also simply start the app with "npm start" and view it in your browser at http://localhost:8000/app/.

Test the SearchController

Create app/search/search_test.js and populate it with the basic test infrastructure. This code sets up a mock SearchService that has the first function we want to test: query(term). It uses $provide to override the default SearchService. Angular unit-test controllers - mocking service inside controller was a useful question on Stack Overflow for figuring out how to mock services in controllers.
```
'use strict';

describe('myApp.search module', function() {
    var mockSearchService;

    beforeEach(module('myApp.search', function($provide) {
        mockSearchService = {query: function(term) {}};
        $provide.value("SearchService", mockSearchService);
    }));
});
```

Modify karma.conf.js (in the root directory) to add the search implementation and test.

files : [
  ...
  'app/components/**/*.js',
  'app/search/search.js',
  'app/search/search_test.js',
  'app/view*/**/*.js'
],

Add the first test to search_test.js. This test verifies that setting a search term and executing the search() function will call the service and return results. You can see the results returned from the service are mocked with deferred.resolve(). The deferred.resolve() call is how to handle promises in unit tests.

describe('search by term', function() {
    var scope, rootScope, controller, deferred;

	// setup the controller with dependencies.
    beforeEach(inject(function($rootScope, $controller, $q) {
        rootScope = $rootScope;
        scope = $rootScope.$new();
        controller = $controller('SearchController', {$scope: scope, SearchService: mockSearchService });
        deferred = $q.defer();
    }));

    it('should search when a term is set and search() is called', function() {
        spyOn(mockSearchService, 'query').andReturn(deferred.promise);
        scope.term = 'M';
        scope.search();
        deferred.resolve({data: {name: "Peyton Manning"}});
        rootScope.$apply();
        expect(scope.searchResults).toEqual({name: "Peyton Manning"});
    });
});

Related: Introduction to Unit Test: Spies is a good introduction to using spies in unit tests.

Run the following command on the command line to start the Karma test runner. You can leave this process running and new tests will be run automatically. You can change the mocked data and expectation to see your test fail.
```
npm test
```
Running Tests from IntelliJ IDEA
See Running Unit Tests on Karma to learn how to run your tests from IntelliJ IDEA.

Add a test to verify a search occurs automatically when the term is in the URL. Notice how the code structure had to change a bit to handle $routeParams.

describe('search by term automatically', function() {
    var scope, rootScope, controller, location, deferred;

    beforeEach(inject(function($rootScope, $controller, $q) {
        rootScope = $rootScope;
        scope = $rootScope.$new();

        // in this case, expectations need to be setup before controller is initialized
        var routeParams = {"term": "peyton"};
        deferred = $q.defer();
        spyOn(mockSearchService, 'query').andReturn(deferred.promise);
        deferred.resolve({data: {name: "Peyton Manning"}});

        controller = $controller('SearchController', {$scope: scope, $routeParams: routeParams, SearchService: mockSearchService });
    }));

    it('should search automatically when a term is on the URL', function() {
        rootScope.$apply();
        expect(scope.searchResults).toEqual({name: "Peyton Manning"});
    });
});

Add a test to verify the EditController works as expected.

describe('edit person', function() {
    var scope, rootScope, controller, location, deferred;

    beforeEach(inject(function($rootScope, $controller, $q) {
        rootScope = $rootScope;
        scope = $rootScope.$new();

        // expectations need to be setup before controller is initialized
        var routeParams = {"id": "1"};
        deferred = $q.defer();
        spyOn(mockSearchService, 'fetch').andReturn(deferred.promise);
        deferred.resolve({data: {name: "Peyton Manning", address: {street: "12345 Blake Street", city: "Denver"}}});

        controller = $controller('EditController', {$scope: scope, $routeParams: routeParams, SearchService: mockSearchService });
    }));

    it('should fetch a single record', function() {
        rootScope.$apply();
        expect(scope.person.name).toBe("Peyton Manning");
        expect(scope.person.address.street).toBe("12345 Blake Street");
    });
});

After adding this test, you'll likely see the following error in your terminal.

Chrome 40.0.2214 (Mac OS X 10.10.2) myApp.search module edit person should fetch a single record FAILED
	fetch() method does not exist
	TypeError: Cannot read property 'name' of undefined

This happens because the mockSearchService does not have a fetch method defined. Modify the beforeEach() on line 7 to add this function.

mockSearchService = {query: function(term) {}, fetch: function(id) {}};

Extra Credit

Create a test for saving a person. Here's a question on Stack Overflow that might help you verify the location after a save has been performed.

Test the Search Feature

To test if the application works end-to-end, you can write scenarios with Protractor. These are also known as integration tests, as they test the integration between all layers of your application.

To verify end-to-end tests work in the project before you begin, run the following command in one terminal window:

npm start

Then in another window, run the following to execute the tests:

npm run protractor

Write your first integration test to verify you can navigate to /search and enter a search term to see results. Add the following to e2e-tests/scenarios.js:

describe('search', function() {
  var searchTerm = element(by.model('term'));
  var searchButton = element(by.id('search'));

  beforeEach(function() {
    browser.get('index.html#/search');
  });

  it('should allow searching at /search', function() {
    searchTerm.sendKeys("M");
    searchButton.click().then(function() {
      expect(element.all(by.repeater('person in searchResults')).count()).toEqual(3);
    });
  });
});

The "searchTerm" variable represents the input field. The by.model() syntax binds to the "ng-model" attribute you defined in the HTML.

Run "npm run protractor". This should fail with the following error.

[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://172.16.6.39:64230/wd/hub
...F
Failures:
  1) my app search should allow searching at /search
   Message:
     NoSuchElementError: No element found using locator: By.id("search")

To fix, you need to add an "id" attribute to the Search button in app/search/index.html.

<form ng-submit="search()">
    <input type="search" name="search" ng-model="term">
    <button id="search">Search</button>
</form>

Run the tests again using "npm run protractor". They should all pass this time.

Write another test to verify editing a user displays their information.

describe('edit person', function() {
  var name = element(by.model('person.name'));
  var street = element(by.model('person.address.street'));
  var city = element(by.model('person.address.city'));

  beforeEach(function() {
    browser.get('index.html#/edit/1');
  });

  it('should allow viewing a person', function() {
    // getText() doesn't work with input elements, see the following for more information:
    // https://github.com/angular/protractor/blob/master/docs/faq.md#the-result-of-gettext-from-an-input-element-is-always-empty
    expect(name.getAttribute('value')).toEqual("Peyton Manning");
    expect(street.getAttribute('value')).toEqual("1234 Main Street");
    expect(city.getAttribute('value')).toEqual("Greenwood Village");
  });
});

Verify it works with "npm run protractor".

Finally, write a test to verify you can save a person and their details are updated. Figuring out how to verify the URL after it changed was assisted by this issue.

describe('save person', function() {
  var name = element(by.model('person.name'));
  var save = element(by.id('save'));

  beforeEach(function() {
    browser.get('index.html#/edit/1');
  });

  it('should allow updating a name', function() {
    name.sendKeys(" Updated");
    save.click().then(function() {
      // verify url set back to search results
      browser.driver.wait(function() {
        return browser.driver.getCurrentUrl().then(function(url) {
          expect(url).toContain('/search/Peyton%20Manning%20Updated');
          return url;
        });
      });
      // verify one element matched this change
      expect(element.all(by.repeater('person in searchResults')).count()).toEqual(1);
    });
  });
});

When you run this test with "npm run protractor", it should fail because there's no element with id="save" in app/search/edit.html. Add it to the Save button in this file and try again. You should see something similar to the following:
```
Finished in 4.478 seconds
6 tests, 9 assertions, 0 failures
```

Source Code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular-tutorial on the testing branch.

There are two commits that make the changes for the two main steps in this tutorial:

Summary

I hope you've enjoyed this quick-and-easy tutorial on testing AngularJS applications. The first couple AngularJS applications I developed didn't have tests and required a lot of manual testing to verify their quality. After learning that testing AngularJS apps is pretty easy, I now do it on all my projects. Hopefully this tutorial motivates you to do do the same.

Getting Started with AngularJS

2015-01-29T11:12:38-07:00

I was hired by my current client in November to help them choose a technology stack for developing modern web applications. In our first sprint, we decided to look at JavaScript MVC frameworks. I suggested AngularJS, Ember.js and React. Since most of the team was new to JavaScript MVC, I decided to create a tutorial for them. I tried to make it easy so they could learn how to write a simple web application with AngularJS. I thought others could benefit from this article as well, so I asked (and received) permission from my client to publish it here.

What you'll build

You'll build a simple web application with AngularJS. You'll also add search and edit features with mock data.

What you'll need

About 15-30 minutes
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and NPM installed.

Create a simple web application

Clone the angular-seed repository using git:

git clone https://github.com/angular/angular-seed.git angular-tutorial
cd angular-tutorial

There are two kinds of dependencies in this project: tools and angular framework code. The tools help manage and test the application.
- To get the tools that depend upon via npm, the node package manager.
- To get the angular code via bower, a client-side code package manager.
The project has preconfigured npm to automatically run bower so you can simply do:
```
npm install
```

Run the application

The project is configured with a simple development web server. The simplest way to start this server is:

npm start

Now browse to the app at http://localhost:8000/app/.

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the directory you cloned angular-seed to.

The Basics

Create an app/search/index.html file with the following HTML:

<form ng-submit="search()">
    <input type="search" name="search" ng-model="term">
    <button>Search</button>
</form>

Create app/search/search.js and define the routes (URLs) and controller for the search feature.

angular.module('myApp.search', ['ngRoute'])

    .config(['$routeProvider', function ($routeProvider) {
        $routeProvider
            .when('/search', {
                templateUrl: 'search/index.html',
                controller: 'SearchController'
            })
    }])

    .controller('SearchController', function () {
        console.log("In Search Controller...");
    });

Modify app/app.js and add the "myApp.search" module you created above.

angular.module('myApp', [
  'ngRoute',
  'myApp.view1',
  'myApp.view2',
  'myApp.version',
  'myApp.search'
])

Modify app/index.html and add a link to the search.js file.

<script src="view2/view2.js"></script>
<script src="search/search.js"></script>
<script src="components/version/version.js"></script>

Refresh your browser and navigate to http://localhost:8000/app/#/search. You should see an input field and search button. You should also see a log message printed in your browser's console. In Chrome, you can view the console using View > Developer > JavaScript Console. You can make it easier to navigate to this page by adding a menu item in app/index.html.
```
<ul class="menu">
  <li><a href="#/view1">view1</a></li>
  <li><a href="#/view2">view2</a></li>
  <li><a href="#/search">search</a></li>
</ul>
```

This section has shown you how to add a new controller and view to a basic AngularJS application. This was fairly simple to create. The next section shows you how to create a fake backend API.

The Backend

To get search results, you're going to create a SearchService that makes HTTP requests. These HTTP requests will be handled by a mock backend using some of Angular's built-in mocking tools. The backend implementation was created using AngularJS Backend-less Development Using a $httpBackend Mock.

Add a SearchService to app/search/search.js. This is done in Angular using its Factory Recipe. Make sure to remove the semicolon from the .controller code block.

.controller('SearchController', function () {
    console.log("In Search Controller...");
})

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        }
    };
    return service;
});

Inject the SearchService into the SearchController and use it to get results from the backend. The form in app/search/index.html calls the search() function. The "term" is defined by the input field in this page with ng-model="term".

.controller('SearchController', function ($scope, SearchService) {
    $scope.search = function () {
        console.log("Search term is: " + $scope.term);
        SearchService.query($scope.term).then(function (response) {
            $scope.searchResults = response.data;
        });
    };
})

If you try to search for a "foo" term now, you'll see the following error in your console.

Search term is: foo
GET http://localhost:8000/search/foo 404 (Not Found)

Create app/search/mock-api.js for the fake backend. Populate it with the following JavaScript.

// We will be using backend-less development
// $http uses $httpBackend to make its calls to the server
// $resource uses $http, so it uses $httpBackend too
// We will mock $httpBackend, capturing routes and returning data
angular.module('myApp')
    .service('ServerDataModel', function ServerDataModel() {
        this.data = [
            {
                id: 1,
                name: "Peyton Manning",
                phone: "(303) 567-8910",
                address: {
                    street: "1234 Main Street",
                    city: "Greenwood Village",
                    state: "CO",
                    zip: "80111"
                }
            },
            {
                id: 2,
                name: "Demaryius Thomas",
                phone: "(720) 213-9876",
                address: {
                    street: "5555 Marion Street",
                    city: "Denver",
                    state: "CO",
                    zip: "80202"
                }
            },
            {
                id: 3,
                name: "Von Miller",
                phone: "(917) 323-2333",
                address: {
                    street: "14 Mountain Way",
                    city: "Vail",
                    state: "CO",
                    zip: "81657"
                }
            }
        ];

        this.getData = function () {
            return this.data;
        };

        this.search = function (term) {
            if (term == "" || term == "*") {
                return this.getData();
            }
            // find the name that matches the term
            var list = $.grep(this.getData(), function (element, index) {
                term = term.toLowerCase();
                return (element.name.toLowerCase().match(term));
            });

            if (list.length === 0) {
                return [];
            } else {
                return list;
            }
        };
    })
    .run(function ($httpBackend, ServerDataModel) {

        $httpBackend.whenGET(/\/search\/\w+/).respond(function (method, url, data) {
            // parse the matching URL to pull out the term (/search/:term)
            var term = url.split('/')[2];

            var results = ServerDataModel.search(term);

            return [200, results, {Location: '/search/' + term}];
        });

        $httpBackend.whenGET(/search\/index.html/).passThrough();
        $httpBackend.whenGET(/view/).passThrough();
    });

This file uses jQuery.grep(), so you'll need to install jQuery. Modify bower.json and add jQuery to the list of dependencies.
```
"dependencies": {
  ...
  "html5-boilerplate": "~4.3.0",
  "jquery": "~1.10.x"
}
```
Stop the npm process, run "npm install" to download and install jQuery, then "npm start" again.

Modify app/app.js and add a dependency on ngMockE2E.

angular.module('myApp', ['ngMockE2E',
  'ngRoute',
  ...

Modify app/index.html to include references to jquery.js, angular-mocks.js and mock-api.js.

<script src="bower_components/angular-route/angular-route.js"></script>
<script src="bower_components/angular-mocks/angular-mocks.js"></script>
<script src="bower_components/jquery/jquery.js"></script>
...
<script src="search/mock-api.js"></script>
<script src="components/version/version.js"></script>

Add the following HTML to app/search/index.html to display the search results.
```
<div>
    <pre>{{ searchResults | json}}</pre>
</div>
```
After making this change, you should be able to search for "p", "d" or "v" and see results as JSON.

To make the results more readable, change the above HTML to use a <table> and Angular's ng-repeat directive.

<table ng-show="searchResults.length" style="width: 100%">
    <thead>
    <tr>
        <th>Name</th>
        <th>Phone</th>
        <th>Address</th>
    </tr>
    </thead>
    <tbody>
    <tr ng-repeat="person in searchResults">
        <td>{{person.name}}</td>
        <td>{{person.phone}}</td>
        <td>{{person.address.street}}<br/>
            {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
        </td>
    </tr>
    </tbody>
</table>

This section has shown you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an Edit Feature

Modify app/search/index.html to add a link for editing a person.

<table ng-show="searchResults.length" style="width: 100%">
    ...
    <tr ng-repeat="person in searchResults">
        <td><a href="" ng-click="edit(person)">{{person.name}}</a></td>
        ...
    </tr>
</table>

Add an edit() function to SearchController. Notice that the $location service dependency has been added to the controller's initialization function.

.controller('SearchController', function ($scope, $location, SearchService) {
    ...

    $scope.edit = function (person) {
        $location.path("/edit/" + person.id);
    }
})

Create a route to handle the edit URL in app/search/search.js.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/edit/:id', {
            templateUrl: 'search/edit.html',
            controller: 'EditController'
        });
}])

Create app/search/edit.html to display an editable form. The HTML below shows how you can use HTML5's data attributes to have valid attributes instead of ng-*.

<form ng-submit="save()">
    <div>
        <label for="name">Name:</label>
        <input type="text" data-ng-model="person.name" id="name">
    </div>
    <div>
        <label for="phone">Phone:</label>
        <input type="text" data-ng-model="person.phone" id="phone">
    </div>
    <fieldset>
        <legend>Address:</legend>
        <address style="margin-left: 50px">
            <input type="text" data-ng-model="person.address.street"><br/>
            <input type="text" data-ng-model="person.address.city">,
            <input type="text" data-ng-model="person.address.state" size="2">
            <input type="text" data-ng-model="person.address.zip" size="5">
        </address>
    </fieldset>
    <div>
        <button type="submit">Save</button>
    </div>
</form>

Modify SearchService to contain functions for finding a person by their id, and saving them.

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        },
        fetch: function (id) {
            return $http.get('/edit/' + id);
        },
        save: function(data) {
            return $http.post('/edit/' + data.id, data);
        }
    };
    return service;
});

Create EditController in app/search/search.js. $routeParams is an Angular service that allows you to grab parameters from a URL.

.controller('EditController', function ($scope, $location, $routeParams, SearchService) {
    SearchService.fetch($routeParams.id).then(function (response) {
        $scope.person = response.data;
    });

    $scope.save = function() {
        SearchService.save($scope.person).then(function(response) {
            $location.path("/search/" + $scope.person.name);
        });
    }
})

At the bottom of app/search/mock-api.js, in its run() function, add the following:

$httpBackend.whenGET(/\/search/).respond(function (method, url, data) {
    var results = ServerDataModel.search("");

    return [200, results];
});

$httpBackend.whenGET(/\/edit\/\d+/).respond(function (method, url, data) {
    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var results = ServerDataModel.find(id);

    return [200, results, {Location: '/edit/' + id}];
});

$httpBackend.whenPOST(/\/edit\/\d+/).respond(function(method, url, data) {
    var params = angular.fromJson(data);

    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var person = ServerDataModel.update(id, params);

    return [201, person, { Location: '/edit/' + id }];
});

$httpBackend.whenGET(/search\/edit.html/).passThrough();

Further up in the same file, add find() and update() methods to ServerDataModel.

this.getData = function () {
    return this.data;
};

this.search = function (term) {
    ...
};

this.find = function (id) {
    // find the game that matches that id
    var list = $.grep(this.getData(), function (element, index) {
        return (element.id == id);
    });
    if (list.length === 0) {
        return {};
    }
    // even if list contains multiple items, just return first one
    return list[0];
};

this.update = function (id, dataItem) {
    // find the game that matches that id
    var people = this.getData();
    var match = null;
    for (var i = 0; i < people.length; i++) {
        if (people[i].id == id) {
            match = people[i];
            break;
        }
    }
    if (!angular.isObject(match)) {
        return {};
    }
    angular.extend(match, dataItem);
    return match;
};

The <form> in app/search/edit.html calls a save() function to update a person's data. You already implemented this above. The function executes the logic below.

$location.path("/search/" + $scope.person.name);

Since the SearchController doesn't execute a search automatically when you execute this URL, add the following logic to do so in app/search/search.js. Note that $routeParams is added to the list of injected dependencies.

.controller('SearchController', function ($scope, $location, $routeParams, SearchService) {
    if ($routeParams.term) {
        SearchService.query($routeParams.term).then(function (response) {
            $scope.term = $routeParams.term;
            $scope.searchResults = response.data;
        });
    }
    ...
})

You'll also need to add a new route so search/term is recognized.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/search/:term', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        ...
}])

After making all these changes, you should be able to refresh your browser and search/edit/update a person's information. If it works - nice job!

Source Code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular-tutorial.

There are three commits that make the changes for the three main steps in this tutorial:

Extra Credit

Deploy your completed app to Heroku. See running version of this tutorial at https://angular-123.herokuapp.com.

Summary

I hope you've enjoyed this quick-and-easy tutorial on how to get started with AngularJS. In a future tutorial, I'll show you how to write unit tests and integration tests for this application. If you're in Denver next Tuesday, I'll be speaking about AngularJS at Denver's Open Source Users Group.

If you're a Java developer that's interested in developing with AngularJS and Spring Boot, you might want to checkout the recently released JHipster 2.0.

Devoxx4Kids - Denver: Introduction to Hardware Concepts with littleBits

2014-10-30T08:17:21-06:00

I'm pleased to announce the second meeting of the Denver Chapter of Devoxx4Kids is now open for registration. It's a two hour class titled Introduction to Hardware Concepts with littleBits and will be taught by Denver's own Tack Mobile. To learn more about littleBits, see http://littlebits.cc. If you or your company would like help by donating a Workshop Set, please contact me.

The class will be held on Saturday, November 22nd, from 10am - 12pm at Assembly Workspace. Cost is $10, but you'll get that back in the form of a t-shirt. Age requirement is 9-18 and kids should have basic computer skills (copy/paste, opening applications, etc.).

I'd like to thank Juan Sanchez for reaching out to me about this class and inspiring his company (and workspace) to make it all happen. It's been great working with you and your team Juan!

When I started Devoxx4Kids Denver, I was hoping to host a class or two per year. Our first meetup in May was a wild success. After taking the summer off to relax, I started looking for more speakers in early October. The response has been great and we'll have another class about GreenFoot on December 13th. We're even in the planning stages for another session on NAO Humanoid Robot programming in Q1 2015.

If you'd like to get involved with Denver's Devoxx4Kids, please join our meetup group.

This site now powered by Java 8, Tomcat 7 and Wufoo Forms

2014-04-09T07:37:59-06:00

I recently upgraded this site to use the latest version of Apache Roller. It was a minor release (5.0.3), but I figured I'd document the steps in case "early onset" comes soon. First of all, to download raibledesigns.com and get it running locally, I perform the following steps:

Backup everything using ~/bin/backup.sh on raibledesigns.com
scp backup file to local hard drive and expand
Copy ROOT, skins and repository directories to local webapps
Make sure activation, mail, mysql and jta JARs are in $CATALINA_HOME/lib
Copy roller-custom.properties from raibledesigns.com's $CATALINA_HOME/lib
Copy context files from hosted $CATALINA_HOME/conf/Catalina to local directory
Import database and change roller-custom.properties to match local credentials

Next, to upgrade to the latest Roller release, I do the following:

Download latest Roller release and expand
Copy JARs (from WEB-INF/lib) to existing install (to upgrade dependencies)
Delete any lower-versioned JARS from WEB-INF/lib directory
Copy JSPs (from WEB-INF/jsps) to existing install
Run database migration scripts from WEB-INF/classes/dbscripts
Use a diff tool (like SmartSynchronize) to compare new vs. existing
Test and troubleshoot (if there's startup errors)

This process has worked well for the last 10 years, and it's been in my head the whole time. It's bound to escape someday.

Contact Form Enhancements
In addition to upgrading Roller, I also upgraded Tomcat 6 to Tomcat 7.0.52. In doing so, I found Jakarta's Mailer Taglib doesn't work with Tomcat 7. As you can tell from the aforementioned thread, I've known this for several years. That's the only thing that's stopped me from upgrading Tomcat the past couple years.

Rather than using my own code to send email, I figured I'd explore my options. I looked into Kontactr, EmailMeForm and Wufoo. The first two allowed me to customize their colors, but Wufoo was the only one that had a transparent background feature. Because Wufoo form's are transparent, and I'd copied their CSS for AppFuse's forms in the past, I decided to use it. To make it work with both of this site's themes (light and dark), I did need to create two forms on Wufoo and do some local JavaScript modifications. You can see the results on my contact page (view-source to see the JavaScript). If you switch the theme (by clicking the little rectangles in the top-right corner), you can see how the form is redrawn with the proper color scheme.

If I'd found it sooner, I likely would've used Brace Forms for my contact form.

Theme Improvements
Ever since I switched to a new look and feel, there's been improvements I wanted to make.

The stylesheet switching doesn't happen as fast as I'd hoped (there's a flash even if using cookies), so I'll likely be converting some theme-setting logic to the server-side. The HTML5 version of the FaceBook Like Button requires you to specify the "data-colorscheme" in markup so this further supports moving to the server.

To fix this issue, I tried creating a cookie-reader plugin for Roller. This worked, but not as well as I'd hoped. The theme cookie was read the first time the page loaded, but then cached for subsequent requests. If a user changed their theme, the new value would not be read by the cached page. My solution now is putting the JavaScript set-default-theme logic right after the stylesheet to set it before the body loads.

Powered by Java 8
To make this site run on Java 8, I had to move to a new server. Keith did a bunch of work on my old server to get Java 8 working, but found Linux JDK 7 code has dependancies on the 2.6 kernel. If you're reading this post, the DNS changes have propagated. This means I have my first Java 8 app in production!

Secured by SSL
The last change I made was to start using my SSL certificate on this site. You can read about the challenges I encountered on the Roller mailing list. The final solution involved changing my Absolute URL to use https and then changing my theme to use the following for its base.

<base href="$absBaseURL.replace('https:', '')" />

I also had to change a number of iframes to use src="//..." instead of src="//...". To force HTTPS for the admin pages, Keith configured Apache to do the redirect. Using Roller to redirect resulted in infinite loop errors.

This means you can now access this site using https://raibledesigns.com or http://raibledesigns.com, whichever you prefer. Speaking of SSL, you might wonder if this new server is affected by The Heartbleed Bug. According to Keith, it is not.

This server is still running openSSL 1.0.0 which was shipped with Centos 6.1, and the Heartbleed vulnerability appeared in OpenSSL 1.0.1 which shipped Centos 6.5.

Summary
This site has undergone quite a few updates this month: Roller 5.0.3, Tomcat 7.0.52, a Wufoo contact form (+ recent tweets on the same page), Java 8 and optional SSL. If you see any issues, please let me know.

May 8, 2014 Update: I decided to revert to "http" for the absolute URL instead of "https". However, I've updated all "src" attributes on <iframe> and <img> tags to use schema-less URLs (// instead of http://). This means that both secure and non-secure URLs should work.

Devoxx4Kids - Denver Chapter Begins!

2014-04-08T10:59:29-06:00

I'm happy to announce the first meeting of the Denver Chapter of Devoxx4Kids is now open for registration. It's a two hour class titled Introduction to Server-Side Minecraft Programming and will be taught by Denver's own Scott Davis. The class will be held on May 3rd, from 10am - 12pm at Thrive's Cherry Creek location, where we can comfortably fit 20 students. Cost is $10, but you'll get that back in the form of a t-shirt. Age requirement is 9-18 and kids should have basic computer skills (copy/paste, opening applications, etc.). Minecraft experience will certainly help too.

I'd like to thank Daniel De Luca for his initial assistance with getting Devoxx4Kids setup in Denver and Arun Gupta for starting Devoxx4Kids USA. Arun has been a great help in getting things going and answering my questions over the last few months. Of course, none of it would be happening without Scott Davis or Thrive. If you join us on May 3rd, you'll see that Scott is an awesome teacher and Thrive has some incredible facilities.

My initial goal with Devoxx4Kids in Denver is to host a class or two this year. If there's enough demand, we can expand. For now, we're starting small and seeing where it takes us. If you're interested in teaching a future class, please let me know. We'd love to teach the kids a number of skills, from Scratch to NAO Humanoid Robot programming to building things with Arduino and Raspberry Pi.

You shouldn't have to worry about front end optimization

2014-01-16T13:49:03-07:00

After writing yesterday's article on optimizing AngularJS apps with Grunt I received an interesting reply from @markj9 on Twitter.

@mraible that might be a bad thing! you should go listen to Igor.. http://t.co/HL2mho7R86
— Mark (@markj9) January 15, 2014

I clicked on the provided link, listened to the podcast (RR HTTP 2.0 with Ilya Grigorik) and discovered some juicy bits around 27:00. The text below is from the podcast's transcript at the bottom of the page.

AVDI: Yeah. If I pushed back a little, it’s only because I just wonder, are the improvements coming solely from the perspective of a Google or Facebook, or are they also coming from the perspective of the hundreds of thousands of people developing smaller applications and websites?

ILYA: Yeah. So, I think it’s the latter, which is to say the primary objective here is actually to make the browsers faster. So, if you open a webpage over HTTP 2.0, it should load faster. And that’s one kind of population that will benefit from it. And the second one is actually developers. We want to make developing web applications easier. You shouldn’t have to worry about things like spriting and concatenating files and doing all this stuff, domain sharding and all this other mess, which are just completely unnecessary and actually makes performance worse in many cases because each one of those has negative repercussions.

Things like, let’s say concatenating your style sheets or JavaScript. Why do we do that? Well, we do that because we want to reduce the number of requests because we have this connection limit with HTTP 1.0. But the downside then is let’s say you’ve — actually Rails does this, you concatenate all of your CSS into one giant bundle. Great, we reduced the number of requests. We can download it faster. Awesome. Then you go in and your designer changes one line from whatever, the background color from blue to red. And now, you have to download the entire bundle. You have to invalidate that certain file and you need to download the whole thing.

Chances are, if you’re doing sound software development today, you already have things split into modules. Like here is my base.css, here is my other page.css. Here are my JavaScript modules. And there’s no reason why we need to concatenate those into one giant bundle and invalidate on every request. This is something that we’ve automated to some degree, but it’s unnecessary. And it actually slows down the browser, too, in unexpected ways.

We recently realized that serving these large JavaScript files actually hurts your performance because we can’t execute the JavaScript until we get the entire file. So, if you actually split it into a bunch of smaller chunks, it actually allows us to execute them incrementally, one chunk at a time. And that makes the site faster. Same thing for CSS, splitting all that stuff. And this may sound trivial, but in practice, it’s actually a giant pain for a lot of applications.

The conversation goes on to talk about how this change in thinking is largely caused by the fact that bandwidth is no longer a problem, latency is.

JAMES: That seems really, really weird to me though. Everything has been moving in that direction and you’re saying our data on that’s just wrong. It’s not faster?

ILYA: Yeah. Part of it is the connectivity profiles are also changing. So when we first started advocating for those sorts of changes back in, whatever it was, 2005, 2007, when this stuff started showing up, the connection speeds were different. We were primarily maybe DSL was state of the art and bandwidth was really an issue there. So, you spend more time just downloading resources. Now that bandwidth is much less of an issue, latency is the problem. And because of that, these “best practices” are changing. And with HTTP 2.0, you actually don’t have to do that at all. And in fact, some of those things will actually hurt your performance.

As you can imagine, this news is quite surprising to me. Optimizations like gzipping and expires headers will continue to be important, but concatenating and minifying might become a "worst" practice? That seems crazy, especially when the tools I test with (YSlow and Page Speed browser plugins) give me higher grades for minifying and reducing the number of requests.

The good news is there's lots of good things coming in HTTP 2.0, and you can use it today.

ILYA: ... any application that’s delivered over HTTP 1.0 will work over HTTP 2.0. There’s nothing changing there. The semantics are all the same. It could be the case that certain optimizations that you’ve done for HTTP 1.1 will actually hurt in HTTP 2.0. And when I say hurt, in practice at least from what I’ve seen today, it doesn’t mean that your site is actually going to be slower. It’s just that it won’t be any better than HTTP 1.0.

Upgrading my servers to support HTTP 2.0 brings up an interesting dilemma. How do I measure that non-minified and non-concatenated assets are actually better? Do I just benchmark page load times or are there better tools for proving things are faster and better?

A Webapp Makeover with Spring 4 and Spring Boot

2013-12-11T12:47:15-07:00

A typical Maven and Spring web application has a fair amount of XML and verbosity to it. Add in Jersey and Spring Security and you can have hundreds of lines of XML before you even start to write your Java code. As part of a recent project, I was tasked with upgrading a webapp like this to use Spring 4 and Spring Boot. I also figured I'd try to minimize the XML.

This is my story on how I upgraded to Spring 4, Jersey 2, Java 8 and Spring Boot 0.5.0 M6.

When I started, the app was using Spring 3.2.5, Spring Security 3.1.4 and Jersey 1.18. The pom.xml had four Jersey dependencies, three Spring dependencies and three Spring Security dependencies, along with a number of exclusions for "jersey-spring".

Upgrading to Spring 4
Upgrading to Spring 4 was easy, I changed the version property to 4.0.0.RC2 and added the new Spring bill of materials to my pom.xml. I also add the Spring milestone repo since Spring 4 won't be released to Maven central until tomorrow.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-framework-bom</artifactId>
            <version>${spring.framework.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<repositories>
    <repository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Next, I removed all the references to ${spring.framework.version} in dependencies since it'd be controlled by Maven's dependency management feature.

     <dependency>
         <groupId>org.springframework</groupId>
         <artifactId>spring-web</artifactId>
-        <version>${spring.framework.version}</version>
     </dependency>

I also changed to use Maven 3's wildcard syntax to exclude multiple dependencies.

    <dependency>
        <groupId>com.sun.jersey.contribs</groupId>
        <artifactId>jersey-spring</artifactId>
        <exclusions>
             <exclusion>
                 <groupId>org.springframework</groupId>
-                    <artifactId>spring</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-core</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-web</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-beans</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-context</artifactId>
+                    <artifactId>*</artifactId>
             </exclusion>
         </exclusions>
     </dependency>

I confirmed the upgrade worked by running "mvn dependency:tree | grep spring", followed by "mvn jetty:run" and viewing the app in my browser.

Upgrading to Jersey 2
The next item I tackled was upgrading to Jersey 2.4.1. I changed the version number in my pom.xml, then added the Jersey BOM.

<dependency>
    <groupId>org.glassfish.jersey</groupId>
    <artifactId>jersey-bom</artifactId>
    <version>${jersey.version}</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

You might ask "why Jersey?" if we already have Spring MVC and its REST support? You might also ask why not Play or Grails instead of a Java + Spring stack? For this particular project, I recommended technology options, and these were certainly among them. However, the team chose differently and I support their decision. The project is creating an iOS app, as well as a responsive HTML5 mobile/desktop app. We figured we had enough risk with new technologies on the front-end that we should play it a bit safer on the backend. To make the backend work a bit sexier, we've decided to allow Spring 4, Java 8 and possibly some reactive principles.

Next, I changed from the old com.sun.jersey dependencies to org.glassfish.jersey and removed jersey-spring.

<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-json-jackson</artifactId>
</dependency>

The last thing I needed to do was change the servlet-class and param-name in web.xml:

<servlet>
    <servlet-name>jersey-servlet</servlet-name>
    <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
    <init-param>
        <param-name>jersey.config.server.provider.packages</param-name>
        <param-value>com.raibledesigns.boot.service</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>

Requiring Java 8
Requiring Java 8 to compile was easy enough. I added the maven-compiler-plugin to enforce a minimum version.

<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>

I downloaded the latest Java 8 SDK and installed it. Then I set my JAVA_HOME to use it.

export JAVA_HOME=`/usr/libexec/java_home -v 1.8`

Integrating Spring Boot
I learned about Spring Boot a few weeks ago at Devoxx. Josh Long gave me a 3-minute demo at the speaker's dinner and showed me enough to pique my interest. To integrate it into my project, I started with the Quick Start. I added the boot-parent, dependencies for web, security and actuator (logging, metrics, etc.) and the Maven plugin. I removed all the Spring and Spring Security dependencies.

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>0.5.0.M6</version>
</parent>
...
<pluginRepositories>
    <pluginRepository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
    </pluginRepository>
</pluginRepositories>
...
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
...
<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
</plugin>

Upon restarting my app, I got an error about spring-security.xml using a 3.1 XSD. I fixed it by changing to 3.2. Next, I wanted to eliminate web.xml. First of all, I created an ApplicationInitializer so the WAR could be started from the command line.

package com.raibledesigns.boot.config;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class ApplicationInitializer extends SpringBootServletInitializer {

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(ApplicationInitializer.class);
    }

    public static void main(String[] args) {
        SpringApplication.run(ApplicationInitializer.class, args);
    }
}

However, after adding this, I received the following error on startup:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 
'org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor': 
Invocation of init method failed; nested exception is 
java.lang.AbstractMethodError: org.hibernate.validator.internal.engine.ConfigurationImpl
.getDefaultParameterNameProvider()Ljavax/validation/ParameterNameProvider;

Adding hibernate-validator as a dependency solved this problem:

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
</dependency>

To configure Spring Security without web.xml and spring-security.xml, I created WebSecurityConfig.java:

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@EnableWebSecurity
@Order(Ordered.LOWEST_PRECEDENCE - 6)
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .antMatchers("/", "/home").permitAll()
                .antMatchers("/v1.0/**").hasRole("USER")
                .anyRequest().authenticated();
        http.httpBasic().realmName("My API");
    }

    @Override
    protected void configure(AuthenticationManagerBuilder authManagerBuilder) throws Exception {
        authManagerBuilder.inMemoryAuthentication()
                .withUser("test").password("test123").roles("USER");
    }
}

To configure Jersey without web.xml, I created a JerseyConfig class:

package com.raibledesigns.boot.config;

import org.glassfish.jersey.filter.LoggingFilter;
import org.glassfish.jersey.jackson.JacksonFeature;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.ServerProperties;

import javax.ws.rs.ApplicationPath;

@ApplicationPath("/v1.0")
public class JerseyConfig extends ResourceConfig {

    public JerseyConfig() {
        packages("com.raibledesigns.boot.service");
        property(ServerProperties.BV_SEND_ERROR_IN_RESPONSE, true);
        property(ServerProperties.JSON_PROCESSING_FEATURE_DISABLE, false);
        property(ServerProperties.MOXY_JSON_FEATURE_DISABLE, true);
        property(ServerProperties.WADL_FEATURE_DISABLE, true);
        register(LoggingFilter.class);
        register(JacksonFeature.class);
    }
}

Finally, I created MvcConfig.java to set the welcome page.

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.ViewControllerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;

@Configuration
public class MvcConfig extends WebMvcConfigurerAdapter {

    @Override
    public void addViewControllers(ViewControllerRegistry registry) {
        registry.addViewController("/").setViewName("index");
    }
}

To cleanup, I deleted src/main/webapp/WEB-INF and created src/main/resources/logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>
    <logger name="org.springframework.boot" level="INFO"/>
    <logger name="org.springframework.security" level="ERROR"/>
</configuration>

Since Boot doesn't support JSP out-of-the-box, I renamed my index.jsp file to index.html and changed the URL in it to point to "/v1.0/hello". I was pleased to see that everything worked nicely. I learned shortly after that I could remove the Spring BOM since Spring Boot uses a <spring.version> property to control its Spring version.

The only issue I found is when started the app with "mvn package && java -jar target/app.war", it failed to initialize Jersey. I tried adding a @Bean for the servlet:

@Bean
public ServletRegistrationBean jerseyServlet() {
    ServletRegistrationBean registration = new ServletRegistrationBean(new ServletContainer(), "/v1.0/*");
    registration.addInitParameter(ServletProperties.JAXRS_APPLICATION_CLASS, JerseyConfig.class.getName());
    return registration;
}

Unfortunately, when running it using "java -jar", I get the following error:

org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. org.glassfish.jersey.server.internal.scanning.ResourceFinderException: 
java.io.FileNotFoundException: /.../target/app.war!/WEB-INF/classes (No such file or directory)
	at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:869)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:814)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:906)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:898)
	at org.glassfish.jersey.server.ApplicationHandler.createApplication(ApplicationHandler.java:300)
	at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:279)
	at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:302)

This seems strange since there is a WEB-INF/classes in my WAR. Regardless, this is not a Boot problem per se, but more of a Jersey issue. From one of the Boot developers:

The whole idea with Boot is that servlets are just a transport - they are a means to an end, and hopefully not the only one - the "container" is Spring, not the servlet container. We probably could add some form of support for SCI but only by hacking the containers since the spec really doesn't allow for much control of their lifecycle. It hasn't been a priority so far.

Summary
I hope this article is useful to see how you to upgrade your Java webapps to use Spring 4 and Spring Boot. I've created a boot-makeover project on GitHub with all the code mentioned. You can also view the commits for each step.

Developing with AngularJS - Part IV: Making it Pop

2013-09-12T10:54:29-06:00

Welcome to the final article in a series on my experience developing with AngularJS. I learned its concepts, beat my head against-the-wall with and finally tamed it enough to create a "My Dashboard" feature for a client. For previous articles, please see the following:

The last mile of development for the My Dashboard feature was to spice things up a bit and make it look better. We hired a design company to come up a new look and feel and they went to work. Within a week, we had a meeting with them and they presented a few different options. We picked the one we liked the best and went to work. Below are screenshots that I used to implement the new design.

At first, I thought implementing this design might take quite a bit of effort, since it looked like it used custom fonts. It's true we could use CSS3's @font-face, but I knew it might take awhile to find the right fonts with the appropriate licenses. When I received the screenshot below, I was pleased to see that all fonts were web-safe.

Design Elements

There are a number of elements in this new design that I had to create. For example, if numbers were only 1 digit, we had to add a leading zero to them in the summary band. Other design elements we needed to implement are listed below:

A background image that filled the page
Fade to white on summary widget titles
A responsive grid for summary widgets
Provide a colored background for odd rows in the summary grid
Add a "Show More" band at the bottom of Tasks, Summary and Reports when there's more items to display

In addition to these elements, there was quite a bit of work to conform to the new colors, fonts and drop-shadows. I implemented all of these using CSS3 (border-radius, box-shadow, box-sizing, linear-gradient), and lots of trial-and-error. To use the best fonts across various devices, I used CSS-Trick's Font Stacks.

New Background

The new background shown in the screenshots above has a light source in the middle of it. Therefore, it's impossible to tile/repeat it across the page since it's not uniform. To make it work, I used a 1024 x 768 image and CSS3's background-size: cover. For more information on background-size, see SitePoint's How to Resize Background Images with CSS3. This worked great on smaller screens, but we noticed some issues on 30" monitors. Therefore, we ended up getting a new repeatable background and stopped using background-size.

LeadingZero filter

For the first leading zero feature, I wrote an Angular filter. I put the code for this in filters.js:

filter('leadingZero', function() {
    return function(input) {
        if (input.length === 1) {
            return "0" + input;
        } else if (input.length > 2) {
            return "+99";
        } else {
            return input;
        }
    }
});

This filter is used in the HTML template as follows:

<div class="summary-value">{{widget.value | leadingZero}}</div>

Text Fade Out

To implement the fade-to-white text in summary titles, I started with this tutorial. I quickly discovered that it worked best for vertical text blocks and not for horizontal text. Then I found Text Ellipsis with Gradient Fade in Pure CSS, which uses :after to position a block over the text that fades to white. Since the title is not the right-most element (the numbers are), I had to figure out best positioning that worked cross-browser. Below is the CSS I used to implement this feature.

.dashboard .summary-title:after {
    display: block;
    position: absolute;
    right: 66px;
    top: 5px;
    bottom: 5px;
    width: 30px;
    background: -moz-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* FF3.6+ */
    background: -webkit-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* Chrome10+,Safari5.1+ */
    background: -o-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* Opera 11.10+ */
    background: -ms-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* IE10+ */
    background: linear-gradient(to right,  rgba(255,255,255,0) 0%, #fff 20px); /* W3C */
    filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#00ffffff', endColorstr='#ffffff',GradientType=1 ); /* IE6-9 */
    content: "";
}

Responsive Grid

To implement the responsive grid of summary widgets, I started with Codrops' Responsive Full Width Grid Tutorial. This proved to be a great model and I used the following CSS to position all the <li>'s appropriately. In the code below, .summary-item is the class on the <li> elements.

.dashboard .summary-item {
    border-right: 1px solid #d1d1d1;
    border-bottom: 1px solid #d1d1d1;
    /* put the border on the inside of the box */
    -webkit-box-sizing: border-box;
    -moz-box-sizing: border-box;
    -ms-box-sizing: border-box;
    box-sizing: border-box;
    font-family: Constantia, "Lucida Bright", Lucidabright, "Lucida Serif", Lucida, "DejaVu Serif", "Bitstream Vera Serif", "Liberation Serif", Georgia, serif;
    font-size: 14px;
    color: #666;
    height: 50px;
    box-shadow: inset 0 0 6px rgba(0,0,0, 0.25);
    /* responsive grid */
    position: relative;
    float: left;
    overflow: hidden;
    width: 25% /* Fallback */
    width: -webkit-calc(100% / 4);
    width: calc(100% / 4);
}

@media screen and (max-width: 1400px) {
    .dashboard .summary-item {
        width: 33.33333333333333%; /* Fallback */
        width: -webkit-calc(100% / 3);
        width: calc(100% / 3);
    }
}

@media screen and (max-width: 1000px) {
    .dashboard .summary-item {
        width: 50%; /* Fallback */
        width: -webkit-calc(100% / 2);
        width: calc(100% / 2);
    }
}

This worked great in most browsers, but we did find an issue with IE9. When squishing/expanding the browser window, sometimes there would be a blank column on the right side. To fix this, I changed the width on the default .summary-item to be 25%, and removed the lines with calc.

.dashboard .summary-item {
    ...
    width: 25%
}

Coloring Odd Rows

Coloring odd rows in a table is easy, but when the rows are in a responsive grid, that's a whole different story. For tables, the CSS rules are extremely simple:

tr:nth-child(even) {background: #CCC}
tr:nth-child(odd) {background: #FFF}

Via Twitter, @tomaslin advised me that the nth-child selector could probably be used for this, but it'd likely require some JavaScript to make it responsive. I found the excellent Master of the :nth-child and began trying to figure it out. The following function is what we now use to color odd rows in the Summary Bar.

function colorRows() {
    var lisInRow = 0;
    var items = $('.summary-items li');
    items.each(function() {
        if($(this).prev().length > 0) {
            if($(this).position().top != $(this).prev().position().top) return false;
            lisInRow++;
        }
        else {
            lisInRow++;
        }
    });
    var rows = items.length / lisInRow;
    for (var i = 0; i < rows; i++) {
        var selector = "nth-child(n+{x}):nth-child(-n+{y})";
        var x = (lisInRow * i) + 1;
        var y = x + (lisInRow - 1);
        selector = selector.replace('{x}', '' + x);
        selector = selector.replace('{y}', '' + y);
        if (i % 2) {
            $('.summary-items li:' + selector).addClass('odd');
        } else {
            $('.summary-items li:' + selector).removeClass('odd');
        }
    }
}

The above code is in dashboard.js and is called anytime the browser window is resized (to adapt to the responsive grid).

$(window).resize(colorRows);

It's also called when summary widgets are re-ordered, in the updateOrder() function of WidgetController.

$scope.updateOrder = function(event, ui) {
    ...
    Preferences.saveWidgetOrder(type, {items: items});
    if (type === 'summary') {
        colorRows();
    }
};

I'd like to figure out how to make this more Angular-esque, but all the "how to hook into window.resize" articles I found make it seem harder than this.

Show More

The last feature I had to implement was the "Show More" bar that appears when widgets are hidden. This was the most difficult thing to implement and I tried many different things before arriving at a solution that works. First of all, the widgets bars that can be expanded are put into their original (collapsed) state using max-height and overflow: hidden. From there, I look at the list inside the bar and compare the height's of the two elements. If the list is taller than the bar, the Show More bar is added.

I originally looked at the list's :last-child to see if it was visible, but jQuery's :hidden selector only works on items that are hidden by display: none rather than ones that are hidden by overflow.

As you can see from the code below, there's special logic needed to expand the min-height of the Summary Bar since it doesn't have enough room at the bottom to add the bar in its collapsed state.

function showMore(element) {
    var bar = element.parent().parent();
    var list = element.parent();
    var barId = bar.attr('id');
    var listHeight = list.height();
    var barHeight = bar.height();
    var isSummaryBar = (barId.indexOf('summary') > -1);
    var summaryBarMinHeight = 260;
    var showMoreShouldBeVisible = (isSummaryBar && element.position().top >= 200) ? true : listHeight > barHeight;
    if (showMoreShouldBeVisible) {
        var messages = {};
        // the variables below are defined in the host page, before this file is loaded
        messages.more = showMoreText;
        messages.less = showLessText;

        var showMore = $('<div class="show-more"/>').html(messages.more + " <b class='caret'></b>");
        showMore.appendTo(bar);
        // summary bar doesn't have enough room for the Show More bar in its collapsed state,
        // so change it from 242 to 260
        if (isSummaryBar) {
            bar.css({'min-height': summaryBarMinHeight + 'px', 'max-height': ''});
        }

        showMore.bind('click', function (e) {
            var element = $(this);
            var parent = element.parent();

            if (element.hasClass('less')) {
                parent.css({"max-height": ''});
                if (isSummaryBar) {
                    parent.css({"min-height": summaryBarMinHeight + 'px'}).animate(200);
                } else {
                    parent.css({"min-height": ''}).animate(200);
                }
                element.removeClass('less');
                element.html(messages.more + ' <b class="caret"></b>');
            } else {
                parent.css({
                    "max-height": 9999,
                    "min-height": 'auto'
                }).animate({
                        "min-height": parent.height() + 19
                    }, 200);

                element.addClass('less');
                element.html(messages.less + ' <b class="caret caret-up"></b>');
            }

            // prevent jump-down
            return false;
        });
    } else {
        // Remove show-more in case it was previously added
        if (bar.find('.show-more').length > 0) {
            if (isSummaryBar) {
                bar.css('min-height', summaryBarMinHeight - 18)
            } else {
                bar.attr('style', '');
            }

            bar.find('.show-more').remove();
        }
    }
}

function showMoreOnResize() {
    var dataItems = $('.task-items,.summary-items,.report-items');
    dataItems.each(function() {
        var lastItem = $(this).find('li:last-child');
        if (lastItem.length > 0) {
            showMore(lastItem);
        }
    });
}

At first, I wrote this logic as a directive, but when I needed it for responsiveness, I moved it into dashboard.js. The showMoreOnResize() function is called on window resize.

$(window).resize(showMoreOnResize);

I also found that I had to add it to the Preferences service after widgets were saved (since the number displayed could change).

factory('Preferences', function ($filter) {
    return {
        ...
        // Save hidden and visible (and order) widgets from config dialog
        saveWidgetPreferences: function (type, widgets) {
            ...
            DWRFacade.saveDashboardWidgetPreference(type, preferences, {
                callback: function() {
                    // recalculate show more bar
                    showMoreOnResize();
                },
                errorHandler: function (errorString) {
                    alert(errorString);
                }
            });
        }
    }
});

To implement the .caret-up (the .caret class is from Bootstrap), I found a caret-right howto and used it to create .caret-up:

.caret-up {
    border-left: 4px solid transparent;
    border-right: 4px solid transparent;
    border-top: 4px solid transparent;
    border-bottom: 4px solid black;
}

Summary

The final My Dashboard feature is something that I'm quite proud of. A fellow developer, Vlad, did an excellent job of implementing the backend and admin portions. The Product Team's vision and desire to make it Pop! created something great. The fact that we didn't have to support IE8 helped a lot in the implementation. Below is a screenshot of how My Dashboard looked when we completed the project.

Angular isn't mentioned much in this article. That's because we didn't have to do much to the existing Angular code to implement the new design. It was just a matter of writing/modifying some CSS as well as introducing some JavaScript for colored rows and show more. If you know how these features could be written in a more Angular Way, I'd love to hear about it.

If you'd still like to learn more about Angular and why it's good to integrate it little by little, I encourage you to read 5 reasons to use AngularJS in the corporate app world.

Upgrading Grails from 2.0 to 2.2

2013-03-22T09:16:37-06:00

In preparation for my Grails vs. Play Smackdown at Devoxx France next week, I recently upgraded my Grails version of Happy Trails from Grails 2.0.3 to Grails 2.2.1. I ran into a few issues along the way and figured I'd document them here to help others out.

Fixing the source
The first issue I ran into was Spock and Groovy 2 incompatibilities.

| Resolving plugin JAR dependencies
| Error WARNING: Dependencies cannot be resolved for plugin [mail] due to error: startup failed:
Could not instantiate global transform class org.spockframework.compiler.SpockTransform specified at jar:file:/Users/mraible/.grails/ivy-cache/org.spockframework/spock-core/jars/spock-core-0.7-groovy-1.8.jar!/META-INF/services/org.codehaus.groovy.transform.ASTTransformation  because of exception org.spockframework.util.IncompatibleGroovyVersionException: The Spock compiler plugin cannot execute because Spock 0.7.0-groovy-1.8 is not compatible with Groovy 2.0.7. For more information, see http://versioninfo.spockframework.org

I posted the problem to StackOverflow and got a response almost immediately. While this pull request helped me quite a bit, it was ultimately caused by my vision: I had two "geb-spock" dependencies listed in BuildConfig.groovy with different groupIds.

At this point, I also moved all my plugin dependencies from application.properties to BuildConfig.groovy.

The next problem I ran into was a unit test and functional tests failing. The unit testing issue was caused by my Direction model not being in the tests @Mock annotation. After I added it, validation kicked and I recognized my test was invalid. I added @Ignore and continued.

The functional test seemed to be seemed to be caused by Geb and it trying to use the Chrome Driver. One of my tests didn't work with the default HtmlUnitDriver, so I used the ChromeDriver for the one test.

| Running 11 spock tests... 6 of 11
| Failure:  signup as a new user(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.WebDriverException: Unable to either launch or connect to Chrome. Please check that ChromeDriver is up-to-date. Using Chrome binary at: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 45.66 seconds
Build info: version: '2.27.0', revision: '18259', time: '2012-12-05 11:30:53'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.8.2', java.version: '1.7.0_04'
Driver info: org.openqa.selenium.chrome.ChromeDriver
    at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:187)
    at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:145)
    at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:533)
    at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:216)
    at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:111)
    at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:115)
    at org.openqa.selenium.chrome.ChromeDriver.(ChromeDriver.java:161)
    at org.openqa.selenium.chrome.ChromeDriver.(ChromeDriver.java:107)
    at happytrails.AuthenticatedUserSpec.signup as a new user(AuthenticatedUserSpec.groovy:25)

Even when running "grails -Dgeb.env=chrome test-app", this still happened. This was caused by the fact that I had GebConfig.groovy in test/functional/happytrails. Move it to test/functional solved the problem. I also discovered that I know longer needed Chrome to get this test to pass. Apparently, the HtmlUnitDriver has issues with Grails 2.2, but it seems to work for me.

After getting the Geb configuration fixed, I ran into a functional test failure:

| Running 11 spock tests... 5 of 11
| Failure:  click signup link(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.ElementNotVisibleException: Element must be displayed to click (WARNING: The server did not provide any stacktrace information)

Even though I could see the "signup" link when I ran "grails run-app", I could see that it didn't show up when running tests in Chrome. This turned out to be caused by an extraneous <div class="nav-collapse"> I had in my main.gsp. Removing it solved the problem. It's strange that this never showed up with Grails 2.0. My only guess is that Geb someone didn't look at the visibility of the element.

The last testing-related issue I ran into was a InvalidElementStateException:

| Running 11 spock tests... 7 of 11
| Failure:  add new route to region(happytrails.AuthenticatedUserSpec)
|  org.openqa.selenium.InvalidElementStateException: Element must be user-editable in order to clear it. (WARNING: The server did not provide any stacktrace information)

I was able to fix this by changing AddRoutePage.groovy from:

static content = {
    createButton(to: ShowRoutePage) { create() }
    name { value("Name") }
    distance { value("Distance") }
    location { value("Location") }
}

To:

static content = {
    createButton(to: ShowRoutePage) { create() }
    form { $("form") }
}

And then referencing name, distance and location accordingly (form.name, etc.) in AuthenticatedUserSpec.groovy.

CloudBees
After I had everything working locally, I logged into Jenkins on CloudBees. Since I hadn't used it in a while, I had to wait a bit while my Jenkins server was re-commissioned. Once it was up, I tried to select Grails 2.2.1 to build with, but found it wasn't available. After a tweeting this, I learned about Grails Wrapper, found that the latest Grails Jenkins plugin supported it and got everything working. I later discovered that CloudBees does support Grails 2.2.1, I just needed to setup another Grails installation to automatically download and install 2.2.1.

Heroku
The last two issues I ran into were with Heroku. Since I was upgrading everything, I wanted Grails to build/run under Java 7 and use Servlet 3. I changed the appropriate properties in BuildConfig.groovy, configured Heroku and deployed. No dice.

Error Compilation error: startup failed:
Invalid commandline usage for javac.
javac: invalid source release: 1.7
Usage: javac  
use -help for a list of possible options

Sidenote: I tried building with Java 8 on CloudBees, but discovered the searchable plugin doesn't support it.

Compile error during compilation with javac.
/scratch/jenkins/workspace/Happy Trails - Grails 2/work/plugins/searchable-0.6.4/src/java/grails/plugin/searchable/internal/compass/index/DefaultUnindexMethod.java:94: error: reference to delete is ambiguous
                    session.delete(query);
                           ^
  both method delete(CompassQuery) in CompassOperations and method delete(CompassQuery) in CompassIndexSession match

As far as Servlet 3, it was pretty obvious that the Jetty version Heroku uses for Grails doesn't support it. Therefore, I reverted back to Servlet 2.5.

java.lang.NoClassDefFoundError: javax/servlet/AsyncContext
	at org.codehaus.groovy.util.LazyReference.getLocked(LazyReference.java:46)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2444)

I sent the Java 7 issue to Heroku Support a few days ago but haven't heard back yet.

Summary
While upgrading Grails from 2.0 to 2.2 wasn't as easy as expected, it is understandable. After all, Grails 2.2 ships with Groovy 2.0, which has a bunch of new features itself. All the issues I ran into were fairly easy to solve, except for Java 7 on Heroku. But hey, what do you expect from a free hosting service?

If you're at Devoxx France next week, I look forward to sharing our research on Grails 2.2.1 vs. Play 2.1.0.

The HTML5 Roadshow Rocks!

2013-03-14T15:16:58-06:00

In early February, as I was creating my Modern Java Web Developer presentation, I came across a training event that was similarly named. Since I happened to know the instructor, I shot him off an email asking more about it. He promptly replied a week later with an intriguing description. The following sentence really stood out for me:

They are NOT beginning workshops -- newbie information is easily found online, in books, at UG meetings, etc. They are targeted at working professionals who want to take their game to the next level.

The other thing that really struck me about this training was it was à la carte; meaning you could pick and pay for just the days you wanted to attend. Scott gave me a two-for-one deal and I signed up for the following workshops:

Normally, attending a training course would have an opportunity cost for me. However, since I work remotely and it doesn't matter where I work, I decided to work from the class. It turned out to be a great idea because it had the same feel of a productive day at the coffeeshop, but it was a coffeeshop full of knowledge. Instead of looking up and doing some people watching or getting distracted by social media, I would look up and learn something new. I actually ended up getting more done in those two training classes than I normally do at my LoDo Office.

Advanced JavaScript
The Advanced JavaScript workshop was taught by Kyle Simpson. This was my first time meeting Kyle and I was immediately impressed by his resume, especially his worked on Firefox Developer Tools for 9 months credentials. He started out talking about good JavaScript documentation on sites such as MDN and Principles of Writing Consistent, Idiomatic JavaScript. Tip: append "mdn" to any searches for JavaScript topics and you're likely to get better results. Not only that, but MDN is a wiki so you can improve it too. From there, we moved onto DOM Events, event propagation (e.g. bubbling vs. capturing), event management, scopes, this, closures and design patterns. It was a very deep dive into JavaScript and I enjoyed every minute of it.

Airplane-Mode HTML5
The Airplane-Mode HTML5 workshop was taught by Scott Davis (his site is currently down, should be fixed next week). Scott talked about many things I was already familiar with, but I still managed to learn a bunch of new tips and tricks. He started by having us install Node, volo and creating projects. He showed how package.json is the JS project's equivalent of pom.xml.

Next, he plunged into viewports, orientation and how to use CSS3 media queries. I've always used min-device-width and max-device-width in my media queries, so orientation: portrait and orientation: landscape was a useful tip. He taught us about CSS3's pseudo-classes (esp. for validation) and pointed us to the Forward Thinking Form Validation article. We learned about using findmebyIP.com and how it can be a useful tool for finding your browser's capabilities, particularly when you're in a black box environment (e.g. a browser on a TV). I knew about Modernizr, but wasn't aware of has.js for JavaScript feature detection.

Scott talked about many more topics, from HTML5 forms and mobile links to geolocation and maps. He ended the day talking about local storage and application cache.

The Good
First of all, the price was perfect at around $250 per day. The best part for me was that I learned more than I expected to. Not only that, but I didn't have to endure any opportunity cost to attend these workshops. There were times I wish I didn't have to work, but it was cool to have "learning stuff" as a distraction rather than social media.

The Bad
The Location. I ride my bike to work year-round and only sit in traffic a few times per year. So I know I'm biased. The location was in Louisville and while it should be only a 30-minute commute, I spent 2 hours in the car each day. Once I got there, the facility was great, the internet was fast and the lunch choices were splendid.

To see if the HTML5 Roadshow is coming to your town, checkout ThirstyHead on Eventbrite or follow them on Twitter.

Integrating GWT into AppFuse

2013-03-07T18:49:28-07:00

I've been interested in integrating GWT into AppFuse ever since I blogged about it 4 years ago. A few months after that post, I wrote about Enhancing Evite.com with GWT and Grails. After Evite, I had a gig near Boston where I developed with GXT for the remainder of the year. When all was said and done, I ended up spending a year with GWT and really enjoyed my experience. I haven't used it much since.

GWT is scheduled to be integrated into AppFuse in version 4.0. That's quite a ways off. The good news is you might not have to wait that long, thanks to Iván García Sainz-Aja. Iván let us know about his work a couple weeks ago in an email to the appfuse-dev mailing list.

It's still work in progress but it has already most of AppFuse functionality..

If you want to give it a try

https://github.com/ivangsa/appfuse.git

the quickest way to have a go would be

web/gwt> mvn -P gwtDebug -Dgwt.inplace=true gwt:compile jetty:run

at the moment it still requires this fork of gwt-bootstrap to be compiled first

https://github.com/ivangsa/gwt-bootstrap.git

It needs a lot of testing yet but it's getting quite there

As you can imagine, I was very excited to hear about Iván's work. So I cloned his repo, built gwt-bootstrap locally and checked it out. Functionality wise, it was great! However, when I dug into the source code, I found a whole lotta code.

To see how the GWT flavor compared to the other implementations in AppFuse, I created a cloc report on the various web frameworks in AppFuse. I'm sure these reports could be adjusted to be more accurate, but I believe they give a good general overview. I posted some graphs that displays my findings in visual form.

When I sent this to the mailing list, Ivan responded that it was a lot of code and estimated 12 new files would be needed to CRUD an entity. This sure seems like a lot to me, but he defended this yesterday and noted that his implementation follows many of GWT's latest best practices: MVP pattern, Activities and Places, EventBus, Gin and Guice. He also shared a wiki page with explanations and diagrams of how things work.

The reason I'm writing this post is to get more feedback on this implementation. First of all, does GWT really require this much code? Secondly, are there other GWT implementations that reduce a lot of the boilerplate? SmartGWT, Vaadin* and Errai come to mind.

If you were starting a new GWT project and using AppFuse, how would you want it implemented?

* Vaadin 7 claims it can be used as a drop-in replacement for GWT. I tried replacing the gwt-servlet and gwt-user dependencies with Vaadin's, but it didn't work.

What's the best way to compare JVM Web Frameworks?

2013-01-09T08:29:17-07:00

I've been comparing web frameworks ever since 2004. It was the first time I'd ever proposed a talk for a conference. ApacheCon was in Vegas that year and my buddy Bruce suggested I speak at it. I submitted the talk, got accepted and went to work learning the frameworks I was talking about. At the time, I had a lot of Struts experience and I'd made a good living learning it, consulting on it and blogging about it. However, there was a new kid on the block (Spring MVC) that was garnishing attention and some other frameworks (WebWork and Tapestry) that had a lot of high praise from developers. I was inspired to learn why so many people hated Struts.

Fast forward 8 years and I'm still comparing web frameworks. Why? Because there still seems to be a large audience that's interested in the topic. Witness InfoQ's Top 20 JVM Web Frameworks, which was one of their most-read articles for two months in a row. One of the beauties of the Java Community is that it's very diverse. There's tons of folks that are part of this community and, like it or not, several folks that are former Java Developers. However, these developers still seem to maintain an interest in the community and it's still one of the largest pools of talent out there. Java is still quite viable and only seems to be getting better with age.

So the topic of web frameworks on the JVM is still hot, and I still like to write about it. For those of you still enthusiastic about the topic, you're in luck. The two best websites for the Java Community, InfoQ and DZone (formerly Javalobby) are still very interested in the topic too!

Both sites emailed me in November to get my advice for their research on the subject. InfoQ (specifically Dio Synodinos) was mostly interested in 1) having me analyze their recently-gathered data, or 2) helping them create a new version. DZone (specifically Mitch Pronschinske) emailed about doing a similar survey to InfoQ's, but with more relevant data points (include GWT, specifying Struts 2 vs. Struts 1, etc.).

My response to Mitch at DZone:

Interestingly enough, the folks at InfoQ contacted me as well as they're thinking of doing a new survey. One of the things I mentioned to them is it'd be interesting to see what folks are using AND which frameworks they admire. Often, devs don't get to choose their web framework at work. I wonder if it'd be possible to collaborate with InfoQ to gather data from developers so it's not being done on two different sites?

I submitted a talk for ApacheCon NA (in February) called Comparing Apache Web Frameworks. When choosing Web Frameworks, I've often found it helps to eliminate frameworks and narrow the scope. Obviously, this makes sense for an Apache Conference, but not for all developers. However, I do plan on analyzing each framework based on a limited set of criteria. Here's what I have so far:

Community, HTML5, REST, Mobile, Performance, Web Performance Optimization

Obviously, community is important for Apache projects, but might not be for the wider audience. It might be good to limit to these 5 criteria, or expand it to 10, but not more. I think it'd be interesting to get the community to rank the various frameworks on these criteria, and also try to find developer's biases while doing it. For example, I wonder if people would be willing to admit they're biased for/against certain frameworks and then take that into account as part of gathering the data?

Unfortunately, ApacheCon denied my submission. This make sense since their schedule seems to be concentrating on HTTPD, Cloud and Big Data.

For InfoQ, Dio asked for a list of web frameworks to include. Below is a list we started with, followed by my response.

Spring MVC
Play
Grails
JSF
Struts 2
Wicket
Lift
Tapestry 5
Seam
JRuby on Rails
Wicket
GWT
Vaadin

I would add Stripes, vert.x and maybe something like Apache Click. VRaptor is probably a good one to add too. There's always a few less-used frameworks that get a lot of complainers if you don't include them.

Comparing to your previous list, I don't think Seam should be in here since they've split the project into separate bundles and are no longer developing Seam as a whole. JRuby on Rails is a tough one because if you say Ruby on Rails, you'll get a ton of responses, but probably not from the Java community. The Ruby community might chime it quite a bit if you can get in touch with them though.

I believe you should include Clojure web frameworks, but I've only heard of one of them: Compojure.

SiteMesh, Netty, etc. - remove them.

I also offered my advice on instructions:

I actually like the two coordinates you used before, importance and adoptability. However, I don't know that everyone read the instructions this way. Most people didn't rank all frameworks and I believe that's part of the point. I only ranked the ones I'd used, but I think it'd be better if people ranked all of them. I also think having these two criteria opens it up to more than just developers. Project/Product Managers and stakeholders that've been successful with certain frameworks should be able to vote too.

For DZone, they wanted to include a set of criteria for ranking:

Project maturity, community support, and documentation quality (one criteria)
UI Features Capability (maybe some are more graphics driven or form driven?)
Code readability
Flexibility (maybe this could be broken down into what types of projects the framework can handle) or several criteria that ask if it is "Good for 'x' type of project"
Performance/Speed
Cross platform support
Extensibility, Plugins, Community Libraries
Architecture (this may just be information for later, not an opinion question)
Web standards support
REST support (is this something you would just rate a yes or no? In that case it wouldn't need to be an opinion question)
Mobile support

My response:

For web frameworks, I believe the 5 I mentioned (Community, HTML5, REST, Mobile, Performance, Web Performance Optimization) are most important, with Security become more and more of a concern. For web standards support, I'd almost change it to "HTML5" and to see how the various frameworks stack up. I think REST is very important, and I think it's cool that Struts 2, Spring MVC and Grails all have great support for it. It'd be interesting to see how the component-based frameworks think of having REST support in the framework (vs. external like Jersey, CXF, etc.).

Of the list you provided, I don't know about Code readability or Flexibility. Code readability is kinda like Learnability. One of the nice things about Spring MVC and Grails is that you can learn how they work very quickly. Then you can use that knowledge and don't have to look things up much. Tapestry and Wicket might be similar for those writing large apps, but I haven't found that to be as true. The more traditional MVC Frameworks just make more sense to me.

UI Feature Capability is a good one because frameworks with widgets are often popular with developers. Flex, GWT, jQuery UI, Sencha all do this very well.

Summary
The reason for this post is to add some transparency to the process of Comparing JVM Web Frameworks. I like to think that I've tried to do this in the past (especially with my reasons for rankings). Now, we'd like to hear from you, the community that uses these web frameworks.

What's the best way to compare JVM Web Frameworks?

While it's nice to hear from the generous folks that create and maintain JVM Web Frameworks, we're mostly interested in hearing from the developers that are using these things on a day-to-day basis. The Blue Collar Developers, if you will. If you could design a JVM Web Framework comparison that answered your questions, how would it look? What questions would it ask? What conclusions would make you happy? Should commercial frameworks like ZK be included?

Your responses are very greatly appreciated.

2012 - A Year in Review

2013-01-08T13:15:24-07:00

I wrote my first year in review blog entry way back in 2005. That means this year's is number 8. Since they keep getting longer every year, I figured I'd try something different this year and use sections similar to Remy Sharp.

Professional

I spent the entirety of the year with one client: Taleo. Oracle bought them in February. In June, the transition to Oracle happened. My tasks and projects haven't changed much since the transition, but it has been a real pain to get paid on time. My contract with them is through the end of May. I hope to take July off (to get married) and August off (to honeymoon) and start a new gig in September.

I did minimal Java work throughout the year and spent most of my time doing CSS and JavaScript. I love doing front-end work much more than back-end, so day-to-day, it was very satisfying.

Business

Raible Designs had a great year, our best ever financially. I hired Trish as an assistant in January, doubling the size of the company. We had our Annual Shareholders Meeting on a Disney Cruise in April. I have no plans to do increase the size of the company in 2013, mostly because I enjoy not having the responsibility of employees. I've employed sub-contractors in the past and it always seems to create more of a headache than it's worth.

We moved our offices from downtown Littleton to Thrive (in LoDo) last April and I plan on staying there. It's an easy 6-mile bicycle commute and the guys there have been great to work with. I highly recommend it if you're looking for co-working space.

Speaking

I spoke at 5 events in 2012:

Silicon Valley Spring User Group on What's New in Spring 3.1.
JFokus on Comparing Web Frameworks and HTML5 with Play Scala.
Spring I/O on Comparing JVM Web Frameworks.
JavaPosse Roundup on How to build a Shot-Ski.
ÜberConf on Play vs. Grails.

Traveling to Stockholm and Madrid for JFokus and Spring I/O was spectacular.

I took the 2nd half of the year off from speaking, celebrating my temporary retirement at a Def Leppard concert in June. When I decided to do this, I was planning on my VW Bus being finished. I was planning on switching from Java Conferences to VW Shows as a hobby. Unfortunately, now it's January and The Bus still isn't done. I have high hopes for it being finished in 2013.

I paid for 2 conferences in 2012, the JavaPosse Roundup in March and Monktoberfest in October. While I enjoyed both, Monktoberfest will likely become a tradition.

Projects

AppFuse: I started the year by refreshing AppFuse's UI with Twitter Bootstrap and blogged about the Twitter's Summit on Bootstrap shortly after. At the same time, I was overhauling Taleo's UI with HTML5, Bootstrap and CSS3, but waited until August to blog about it.

In the fall, the AppFuse Team migrated to GitHub and integrated Hibernate Search. AppFuse 2.2.1 was released in December.

Happy Trails: In May and June, I worked on Happy Trails with James Ward as part of our Play vs. Grails Smackdown. This was a very enjoyable experience and I learned a lot about GitHub, Cloudbees and Heroku.

Play More: In June, I wrote about Upgrading to Play 2: Anorm and Testing. A couple weeks later, I wrote about integrating Validation and Secure Social into Play More. In the aforementioned post, I also posted my ÜberConf presentation, the source code and my thoughts on Play 2.

I found Anorm and Scalate to be huge time sinks and don't know if I'd recommend using either one in a Play 2 project. I'm sure Scalate will be easier to use as its Play 2 integration gets more refined, but I don't know if there's any hope for a JDBC abstraction that doesn't produce error messages when things go south.

Roller: Although I didn't contribute much to Apache Roller in 2012, I do continue to use it frequently. This site celebrated its 10-year anniversary in August. It got a new look and feel shortly after.

Personal

Last year was a fantastic year for family time. Our cruise around the Western Caribbean was money well spent.

My goal was to spend a month with my parents this year and we satisfied that by July. In total, we enjoyed their company for 8 weeks in 2012.

The worst ski season on record ended in April and Trish and I headed for wine country in May.

Abbie and Jack's school year ended on June 5th and we celebrated Father's Day on the Oregon coast.

Raible Road Trip #17 began in late June and we spent two weeks parading, rafting and golfing in Montana. From there, we flew to Maui for two weeks. I wrote about it all in Summer Vacation 2012 in Montana and Maui.

After returning from Hawaii, we dropped the kids off at their Mom's and headed to my cousin's wedding in Beacon, NY for the weekend. From there, my parents, Trish and I rode the train to Boston, spent a few days in Cape Cod and returned home in time for the beginning of school.

The kids started the school year in mid-August and we celebrated Jack's 8th birthday a couple weeks later.

My lack of speaking engagements and work-related travel gave us more time to frolic in the Colorado mountains, so we bought a boat for Labor Day weekend. That Saturday, we took it on its Maiden Voyage on the Colorado River, floating from Radium to Rancho del Rio. I guided and rowed the boat most of the time while our 7 passengers (and 2 dogs) enjoyed cold beverages, great scenery and relaxing in the sun.

The next day, Abbie played her first game of golf.

In October, I finished our deck project and work continued on our kitchen remodel (started in July by Ted Pepper). Our kitchen was far enough along for us to host our 1st Annual Halloween Party.

Abbie turned 10 the first week in November. That Friday, we headed to Kauai to celebrate my parents' 40th Anniversary. We spent Thanksgiving in Wenatchee, Christmas in Boston/Vermont and New Years with good friends in Fraser.

2013

My goals for 2013 are to be happy and to slow down. I don't plan on traveling any less, or spending less time with my family. I'd just like to walk slower, smell the good ol' Rocky Mountain air more and enjoy life. I believe our raft will help us facilitate this and we're already planning trips to the Green River in Utah and Schafer Meadows in the Bob.

The big event for the year will be Trish and my wedding this summer. We'll be doing it in my hometown in Montana and we expect it to be quite the shindig. Afterward, we plan on traveling the world for our honeymoon.

Of course, if the bus gets finished, that's a huge bonus too.

Professionally, I'm starting an exciting project at Taleo that will involve heavy JavaScript, CSS3 and maybe even Canvas. I have speaking engagements lined up at Denver JUG in February and HTML5 Denver in April. Also, I just submitted a couple talks to Devoxx France.

For AppFuse, I'd love to get 3-4 releases out this year, but 2 is probably more realistic. The AppFuse Roadmap shows our future releases + features. I'd also like to work a bit more on my Play 2 app; finishing the mobile client and possibly switching from Anorm to Slick or Spring Data.

While Oracle has been good to me, I look forward to finding and working for a new client in September.

Last year at this time, we watched the Broncos execute the Mile High Miracle over the Steelers. With tickets to both this week's game (vs. The Ravens) and next week's game (vs. The Patriots), I wouldn't be surprised if January turns out to be a great month. However, with the Powder Days, The Super Bowl, the raft trips, the Bus Shows and The Wedding, there's a good chance that all the months in 2013 are epic.

We bought a boat!

2012-09-05T11:12:02-06:00

I've always enjoyed whitewater rafting. I think the first time I did it was in college and I immediately fell in love. Through the years, I've been on many trips with family and friends. However, it wasn't until this summer that I realized it was something I should do more often. It was Trish's friends, Chris and Bryce, that started it all. They bought a raft last year and we floated down the Colorado River with them a couple times over Memorial Day Weekend. Then we went to Montana and enjoyed a couple days on the Middle Fork of the Flathead with Dr. Barton and a bunch of raft guides. That weekend in July, we realized we'd done more rafting than any other outdoor activities (mountain biking, camping and even golfing). That's when we decided to buy our own.

We had a lot of help in the process of buying a raft. First of all, I sat down with my friend Dr. Barton and made a list of all the things we'd need. The good doctor was a whitewater guide in Montana for 5 years, has rescued trips from the wilderness and has even rafted the Grand Canyon - so I considered him a good source of information. After composing the list of necessary gear, we headed to Down River Equipment on August 26th, the last day of their end-of-season sale. It took us an hour to pick out the raft we wanted (a Pro 140) and gather up all the gear (frame, cooler, oars, dry box/bags, lifejackets, koozies, etc.). We asked them to have it ready by Friday and headed home.

Last Friday, we picked up a raft trailer from Trailer Source an hour before they closed, then journeyed to Down River where Mike (the owner) and Matt (the guy who helped us the previous Sunday) helped us setup our oars and load up our new boat. There was much rejoicing.

Saturday, we took it on its Maiden Voyage on the Colorado River, floating from Radium to Rancho del Rio. According to this page, there were some Class III rapids, but they all felt like Class II. I guided and rowed the boat most of the time while our 7 passengers (and 2 dogs) enjoyed cold beverages, great scenery and relaxing in the sun. It took us a bit longer (4 hours) than expected (2 hours), but we all thought it was well worth it.

Abbie's First Golf Game
After a long day of floating on Saturday, we decided to chill on Sunday with a little golf. We split the kids up for the weekend (Jack went with his mom), so we figured the proper way to treat our only child was to take Abbie to play her first real game of golf at Pole Creek. We played 9 holes and both Abbie and I had a great time trying out our new clubs. We received a nice kids golfing tip from someone at the driving range: have them tee off from the 150 marker so they have a chance to par the hole.

The course had a 50% discount for kids and we never saw anyone behind us the entire game. We were especially impressed when the course photographer offered us a framed set of Abbie pics for $15.

We don't know how many more days of rafting we'll get in this year, but next year should be epic. We're hoping to do multi-day trips on the Green River, the Smith River and fly into Schafer Meadows for a journey through the Bob Marshall Wilderness. I grew up only 10 miles from "The Bob" and I've never been in it. I can't wait!

New Look and Feel, Designed by Gillen's Army

2012-08-14T22:58:11-06:00

As part of my 10-year blogiversary, I was hoping to refresh this site with a new look and feel. A few months ago, I contacted my friend Mark Waggoner to see about getting his design help. We promptly worked out a logo/business card/website deal and Gillen's Army went to work.

I picked a logo from numerous choices in late June, finalized a business card for printing in July and received the HTML and CSS for the site on August 2nd. I started converting it to a Roller theme last week and did a whole bunch of other modifications in the process.

Upgraded to Roller 5.0.1.
Upgraded wro4j to the latest version (1.4.8.1) to workaround using a → (\2192) in CSS.
Changed to use jQuery and Lightbox2 for pictures.
Upgraded to the latest version (3.0.83) of SyntaxHighlighter. You might notice there is no longer a toolbar in this version. However, you can still double-click on code and easily copy/paste it.

In addition to these upgrades, I made a few enhancements. I converted to HTML5 (by switching the doctype), added Modernizr and a feature that detects if the sun is up in your location. If you allow your browser to send me your lat and long, I'll give you a dark theme when the sun is down and a light theme when it's daylight. I used Preston's Hunt's JavaScript Class for Sunrise and Sunset Calculations to determine isDaylight. You can also change the theme to light or dark using the small rectangles above the search box on the right. This sets a cookie and overrides the HTML5 Geo check. You can see the implementation of this logic in site.js.

The stylesheet switching doesn't happen as fast as I'd hoped (there's a flash even if using cookies), so I'll likely be converting some theme-setting logic to the server-side. The HTML5 version of the FaceBook Like Button requires you to specify the "data-colorscheme" in markup so this further supports moving to the server.

I have other minor adjustments I'd like to make, but more importantly - I wanted to get it out to you all. Tell me what you like and don't like. Among other things, the form inputs for comments and contact forms have backgrounds that might not be great for those color-impaired. Also, you can see how the iframe on the contact page has a white background instead of one based on the theme.

Here's some stats comparing my old andreas08 theme to the new darklight:

Metric	andreas08	darklight
Size and Speed	167 requests, 3.6MB, 9.89s	148 requests, 3.2MB, 7.34s
YSlow	76	87
PageSpeed	91	96

Sweet! It looks like this site is faster than ever. Cheers to Mark and Gillen's Army for the new design. I dig it!

Summer Vacation 2012 in Montana and Maui

2012-08-01T09:30:27-06:00

I started writing about my summer vacations in Montana as soon as this blog gave me an opportunity: July 2003. In July 2004, I didn't travel to the cabin for the 4th, but I did in 2005. Then, 5 years ago, I started writing longer blog posts about our trips. 2008 was Raible Road Trip #12. In 2009, I spent a whole month in Montana. In 2010, I visited after my parents moved there full-time. Last year, I took Trish for the first time.

I don't know what Raible Road Trip number this year's trip was, but I'm going to go ahead and assume #17. This year wasn't exactly a vacation for me. It was what I like to call a workation. I tried to take the whole month off, but my current client needed my services. As a compromise, they agreed to let me work 20 hours per week during July. While I didn't get a chance to fully unplug, I did enjoy the reduced workload and time to spend with friends and family. And the views from The Cabin and Sugar Beach weren't bad either.

Below is our trip itinerary in condensed form, complete with photos from my incredibly talented fiancé.

We embarked on our drive to Montana on June 30th, stopped for fireworks in Wyoming and spent the night in Yellowstone park. Arrived at the cabin, got to sleep in the new house for the first time, built a cabin float, drove it in the parade, went boating at Holland Lake, played golf. Journeyed to West Glacier to celebrate Dr. Barton's PhD at his sister's house. Rafted the Middlefork of the Flathead all weekend. Played some more golf and went waterskiing.



→ Trish's Montana Summer 2012 Album

On July 14th, we flew to Maui and continued to explore. Our good friend's parents have a timeshare on Sugar Beach and got us a screaming deal on lodging. We had beachfront accommodations with a pool, hot tub and sauna. We played golf on my birthday (July 16th). Visited the Old Lahaina Luau, Maui Aquarium and Coconut Festival at Grand Wailea. Drove to the summit of Haleakala. Sipped cold handcrafted ales while watched the kids boogie board. Embarked on a sunset cruise, hiked in Iao Valley, did some horseback riding along the beach, sea kayaked and snorkeled with sea turtles. Savored Shawn's excellent cooking. Flew back to Missoula (Montana), slept for 6 hours and drove 12.5 hours back to Denver on July 27th.



→ Trish's Maui Summer 2012 Album

Mahalo Montana and Maui. Thank you to our awesome friends: Mom, Dad, Owen, Jason, Ryan and Shawn. It was an incredible adventure.

Migrating to Play 2 and My ÜberConf Presentation

2012-06-21T16:09:22-06:00

In my last post about migrating to Play 2, I said I'd write another post on the rest of my experience. While I'm not completely finished with migrating to Play 2, I feel like I've done enough to talk about the issues I encountered.

Validation and Displaying Errors
With Play 1, I can't help but think validation was a bit more intuitive. For example, here's how I populated an object from request parameters, converted a value and validated its data was fit to put in a database.

var workout = params.get("workout", classOf[Workout])

// change duration to time
var duration = params.get("workout.duration")
workout.duration = convertWatchToTime(duration)

Validation.valid("workout", workout)

if (Validation.hasErrors) {
  renderArgs.put("template", "Profile/edit")
  edit(id);
  ...
} else { // put into db

With Play Scala 2, you have to define a Form structure and bind it from the request. Based on what I was able to conjure up, I ended up writing the following code to accomplish the same thing:

val workoutForm = Form(
  mapping(
    "id" -> ignored(NotAssigned: anorm.Pk[Long]),
    "title" -> text,
    "description" -> text,
    "duration" -> nonEmptyText,
    "distance" -> nonEmptyText,
    "postedAt" -> optional(date),
    "athleteId" -> optional(longNumber)
  )((id, title, description, duration, distance, postedAt, athleteId) =>
    Workout(id, title, description, convertWatchToTime(duration), distance.toDouble, null, 0))
    ((w: Workout) =>
      Some((w.id, w.title, w.description, w.duration.toString, w.distance.toString, null, Some(0))))
)
...
workoutForm.bindFromRequest.fold(
  form => {
    Ok(Scalate("/Profile/edit.jade").render(request, 'errors -> form.errors))
  },
  workout => { // put into db

First of all, the Play 2 version is quite a bit more verbose, but most of that comes from the re-defining of my model object as a form. It seems strange that the Java API allows you to do it in one line whereas the Scala version does not. Also, I was unable to figure out how to get the data from my "form" back into the request so I could refill input fields. I'll admit, I didn't spend a lot of time trying to figure it out, but it did fail the 10 minute test. Note to self: use HTML5's required attribute to reduce the need for server-side validation on modern browsers.

On a more positive note, I did like the way I was able to use routes in my Jade templates. It was as simple as importing the routes class and using it as you would in Play's Scala Templates:

-import controllers._

form(method="post" class="form-stacked" id="workoutForm"
  action={routes.Profile.postWorkout(workout.map(_.id.get))})
  input(type="hidden" name="id" value="#{workout.map(_.id)}")

Secure Social
After getting most of my UI working, I started looking at the Secure Social Module for Play 2. Below are the steps I had to go through to install it:

Cloned GitHub project to my hard drive.
Copied module-code/* into my projects' modules/securesocial directory.

Modified project/Build.scala to add secureSocial and dependsOn to my project.

val secureSocial = PlayProject(
  appName + "-securesocial", appVersion, mainLang = SCALA, path = file("modules/securesocial")
)

val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings(
  // Add your own project settings here
).dependsOn(secureSocial).aggregate(secureSocial)

Added a conf/securesocial.conf and included it in my application.conf with the following line:
```
include "securesocial.conf"
```
Added a conf/play.plugins with the following to get Twitter to load as a provider:
```
10000:securesocial.core.providers.TwitterProvider
```
Created an InMemoryUserService.scala and referenced it in my play.plugins file:
```
9999:services.InMemoryUserService
```
Added Secure Social's routes to my conf/routes file.

Once I'd finished all these steps, I fired up my app and was pleasantly surprised to find I could navigate to /login and successfully authenticate via Twitter. Installing Secure Social in a Play 2 app is quite a bit harder than adding it as a dependency in Play 1, but I was thankful that I got it to work in under 10 minutes.

Heroku
The next thing I did was attempt to deploy my app to Heroku. I knew there might be some issues with Scalate after reading Jan Helwich's blog post about Scalate on Heroku. The first things I encountered were 1) a successful startup and 2) an error in my browser.

I was able to reproduce this issue locally by running "play clean stage" and starting the app with "target/start". After 30 minutes of banging my head against the wall, I guessed it might be caused by Secure Social. Removing Secure Social solved the problem and I was back in business. However, this time when I deployed, I received the error that Jan had mentioned might happen.

2012-06-21T07:07:12+00:00 app[web.1]: [error] o.f.s.l.DefaultLayoutStrategy - Unhandled: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.jade.scala (No such file or directory)
2012-06-21T07:07:12+00:00 app[web.1]: [error] application - 
2012-06-21T07:07:12+00:00 app[web.1]: 
2012-06-21T07:07:12+00:00 app[web.1]: ! @6amfgf02h - Internal server error, for request [GET /] ->
2012-06-21T07:07:12+00:00 app[web.1]: 
2012-06-21T07:07:12+00:00 app[web.1]: play.core.ActionInvoker$$anonfun$receive$1$$anon$1: Execution exception [[TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.jade.scala (No such file or directory)]]
2012-06-21T07:07:12+00:00 app[web.1]:   at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:134) [play_2.9.1-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]:   at play.core.ActionInvoker$$anonfun$receive$1.apply(Invoker.scala:115) [play_2.9.1-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]:   at akka.actor.Actor$class.apply(Actor.scala:311) [akka-actor-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]:   at play.core.ActionInvoker.apply(Invoker.scala:113) [play_2.9.1-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]:   at akka.actor.ActorCell.invoke(ActorCell.scala:619) [akka-actor-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]:   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:196) [akka-actor-2.0.1.jar:2.0.1]
2012-06-21T07:07:12+00:00 app[web.1]: Caused by: org.fusesource.scalate.TemplateException: target/../tmp/src/app/target/../app/views/layouts/default.jade.scala (No such file or directory)
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:834) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.liftedTree1$1(TemplateEngine.scala:411) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:405) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.load(TemplateEngine.scala:475) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.layout.DefaultLayoutStrategy.org$fusesource$scalate$layout$DefaultLayoutStrategy$$tryLayout(DefaultLayoutStrategy.scala:77) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]: Caused by: java.io.FileNotFoundException: target/../tmp/src/app/target/../app/views/layouts/default.jade.scala (No such file or directory)
2012-06-21T07:07:12+00:00 app[web.1]:   at java.io.FileOutputStream.open(Native Method) ~[na:1.6.0_20]
2012-06-21T07:07:12+00:00 app[web.1]:   at java.io.FileOutputStream.(FileOutputStream.java:209) ~[na:1.6.0_20]
2012-06-21T07:07:12+00:00 app[web.1]:   at java.io.FileOutputStream.(FileOutputStream.java:160) ~[na:1.6.0_20]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.util.IOUtil$.writeBinaryFile(IOUtil.scala:111) ~[scalate-util-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.compileAndLoad(TemplateEngine.scala:747) ~[scalate-core-1.5.3.jar:1.5.3]
2012-06-21T07:07:12+00:00 app[web.1]:   at org.fusesource.scalate.TemplateEngine.compileAndLoadEntry(TemplateEngine.scala:691) ~[scalate-core-1.5.3.jar:1.5.3]

I tried his suggestion (removing the first slash on my Scalate paths) but it didn't work. I tried adding in Scalate pre-compilation, but that didn't solve the problem either. The good news is I did solve it this afternoon by changing my Scalate object to use a canonical path instead of an absolute one.

iPhone App
In addition to the changes mentioned here, I re-wrote the iPhone app for Play More. I upgraded it to PhoneGap 1.8.1, used jQTouch, developed with AppCode (instead of Xcode) and had a pretty good experience. The only issue I ran into was with the jqt.bars extension from DataZombies. I briefly tried to integrate it and then decided not to. However, I left all its JS and CSS in my page and this caused scrolling to not work and made the app sluggish. Removing the files solved the problem. The other big improvement I made was moving all the static assets (JS, CSS, images) into the mobile app instead of referencing them from http://play-more.com. This reduced the startup time from 30-40 seconds to 3-4 seconds!

Presentation and Source Code
I presented all of these findings and told my story at ÜberConf this morning. In addition, I announced that the code is now open source and available on GitHub. You can view my presentation below or on Slideshare.

Conclusion
Would I do it again? Learning Scala was my primary motivator for digging into Play. When Play 2 was announced, I thought migrating my app to the new version would be easy. Unfortunately, the Play Developers decided to break backwards-compatibility and wrote a whole new framework that still seems to be in its infancy. I think you can see from my last couple of posts that migrating from Play 1.x to 2.x was not an easy task. It's been nice to learn more about Play and Scala in the process, but living on the bleeding edge was also quite frustrating at times. Play Scala 1.x seemed to be quite a bit more productive than Play 2, especially because of the Magic[T] in Anorm, but also because it required less code in Controllers.

On the upside, my experience with HTML5 and CoffeeScript was wonderful. They did what I asked them to do and didn't cause much pain. When a browser-based webapp couldn't handle geo running in the background, PhoneGap came to the rescue.

I plan on continuing to develop Play More! If you'd like to help, checkout the open issues and viva la open source!

Twitter's Open Source Summit: Bootstrap 2.0 Edition

2012-02-01T11:28:40-07:00

Every few months, Twitter hosts an Open Source Summit to talk about tools they're using. Since I happened to be near San Fransisco, I happily attended their latest #ossummit to learn about Bootstrap 2.0. Below are my notes from last night's event.

95% of Twitter's infrastructure is powered by open source. They hope to contributing back to open source by doing 2-3 summits per year. Without open source, there would be no Twitter. You can find a bunch of Twitter's open source contributions at twitter.github.com. They're also big fans of Apache and commit to a wide variety of projects there.

Bootstrap
Bootstrap is developed by two main guys: @mdo and @fat. Mark (@mdo) has been a designer at Twitter for 2 years. He started on the Revenue Team with ads, but has been working on redesign for last 4 months. Has been doing HTML and CSS for about 11 years. He used Notepad on Windows to build his GeoCities site.

boot·strap: simple and flexible HTML, CSS, and JavaScript for popular user interface components and interactions.

Work on Bootstrap started about a 1.5 years ago. Internal tools didn't get the proper attention they needed. They figured out there was a lot of people that wanted good looking UIs and interactions. It became Twitter Blueprint and was mostly used internally. Jacob (@fat) started shortly after first release and decided to add some JavaScript on top of it. The JavaScript for Bootstrap was originally the "Twitter Internal Toolkit" or "TIT" and was built on Moo Tools. Jacob was like "we gotta open source this, it's gonna be huge!" (he was right).

The 1.0 release supported Chrome, Safari and Firefox (everyone at Twitter was on Macs). 1.3 added cross-browser support and JavaScript plugins.

Now there's Bootstrap 2 (just released!). They rewrote all the documentation and components and removed legacy code.

So, what's new? The biggest thing is the docs. Previously had live examples, now shows live examples and why you would do something, as well as additional options. The "topbar" has been renamed to "navbar", but it's still got all the hotness. It's responsive with CSS media queries for small devices, tablets, small desktops and large desktops. This means the layout breaks at certain points and stacks elements to fit on smaller screens.

CSS: smarter defaults, better classes. In 1.4, all forms were stacked. Now they can flow horizontally. Tables are now namespaced so Bootstrap's styles don't apply to all tables. The available table, form and navigation classes are as follows:

// Tables
.table { ... }
.table-striped { ... }
.table-bordered { ... }
.table-condensed { ... }

// Forms
.form-inline { ... }
.form-search { ... }
.form-horizontal { ... }

// Nav
.nav { ... }
.nav-tabs { ... }
.nav-pills { ... }

The goals with 2.0 are consistency, simplification and future-proofing styles. With 1.4, buttons used "btn primary" and it caused problems if you wanted to have a "primary" class in your project. With 2.0, buttons and all elements are namespaced to avoid collisions (now it's .btn-primary).

After Mark finished talking about the design of Bootstrap, Jacob (@fat) started talking about Bootstrap's JavaScript. Jacob works on The Platform Team at Twitter and claims he made a lot of mistakes with 1.x. However, thanks to semantic versioning, 2.0 is a new version and he got to start over!

The biggest change in 2.0 is the use of data attributes (a.k.a. data-*). They were using them in 1.x, but not to the full potential of what they can be and should be. The first class API for Bootstrap JavaScript is data attributes (or HTML), not JavaScript.

With 1.x, you could add an anchor to close modals and alerts.

// 1.x closing modal/alerts
<a class="close" href="#">×</a>

However, if you put your alerts in your modals, you close them all when you likely only wanted to close one. 2.0 uses a "data-dismiss" attribute.

<a class="close" data-dismiss="model">×</a>

This allows you to target what you want closed (modals or alerts, etc.). You know exactly what's going to happen just by reading the code. Another example is the "href" attribute of an anchor. Rather than using "href", you can now use "data-target".

// 1.x href = target
<a href="#myModal" data-toggle="modal">Launch</a>

// 2.x data-target = target
<a data-target=".fat" data-toggle="modal">Launch</a>

If you'd rather turn off the data attribute API, or just part of it, you can do so by using the following:

// Turn off all data-api
$('body').off('.data-api')

// Turn off alert data-api
$('body).off('.alert.data-api')

2.0's JavaScript API has the same stuff, but better. You can turn off the data-api and do everything with JavaScript. They copied jQuery UI in a lot of ways (defaults, constructors, etc.). Bootstrap's JavaScript has 12 plugins. New ones include collapse, carousel and typeahead.

Customize - a new tab that lets you customize and download Bootstrap. It's basically an alternative to customizing .less files and allows you to choose components, select jQuery plugins, customize variables (colors, font-sizes, backgrounds) and download.

What does the future hold? Internationalization, improving responsiveness, more new features and bug fixes.

After both Mark and Jacob gave their talks, they talked together about Community and how great it's been. Even if you're not into writing CSS and JavaScript, they mentioned they still wanted to hear from you. To give an example of great community contributions, one guy opened 50 issues in the last 2 days.

Someone in the audience asked why they used LESS over SASS. Jacob said the main reason they use LESS is because they're good friends with the guy who invented it (Alexis). SASS turns CSS into a programming language, but they wanted to maintain the approachability of CSS, which LESS does. There's no plans to do an official SASS port, but there is talk of doing one. One advantage of the current LESS compiler is they rewrote it to have better output so it's far more readable.

NASA
After Mark and Jacob finished, there was a 5 minute break to grab beers and snacks. Then Sean Herron (@seanherron) (a.k.a. "NASA Bro") talked about Bootstrap at NASA. He actually didn't talk about Bootstrap much, except that they used it for code.NASA. He talked about NASA and how it's playing a key role in the movement towards open data, open source and open standards in our federal government. He mentioned how data.NASA was launched last August and that they helped develop OpenStack. Finally, he mentioned open.NASA, which is a collaborative approach to open, direct and transparent communication about our space program.

Hogan.js
Next up, Rob Sayre (@sayrer) talked about Hogan.js. Rob has been at Twitter for a few months, before that he wrote JavaScript at other places. Hogan.js is a compiler for Mustache templates.

Why Mustache? Because it's similar to HTML and easy to edit. You can mock data as JSON files and programmers are not required.

At Twitter, designers can do the CSS and Mustache without connecting to the backend. It has cross-language support in Ruby, Java and JavaScript. However, client-side template compilation has performance problems, especially in IE7 on a Windows XP box with 4 viruses.

So they had a few choices: work on mustache.js, or use Dust.js or Handlebars.js. The compilers are very nice for Dust.js and Handlebars.js, but they're huge. Handlebar's parser is 4000 lines. The entire Hogan.js file is 500 lines. They decided they were too large to send to the browser's of their users, so they chose to write a better compiler for Mustache.

Hogan.js's main features:

Compile on the server
Parser API
Performance

Performance is much better with Hogan.js than Mustache.js. On IE7 - Hogan is 5x faster than Mustache. On an iPhone, it's about the same (and an iPhone's browser is faster than IE7 on a decent computer). With modern browsers (Chrome 17, Safari 5 and Firefox 10), it's more than 10x faster.

Hogan.js is currently used at Twitter for Tweet embedding, the Bootstrap build process and soon, Twitter.com.

It's been awhile since I got excited about an open source project. Bootstrap has helped me a lot recently, in my Play More! mobile app, on some client projects and I'm in the process of refreshing AppFuse's UI to use it. I love how you can add a class or two to an element and all of a sudden they pop with good looks. The main problem with Bootstrap at this point is that a lot of Bootstrapped apps look the same. There's talk of adding themes in a future release to help alleviate this problem. In the meantime, there's a lot to get excited about with 2.0.

Thanks to Twitter for hosting this event and kudos to Mark and Jacob (and the community!) for such a fantastic project.

Refreshing AppFuse's UI with Twitter Bootstrap

2012-01-31T17:12:17-07:00

The last time AppFuse had an update done to its look and feel was in way back in 2006. I've done a lot of consulting since then, which has included a fair bit of page speed optimization, HTML5 development and integrating smarter CSS. It was way back in '05 when we first started looking at adding a CSS Framework to AppFuse. It was Mike Stenhouse's CSS Framework that provided the inspiration and my CSS Framework Design Contest that provided its current themes (puzzlewithstyle, andreas01 and simplicity).

Since then, a lot of CSS Frameworks have been invented, including Blueprint in 2007 and Compass in 2008. However, neither has taken the world by storm like Twitter Bootstrap. From Building Twitter Bootstrap:

A year-and-a-half ago, a small group of Twitter employees set out to improve our team’s internal analytical and administrative tools. After some early meetings around this one product, we set out with a higher ambition to create a toolkit for anyone to use within Twitter, and beyond. Thus, we set out to build a system that would help folks like us build new projects on top of it, and Bootstrap was conceived.
...
Today, it has grown to include dozens of components and has become the most popular project on GitHub with more than 13,000 watchers and 2,000 forks.

The fact that Bootstrap has become the most popular project on GitHub says a lot. For AppFuse.next, I'd like to integrate a lot of my learnings over the past few years, as well as support HTML5 and modern browsers as best we can. This means page speed optimizations, getting rid of Prototype and Scriptaculous in favor of jQuery, adding wro4j for resource optimization and integrating HTML5 Boilerplate. I've used Twitter Bootstrap for my Play More! app, as well as some recent client projects. Its excellent documentation has made it easy to use and I love the way you can simply add classes to elements to make them transform into something beautiful.

Last week, I spent a couple late nights integrating Twitter Bootstrap 2.0 into the Struts 2 and Spring MVC versions of AppFuse. The layout was pretty straightforward thanks to Scaffolding. Creating the Struts Menu Velocity template to produce dropdowns wasn't too difficult. I added class="table table-condensed" to the list screen tables, class="well form-horizontal" to forms and class="btn primary" to buttons.

I also added validation errors with the "help-inline" class. This is also where things got tricky with Struts and Spring MVC. For the form elements in Bootstrap, they recommend you use a "control-group" element that contains your label and a "controls" element. The control contains the input/select/textarea and also the error message if there is one. Here's a sample element waiting for data:

<div class="control-group">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name">
    </div>
</div>

Below is what that element should look like to display a validation error:

<div class="control-group error">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name" value="">
        <span class="help-inline">Please enter your name.</span>
    </div>
</div>

You can see this markup is pretty easy, you just need to add an "error" class to control-group and span to show the error message. With Struts 2, this was pretty easy thanks to its customizable templates for its tags. All I had to do was create a "template/css_xhtml" directory in src/main/webapp and modify checkbox.ftl, controlfooter.ftl, controlheader-core.ftl and controlheader.ftl to match Bootstrap's conventions.

Spring MVC was a bit trickier. Since its tags don't have the concept of writing an entire control (label and field), I had to do a bit of finagling to get things to work. In the current implementation, Struts 2 forms have a single line for a control-group and its control-label and controls.

<s:textfield key="user.firstName" required="true"/>

With Spring MVC, it's a bit more complex:

<spring:bind path="user.firstName">
<fieldset class="control-group${(not empty status.errorMessage) ? ' error' : ''}">
</spring:bind>
    <appfuse:label styleClass="control-label" key="user.firstName"/>
    <div class="controls">
        <form:input path="firstName" id="firstName" maxlength="50"/>
        <form:errors path="firstName" cssClass="help-inline"/>
    </div>
</fieldset>

You could probably overcome this verbosity with Tag Files.

Figuring out if a control-group needed an error class before the input tag was rendered was probably the hardest part of this exercise. This was mostly due to Bootstrap's great documentation and useful examples (viewed by inspecting the markup). Below are some screenshots of the old screens and new ones.

Check out the full set on Flickr if you'd like a closer look.

Even though I like the look of the old UI, I can't help but think a lot of the themes are designed for blogs and content sites, not webapps. The old Wufoo forms were a lot better looking though. And if you're going to develop kick-ass webapps, you need to make them look good. Bootstrap goes a long way in doing this, but it certainly doesn't replace a good UX Designer. Bootstap simply helps you get into HTML5-land, start using CSS3 and it takes the pain out of making things work cross-browser. Its fluid layouts and responsive web design seems to work great for business applications, which I'm guessing AppFuse is used for the most.

I can't thank the Bootstrap developers enough for helping me make this all look good. With Bootstrap 2 dropping this week, I can see myself using this more and more on projects. In the near future, I'll be helping integrate Bootstrap into AppFuse's Tapestry 5 and JSF versions.

What do you think of this CSS change? Do you change your CSS and layout a fair bit when starting with AppFuse archetypes? What can we do to make AppFuse apps look better out-of-the-box?

Update: I updated AppFuse to the final Bootstrap 2.0 release. Also, Johannes Geppert wrote a Struts 2 Bootstrap Plugin. I hope to integrate this into AppFuse in the near future.

Play 2.0, A web framework for a new era

2011-11-16T05:58:09-07:00

This week, I'm in Antwerp, Belgium for the annual Devoxx conference. After traveling 21 hours door-to-door yesterday, I woke up and came to the conference to attend some talks on Play and PhoneGap. I just got out of the session on Play 2.0, which was presented by Sadek Drobi and Guillaume Bort. Below are my notes from this presentation.

The Play 2.0 beta is out! You can read more about this release on the mailing list. This beta includes native support for both Scala and Java, meaning you can use both in the same project. The release also bundles Akka and SBT by default.

In other news, Play 2.0 is now part of the Typesafe Stack. Typesafe is the Scala company, started by the founder of Scala (Martin Odersky) and the founder of Akka (Jonas Bonér). Guillaume is also joining the Typesafe Advisory Board.

Sadek and Guillaume both work at zenexity, where Play is the secret weapon for the web applications they've built for the last decade. Play was born in the real world. They kept listening to the market to see what they should add to the project. At some point, they realized they couldn't keep adding to the old model and they needed to create something new.

The web has evolved from static pages to dynamic pages (ASP, PHP). From there, we moved to structured web applications with frameworks and MVC. Then the web moved to Ajax and long-polling to more real-time, live features. And this changes everything.

Now we need to adapt our tools. We need to handle tremendous flows of data. Need to improve expressiveness for concurrent code. We need to pick the appropriate datastore for the problem (not only SQL). We need to integrate with rapidly-evolving client side technologies like JavaScript, CoffeeScript, and Dart. We need to use elastic deployment that allows scaling up and scaling down.

zenexity wanted to integrated all of these modern web needs into Play 2.0. But they also wanted to keep Play approachable. They wanted to maintain fast turnaround so you can change your code and hit reload to see the changes. They wanted to keep it as a full stack framework with support for JSON, XML, Web Services, Jobs, etc. And they wanted to continue to use and conventions over configuration.

At this point, Guillaume did a Play 2.0 Beta demo, show us how it uses SBT and has a console so everything so it runs really fast. You can have both Scala and Java files in the same project. Play 2.0 templates are based on Scala, but you don't need to know Scala to use them. You might have to learn how to write a for loop in Scala, but it's just a subset of Scala for templates and views. SBT is used for the build system, but you don't have to learn or know SBT. All the old play commands still work, they're just powered by a different system.

After the demo, Sadek took over and started discussing the key features of Play 2.0.

To handle tremendous amounts of data, you need to do chunking of data and be able to process a stream of data, not just wait until it's finished. Java's InputStream is outdated and too low level. Its read() method reads the next byte of data from the input and this method can block until input data is available.

To solve this, Play includes a reactive programming feature, which they borrowed from Haskell. It uses Iteratee/Enumerator IO and leverages inversion of control (not like dependency injection, but more like not micro-managing). The feature allows you to have control when you need it so you don't have to wait for the input stream to complete. The Enumerator is the component that sends data and the Iteratee is the component that receives data. The Iteratee does incremental processing and can tell the Enumerator when it's done. The Iteratee can also send back a continuation, where it tells the Enumerator it wants more data and how to give it. With this paradigm, you can do a lot of cool stuff without consuming resources and blocking data flow.

Akka is an actor system that is a great model for doing concurrent code. An Actor could be both an Enumerator and an Iteratee. This vastly improves the expressiveness for concurrent code. For example, here's how you'd use Akka in Play:

def search(keyword: String) = Action {
  AsyncResult {
    // do something with result
  }
}

Play does not try to abstract data access because datastores are different now. You don't want to think of everything as objects if you're using something like MongoDB or navigating a Social Graph. Play 2.0 will provide some default modules for the different datastores, but they also expect a lot of contributed modules. Anorm will be the default SQL implementation for Play Scala and Ebean will be the default ORM implementation for Play Java. The reason they've moved away from Hibernate is because they needed something that was more stateless.

On the client side, there's so many technologies (LESS, CoffeeScript, DART, Backbone.js, jQuery, SASS), they didn't want to bundle any because they move too fast. Instead, there's plugins you can add that help you leverage these technologies. There's a lot of richness you can take advantage of on the client side and you need to have the tools for that.

Lastly, there's a new type of deployment: container-less deployment to the cloud. Akka allows you to distribute your jobs across many servers and Heroku is an implementation of elastic deployment that has built-in support for Play.

They've explained what they tried to design and the results of this new, clean architecture have been suprising. Side effects include: type-safety everywhere for rock-solid applications. There's an awesome performance boost from Scala. There's easier integration with existing projects via SBT. And it only takes 10 lines of code to develop an HTTP Server that responds to web requests.

The memory consumption is amazing: only 2MB of heap is used when a Play 2.0 app is started. Tests on Guillaume's laptop have shown that it can handle up to 40,000 requests per second, without any optimization of the JVM. Not only that, but after the requests subside, garbage collection cleans up everything and reduces the memory consumption back to 2MB.

At this point, Guillaume did another demo, showing how everything is type-safe in 2.0, including the routes file. If you mistype (or comment one out) any routes, the compiler will find it and notify you. Play 2.0 also contains a compiled assets feature. This allows you to use Google's Closure Compiler, CoffeeScript and LESS. If you put your LESS files in app/assets/stylesheets, compilation errors will show up in your browser. If you put JavaScript files in app/assets/javascripts, the Closure compiler will be used and compilation errors will show up in your browser.

Play 2.0 ships with 3 different sample applications, all implemented in both Java and Scala. HelloWorld is more than just text in a browser, it includes a form that shows how validation works. Another app is computer-database. When Guillaume started it, we saw how evolutions were used to create the database schema from the browser. The Play Team has done their best to make the development process a browser-based experience rather than having to look in your console. The computer-database is a nice example of how to do CRUD and leverages Twitter's Bootstrap for its look and feel.

The last sample application is zentasks. It uses Ajax and implements security so you can see how to create a login form. It uses LESS for CSS and CoffeeScript and contains features like in-place editing. If you'd like to see any of these applications in action, you can stop by the Typesafe booth this week at Devoxx.

Unfortunately, there will be no migrating path for Play 1.x applications. The API seems very similar, but there are subtle changes that make this difficult. The biggest thing is how templating has changed from Groovy to Scala. To migrate from 1.2.x would be mostly a copy/paste, modify process. There are folks working on getting Groovy templates working in 2.0. The good news is zenexity has hundreds of 1.x applications in production, so 1.x will likely be maintained for many years.

Summary
This was a great talk on what's new in Play 2.0. I especially like the native support for LESS and CoffeeScript and the emphasis on trying to keep developers using two tools: their editor and the browser. The sample apps look great, but the documentation look sparse. I doubt I'll get a chance to migrate my Play 1.2.3 app to 2.0 this month, but I hope to try migrating sometime before the end of the year.

More Scalate Goodness for Play

2011-11-07T14:07:40-07:00

This article is the 6th in a series on about my adventures developing a web application with HTML5, Play Scala, CoffeeScript and Jade. Previous articles can be found at:

Last week, I wrote about my adventures with Anorm and mentioned I'd made some improvements to Scalate Play interoperability. First of all, I've been using a Scalate trait and ScalateTemplate class to render Jade templates in my application. I described this setup in my first article on Scalate and Play.

Adding SiteMesh Features and Default Variables
When I started making my app look good with CSS, I started longing for a feature I've used in SiteMesh. That is, to have a body id or class that can identify the page and allow per-page CSS rules. To do this with SiteMesh, you'd have something like the following in your page:

  
<body id="signup"/>

And then read it in your decorator:

<body<decorator:getProperty property="body.class" writeEntireProperty="true"/>>

As I started looking into how to do this, I came across Play Scala's ScalaController and how it was populating Play's default variables (request, response, flash, params, etc.). Based on this newfound knowledge, I added a populateRenderArgs() method to set all the default variables and my desired bodyClass variable.

def populateRenderArgs(args: (Symbol, Any)*): Map[String, Any] = {
  val renderArgs = Scope.RenderArgs.current();

  args.foreach {
    o =>
      renderArgs.put(o._1.name, o._2)
  }

  renderArgs.put("session", Scope.Session.current())
  renderArgs.put("request", Http.Request.current())
  renderArgs.put("flash", Scope.Flash.current())
  renderArgs.put("params", Scope.Params.current())
  renderArgs.put("errors", validationErrors)
  renderArgs.put("config", Play.configuration)

  // CSS class to add to body
  renderArgs.put("bodyClass", Http.Request.current().action.replace(".", " ").toLowerCase)
  renderArgs.data.toMap
}

implicit def validationErrors:Map[String,play.data.validation.Error] = {
  import scala.collection.JavaConverters._
  Map.empty[String,play.data.validation.Error] ++ 
    Validation.errors.asScala.map( e => (e.getKey, e) )
}

After adding this method, I was able to access these values in my templates by defining them at the top:

-@ val bodyClass: String 
-@ val params: play.mvc.Scope.Params
-@ val flash: play.mvc.Scope.Flash

And then reading their values in my template:

body(class=bodyClass)
...
- if (flash.get("success") != null) {
  div(class="alert-message success" data-alert="alert")
    a(class="close" href="#") &×
    | #{flash.get("success")}
- }
...
  fieldset
    legend Leave a comment →
    div.clearfix
      label(for="author") Your name:
      input(type="text" name="author" class="xlarge" value={params.get("author")})
    div.clearfix
      label(for="content") Your message:
      textarea(name="content" class="xlarge") #{params.get("content")}
    div.actions
      button(type="submit" class="btn primary") Submit your comment
      button(type="reset" class="btn") Cancel

For a request like Home/index, the body tag is now rendered as:

<body class="home index">

This allows you to group CSS styles by Controller names as well as by method names.

Next, I started developing forms and validation logic. I quickly discovered I needed an action() method like the one defined in ScalaTemplate's TemplateMagic class.

def action(action: => Any) = {
  new play.mvc.results.ScalaAction(action).actionDefinition.url
}

Since TemplateMagic is an inner class, I determined that copying the method into my ScalateTemplate class was the easiest workaround. After doing this, I was able to import the method and use it in my templates.

-import controllers.ScalateTemplate._
...
form(method="post" class="form-stacked" id="commentForm"
     action={action(controllers.Profile.postComment(workout._1.id()))})

After getting the proper URL written into my form's action attribute, I encountered a new problem. The Play Scala Tutorial explains validation flow as follows:

if (Validation.hasErrors) {
  show(postId)
} else {
  Comment.create(Comment(postId, author, content))
  Action(show(postId))
}

However, when I had validation errors, I end up with the following error:

Could not load resource: [Timeline/postComment.jade]

To fix this, I added logic to my Scalate trait that looks for a "template" variable before using Http.Request.current().action.replace(".", "/") for the name. After making this change, I was able to use the following code to display validation errors.

if (Validation.hasErrors) {
  renderArgs.put("template", "Timeline/show")
  show(postId)
} else {
  Comment.create(Comment(postId, author, content))
  Action(show(postId))
}

Next, I wanted to give child pages the ability to set content in parent pages. With SiteMesh, I could use the <content> tag as follows:

<content tag="underground">
  HTML goes here
</content>

This HTML could then be retrieved in the decorator using the <decorator:getProperty> tag:

<decorator:getProperty property="page.underground"/>

With Scalate, I found it equally easy using the captureAttribute() method. For example, here's how I captured a list of an athlete's workouts for display in a sidebar.

- captureAttribute("sidebar")
  - Option(older).filterNot(_.isEmpty).map { workouts =>
    .older-workouts
      h3
        | Older workouts
        span.from from this app
      - workouts.map { workout =>
        - render("workout.jade", Map('workout -> workout, 'mode -> "teaser"))
      - }
  - }
- }

Then in my layout, I was able to retrieve this and display it. Below is a snippet from the layout I'm using (copied from Twitter's Bootstrap example). You can see how the sidebar is included in the .span4 at the end.

-@ val sidebar: String = ""
...
.container
  .content
    .page-header
      h1
        = pageHeader
        small
          = pageTagline
    .row
      .span10
        !~~ body
      .span4
        = unescape(sidebar)
  footer

View vs. Render in Scalate
In the sidebar code above, you might notice the render() call. This is the Scalate version of server-side includes. It works well, but there's also a view() shortcut you can use if you want to have templates for rendering your model objects. I quickly discovered it might be difficult to use this feature in my app because my object was Option[(models.Workout, models.Athlete, Seq[models.Comment])] instead of a simple object. You can read the view vs. render thread on the Scalate Google Group if you're interested in learning more.

Scalate as a Module
The last enhancement I attempted to make was to put Scalate support into a Play module. At first, I tried overriding Play's Template class but ran into compilation issues. Then Guillaume Bort (Play's lead developer) recommended I stick with the trait approach and I was able to get everything working. I looked at the outdated play-scalate module to figure out how to add Scala support to build.xml and copied its 500.scaml page for error reporting.

In order to get line-precise error reporting working, I had to wrap a try/catch around calling Scalate's TemplateEngine.layout() method. Again, most of this code was copied from the outdated play-scalate module.

case class Template(name: String) {
  
  def render(args: (Symbol, Any)*) = {
    val argsMap = populateRenderArgs(args: _*)
    
    val buffer = new StringWriter()
    var context = new DefaultRenderContext(name, scalateEngine, new PrintWriter(buffer))
    
    try {
      val templatePath = new File(Play.applicationPath+"/app/views","/"+name).toString
        .replace(new File(Play.applicationPath+"/app/views").toString,"")
      scalateEngine.layout(templatePath + scalateType, argsMap)
    } catch {
      case ex:TemplateNotFoundException => {
        if(ex.isSourceAvailable) {
          throw ex
        }
        val element = PlayException.getInterestingStrackTraceElement(ex)
        if (element != null) {
           throw new TemplateNotFoundException(name, 
             Play.classes.getApplicationClass(element.getClassName()), element.getLineNumber());
        } else {
           throw ex
        }
      }  
      case ex:InvalidSyntaxException => handleSpecialError(context,ex)
      case ex:CompilerException => handleSpecialError(context,ex)
      case ex:Exception => handleSpecialError(context,ex)
    } finally {
      if (buffer.toString.length > 0)
        throw new ScalateResult(buffer.toString,name)
    }
  }
}
...
private def handleSpecialError(context:DefaultRenderContext,ex:Exception) {
  context.attributes("javax.servlet.error.exception") = ex
  context.attributes("javax.servlet.error.message") = ex.getMessage
  try {
    scalateEngine.layout(scalateEngine.load(errorTemplate), context)
  } catch {
    case ex:Exception =>
      // TODO use logging API from Play here...
      println("Caught: " + ex)
      ex.printStackTrace
  }
}

private def errorTemplate:String = {
  val fullPath = new File(Play.applicationPath,"/app/views/errors/500.scaml").toString 
  fullPath.replace(new File(Play.applicationPath+"/app/views").toString,"")
}

Once I had this in place, error messages from Scalate are much better. Not only do I see the error in my browser, but I can click on the offending line to open it directly in TextMate.

I've published my play-scalate module on GitHub so others can try it out. To give it a whirl, add the following to your dependencies.yml:

    - upgrades -> play-scalate 0.1

repositories:
    - upgrades:
        type: http
        artifact: "http://static.raibledesigns.com/[module]-[revision].zip"
        contains:
            - upgrades -> *

Then add with play.modules.scalate.Scalate to your controllers and call the render() method.

Summary
After using Scalate and Play for almost 3 months, I'm really enjoying the combination. When I first integrated Scalate with a simple trait, the error messages were always in the console. Now that I've borrowed some smarts from Play's ScalaController and play-scalate's error reporting, I feel like it's practically a built-in solution. I was easily able to integrate my desired SiteMesh features and it even allows reusable template blocks. In the end, it's just Scala and Scalate does a good job of allowing you to leverage that.

Other thoughts:

If you're writing a lot of Jade and familiar with HTML, Don Park's html2jade is a great tool that comes with Scalate support.
I'm really enjoying writing CSS with LESS, particularly the ability to nest rules and have programming features. The only issue I've seen is IntelliJ's LESS plugin only does code-completion for variables rather than CSS values.
The Play Framework Cookbook is a great reference for learning how to write modules. Not only does it explain how to create modules, it has some great real-world examples for doing bytecode enhancement, implementing message queues, using Solr and how to do production monitoring.

If this series of articles has intrigued you and you'll be at Devoxx next week, you should stop by my talk on Thursday afternoon. In addition, there's several other Play talks at Devoxx and a possible meetup on Wednesday. Check out the Devoxx, anyone? thread for more information.

Update: There's one thing I forgot to mention about the Play Scalate Module. When I had Scalate integrated in my app with a trait, I only included the scalate-core and scalate-util JARs in dependencies.yml:

- org.fusesource.scalate -> scalate-core 1.5.2-scala_2.8.1:
    transitive: false
- org.fusesource.scalate -> scalate-util 1.5.2-scala_2.8.1:
    transitive: false

However, when I created the play-scalate module, I allowed more dependencies.

- org.fusesource.scalate -> scalate-core 1.5.2-scala_2.8.1:
    exclude:
        - javax.servlet -> *
        - com.sun.jersey -> *
        - org.osgi -> *
- org.fusesource.scalate -> scalate-util 1.5.2-scala_2.8.1

Because Scalate depends on Logback, debug messages started showing up in my console. To fix this, I created conf/logback.xml in my project and filled it with the following XML.

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
      <encoder>
          <pattern>%msg%n</pattern>
      </encoder>
  </appender>

  <root level="info">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

This reduces the logging and allows me to increase Scalate's logging if I ever have the need.

Developing with HTML5, CoffeeScript and Twitter's Bootstrap

2011-10-20T14:47:36-06:00

This article is the fourth in a series about my adventures developing a Fitness Tracking application with HTML5, Play Scala, CoffeeScript and Jade. Previous articles can be found at:

Developing Features
After getting my desired infrastructure setup, I started coding like a madman. The first feature I needed was a stopwatch to track the duration of a workout, so I started writing one with CoffeeScript. After spending 20 minutes playing with dates and setTimeout, I searched and found a stopwatch jQuery plug-in. I added this to my app, deployed it to Heroku, brought up the app on my iPhone 3G, clicked Start and started riding my bike to work.

When I arrived, I unlocked my phone and discovered that the time had stopped. At first, I thought this was a major setback. My disappointed disappeared when I found a Super Neat JavaScript Stopwatch and Kåre Byberg's version that worked just fine. This stopwatch used setTimeout, so by keeping the start time, the app on the phone would catch up as soon as you unlocked it. I ported Kåre's script to CoffeeScript and rejoiced in my working stopwatch.

# Created by Kåre Byberg © 21.01.2005. Please acknowledge if used 
# on other domains than http://www.timpelen.com.
# Ported to CoffeeScript by Matt Raible. Also added hours support.
flagClock = 0
flagStop = 0
stopTime = 0
refresh = null
clock = null

start = (button, display) ->
  clock = display
  startDate = new Date()
  startTime = startDate.getTime()
  if flagClock == 0
    $(button).html("Stop")
    flagClock = 1
    counter startTime, display
  else
    $(button).html("Start")
    flagClock = 0
    flagStop = 1

counter = (startTime) ->
  currentTime = new Date()
  timeDiff = currentTime.getTime() - startTime
  timeDiff = timeDiff + stopTime  if flagStop == 1
  if flagClock == 1
    $(clock).val formatTime timeDiff, ""
    callback = -> counter startTime
    refresh = setTimeout callback, 10
  else
    window.clearTimeout refresh
    stopTime = timeDiff

formatTime = (rawTime, roundType) ->
  if roundType == "round"
    ds = Math.round(rawTime / 100) + ""
  else
    ds = Math.floor(rawTime / 100) + ""
  sec = Math.floor(rawTime / 1000)
  min = Math.floor(rawTime / 60000)
  hour = Math.floor(rawTime / 3600000)
  ds = ds.charAt(ds.length - 1)
  start() if hour >= 24
  sec = sec - 60 * min + ""
  sec = prependZeroCheck sec
  min = min - 60 * hour + ""
  min = prependZeroCheck min
  hour = prependZeroCheck hour
  hour + ":" + min + ":" + sec + "." + ds

prependZeroCheck = (time) ->
  time = time + "" # convert from int to string
  unless time.charAt(time.length - 2) == ""
    time = time.charAt(time.length - 2) + time.charAt(time.length - 1)
  else
    time = 0 + time.charAt(time.length - 1)

reset = ->
  flagStop = 0
  stopTime = 0
  window.clearTimeout refresh
  if flagClock == 1
    resetDate = new Date()
    resetTime = resetDate.getTime()
    counter resetTime
  else
    $(clock).val "00:00:00.0"

@StopWatch = {
  start: start
  reset: reset
}

The Scalate/Jade template to render this stopwatch looks as follows:

script(type="text/javascript" src={uri("/public/javascripts/stopwatch.coffee")})

#display
  input(id="clock" class="xlarge" type="text" value="00:00:00.0" readonly="readonly")
#controls
  button(id="start" type="button" class="btn primary") Start
  button(id="reset" type="button" class="btn :disabled") Reset

:plain
  <script type="text/coffeescript">
    $(document).ready ->
      $('#start').click ->
        StopWatch.start this, $('#clock')

      $('#reset').click ->
        StopWatch.reset()
  </script>

Next, I wanted to create a map that would show your location. For this, I used Merge Design's HTML 5 Geolocation Demo as a guide. The HTML5 Geo API is pretty simple, containing only three methods:

// Gets the users current position
navigator.geolocation.getCurrentPosition(successCallback,
                                         errorCallback,
                                         options);
// Request repeated updates of position
watchId = navigator.geolocation.watchPosition(successCallback, errorCallback);

// Cancel the updates
navigator.geolocation.clearWatch(watchId);

After rewriting the geolocation example in CoffeeScript, I ended up with the following code in my map.coffee script. You'll notice it uses Google Maps JavaScript API to show an actual map with a marker.

# Geolocation with HTML 5 and Google Maps API based on example from maxheapsize: 
# http://maxheapsize.com/2009/04/11/getting-the-browsers-geolocation-with-html-5/
# This script is by Merge Database and Design, http://merged.ca/ -- if you use some, 
# all, or any of this code, please offer a return link.

map = null
mapCenter = null
geocoder = null
latlng = null
timeoutId = null

initialize = ->
  if Modernizr.geolocation
    navigator.geolocation.getCurrentPosition showMap

showMap = (position) ->
  latitude = position.coords.latitude
  longitude = position.coords.longitude
  mapOptions = {
    zoom: 15,
    mapTypeId: google.maps.MapTypeId.ROADMAP
  }
  map = new google.maps.Map(document.getElementById("map"), mapOptions)
  latlng = new google.maps.LatLng(latitude, longitude)
  map.setCenter(latlng)

  geocoder = new google.maps.Geocoder()
  geocoder.geocode({'latLng': latlng}, addAddressToMap)

addAddressToMap = (results, status) ->
  if (status == google.maps.GeocoderStatus.OK) 
    if (results[1]) 
      marker = new google.maps.Marker({
          position: latlng,
          map: map
      })
      $('#location').html('Your location: ' + results[0].formatted_address)
  else 
    alert "Sorry, we were unable to geocode that address."

start = ->
  timeoutId = setTimeout initialize, 500

reset = ->
  if (timeoutId)
    clearTimeout timeoutId

@Map = {
  start: start
  reset: reset
}

The template to show the map is a mere 20 lines of Jade:

script(type="text/javascript" src="//www.google.com/jsapi")
script(type="text/javascript" src="//maps.googleapis.com/maps/api/js?sensor=false")

:css
  .demo-map {
    border: 1px solid silver;
    height: 200px;
    margin: 10px auto;
    width: 280px;
  }

#map(class="demo-map")

p(id="location")
  span(class="label success") New
  | Fetching your location with HTML 5 geolocation...

script(type="text/javascript" src={uri("/public/javascripts/map.coffee")})
:javascript
    Map.start();

The last two features I wanted were 1) distance traveled and 2) drawing the route taken on the map. For this I learned from A Simple Trip Meter using the Geolocation API. As I was beginning to port the JS to CoffeeScript, I thought, "there's got to be a better way." I searched and found Js2coffee to do most of the conversion for me. If you know JavaScript and you're learning CoffeeScript, this is an invaluable tool.

I tried out the trip meter that evening on a bike ride and noticed it said I'd traveled 3 miles when I'd really gone 6. I quickly figured out it was only calculating start point to end point and not taking into account all the turns in between. To view what was happening, I integrated my odometer.coffee with my map using Google Maps Polylines. Upon finishing the integration, I discovered two things, 1) HTML5 geolocation was highly inaccurate and 2) geolocation doesn't run in the background.

I was able to solve the first problem by passing in {enableHighAccuracy: true} to navigator.geolocation.watchPosition(). Below are two screenshots showing before high accuracy and after. Both screenshots are from the same two-block walk.

The second issue is a slight show-stopper. PhoneGap might be able to solve the problem, but I'm currently using a workaround → turning off auto-lock and keeping Safari in the foreground.

Making it look good
After I got all my desired features developed, I moved onto making the app look good. I started by using SASS for my CSS and installed Play's SASS module. I then switched to LESS when I discovered and added Twitter's Bootstrap to my project. At first I used Play's LESS module (version 0.3), but ran into compilation issues. I then tried Play's GreenScript module, but gave up on it when I found it was incompatible with the CoffeeScript module. Switching back to the LESS module and using the "0.3.compatibility" version solved all remaining issues.

You might remember that I integrated HTML5 Boilerplate and wondering why I have both Bootstrap and Boilerplate in my project. At this point, I don't think Boilerplate is needed, but I've kept it just in case it's doing something for HTML5 cross-browser compatibility. I've renamed its style.css to style.less and added the following so it has access to Bootstrap's variables.

/* Variables from Bootstrap */
@import "libs/variables.less";

Then I made my app look a lot better with layouts, stylish forms, a fixed topbar and alerts. For example, here's the CoffeeScript I wrote to display geolocation errors:

geolocationError = (error) ->
  msg = 'Unable to locate position. '
  switch error.code
    when error.TIMEOUT then msg += 'Timeout.'
    when error.POSITION_UNAVAILABLE then msg += 'Position unavailable.'
    when error.PERMISSION_DENIED then msg += 'Please turn on location services.'
    when error.UNKNOWN_ERROR then msg += error.code
  $('.alert-message').remove()
  alert = $('<div class="alert-message error fade in" data-alert="alert">')
  alert.html('<a class="close" href="#">×</a>' + msg);
  alert.insertBefore($('.span10'))

Then I set about styling up the app so it looked good on a smartphone with CSS3 Media Queries. Below is the LESS code I used to hide elements and squish the widths for smaller devices.

@media all and (max-device-width: 480px) {
  /* hide scrollbar on mobile */
  html { overflow-y:hidden }
  /* hide sidebar on mobile */
  .home .span4, .home .page-header, .topbar form {
    display: none
  }
  .home .container {
    width: 320px;
  } 
  .about {
    .container, .span10 {
      width: 280px;
    }
    .span10 {
      padding-top: 0px;
    }
  }

Tools
In the process of developing a stopwatch, odometer, displaying routes and making everything look good, I used a number of tools. I started out primarily with TextMate and its bundles for LESS, CoffeeScript and Jade. When I started writing more Scala, I installed the Scala TextMate Bundle. When I needed some debugging, I switched to IntelliJ and installed its Scala plugin. CoffeeScript, LESS and HAML plugins (for Jade) were already installed by default. I also used James Ward's Setup Play Framework with Scala in IntelliJ.

Issues
I think it's obvious that my biggest issue so far is the fact that a webapp can't multitask in the background like a native app can. Beyond that, there's accuracy issues with HTML5's geolocation that I haven't seen in native apps.

I also ran into a caching issue when calling getCurrentPosition(). It only worked the first time and I had to refresh my browser to get it to work again. Strangely enough, this only happened on my desktop (in Safari and Firefox) and worked fine on my iPhone. Unfortunately, it looks like PhoneGap has issues similar to this.

My workaround for no webapp multitasking is turning off auto-lock and leaving the browser in the foreground while I exercise. The downside to this is it really drains the battery quickly (~ 3 hours). I constantly have to charge my phone if I'm testing it throughout the day. The testing is a real pain too. I have to deploy to Heroku (which is easy enough), then go on a walk or bike ride. If something's broke, I have to return home, tweak some things, redeploy and go again. Also, there's been a few times where Safari crashes halfway through and I lose all the tracking data. This happens with native apps too, but seemingly not as often.

If you'd like to try the app on your mobile phone and see if you experience these issues, checkout play-more.com.

Summary
Going forward, there's still more HTML5 features I'd like to use. In particular, I'd like to play music while the fitness tracker is running. I'd love it if cloud music services (e.g. Pandora or Spotify) had an API I could use to play music in a webapp. Soundcloud might be an option, but I've also thought of just uploading some MP3s and playing them with the <audio> tag.

I've really enjoyed developing with all these technologies and haven't experienced much frustration so far. The majority has come from integrating Scalate into Play, but I've resolved most problems. Next, I'll talk about how I've improved Play's Scalate support and my experience working with Anorm.

Integrating OAuth with AppFuse and its REST API

2011-07-05T10:56:48-06:00

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
Integrate Spring Security and OAuth
Authenticate and Retrieve Data with Client

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Java Web Application Security - Part V: Penetrating with Zed Attack Proxy

2011-06-21T07:45:41-06:00

Web Application Security is an important part of developing applications. As developers, I think we often forget this, or simply ignore it. In my career, I've learned a lot about web application security. However, I only recently learned and became familiar with the rapidly growing "appsec" industry.

I found a disconnect between what appsec consultants were selling and what I was developing. It seemed like appsec consultants were selling me fear, mostly because I thought my apps were secure. So I set out on a mission to learn more about web application security and penetration testing to see if my apps really were secure. This article is part of that mission, as are the previous articles I've written in this series.

When I first decided I wanted to do a talk on Webapp Security, I knew it would be more interesting if I showed the audience how to hack and fix an application. That's why I wrote it into my original proposal:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

At the time, I hadn't done much webapp pentesting. You can tell this from the fact that I mentioned WebGoat as the pentesting tool. From WebGoat's Project page:

WebGoat is a deliberately insecure J2EE web application maintained by OWASP designed to teach web application security lessons. In each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the WebGoat application. For example, in one of the lessons the user must use SQL injection to steal fake credit card numbers. The application is a realistic teaching environment, providing users with hints and code to further explain the lesson.

What I really meant to say and use was Zed Attack Proxy, also known as OWASP ZAP. ZAP is a Java Desktop application that you setup as a proxy for your browser, then use to find vulnerabilities in your application. This article explains how you can use ZAP to pentest a web applications and fix its vulnerabilities.

The application I'll be using in this article is the Ajax Login application I've been using throughout this series. I think it's great that projects like Damn Vulnerable Web App and WebGoat exist, but I wanted to test one that I think is secure, rather than one I know is not secure. In this particular example, I'll be testing the Spring Security implementation, since that's the framework I most often use in my open source projects.

Zed Attack Proxy Tutorial

Download and Run the Application
Install and Configure ZAP
Perform a Scan
Fix Vulnerabilities
Summary

Download and Run the Application
To begin, download the application and expand it on your hard drive. This app is the completed version of the Ajax Login application referenced in Java Web Application Security - Part II: Spring Security Login Demo. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and you need to login to do anything.

Install and Configure ZAP
The Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. Download the latest version (I used 1.3.0) and install it on your system. After installing, launch the app and change the proxy port to 9000 (Tools > Options > Local Proxy). Next, configure your browser to proxy requests through port 9000 and allow localhost requests to be proxied. I used Firefox 4 (Preferences > Advanced > Network > Connection Settings). When finished, your proxy settings should look like the following screenshot:

Another option (instead of removing localhost) is to add an entry to your hosts file with your production domain name. This is what I've done for this demo.

127.0.0.1       demo.raibledesigns.com

I've also configured Apache to proxy requests to Jetty with the following mod_proxy settings in my httpd.conf:

<IfModule mod_proxy.c>
    ProxyRequests Off 
    ProxyPreserveHost Off 

    <VirtualHost *:80>
       ProxyPass  /  http://localhost:8080/
    </VirtualHost>

    <VirtualHost *:443>
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile "/etc/apache2/ssl.key/server.crt"
        SSLCertificateKeyFile "/etc/apache2/ssl.key/server.key"

        ProxyPass  /  https://localhost:8443/
    </VirtualHost>
</IfModule>

Perform a Scan
Now you need to give ZAP some data to work with. Using Firefox, I navigated to http://demo.raibledesigns.com and browsed around a bit, listing users, added a new one and deleted an existing one. After doing this, I noticed a number of flags in the ZAP UI under Sites. I then right-clicked on each site (one for http and one for https) and selected Attack > Active Scan site. You should be able to do this from the "Active Scan" tab at the bottom of ZAP, but there's a bug when the URLs are the same. After doing this, I received a number of alerts, ranging from high (cross-site scripting) to low (password autocomplete). The screenshot below shows the various issues.

Now let's take a look at how to fix them.

Fix Vulnerabilities
One of the things not mentioned by the scan, but #1 in Seven Security (Mis)Configurations in Java web.xml Files, is Custom Error Pages Not Configured. Custom error pages are configured in this app, but error.jsp contains the following code:

<% if (exception != null) { %>
    <% exception.printStackTrace(new java.io.PrintWriter(out)); %>
<% } else { %>
    Please check your log files for further information.
<% } %>

Stack traces can be really useful to an attacker, so it's important to start by removing the above code from src/main/webapp/error.jsp.

The rest of the issues have to do with XSS, autocomplete, and cookies. Let's start with the easy ones. Fixing autocomplete is easy enough; simply changed the HTML in login.jsp and userform.jsp to have autocomplete="off" as part of the <form> tag.

Then modify web.xml so http-only and secure cookies are used. While you're at it, add session-timeout and tracking-mode as recommended by the aforementioned web.xml misconfigurations article.

<session-config>
    <session-timeout>15</session-timeout>
    <cookie-config>
        <http-only>true</http-only>
        <secure>true</secure>
    </cookie-config>
    <tracking-mode>COOKIE</tracking-mode>
</session-config>

Next, modify Spring Security's Remember Me configuration so it uses secure cookies. To do this, add use-secure-cookies="true" to the <remember-me> element in security.xml.

<remember-me user-service-ref="userService" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"
             use-secure-cookie="true"/>

Unfortunately, Spring Security doesn't support HttpOnly cookies, but will in a future release.

The next issue to solve is disabling directory browsing. You can do this by copying Jetty's webdefault.xml (from the org.eclipse.jetty:jetty-webapp JAR) into src/test/resources and changing its "dirAllowed" <init-param> to false:

<servlet>
  <servlet-name>default</servlet-name>
  <servlet-class>org.mortbay.jetty.servlet.DefaultServlet</servlet-class>
  <init-param>
    <param-name>acceptRanges</param-name>
    <param-value>true</param-value>
  </init-param>
  <init-param>
    <param-name>dirAllowed</param-name>
    <param-value>false</param-value>
  </init-param>
  <init-param>

You'll also need to modify the plugin's configuration to point to this file by adding it to the <webAppConfig> section in pom.xml.

<configuration>
    <webAppConfig>
        <contextPath>/</contextPath>
        <defaultsDescriptor>src/test/resources/webdefault.xml</defaultsDescriptor>
    </webAppConfig>

Of course, if you're running in production you'll want to configure this in your server's settings rather than in your pom.xml file.

Next, I set out to fix secure page browser cache issues. I had the following settings in my SiteMesh decorator:

<meta http-equiv="Cache-Control" content="no-store"/>
<meta http-equiv="Pragma" content="no-cache"/>

However, according to ZAP, the first meta tag should have "no-cache" instead of "no-store", so I changed it to "no-cache".

After making all these changes, I created a new ZAP session and ran an active scan on both sites again. Below are the results:

I believe the first issue (parameter tampering) is because I show the error page when a duplicate user exists. To fix this, I changed UserFormController so it catches a UserExistsException and sends the user back to the form.

try {
    userManager.saveUser(user);
} catch (UserExistsException uex) {
    result.addError(new ObjectError("user", uex.getMessage()));
    return "userform";
}

However, this still doesn't seem to cause the alert to go away. This is likely because I'm not filtering/escaping HTML when it's first submitted. I believe the best solution for this would be to use something like OWASP's ESAPI to filter parameter values. However, I was unable to find integration with Spring MVC's data binding, so I decided not to try and fix this vulnerability.

Finally, I tried to disable jsessionid in URLs using suggestions from Stack Overflow. The previous setting in web.xml (<tracking-mode>COOKIE</tracking-mode>) should do this, but it doesn't seem to work with Jetty 8. The other issues (secure page browser cache, HttpOnly cookies and secure cookies), I was unable to solve. The last two are issues caused by Spring Security as far as I can tell.

Summary
In this article, I've shown you how to pentest a web application using Firefox and OWASP's Zed Attack Proxy (ZAP). I found ZAP to be a nice tool for figuring out vulnerabilities, but it'd be nice if it had a "retest" feature to see if you fixed an issue for a particular URL. It does have a "resend" feature, but running it didn't seem to clear alerts after I'd fixed them.

The issues I wasn't able to solve seemed to be mostly related to frameworks (e.g. Spring Security and HttpOnly cookies) or servers (Jetty not using cookies for tracking). My suspicion is the Jetty issues are because it doesn't support Servlet 3 as well as it advertises. I believe this is fair; I am using a milestone release after all. I tried scanning http://demo.raibledesigns.com/ajax-login (which runs on Tomcat 7 at Contegix) and confirmed that no jsessionid exists.

Hopefully this article has helped you understand how to figure out security vulnerabilities in your web applications. I believe ZAP will continue to get more popular as developers become aware of it. If you feel ambitious and want to try and solve all of the issues in my Ajax Login application, feel free to fork it on GitHub.

If you're interested in talking more about Webapp Security, please leave a comment, meet me at Jazoon later this week or let's talk in July at Über Conf.

Java Web Application Security - Part III: Apache Shiro Login Demo

2011-05-26T16:43:22-06:00

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Store Users in a Database
Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

Java Web Application Security - Part II: Spring Security Login Demo

2011-05-13T09:20:51-06:00

Last week, I wrote a tutorial on how to implement Security in Java EE 6. This week, I'd like to show you how to implement the same features using Spring Security. Before I begin, I'd like to explain my reason for writing this article.

Last month, I presented a talk on Java Web Application Security at the Utah JUG (UJUG). As part of that presentation, I did a number of demos about how to implement security with Java EE 6, Spring Security and Apache Shiro. I told the audience that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense.

Today, I've finished the second screencast showing how to implement security with Spring Security. Below is the presentation (with the screencast embedded on slide 16) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Add Remember Me
Store Users in a Database
Summary

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a Spring context file that contains Spring Security's configuration. Create src/main/webapp/WEB-INF/security.xml and populate it with the contents below:

  <?xml version="1.0" encoding="UTF-8"?>
  <beans:beans xmlns="http://www.springframework.org/schema/security"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:beans="http://www.springframework.org/schema/beans"
               xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">

      <!-- New in Spring Security 3.1 -->
      <!-- <http pattern="/css/**" security="none"/> -->

      <http auto-config="true">
          <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN"/>
          <http-basic/>
      </http>

      <authentication-manager alias="authenticationManager">
          <authentication-provider>
              <password-encoder hash="sha"/>
              <user-service>
                  <user name="user" password="12dea96fec20593566ab75692c9949596833adc9" authorities="ROLE_USER"/>
                  <user name="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" authorities="ROLE_ADMIN"/>
              </user-service>
          </authentication-provider>
      </authentication-manager>

      <!-- Override userSecurityAdvice bean in appfuse-service to allow any role to update a user. -->
      <beans:bean id="userSecurityAdvice" class="org.appfuse.examples.webapp.security.UserSecurityAdvice"/>
  </beans:beans>

The last bean, userSecurityAdvice, is an aspect that's needed to override some behavior in AppFuse. You won't need this normally when implementing Spring Security.

Next, open src/main/webapp/WEB-INF/web.xml and add Spring's DelegatingFilterProxy:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

You don't need to add any dependencies in your pom.xml is because this project depends on AppFuse, which already contains these dependencies.

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Spring Security is a bit easier to configure than Java EE 6 out-of-the-box, mostly because it doesn't require you to configure your container.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Spring Security has a way to configure "logout" to match a URL and get rid of a class like LogoutController. Since it was already in the project, I don't cover that in this tutorial.

Force SSL
Spring Security allows you to switch between secure (https) and non-secure (http) protocols using a simple requires-channel attribute on the <intercept-url> element. Possible values are "http", "https" and "any". Add requires-channel="https" to your security.xml file:

<intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

You can switch between http and https protocols. With Java EE, you can only force https. You have to write a custom filter to switch back to http.
Redirecting to https actually works. With Java EE (on Jetty at least), a 403 is returned instead of redirecting the request.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a <form-login> element in security.xml's <http> element:

<http auto-config="true">
    <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>
    <form-login login-page="/login" authentication-failure-url="/login?error=true"
                login-processing-url="/j_security_check"/>
    <http-basic/>
</http>

You can leave the <http-basic> element since Spring Security is smart enough to serve up the form for browsers and use Basic Authentication for clients such as web services. The login.jsp page (that /login forwards to) already exists in the project, in the src/main/webapp directory. The forwarding is done by the UrlRewriteFilter with the following configuration in src/main/webapp/WEB-INF/urlrewrite.xml.

<rule>
    <from>/login</from>
    <to>/login.jsp</to>
</rule>

This JSP has 3 important elements: 1) a form that submits to "/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Add Remember Me
Remember Me is a feature you see in many web applications today. It's usually a checkbox on the login form that allows you to auto-login the next time you visit a site. This feature doesn't exist in Java EE security, but it does exist in Spring Security. To enable it, add the following just below <form-login> in security.xml:

<remember-me user-service-ref="userDao" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"/>

Next, open src/main/webapp/login.jsp and change the name of the "remember me" checkbox to be _spring_security_remember_me:

<input type="checkbox" name="_spring_security_remember_me" id="rememberMe"/>

After making these changes, you should be able to restart Jetty, go to http://localhost:8080/users, enter admin/adminjdbc, check the Remember Me checkbox and login. Then close your browser, and repeat the process. This time, you won't be prompted to login. For more information on this feature, see Spring Security's Remember Me documentation.

While storing usernames and passwords in a file is convenient for demos, it's not very real-world-ish. The next section shows you how to configure Spring Security to use a database for its user store.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a user-service-ref attribute to the <authentication-provider> element. You can also delete the <user-service> element.

<authentication-manager alias="authenticationManager">
    <authentication-provider user-service-ref="userDao">
        <password-encoder hash="sha"/>
    </authentication-provider>
</authentication-manager>

The "userDao" bean is provided by AppFuse and its UserDaoHibernate.java class. This class implements Spring Security's UserDetailsService interface. With Java EE, I had to configure a database connection and make sure the JDBC Driver was in my container's classpath. With Spring Security, you can talk to the database you already have configured in your application.

Of course, you could do this with Java EE too. One thing I neglected to show in my last tutorial was that 1) the app uses H2 and 2) I had to configure Java EE's database to be MySQL. This was because when I tried to access my H2 instance, I got an error about two threads trying to access it at once.

2011-05-13 08:47:29.081:WARN::UserRealm Java EE Login could not connect to database; will try later
org.h2.jdbc.JdbcSQLException: Database may be already in use: "Locked by another process". 
        Possible solutions: close all other connection(s); use the server mode [90020-154]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:167)
	at org.h2.message.DbException.get(DbException.java:144)
	at org.h2.store.FileLock.getExceptionAlreadyInUse(FileLock.java:443)
	at org.h2.store.FileLock.lockFile(FileLock.java:338)
	at org.h2.store.FileLock.lock(FileLock.java:134)
	at org.h2.engine.Database.open(Database.java:535)
	at org.h2.engine.Database.openDatabase(Database.java:218)

The password for the "admin" user is configured in src/test/resources/sample-data.xml and it's loaded by DbUnit before the application starts. You can view your pom.xml and the dbunit-maven-plugin's configuration if you're interested in learning how this is done. The password is currently configured to "adminjdbc", but you can reset it by generating a new password and modifying sample-data.xml.

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Spring Security 3.0.5. In addition to the basic XML configuration, Spring Security also provides a AOP support and annotations you can use to secure methods. It also has many more features than standard Java EE Security. In my opinion, it's the most mature security framework we have in Java today. Currently, I think its reference documentation is the best place to learn more.

There are a few limitations I found with Spring Security:

The authentication mechanism (file, database, ldap, etc.) is contained in the WAR
Securing methods only works on Spring beans
Remember Me doesn't work in my screencast (because I forgot to rename the checkbox in login.jsp)

Of course, you can configure Spring to load its configuration from outside the WAR (e.g. a file or JNDI), but it's not as easy as including the configuration in your app.

In the next couple weeks, I'll post Part III of this series, where I'll show you how to implement this same set of features using Apache Shiro. In the meantime, please let me know if you have any questions.

I created the screencasts with Camtasia. For small screens, and embedding in the presentation, I created it at 50% and used the SmartFocus feature to zoom in and out during the demo. For larger screens, I published another screencast at 100%, in HD. If you have a preference for which screencast is better, I'd love to hear about it.

Java Web Application Security - Part I: Java EE 6 Login Demo

2011-05-05T16:58:00-06:00

Back in February, I wrote about my upcoming conferences:

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks:

Implement Basic Authentication
Implement Form-based Authentication
Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs)
Force SSL for certain URLs
Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone)
Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone)
Encrypt Passwords
Secure methods with annotations

For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities.

I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml:

<security-constraint>
    <web-resource-collection>
        <web-resource-name>users</web-resource-name>
        <url-pattern>/users</url-pattern>
        <http-method>GET</http-method>
        <http-method>POST</http-method>
    </web-resource-collection>
    <auth-constraint>
        <role-name>ROLE_ADMIN</role-name>
    </auth-constraint>
</security-constraint>

<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Java EE Login</realm-name>
</login-config>

<security-role>
    <role-name>ROLE_ADMIN</role-name>
</security-role>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after </webAppConfig> in the Jetty plugin's configuration.

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.HashLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/realm.properties</config>
    </loginService>
</loginServices>

The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath());
}

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to replace the <login-config> in your web.xml with the following:

<login-config>
    <auth-method>FORM</auth-method>
    <form-login-config>
        <form-login-page>/login.jsp</form-login-page>
        <form-error-page>/login.jsp?error=true</form-error-page>
    </form-login-config>
</login-config>

The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Force SSL
Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same <security-constraint> you already have in web.xml, add the following after </auth-constraint>:

<user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>

To configure Jetty to listen on an SSL port, add the following just after </loginServices> in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you.

Store Users in a Database
Finally, to store your users in a database instead of file, you'll need to change the <loginService> in the Jetty plugin's configuration. Replace the existing <loginService> element with the following:

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.JDBCLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/jdbc-realm.properties</config>
    </loginService>
</loginServices>

The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information.

jdbcdriver = com.mysql.jdbc.Driver
url = jdbc:mysql://localhost/appfuse
username = root
password =
usertable = app_user
usertablekey = id
usertableuserfield = username
usertablepasswordfield = password
roletable = role
roletablekey = id
roletablerolefield = name
userroletable = user_role
userroletableuserkey = user_id
userroletablerolekey = role_id
cachetime = 300

Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands:

mysql -u root -p -e 'create database appfuse'
curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql
mysql -u root -p appfuse < create-appfuse.sql

Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the <configuration> element (before <executions>) in pom.xml:

<dependencies>
    <!-- MySQL for JDBC Realm -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.14</version>
    </dependency>
</dependencies>

Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example:

$ sh jetty-password.sh javaeelogin
javaeelogin
OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur
MD5:53b176e6ce1b5183bc970ef1ebaffd44

The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement.

UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin';

Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0:

authenticate(response)
login(user, pass)
logout()

This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely:

No error messages for failed logins
No Remember Me
No auto-redirect from HTTP to HTTPS
Container has to be configured
Doesn’t support regular expressions for URLs

Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain.

In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions.

Adding Search to AppFuse with Compass

2011-03-15T17:11:12-06:00

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

Added Searchable annotations to the User and Address.
Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
Defined CompassTemplate (similar in concept to HibernateTemplate).
Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

To index on startup, I added compassGps.index() to the StartupListener..
In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code.

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

Upgrading to JSF 2

2011-03-07T13:24:53-07:00

Last week, I spent a few hours upgrading AppFuse from JSF 1.2 to JSF 2.0. In reality, I upgraded from MyFaces 1.2.7 to 2.0.4, but all JSF implementations should be the same, right? All in all, it was a pretty easy upgrade with a few minor AppFuse-specific things. My goal in upgrading was to do the bare minimum to get things working and to leave integration of JSF 2 features for a later date.

In addition to upgrading MyFaces, I had to upgrade Tomahawk by changing the dependency's artifactId to tomahawk20. I was also able to remove the following listener from my web.xml:

<listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
<listener>

After that, I discovered that MyFaces uses a new URI (/javax.faces.resource/) for serving up some of its resource files. I kindly asked Spring Security to ignore these requests by adding the following to my security.xml file.

<intercept-url pattern="/javax.faces.resource/**" filters="none"/>

Since JSF 2 includes Facelets by default, I tried removing Facelets as a dependency. After doing this, I received the following error:

ERROR [308855416@qtp-120902214-7] ViewHandlerWrapper.fillChain(158) | Error instantiation parent Faces ViewHandler
java.lang.ClassNotFoundException: com.sun.facelets.FaceletViewHandler
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at org.ajax4jsf.framework.ViewHandlerWrapper.fillChain(ViewHandlerWrapper.java:144)
        at org.ajax4jsf.framework.ViewHandlerWrapper.calculateRenderKitId(ViewHandlerWrapper.java:68)
        at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:179)
        at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:113)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:171)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Figuring this was caused by the following element in my web.xml ...

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

... I removed it and tried again. This time I received a NoClassDefFoundError:

java.lang.NoClassDefFoundError: com/sun/facelets/tag/TagHandler
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:392)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.myfaces.shared_impl.util.ClassUtils.classForName(ClassUtils.java:184)
        at org.apache.myfaces.view.facelets.util.ReflectionUtil.forName(ReflectionUtil.java:67)

Since everything seemed to work with Facelets in the classpath, I decided to save this headache for a later date. I entered two issues in AppFuse's JIRA, one for removing Facelets and one for replacing Ajax4JSF with RichFaces.

The next issue I encountered was redirecting from AppFuse's password hint page. The navigation-rule for this page is as follows:

<navigation-rule>
    <from-view-id>/passwordHint.xhtml</from-view-id>
    <navigation-case>
        <from-outcome>success</from-outcome>
        <to-view-id>/login</to-view-id>
        <redirect/>
    </navigation-case>
</navigation-rule>

With JSF 2.0, the rule changes the URL to /login.xhtml when redirecting (where it was left as /login with 1.2) and it was caught by the security setting in my web.xml that prevents users from viewing raw templates.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Protect XHTML Templates</web-resource-name>
        <url-pattern>*.xhtml</url-pattern>
    </web-resource-collection>
    <auth-constraint/>
</security-constraint>

To solve this issue, I had to make a couple of changes:

Comment out the security-constraint in web.xml and move it to Spring Security's security.xml file.
```
<intercept-url pattern="/**/*.xhtml" access="ROLE_NOBODY"/>
```
Add a rule to urlrewrite.xml that redirects back to login (since login.xhtml doesn't exist and I'm using extensionless URLs).
```
<rule match-type="regex">
    <from>^/login.xhtml$</from>
    <to type="redirect">%{context-path}/login</to>
</rule>
```

After getting the Password Hint feature passing in the browser, I tried running the integration tests (powered by Canoo WebTest). The Password Hint test kept failing with the following error:

[ERROR] /Users/mraible/dev/appfuse/web/jsf/src/test/resources/web-tests.xml:51: JavaScript error loading
page http://localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/passwordHint?username=admin: syntax error (http://
localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/javax.faces.resource/oamSubmit.js.jsf?ln=org.apache.myfaces#122)

Figuring this was caused by my hack to submit the form when the page was loaded, I turned to Pretty Faces, which allows you to call a method directly from a URL. After adding the Pretty Faces dependencies to my pom.xml, I created a src/main/webapp/WEB-INF/pretty-config.xml file with the following XML:

<url-mapping>
    <pattern value="/editProfile"/>
    <view-id value="/userForm.jsf"/>
    <action>#{userForm.edit}</action>
</url-mapping>

<url-mapping>
    <pattern value="/passwordHint/#{username}"/>
    <view-id value="/passwordHint.jsf"/>
    <action>#{passwordHint.execute}</action>
</url-mapping>

This allowed me to remove both editProfile.xhtml and passwordHint.xhtml, both of which simply auto-submitted forms.

At this point, I figured I'd be good to go and ran my integration tests again. The first thing I discovered was that ".jsf" was being tacked onto my pretty URL, most likely by the UrlRewriteFilter. Adding the following to my PasswordHint.java class solved this.

if (username.endsWith(".jsf")) {
    username = username.substring(0, username.indexOf(".jsf"));
}

The next thing was a cryptic error that took me a while to figure out.

DEBUG [1152467051@qtp-144702232-0] PasswordHint.execute(38) | Processing Password Hint...
2011-03-05 05:48:52.471:WARN::/passwordHint/admin
com.ocpsoft.pretty.PrettyException: Exception occurred while processing <:#{passwordHint.execute}> null
        at com.ocpsoft.pretty.faces.beans.ActionExecutor.executeActions(ActionExecutor.java:71)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.processEvent(PrettyPhaseListener.java:214)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.afterPhase(PrettyPhaseListener.java:108)
        at org.apache.myfaces.lifecycle.PhaseListenerManager.informPhaseListenersAfter(PhaseListenerManager.java:111)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:185)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Digging into the bowels of MyFaces, I discovered a class was looking for a viewId with an extension and no view-id was being set. Adding the following to the top of my execute() method solved this.

getFacesContext().getViewRoot().setViewId("/passwordHint.xhtml");

After making this change, all AppFuse's integration tests are passing and the upgrade seems complete. The only other issues I encountered were logging-related. The first is an error about Tomahawk that doesn't seem to affect anything.

Mar 5, 2011 6:44:01 AM com.sun.facelets.compiler.TagLibraryConfig loadImplicit
SEVERE: Error Loading Library: jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml
java.io.IOException: Error parsing [jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml]: 
        at com.sun.facelets.compiler.TagLibraryConfig.create(TagLibraryConfig.java:410)
        at com.sun.facelets.compiler.TagLibraryConfig.loadImplicit(TagLibraryConfig.java:431)
        at com.sun.facelets.compiler.Compiler.initialize(Compiler.java:87)
        at com.sun.facelets.compiler.Compiler.compile(Compiler.java:104)

The second is excessive logging from MyFaces. As far as I can tell, this is because MyFaces switched to java.util.logging instead of commons logging. With all the frameworks that AppFuse leverages, I think it has all the logging frameworks in its classpath now. I was hoping to fix this by posting a message to the mailing list, but haven't received a reply yet.

[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider newInstance
[WARNING] [talledLocalContainer] INFO: Creating instance of org.appfuse.webapp.action.BasePage
[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider destroyInstance
[WARNING] [talledLocalContainer] INFO: Destroy instance of org.appfuse.webapp.action.BasePage

After successfully upgrading AppFuse, I turned to AppFuse Light, where things were much easier.

Now that AppFuse uses JSF 2, I hope to start leveraging some of its new features. If you're yearning to get started with them today, I invite you to grab the source and start integrating them yourself.

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

2011-02-10T16:53:27-07:00

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter Name	Dispatchers
securityFilter	request
rewriteFilter	request, forward
struts-prepare	request
sitemesh	request, forward, include
staticFilter	request, forward
struts	request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter Name	Dispatchers
rewriteFilter	request
securityFilter	request, forward, include
struts-prepare	request, forward
sitemesh	request, forward, include
struts	forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

2010-10-16T15:19:07-06:00

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

Highly Interactive Software with Java and Flex

2010-03-18T12:29:26-06:00

This morning at TSSJS, I attended James Ward's talk about Highly Interactive Software with Java and Flex. Below are my notes from his talk.

Application have moved from mainframes (hard to deploy, limited clients) to client/server (hard to deploy, full client capabilities) to web applications (easy to deploy, limited clients) to rich internet applications (easy to deploy, full client capabilities).

Shortly after showing a diagram of how applications have changed, James showed a demo of a sample Flex app for an automobile insurance company. It was very visually appealing, kinda like using an iPhone app. It was a multi-form application that slides right-to-left as you progress through the wizard. It also allowed you to interact with a picture of your car (to indicate where the damage happened) and a map (to indicate how the accident happened). Both of these interactive dialogs still performed data entry, they just did it in more of a visual way.

Adobe's developer technology for building RIAs is Flex. There's two different languages in Flex: ActionScript and MXML. ActionScript was originally based on JavaScript, but now (in ActionScript 3) uses features from Java and C#. On top of ActionScript is MXML. It's a declarative language, but unlike JSP taglibs. All you can do with MXML is instantiate objects and set properties. It's merely a convenience language, but also allows tooling. The open source SDK compiler takes Flex files and compiles it into a *.swf file. This file can then be executed using the Flash Player (in browser) or Air (desktop).

The reason Adobe developed two different runtimes was because they didn't want to bloat the Flash Player. Once the applications are running client-side, the application talks to the web server. Protocols that can be used for communication: SOAP, HTTP/S, AMF/S and RTMP/S. The web server can be composed of REST or SOAP Web Services, as well as BlazeDS or LC Data Services to talk directly to Java classes.

To see all the possible Flex components, see Tour de Flex. It contains a number of components: core components, data access controls, AIR capabilities, cloud APIs, data visualization. The IBM ILOG Elixir real-time dashboard is particularly interesting, as is Doug McCune's Physics Form.

Next James showed us some code. He used Flex Builder to create a new Flex project with BlazeDS. The backend for this application was a JSP page that talks to a database and displays the results in XML. In the main .mxml file, he used <s:HTTPService> with a URL pointing to the URI of the JSP. Then he added an <mx:DataGrid> and the data binding feature of Flex. To do this, he added dataProvider="{srv.lastResult.items.item}" to the DataGrid tag, where "srv" is the id of the HTTPService. Then he added a Button with click="srv.send()" and set the layout to VerticalLayout. This was a simple demo to show how to hook in a backend with XML.

To show that Flex can interact with more than XML over HTTP, James wrote a SOAP service and changed <s:HTTPService> to <s:WebService> and changed the "url" attribute to "wsdl" (and adjusted the value as appropriate). Then rather than using {srv.lastResult.*}, he had to bind to a particular method and change it to {srv.getElements.lastResults}. The Button's click value also had to change to "srv.getElements(0, 2000)" (since the method takes 2 parameters).

After doing coding in Flex Builder, James switched to his Census to compare server-execution times. In the first example (Flash XML AS), most of the time was spent gzipping the 1MB XML file, but the transfer time is reduced because of this. The server execution time is around 800ms. Compare this to the Flex AMF3 example where the server execution time is 49ms. This is because the AMF (binary) protocol streamlines the data and doesn't include repeated metadata.

To integrate BlazeDS in your project, you add the dependencies and then map the MessageBrokerServlet in your web.xml. Then you use a services-config.xml to define the protocol and remoting-config.xml to define what Java classes to export as services. To use this in the Flex aplication, James changed <s:WebService> to <s:RemoteObject>. He changed the "wsdl" attribute to "endpoint" and added a "destination" attribute to specify the name of the aliased Java class to talk to. Next, James ran the demo and showed that he could change the number of rows from 2,000 to 20,000 and the load time was still much, much faster than the XML and SOAP versions.

There's also a Spring BlazeDS Integration project that allows you to simply annotate beans to expose them as AMF services.

BlazeDS also includes a messaging service that you can use to create publishers and subscribers. The default channels in BlazeDS uses HTTP Streaming and HTTP Long Polling (comet), but it can be configurable (e.g. to use JMS). There's also an Adobe commercial product that keeps a connection open using NIO on the server and has a binary protocol. This is useful for folks that need more real-time data in their applications (e.g. trading floors).

I thought this was a really good talk by James. It had some really cool visual demos and the demo was interesting in showing how easy it was to switch between different web services and protocols. This afternoon, I'll be duking it out with James at the Flex vs. GWT Smackdown. If you have deficiencies of Flex you'd like me to share during that talk, please let me know.

17" MacBook Pro Stolen from Living Room

2010-02-03T09:45:52-07:00

Almost 3 years ago, I bought a 17" MacBook Pro. This laptop served me well for several years, mostly as a home computer. A few months after I bought it, I started working at LinkedIn and got a brand new laptop as part of my first day on the job. After working with LinkedIn's 15" for almost 2 years, I grew to love the form factor and purchased another one almost a year ago. I found the 17" was too big for planes and the 15" fits me perfectly.

Fast forward to last night.

I attended the first half of the Ignite talks at DOSUG and left around 7. When I arrived home, I suspected something might be out of whack when my front door was unlocked. Julie had come over to pick up Abbie's dance shoes around 6, so I figured she must've forgot to lock the door on her way out. When I got inside and saw my coat closet open, I justified it by thinking she grabbed one of the kids coats out of the closet and forgot to close the door. When I walked into my living room and saw my space heater's remote in the middle of the floor, busted open with batteries out, it clicked that a stranger was in my house. The first thing that jumped into my head was "Where's my laptop?" As I looked at the bare mini-desk in my living room, I realized it had been stolen.

I called Julie and asked her if she left the front door open. She said no, but when she arrived at my house (and came through the back), the garage's light was on and my back door was wide open. I asked her if she saw the busted remote on the floor or if the closet door was open. She said no. Putting all the pieces together, it appears that the burglar was actually hiding in my closet when Julie came into my house. Naturally, she's a little a freaked out by this, but she also saved the day by scaring off perpetrator before they took anything else.

This isn't a new trend for me and this incident is mostly my fault. I left my backdoor unlocked. Two years ago, my truck's stereo was stolen and there's a good chance I left the doors unlocked (and didn't turn on the alarm). Last year, my bike was stolen and the lock was still there, indicating I missed the frame when locking it up. So getting robbed in the first part of every year seems somewhat par for the course.

With my truck's stereo, it worked out well because the rig needed a new stereo. My bike last year? There wasn't any silver lining to that incident, so I made myself earn a new one. With this laptop incident, there is a silver lining in that I've been thinking about getting a 27" iMac for a home computer. Other options include a Mac Pro for my office (and use my laptop for traveling/home use) or a Mac Mini for home and hook it up to my TV with a wireless keyboard and mouse.

The home iMac seems like the best option, but I'd also be interested to hear what others recommend. Of course, I'll be keeping my doors locked from now on.

Grails OAuth and LinkedIn APIs

2009-12-22T15:37:57-07:00

Back in November, I wrote about how to talk to LinkedIn APIs with GWT. A week later, I figured out how to do it with Grails and contributed a patch to the grails-oauth plugin.

Since then, a few folks have asked how I did it. Since code speaks louder than words, I took some time and 1) verified the oauth plugin works as expected and 2) created an example application demonstrating functionality. You can find the results in my fork of grails-oauth on GitHub. You can also view the example online.

Below is a quick tutorial explaining how to integrate LinkedIn into your Grails application.

Download and install Grails 1.1.2.
Run grails create-app to create your application.

Add the following to the bottom of grails-app/conf/Config.groovy:

oauth {
    linkedin {
        requestTokenUrl="https://api.linkedin.com/uas/oauth/requestToken"
        accessTokenUrl="https://api.linkedin.com/uas/oauth/accessToken"
        authUrl="https://api.linkedin.com/uas/oauth/authorize"
        consumer.key="XXX"
        consumer.secret="XXX"
    }
}

You can get your consumer.key and consumer.secret at https://www.linkedin.com/secure/developer. Make sure to set the OAuth Redirect URL to http://localhost:8080/{your.app.name}/oauth/callback for testing.

Download the oauth-plugin, extract it and build it using grails package-plugin. Install it in your project using grails install-plugin path/to/zip.

Add a link to the GSP you want to invoke LinkedIn Authentication from:

<g:oauthLink consumer='linkedin' returnTo="[controller:'profile']">
    Login with LinkedIn
</g:oauthLink>

Create grails-app/controllers/ProfileController.groovy to access your LinkedIn Profile.

class ProfileController {
    def apiUrl = "http://api.linkedin.com/v1/people/~"
    def oauthService
    
    def index = {
 
        if (session.oauthToken == null) {
            redirect(uri:"/")
        }
 
        if (params?.apiUrl) apiUrl = params.apiUrl
        
        def response = oauthService.accessResource(
                apiUrl, 'linkedin', [key:session.oauthToken.key, secret:session.oauthToken.secret], 'GET')
 
        render(view: 'index', model: [profileXML: response, apiUrl: apiUrl])
    }
 
    def change = {
        if (params?.apiUrl) {
            println("Setting api url to " + params.apiUrl)
            apiUrl = params.apiUrl
        }
        
        redirect(action:index,params:params)
    }
}

Create grails-app/views/profile/index.gsp to display the retrieved profile and allow subsequent API calls.

<html>
<head><title>Your Profile</title></head>
<body>
<a class="home" href="${createLinkTo(dir:'')}">Home</a>
<g:hasOauthError>
    <div class="errors">
        <g:renderOauthError/>
    </div>
</g:hasOauthError>

<g:form url="[action:'change',controller:'profile']" method="get">
    Your LinkedIn Profile:
    <textarea id="payload" style="width: 100%; height: 50%; color: red">${profileXML}</textarea>
    <p>
        <g:textField name="apiUrl" value="${apiUrl}" size="100%"/>
        <br/>
        <g:submitButton name="send" value="Send Request"/>
    </p>
</g:form>
</body>
</html>

Start your app using grails run-app and enjoy.

As mentioned earlier, you can download the grails-oauth-example or view it online.

One improvement I'd like to see is to simplify the parsing of XML into a Profile object, much like the linkedin gem does for Rails.

If you're interested in learning more about LinkedIn and OAuth, I encourage you to checkout Taylor Singletary's presentation LinkedIn OAuth: Zero to Hero.

Update: I updated the oauth-plugin so it's backwards-compatible with OAuth 1.0 and added Twitter to the example application to prove it. If you're seeing "Cannot invoke method remove() on null object", it's likely caused by your redirect URL pointing to an application on a different domain.

GWT OAuth and LinkedIn APIs

2009-11-24T15:46:05-07:00

When I worked at LinkedIn last year, I received a lot of inquiries from friends and developers about LinkedIn's APIs. After a while, I started sending the following canned response:

For API access to build LinkedIn features into your application, fill out the following form:

http://www.linkedin.com/static?key=developers_apis

For requests to build an application, go to:

http://www.linkedin.com/static?key=developers_opensocial

I talked with the API team and they did say they look at every request that's sent via these forms. They don't respond to all of them b/c they know that many people would be angry if they told them "no", so they'd rather not have that headache.

Yesterday, I was pumped to see that they've finally decided to open up their API to Developers.

Starting today, developers worldwide can integrate LinkedIn into their business applications and Web sites. Developer.linkedin.com is now live and open for business.

First of all, congratulations to the API team on finally making this happen! I know it's no small feat. Secondly, it's great to see them using Jive SBS for their API documentation and developer community. My current client uses this to facilitate development and I love how it integrates a wiki, JIRA, FishEye, Crucible and Bamboo into one central jumping off point.

I've always been a fan of LinkedIn, ever since I joined way back in May 2003. However, I've longed for a way to access my data. LinkedIn Widgets are nice, but there's something to be said for the full power of an API. Last night, I sat down for a couple hours and enhanced my Implementing OAuth with GWT example to support LinkedIn's API.

I'm happy to report my experiment was a success and you can download GWT OAuth 1.2 or view it online. For now, I'm simply authenticating with OAuth and accessing the Profile API.

In the process, I learned a couple things:

LinkedIn's OAuth implementation returns an oauth_verifier parameter after authenticating, whereas Google and Twitter do not. This parameter needs to be included when calling the Access token path.
The Profile API example I implemented gets the current user's profile with http://api.linkedin.com/v1/people/~. This returns a "light" version of your profile. To get a more detailed version, you need to use Field Selectors. For example: http://api.linkedin.com/v1/people/~:(id,first-name,last-name,picture-url,headline,summary,positions,educations)
LinkedIn's API only supports passing OAuth parameters in a header, rather than query parameters. To make this work, I modified my ProxyServlet to convert query parameters to an "Authorization" header at the end of the setProxyRequestHeaders() method.

// For LinkedIn's OAuth API, convert request parameters to an AuthorizationHeader
if (httpServletRequest.getRequestURL().toString().contains("linkedin-api")) {
    String[] parameters = httpServletRequest.getQueryString().split("&");
    StringBuilder sb = new StringBuilder("OAuth realm=\"http://api.linkedin.com/\",");
    for (int i = 0; i < parameters.length; i++) {
        sb.append(parameters[i]);
        if (i < parameters.length - 1) {
            sb.append(",");
        }
    }

    Header authorization = new Header("Authorization", sb.toString());
    httpMethodProxyRequest.setRequestHeader(authorization);
}

You might recall that my previous example had issues authenticating with Google, but worked well with Twitter. LinkedIn's authentication seems to work flawlessly. This leads me to believe that Twitter and LinkedIn have a much more mature OAuth implementation than Google.

Related OAuth News: Apache Roller 5 will be shipping with OAuth support. See Dave Johnson's What's New in Roller 5 presentation for more information.

Update December 6, 2009: I modified the gwt-oauth project to use GWT 1.7.1 and changed to the Maven GWT Plugin from Codehaus. Download GWT OAuth 1.3 or view it online.

User Interface Schema Definitions

2009-11-06T15:18:25-07:00

On my current project, we're developing a "designer" tool that allows users to build forms in their browser. These forms are displayed in various channels (e.g. web, mobile, sms) to capture data and make decisions based on user input.

When I joined the project, it was in a proof-of-concept phase and the form definitions created where serialized as XML (using JAXB) with element names that seemed logical. Now that we're moving the PoC to production mode, we're thinking it might be better to change our form definitions to leverage something that's more "industry standard". If nothing else, it'll help with marketing. ;-)

This week, I was tasked with doing research on "existing user interface schema definitions". I'm writing this post to see if there's any major specifications I'm missing. I plan on providing my recommendation to my team on Monday. Here's what I've found so far:

User Interface Markup Language (UIML): At 120 pages, the 4.0 spec seems very detailed. I'm not sure we'd use all of it, but it's interesting how the spec allows you to describe the initial tree structure (<structure>) and to dynamically modify its structure (<restructure>). It also has the notion of templates, which mirrors a similar concept we have in our application. Furthermore, it has VoiceXML support, which could be useful if we use call centers as a channel.
USer Interface eXtensible Markup Language (UsiXML): I admit that I haven't read much of this specification -- mostly because the UIML spec seemed to cover most of what we needed (especially since we're most interested in describing forms). As far as a I can tell, the major difference between UsiXML is its being submitted to the W3C for standardization (according to Wikipedia), while UIML is being standardized by OASIS. Beyond that, I find it strange that UIML's spec is 120 pages and UsiXML is 121. Neither project seems to have any activity this year.
Numerous others: including AAIML, AUIML, XIML, XUL, XAML and XForms. XForms seems like it may be the most logical if we're only interested in form layout and describing elements within them.

If all we're interested in is an XSD to define our forms, the most appealing specs have them: UIML, UsiXML and XForms. If activity is any sort of motivator for adoption, it's interesting to note that XForms 1.1 was submitted as a W3C recommendation a couple weeks ago (October 20, 2009).

If you've developed some sort of "form designer" tool that renders to multiple channels, I'd love to hear about your experience. Did you use some sort of industry standard to define your form elements, layout, etc. or did you come up with your own?

A Letter to the AppFuse Community

2009-11-04T00:17:17-07:00

The last AppFuse release was way back in May 2008. Many folks have asked when the next release would be ever since. Often, I've said "sometimes this quarter", but obviously, that's never happened. For that, I apologize.

There are many reasons I haven't worked on AppFuse for the past 18 months, but it mostly comes down to the fact that I didn't make time for it. The good news is I'm working on it again and will have a release out sometime this month. Unfortunately, it probably won't be a 2.1 final release, but there's so many things that've changed, I feel like a milestone release is a good idea. Here's a brief summary of changes so far:

Changed archetypes to include all source and tests for the "webapp" portion of the application. No more warpath plugin, merging wars and IDE issues. Using "mvn jetty:run" should work as expected.
Moved from Spring XML to Annotations.
AppFuse Light converted to Maven modules and now depends on AppFuse's backend.
Published easier to use archetype selection form in the QuickStart Guide.
Published archetype selection form for AppFuse Light. I do plan on combining these forms as soon as I figure out the best UI and instructions for users to choose AppFuse or AppFuse Light.
Upgraded all libraries to latest released versions (Spring 3 hasn't had a final release yet).
Upgraded to Tapestry 5 thanks to Serge Eby. I still need to complete tests and code generation for tests.
Added Compass support thanks to a patch from Shay Banon.
Upgraded from XFire to CXF for Web Services.
Moved Maven repository to Sonatype's OSS Repository Hosting for snapshots and releasing to Maven Central. There are no longer any AppFuse-specific artifacts, all are available in central.

I realize there's many full-stack frameworks that do the same thing as AppFuse with less code. Examples include Ruby on Rails, Grails, Seam, Spring Roo and the Play framework. However, there seems to be quite a few folks that continue to use AppFuse and it stills serves the community as a nice example of how to integrate frameworks. Furthermore, it helps me keep up with the latest framework releases, their quirks and issues that happen when you try to integrate them. In short, working on it helps me stay up to speed with Java open source frameworks.

For those folks that like the 1.x, Ant-based version of AppFuse, there will not be a 1.9.5 release. I know I promised it for years, but it's simply something I will not use, so I'd rather not invest my time in it. I'm sorry for lying to those that expected it.

So what's the future of AppFuse? Will it continue to integrate web frameworks with Spring and popular persistence frameworks? Possibly, but it seems more logical to align it with the types of Ajax + REST applications I'm creating these days. I'm currently thinking AppFuse 3.0 would be nice as a RESTful backend with GWT and Flex UIs. I might create the backend with CXF, but it's possible I'd use one of the frameworks mentioned above and simply leverage it to create the default features AppFuse users have come to expect.

More than anything, I'm writing this letter to let you know that the AppFuse project is not dead and you can expect a release in the near future.

Thanks for your support,

Matt

Lean Teams: Doing more with less

2009-09-23T21:20:28-06:00

This evening I attended the Denver Rails User Group (a.k.a. DeRailed) to hear a presentation by Marty Haught. It was titled "Lean Teams: Doing more with less" and the following are my notes from the event.

Today's talk is about "Rocking with Ramen" - a.k.a. working with less funds to make great things. Lean comes from the manufacturing world in that you should Add Nothing but Value. The most important thing you should do is add business value.

The Seven Wasteful Sins for manufacturing are:

Overproduction
Inventory
Extra Processing Steps
Motion
Defects
Waiting
Transportation

The key to fighting overproduction in software is to trim features to those that achieve the greatest value. You should do "the simplest thing that could possibly work" and delay commitment as long as you can because YAGNI.

A minimum viable product is a starting place for validated learning with the least amount of effort. It should be embarrassing. Early adopters see the potential. Rails Rumble and Startup Weekend are good examples of promoting this type of development.

Unused and useless features are best solved by feedback-driven development. This is a process for validating value and creating software that people use. The end result is that you create software that people use and you're able to pivot your plan as you learn. The benefit of this is you stay humble and you don't drink the Kool-Aid (e.g. VC's tell you you're going to be the next Twitter).

The first part of feedback is "Pirate Metrics" by Dave McClure. The main things to track are acquisition, activation, retention, referral and revenue (AARRRR!). The main things you should gather from metrics is they're actionable and should help you make decisions. Vanity metrics like hits-per-month and such should be ignored.

Other feedback options include net promoter score (popup question to ask if users would recommend to a friend), feedback form (make it easy for users to tell you what you think about your product), A/B testing, and usability testing.

The final point is that it's OK to remove features.

To reduce extra processing and waiting, you should implement "Kanban". It's a pull-based system for a continuous flow of work and can be used in software projects to manage/schedule work for cross functional teams. It's an expression of just-in-time and has an emphasis on flow. It's all about getting across the board as fast as possible. In agile development, this is often expressed as a card-based system on a wall in the same room as your development team. Things can only move from the left-to-right as there is space for them. Marty is showing a screenshot of a "Zen" tool he uses on his projects. It has 3 columns (Definition, Work and Verification) from left-to-right that allows you to easily move stuff.

The most important thing about Kanban is it helps to eliminate constraints. The Zen tool only allows a certain amount of items in the "Work" column and it visually communicates blocked items by moving them to the top and highlighting them with a red border. The Zen tool that Marty is showing looks similar to Rally, but is much more visually appealing.

The benefits of Kanban include:

simple, less process
less inventory of requirements/stories
limit work in progress, maximize throughput
less time in meetings
more naturally represents story lifecycle
more easily spot bottlenecks
estimate only if it adds value

Kanban promotes tracking how long it takes for a story get across the board and into production vs. tracking velocity of a team.

On my current project, we use Rally, a small team and have two week iterations. Because the things that Marty is talking about seem to be things we're already doing, I asked him how Kanban differs from Scrum with small teams. He explained that this biggest difference is Kanban is most useful when you're pushing things to production with each iteration.

The most controversial practice that Marty promotes is Continuous Deployment. This is the automated deployment of code to production. It includes automated testing and continuous integration, simple deployment/rollback scripts, a successful CI build triggers deployment, and there's real-time alerts in production. When shit goes wrong, you should use the "five whys" to perform root cause analysis. Marty admits that this is only a good idea when there's a high-level of trust in your development team and lots of tests to prove nothing is broken.

The benefits of continuous deployment is there's a lower story cycle time, you eliminate waste in deploying code, you deliver features/bugs fixes faster and you find integration issues quicker and in isolation. It's also a great way to promote not checking in shitty code.

The skeptics think this is a bad idea because 1) it's scary, 2) they believe it causes lower quality and 3) it causes more issues in production. The good news is you can still control production deployments with your source control system (e.g. branches and such). More than anything, it forces you to have a high quality continuous integration system that acts as the gatekeeper for what goes to production.

You can learn more about topics Marty covered in this talk at the following sites:

Eric Ries - http://www.startuplessonslearned.com
Steve Blank - The Four Steps to the Epiphany
Mary Poppendieck - Lean Software Development
Dave McClure - http://davemcclure.com
Kanban Applied to Software Development: from Agile to Lean

If you're lucky enough to be attending Aloha on Rails, Marty will be presenting there. I recommend you attend his talk if you're trying to get stuff done quickly and get it into production even quicker. His techniques seem to be invaluable for developers that are trying to maximize their efficiency and reduce the time it takes to get their code into production.

My Experience with Java REST Frameworks (specifically Jersey and CXF)

2009-08-27T14:20:51-06:00

Recently I was tasked with developing a server-side REST strategy for a client. When I started working with them, they were using GWT RPC for exposing services. They wanted to move to RESTful services to allow for a more open platform, that multiple types of clients could talk to. There were interested in supporting SOAP and GWT RPC as well, but it wasn't required. They are using Spring, a custom namespace (for easily creating remote services) and an HTML documentation generator to expose their API in human readable form.

When I first starting developing, I chose to try Enunciate. From Enunciate's homepage:

Enunciate is an engine for creating, maintaining, and deploying your rich Web service API on the Java platform. If that sounds complicated, it's not. All you have to do is define your service interfaces in Java source code. ... Then invoke Enunciate.

Sounds pretty sweet, eh? At first glance, the things I liked about Enunciate were:

The ability to generate multiple endpoints (and clients).
Generates nice-looking documentation.
Allows selecting different frameworks (for example, CXF instead of JAX-WS RI).

Initially, the hardest part of using Enunciate was integrating it into my project. This was somewhat related to having a multi-module project, but moreso related to getting the settings right in my enunciate.xml file. After getting everything working, I encountered a few Spring wiring issues with the GWT Endpoints and with Jersey's Spring support.

The good news is I believe most of these issues were related to my project and how it proxies Spring beans that Jersey couldn't find. Jersey's Spring support only supports annotations at this time, so I was unable to tell it the proxied bean names via XML (or even its own annotations). I'm sure this problem is solvable, but after struggling with it for a day or two, I decided to give up on Enunciate. I had to get something working, and fast.

At this point, I was back to the drawing board. I knew there were plenty of good Java REST frameworks available, but Spring and CXF (for SOAP) were already available dependencies in my project, so I chose that route. I was able to get something up and running fairly quickly with this tutorial and CXF's JAX-RS documentation. I ran into an Invalid JSON Namespace issue, but was able to solve it by adding a custom namespace.

It's not all rosy though, there are still a couple CXF issues I haven't solved:

I'd like to remove the namespace prefixes on returned JSON. I've never seen JSON with "prefix." notation before and it doesn't seem right.
I'm unable to return String[] from services.

While CXF does take a bit of XML for each service, it's kinda slick in that it only requires you to annotate your service interfaces. It also generates documentation for your services (WSDL and WADL), but it's not as pretty as Enunciate's. To solve this, I added Enunciate for documentation only. To make Enunciate work, I did have to add some annotations to my service implementation classes and parse the generated HTML to fix some links to CXF's services, but ultimately it works pretty well.

In the end, I recommended my client not use Enunciate for generating endpoints. This was primarily related to their unique Spring configuration, but also because I was able to easily use the same classes for REST, SOAP and GWT Endpoints. However, I will continue to keep my eye on Enunciate. It could be very useful for the upcoming appfuse-ws archetype. I'm also eager to see better GWT support and the ability to generate Overlay types in future releases.

My attempt at browser-based username/password autocomplete with GWT

2009-08-07T08:09:11-06:00

Last night, I did a quick spike to try and implement username/password autocomplete in my GXT application. By "autocomplete", I don't mean Ajax-style autocomplete, but rather browser-based autocomplete. The best information I found on doing this is in the following post:

http://osdir.com/ml/GoogleWebToolkit/2009-04/msg01838.html

I didn't use this technique (wrapping an existing form with a FormPanel) because I'm using GXT and didn't want to lose the look-and-feel of my login form.

I was successful in getting everything to work in Firefox (it populates both the username and password). In IE, it only populates the username, not the password. In Safari/Chrome, it doesn't work at all. Here's how I did it:

1. Created a hidden HTML form on my HTML page that embeds GWT.

<form method="post" action="javascript:void(0)" style="display: none">
    <input type="text" id="username" name="username" value=""/>
    <input type="password" id="password" name="password" value=""/>
    <input type="submit" value="Login" id="login"/>
</form>

2. When a user clicks on the "Login" button in my GWT application, populate the fields in this hidden form and "click" on the Login button (which will do nothing since the action="javascript:void(0)").

// Set the hidden fields to trigger the browser to remember
DOM.getElementById("username").setAttribute("value", username.getValue());
DOM.getElementById("password").setAttribute("value", password.getValue());
clickFormLogin();

...

public static native void clickFormLogin() /*-{
$doc.getElementById("login").click();
}-*/;

This works in Firefox 3.5 and prompts me to save the user/pass at the top of the screen. Why doesn't this work in Safari/Chrome? My guess is because the form's action doesn't go anywhere and the form is not submitted. If I change the action to be an actual URL and show the form, clicking on the form's Login button will save it in those browsers.

Grabbing the actual populated values when the screen loads did turn out to be a bit tricky. With IE, I was able to grab the username, but not the password. It seems that keyboard or mouse interaction (tabbing off or focusing/blurring) with the username field is necessary to get the password field populated. In Firefox, I was unable to grab the values if I did it straight away. The solution turned out to be wrapping everything in a Timer and waiting a 1/2 second.

// grab user/pass from browser and hidden login form (if user has stored them)
// runs 1/2 second after page loads so cursor goes at end of username and works in Firefox
Timer t = new Timer() {
    public void run() {
        username.setValue(getElementValue("username"));
        password.setValue(getElementValue("password"));
        if (username.getValue() != null) {
            password.focus();
        }
        validate();
    }
};

t.schedule(500);

...

public static native String getElementValue(String domId) /*-{
    return $doc.getElementById(domId).value;
}-*/;

While I'm glad I got it working in Firefox, I'm disappointed with IE's lack of password autocompletion. More than anything, I can't help but think there's a way to make this work in WebKit-based browsers. If you've successfully implemented cross-browser username/password autocomplete in GWT, please share your experience. If you share your experience about successfully doing it in Safari/Chrome, I might even buy you a beer or two.

Integrating GWT with Spring Security

2009-08-06T08:50:15-06:00

Yesterday, I wrote about How to do cross-domain GWT RPC with a ProxyServlet. Today I'll be discussing how to modify the ProxyServlet to authenticate with Spring Security. For the application I'm working on, the ProxyServlet is only used in development (when running GWT's hosted mode) and isn't necessary when deploying the client and server on the same server. Using the ProxyServlet allows cross-domain requests so you can run GWT in hosted mode and talk to your backend running on another server. This setup can be especially handy in that you can easily point your hosted client at different backends (for example, if you have testing and staging environments).

In this example, the backend application is a JSF/Spring application that has Spring Security wired in to protect services with both Basic and Form-based authentication. Basic authentication will kick in if a "Authorization" header is sent, otherwise Form-based authentication is used. Here's the Spring Security context file that makes this happen:

<?xml version="1.0" encoding="UTF-8"?>

<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="...">

    <http auto-config="true" realm="My Web Application">
        <intercept-url pattern="/faces/welcome.jspx" access="ROLE_USER"/>
        <intercept-url pattern="/*.rpc" access="ROLE_USER"/>
        <http-basic/>
        <form-login login-page="/faces/login.jspx" authentication-failure-url="/faces/accessDenied.jspx"
                    login-processing-url="/j_spring_security_check" default-target-url="/redirect.jsp"
                    always-use-default-target="true"/>
    </http>

    <authentication-provider>
        <user-service >
            <user name="admin" password="admin" authorities="ROLE_USER"/>
        </user-service>
    </authentication-provider>
</beans:beans>

The easiest way to configure your GWT application to talk to a Spring Security protected resource is to protect your HTML page that GWT is embedded in. This is the documented way to integrate GWT with Spring Security (ref: GWT's LoginSecurityFAQ, search for "Acegi"). This works well for production, but not for hosted-mode development.

Basic Authentication
To authenticate with Basic Authentication, you can use GWT's RequestBuilder and set an "Authentication" header that contains the user's (base64-encoded) credentials.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/faces/welcome.jspx";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Authorization", createBasicAuthToken());
        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

protected String createBasicAuthToken() {
    byte[] bytes = stringToBytes(username.getValue() + ":" + password.getValue());
    String token = Base64.encode(bytes);
    return "Basic " + token;
}

protected byte[] stringToBytes(String msg) {
    int len = msg.length();
    byte[] bytes = new byte[len];
    for (int i = 0; i < len; i++)
        bytes[i] = (byte) (msg.charAt(i) & 0xff);
    return bytes;
}

To use this LoginRequest class, create it with a callback and look for a 401 response code to determine if authentication failed.

new LoginRequest(new RequestCallback() {
    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_UNAUTHORIZED &&
                response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }

        if (response.getStatusCode() == Response.SC_UNAUTHORIZED) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

If your GWT application is included in the "services" war, everything should work at this point. However, if you try to login with invalid credentials, your browser's login dialog will appear. To suppress this in the aforementioned ProxyServlet, you'll need to make a change in its executeProxyRequest() method so the "WWW-Authenticate" header is not copied.

// Pass the response code back to the client
httpServletResponse.setStatus(intProxyResponseCode);

// Pass response headers back to the client
Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
for (Header header : headerArrayResponse) {
    if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
            header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") ||
            header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header
    } else {
        httpServletResponse.setHeader(header.getName(), header.getValue());
    }
}

I'm not sure how to suppress the browser prompt when not using the ProxyServlet. If you have a solution, please let me know.

Basic Authentication works well for GWT applications because you don't need additional logic to retain the authenticated state after the initial login. While Basic Authentication over SSL might offer a decent solution, the downside is you can't logout. Form-based Authentication allows you to logout.

Form-based Authentication

Before I show you how to implement form-based authentication, you should be aware that Google does not recommend this. Below is a warning from their LoginSecurityFAQ.

Do NOT attempt to use the Cookie header to transfer the sessionID from GWT to the server; it is fraught with security issues that will become clear in the rest of this article. You MUST transfer the sessionID in the payload of the request. For an example of why this can fail, see CrossSiteRequestForgery.

In my experiment, I didn't want to change the server-side Spring Security configuration, so I ignored this warning. If you know how to configure Spring Security so it looks for the sessionID in the payload of the request (rather than in a cookie), I'd love to hear about it. The upside of the example below is it should work with container-managed authentication as well.

The LoginRequest class for form-based authentication is similar to the previous one, except it has a different URL and sends the user's credentials in the request body.

private class LoginRequest {
    public LoginRequest(RequestCallback callback) {
        String url = "/services/j_spring_security_check";

        RequestBuilder rb = new RequestBuilder(RequestBuilder.POST, url);
        rb.setHeader("Content-Type", "application/x-www-form-urlencoded");
        rb.setRequestData("j_username=" + URL.encode(username.getValue()) +
                    "&j_password=" + URL.encode(password.getValue()));

        rb.setCallback(callback);
        try {
            rb.send();
        } catch (RequestException e) {
            Window.alert(e.getMessage());
        }
    }
}

If you deploy your GWT application in the same WAR your services are hosted in, this is all you'll need to do. If you're using the ProxyServlet, there's a couple of changes you'll need to make in order to set/send cookies when running in hosted mode.

First of all, you'll need to make sure you've configured the servlet to follow redirects (by subclassing or simply modifying its default). After that, add the following logic on line 358 (or just look for "if (followRedirects)") to expose the sessionID to the client. The most important part is setting the cookie's path to "/" so the client (running at localhost:8888) can see it.

if (followRedirects) {
    // happens on first login attempt
    if (stringLocation.contains("jsessionid")) { 
        Cookie cookie = new Cookie("JSESSIONID",
                stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    // the following happens if you refresh your GWT app after already logging in once
    } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
        String[] cookieDetails = header.getValue().split(";");
        String[] nameValue = cookieDetails[0].split("=");

        Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
        cookie.setPath("/");
        httpServletResponse.addCookie(cookie);
    }
    httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() +
            this.getProxyPath(), stringMyHostName));
    return;
}

Click here to see a screenshot of the diff of the ProxyServlet after this code has been added.

Figuring out that headers needed to be parsed after authenticating successfully and before redirecting was the hardest part for me. If you grab the JSESSIONID from the "Set-Cookie" header anywhere else, the JSESSIONID is one that hasn't been authenticated. While the login will work, subsequent calls to services will fail.

To make subsequent calls with the cookie in the header, you'll need to make an additional modification to ProxyServlet to send cookies as headers. First of all, add a setProxyRequestCookies() method:

/**
 * Retrieves all of the cookies from the servlet request and sets them on
 * the proxy request
 *
 * @param httpServletRequest The request object representing the client's
 *                            request to the servlet engine
 * @param httpMethodProxyRequest The request that we are about to send to
 *                                the proxy host
 */
@SuppressWarnings("unchecked")
private void setProxyRequestCookies(HttpServletRequest httpServletRequest, 
                                    HttpMethod httpMethodProxyRequest) {
    // Get an array of all of all the cookies sent by the client
    Cookie[] cookies = httpServletRequest.getCookies();
    if (cookies == null) {
        return;
    }
    
    for (Cookie cookie : cookies) {
        cookie.setDomain(stringProxyHost);
        cookie.setPath(httpServletRequest.getServletPath());
        httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() +  
                "=" + cookie.getValue() + "; Path=" + cookie.getPath());
    }
}

Next, in the doGet() and doPost() methods, add the following line just after the call to setProxyRequestHeaders().

setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);

After making these modifications to ProxyServlet, you can create LoginRequest and attempt to authenticate. To detect a failed attempt, I'm looking for text in Spring Security's "authentication-failure-url" page.

new LoginRequest(new RequestCallback() {

    public void onResponseReceived(Request request, Response response) {
        if (response.getStatusCode() != Response.SC_OK) {
            onError(request, new RequestException(response.getStatusText() + ":\n" + response.getText()));
            return;
        }
        
        if (response.getText().contains("Access Denied")) {
            Window.alert("You have entered an incorrect username or password. Please try again.");
        } else {
            // authentication worked, show a fancy dashboard screen
        }
    }

    public void onError(Request request, Throwable throwable) {
        Window.alert(throwable.getMessage());
    }
});

After making these changes, you should be able to authenticate with Spring Security's form-based configuration. While this example doesn't show how to logout, it should be easy enough to do by 1) deleting the JSESSIONID cookie or 2) calling the Logout URL you have configured in your services WAR.

Hopefully this howto gives you enough information to configure your GWT application to talk to Spring Security without modifying your existing backend application. It's entirely possible that Spring Security offers a more GWT-friendly authentication mechanism. If you know of a better way to integrate GWT with Spring Security, I'd love to hear about it.

Update on October 7, 2009: I did some additional work on this and got Remember Me working when using form-based authentication. I found I didn't need as much fancy logic in my ProxyServlet and was able to reduce the "followRequests" logic to the following:

if (followRedirects) {
    if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
        Header[] headers = httpMethodProxyRequest.getResponseHeaders("Set-Cookie");
        if (headers.length == 1) {
            extractCookieFromHeader(httpServletResponse, headers[0]);
        } else {
            // ignore the first header since there always seems two jsessionid headers
            // and the 2nd is the valid one
            for (int i = 1; i < headers.length; i++) {
                extractCookieFromHeader(httpServletResponse, headers[i]);
            }
        }
    }
    httpServletResponse.sendRedirect(
            stringLocation.replace(getProxyHostAndPort() + getProxyPath(), stringMyHostName));
    return;
}

I was also able to remove the setProxyRequestCookies() method completely as it no longer seems necessary.

Next, I'd like to figure out how to make Spring Security more Ajax-friendly where it can read an authentication token in the request body or header (instead of from a cookie). Also, it'd be sweet if I could convince it to return error codes instead of the login page (for example, when a certain header is present).

How to do cross-domain GWT RPC with a ProxyServlet

2009-08-05T16:06:12-06:00

Last week, I started working on a new project using GWT. On my last project, we used GWT HTTP Calls and my new project is using RPC. We'll likely migrate to a JSON backend eventually, but in the meantime, I wanted to be able to develop in hosted mode (localhost:8888) and call services on another host (localhost:8080), where the services are running in a JSF/Spring webapp.

At first, I thought it'd be easy thanks to the handy-dandy ProxyServlet I mentioned in Implementing OAuth with GWT. However, when I tried to hook it in and use it, I saw the following error in my server-side logs.

java.lang.NullPointerException
        at javax.servlet.GenericServlet.getServletName(GenericServlet.java:322)
        at javax.servlet.GenericServlet.log(GenericServlet.java:277)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doGetSerializationPolicy(RemoteServiceServlet.java:219)
        at com.google.gwt.user.server.rpc.RemoteServiceServlet.getSerializationPolicy(RemoteServiceServlet.java:117)
        at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamReader.prepareToRead(ServerSerializationStreamReader.java:429)
        at com.google.gwt.user.server.rpc.RPC.decodeRequest(RPC.java:234)

Looking at RemoteServiceServlet.java:219, there's a logging call that fails for some reason (at least in my application).

/*
 * Check that the module path must be in the same web app as the servlet
 * itself. If you need to implement a scheme different than this, override
 * this method.
 */
if (modulePath == null || !modulePath.startsWith(contextPath)) {
  String message = "ERROR: The module path requested, "
      + modulePath
      + ", is not in the same web application as this servlet, "
      + contextPath
      + ".  Your module may not be properly configured or your client and server code maybe out of date.";
  log(message, null);
}

In the above code, you might notice that GWT is checking to make sure the client is hosted in the same application as the server. After I figured this out, it was pretty easy to modify my ProxyServlet to trick GWT RPC into thinking the client was in the same web application. In the ProxyServlet's handleContentPost method, I added the following code to replace "localhost:8888/" with "localhost:8080/services/" (in the content of the post to the server).

if (contentType.startsWith("text/x-gwt-rpc")) {
    String clientHost = httpServletRequest.getLocalName();
    if (clientHost.equals("127.0.0.1")) {
        clientHost = "localhost";
    }

    int clientPort = httpServletRequest.getLocalPort();
    String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                       clientPort : "");
    String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                       intProxyPort : "") + httpServletRequest.getServletPath();
    postContent = postContent.replace(clientUrl , serverUrl);
}

After manipulating the posted content, I was successfully able to use GWT RPC cross-domain.

Woo hoo!

For your convenience, the full handleContentPost() method is listed below.

private void handleContentPost(PostMethod postMethodProxyRequest, 
                               HttpServletRequest httpServletRequest) 
            throws IOException, ServletException {
    StringBuilder content = new StringBuilder();
    BufferedReader reader = httpServletRequest.getReader();
    for (;;) {
        String line = reader.readLine();
        if (line == null) break;
        content.append(line);
    }

    String contentType = httpServletRequest.getContentType();
    String postContent = content.toString();

    if (contentType.startsWith("text/x-gwt-rpc")) {
        String clientHost = httpServletRequest.getLocalName();
        if (clientHost.equals("127.0.0.1")) {
            clientHost = "localhost";
        }

        int clientPort = httpServletRequest.getLocalPort();
        String clientUrl = clientHost + ((clientPort != 80) ? ":" + 
                           clientPort : "");
        String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + 
                           intProxyPort : "") + httpServletRequest.getServletPath();
        postContent = postContent.replace(clientUrl , serverUrl);
    }

    String encoding = httpServletRequest.getCharacterEncoding();
    debug("POST Content Type: " + contentType + " Encoding: " + encoding,
          "Content: " + postContent);
    StringRequestEntity entity;
    try {
        entity = new StringRequestEntity(postContent, contentType, encoding);
    } catch (UnsupportedEncodingException e) {
        throw new ServletException(e);
    }
    // Set the proxy request POST data
    postMethodProxyRequest.setRequestEntity(entity);
}

Update: In the comments, Ganesh asked for more details, so I figured it'd be a good idea to post the full source code. First of all, click here to see the code for the ProxyServlet:

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Map;
import java.util.zip.GZIPInputStream;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.FileUploadException;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.StringRequestEntity;
import org.apache.commons.httpclient.methods.multipart.ByteArrayPartSource;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.StringPart;

/**
 * ProxyServlet from http://edwardstx.net/wiki/attach/HttpProxyServlet/ProxyServlet.java
 * (This seems to be a derivative of Noodle -- http://noodle.tigris.org/)
 * 
 * Patched to skip "Transfer-Encoding: chunked" headers, avoid double slashes
 * in proxied URLs, handle GZip and allow GWT RPC.
 */
public class ProxyServlet extends HttpServlet {

    private static final int FOUR_KB = 4196;

    /**
     * Serialization UID.
     */
    private static final long serialVersionUID = 1L;
    /**
     * Key for redirect location header.
     */
    private static final String STRING_LOCATION_HEADER = "Location";
    /**
     * Key for content type header.
     */
    private static final String STRING_CONTENT_ENGINE_HEADER_NAME = "Content-Type";
    /**
     * Key for content length header.
     */
    private static final String STRING_CONTENT_LENGTH_HEADER_NAME = "Content-Length";
    /**
     * Key for host header
     */
    private static final String STRING_HOST_HEADER_NAME = "Host";
    /**
     * The directory to use to temporarily store uploaded files
     */
    private static final File FILE_UPLOAD_TEMP_DIRECTORY = new File(System.getProperty("java.io.tmpdir"));

    // Proxy host params
    /**
     * The host to which we are proxying requests. Default value is "localhost".
     */
    private String stringProxyHost = "localhost";
    /**
     * The port on the proxy host to wihch we are proxying requests. Default value is 80.
     */
    private int intProxyPort = 80;
    /**
     * The (optional) path on the proxy host to wihch we are proxying requests. Default value is "".
     */
    private String stringProxyPath = "";
    /**
     * Setting that allows removing the initial path from client. Allows specifying /twitter/* as synonym for twitter.com.
     */
    private boolean removePrefix;
    /**
     * The maximum size for uploaded files in bytes. Default value is 5MB.
     */
    private int intMaxFileUploadSize = 5 * 1024 * 1024;
    private boolean isSecure;
    private boolean followRedirects;

    /**
     * Initialize the <code>ProxyServlet</code>
     * @param servletConfig The Servlet configuration passed in by the servlet container
     */
    public void init(ServletConfig servletConfig) {
        // Get the proxy host
        String stringProxyHostNew = servletConfig.getInitParameter("proxyHost");
        if (stringProxyHostNew == null || stringProxyHostNew.length() == 0) {
            throw new IllegalArgumentException("Proxy host not set, please set init-param 'proxyHost' in web.xml");
        }
        this.setProxyHost(stringProxyHostNew);
        // Get the proxy port if specified
        String stringProxyPortNew = servletConfig.getInitParameter("proxyPort");
        if (stringProxyPortNew != null && stringProxyPortNew.length() > 0) {
            this.setProxyPort(Integer.parseInt(stringProxyPortNew));
        }
        // Get the proxy path if specified
        String stringProxyPathNew = servletConfig.getInitParameter("proxyPath");
        if (stringProxyPathNew != null && stringProxyPathNew.length() > 0) {
            this.setProxyPath(stringProxyPathNew);
        }
        // Get the maximum file upload size if specified
        String stringMaxFileUploadSize = servletConfig.getInitParameter("maxFileUploadSize");
        if (stringMaxFileUploadSize != null && stringMaxFileUploadSize.length() > 0) {
            this.setMaxFileUploadSize(Integer.parseInt(stringMaxFileUploadSize));
        }
    }

    /**
     * Performs an HTTP GET request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a GET request
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("GET Request URL: " + httpServletRequest.getRequestURL(),
              "Destination URL: " + destinationUrl);
        GetMethod getMethodProxyRequest = new GetMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, getMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, getMethodProxyRequest);
        // Execute the proxy request
        this.executeProxyRequest(getMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Performs an HTTP POST request
     * @param httpServletRequest The {@link HttpServletRequest} object passed
     *                            in by the servlet engine representing the
     *                            client request to be proxied
     * @param httpServletResponse The {@link HttpServletResponse} object by which
     *                             we can send a proxied response to the client
     */
    public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a standard POST request
        String contentType = httpServletRequest.getContentType();
        String destinationUrl = this.getProxyURL(httpServletRequest);
        debug("POST Request URL: " + httpServletRequest.getRequestURL(),
              "    Content Type: " + contentType,
              " Destination URL: " + destinationUrl);
        PostMethod postMethodProxyRequest = new PostMethod(destinationUrl);
        // Forward the request headers
        setProxyRequestHeaders(httpServletRequest, postMethodProxyRequest);
        setProxyRequestCookies(httpServletRequest, postMethodProxyRequest);
        // Check if this is a mulitpart (file upload) POST
        if (ServletFileUpload.isMultipartContent(httpServletRequest)) {
            this.handleMultipartPost(postMethodProxyRequest, httpServletRequest);
        } else {
            if (contentType == null || PostMethod.FORM_URL_ENCODED_CONTENT_ENGINE.equals(contentType)) {
                this.handleStandardPost(postMethodProxyRequest, httpServletRequest);
            } else {
                this.handleContentPost(postMethodProxyRequest, httpServletRequest);
            }
        }
        // Execute the proxy request
        this.executeProxyRequest(postMethodProxyRequest, httpServletRequest, httpServletResponse);
    }

    /**
     * Sets up the given {@link PostMethod} to send the same multipart POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a multipart POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the mutlipart POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleMultipartPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest)
            throws ServletException {
        // Create a factory for disk-based file items
        DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
        // Set factory constraints
        diskFileItemFactory.setSizeThreshold(this.getMaxFileUploadSize());
        diskFileItemFactory.setRepository(FILE_UPLOAD_TEMP_DIRECTORY);
        // Create a new file upload handler
        ServletFileUpload servletFileUpload = new ServletFileUpload(diskFileItemFactory);
        // Parse the request
        try {
            // Get the multipart items as a list
            List<FileItem> listFileItems = (List<FileItem>) servletFileUpload.parseRequest(httpServletRequest);
            // Create a list to hold all of the parts
            List<Part> listParts = new ArrayList<Part>();
            // Iterate the multipart items list
            for (FileItem fileItemCurrent : listFileItems) {
                // If the current item is a form field, then create a string part
                if (fileItemCurrent.isFormField()) {
                    StringPart stringPart = new StringPart(
                            fileItemCurrent.getFieldName(), // The field name
                            fileItemCurrent.getString() // The field value
                            );
                    // Add the part to the list
                    listParts.add(stringPart);
                } else {
                    // The item is a file upload, so we create a FilePart
                    FilePart filePart = new FilePart(
                            fileItemCurrent.getFieldName(), // The field name
                            new ByteArrayPartSource(
                            fileItemCurrent.getName(), // The uploaded file name
                            fileItemCurrent.get() // The uploaded file contents
                            ));
                    // Add the part to the list
                    listParts.add(filePart);
                }
            }
            MultipartRequestEntity multipartRequestEntity = new MultipartRequestEntity(
                    listParts.toArray(new Part[]{}),
                    postMethodProxyRequest.getParams());
            postMethodProxyRequest.setRequestEntity(multipartRequestEntity);
            // The current content-type header (received from the client) IS of
            // type "multipart/form-data", but the content-type header also
            // contains the chunk boundary string of the chunks. Currently, this
            // header is using the boundary of the client request, since we
            // blindly copied all headers from the client request to the proxy
            // request. However, we are creating a new request with a new chunk
            // boundary string, so it is necessary that we re-set the
            // content-type string to reflect the new chunk boundary string
            postMethodProxyRequest.setRequestHeader(STRING_CONTENT_ENGINE_HEADER_NAME, multipartRequestEntity.getContentType());
        } catch (FileUploadException fileUploadException) {
            throw new ServletException(fileUploadException);
        }
    }

    /**
     * Sets up the given {@link PostMethod} to send the same standard POST
     * data as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    @SuppressWarnings("unchecked")
    private void handleStandardPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) {
        // Get the client POST data as a Map
        Map<String, String[]> mapPostParameters = (Map<String, String[]>) httpServletRequest.getParameterMap();
        // Create a List to hold the NameValuePairs to be passed to the PostMethod
        List<NameValuePair> listNameValuePairs = new ArrayList<NameValuePair>();
        // Iterate the parameter names
        for (String stringParameterName : mapPostParameters.keySet()) {
            // Iterate the values for each parameter name
            String[] stringArrayParameterValues = mapPostParameters.get(stringParameterName);
            for (String stringParamterValue : stringArrayParameterValues) {
                // Create a NameValuePair and store in list
                NameValuePair nameValuePair = new NameValuePair(stringParameterName, stringParamterValue);
                listNameValuePairs.add(nameValuePair);
            }
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestBody(listNameValuePairs.toArray(new NameValuePair[]{}));
    }

    /**
     * Sets up the given {@link PostMethod} to send the same content POST
     * data (JSON, XML, etc.) as was sent in the given {@link HttpServletRequest}
     * @param postMethodProxyRequest The {@link PostMethod} that we are
     *                                configuring to send a standard POST request
     * @param httpServletRequest The {@link HttpServletRequest} that contains
     *                            the POST data to be sent via the {@link PostMethod}
     */
    private void handleContentPost(PostMethod postMethodProxyRequest, HttpServletRequest httpServletRequest) throws IOException, ServletException {
        StringBuilder content = new StringBuilder();
        BufferedReader reader = httpServletRequest.getReader();
        for (;;) {
            String line = reader.readLine();
            if (line == null) break;
            content.append(line);
        }

        String contentType = httpServletRequest.getContentType();
        String postContent = content.toString();

        if (contentType.startsWith("text/x-gwt-rpc")) {
            String clientHost = httpServletRequest.getLocalName();
            if (clientHost.equals("127.0.0.1")) {
                clientHost = "localhost";
            }

            int clientPort = httpServletRequest.getLocalPort();
            String clientUrl = clientHost + ((clientPort != 80) ? ":" + clientPort : "");
            String serverUrl = stringProxyHost + ((intProxyPort != 80) ? ":" + intProxyPort : "") + httpServletRequest.getServletPath();
            //debug("Replacing client (" + clientUrl + ") with server (" + serverUrl + ")");
            postContent = postContent.replace(clientUrl , serverUrl);
        }

        String encoding = httpServletRequest.getCharacterEncoding();
        debug("POST Content Type: " + contentType + " Encoding: " + encoding,
              "Content: " + postContent);
        StringRequestEntity entity;
        try {
            entity = new StringRequestEntity(postContent, contentType, encoding);
        } catch (UnsupportedEncodingException e) {
            throw new ServletException(e);
        }
        // Set the proxy request POST data
        postMethodProxyRequest.setRequestEntity(entity);
    }

    /**
     * Executes the {@link HttpMethod} passed in and sends the proxy response
     * back to the client via the given {@link HttpServletResponse}
     * @param httpMethodProxyRequest An object representing the proxy request to be made
     * @param httpServletResponse An object by which we can send the proxied
     *                             response back to the client
     * @throws IOException Can be thrown by the {@link HttpClient}.executeMethod
     * @throws ServletException Can be thrown to indicate that another error has occurred
     */
    private void executeProxyRequest(
            HttpMethod httpMethodProxyRequest,
            HttpServletRequest httpServletRequest,
            HttpServletResponse httpServletResponse)
            throws IOException, ServletException {
        // Create a default HttpClient
        HttpClient httpClient = new HttpClient();
        httpMethodProxyRequest.setFollowRedirects(false);
        // Execute the request
        int intProxyResponseCode = httpClient.executeMethod(httpMethodProxyRequest);
        String response = httpMethodProxyRequest.getResponseBodyAsString();

        // Check if the proxy response is a redirect
        // The following code is adapted from org.tigris.noodle.filters.CheckForRedirect
        // Hooray for open source software
        if (intProxyResponseCode >= HttpServletResponse.SC_MULTIPLE_CHOICES /* 300 */ && intProxyResponseCode < HttpServletResponse.SC_NOT_MODIFIED /* 304 */) {
            String stringStatusCode = Integer.toString(intProxyResponseCode);
            String stringLocation = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
            if (stringLocation == null) {
                throw new ServletException("Received status code: " + stringStatusCode + " but no " + STRING_LOCATION_HEADER + " header was found in the response");
            }
            // Modify the redirect to go to this proxy servlet rather that the proxied host
            String stringMyHostName = httpServletRequest.getServerName();
            if (httpServletRequest.getServerPort() != 80) {
                stringMyHostName += ":" + httpServletRequest.getServerPort();
            }
            stringMyHostName += httpServletRequest.getContextPath();
            if (followRedirects) {
                if (stringLocation.contains("jsessionid")) {
                    Cookie cookie = new Cookie("JSESSIONID", stringLocation.substring(stringLocation.indexOf("jsessionid=") + 11));
                    cookie.setPath("/");
                    httpServletResponse.addCookie(cookie);
                    //debug("redirecting: set jessionid (" + cookie.getValue() + ") cookie from URL");
                } else if (httpMethodProxyRequest.getResponseHeader("Set-Cookie") != null) {
	                Header header = httpMethodProxyRequest.getResponseHeader("Set-Cookie");
                    String[] cookieDetails = header.getValue().split(";");
					String[] nameValue = cookieDetails[0].split("=");

					Cookie cookie = new Cookie(nameValue[0], nameValue[1]);
					cookie.setPath("/");
					//debug("redirecting: setting cookie: " + cookie.getName() + ":" + cookie.getValue() + " on " + cookie.getPath());
					httpServletResponse.addCookie(cookie);
                }
                httpServletResponse.sendRedirect(stringLocation.replace(getProxyHostAndPort() + this.getProxyPath(), stringMyHostName));
                return;
            }
        } else if (intProxyResponseCode == HttpServletResponse.SC_NOT_MODIFIED) {
            // 304 needs special handling.  See:
            // http://www.ics.uci.edu/pub/ietf/http/rfc1945.html#Code304
            // We get a 304 whenever passed an 'If-Modified-Since'
            // header and the data on disk has not changed; server
            // responds w/ a 304 saying I'm not going to send the
            // body because the file has not changed.
            httpServletResponse.setIntHeader(STRING_CONTENT_LENGTH_HEADER_NAME, 0);
            httpServletResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
            return;
        }

        // Pass the response code back to the client
        httpServletResponse.setStatus(intProxyResponseCode);

        // Pass response headers back to the client
        Header[] headerArrayResponse = httpMethodProxyRequest.getResponseHeaders();
        for (Header header : headerArrayResponse) {
            if (header.getName().equals("Transfer-Encoding") && header.getValue().equals("chunked") ||
                    header.getName().equals("Content-Encoding") && header.getValue().equals("gzip") || // don't copy gzip header
                    header.getName().equals("WWW-Authenticate")) { // don't copy WWW-Authenticate header so browser doesn't prompt on failed basic auth
                // proxy servlet does not support chunked encoding
            } else {
                httpServletResponse.setHeader(header.getName(), header.getValue());
            }
        }

        List<Header> responseHeaders = Arrays.asList(headerArrayResponse);

        if (isBodyParameterGzipped(responseHeaders)) {
            debug("GZipped: true");
            if (!followRedirects && intProxyResponseCode == HttpServletResponse.SC_MOVED_TEMPORARILY) {
                response = httpMethodProxyRequest.getResponseHeader(STRING_LOCATION_HEADER).getValue();
                httpServletResponse.setStatus(HttpServletResponse.SC_OK);
                intProxyResponseCode = HttpServletResponse.SC_OK;
                httpServletResponse.setHeader(STRING_LOCATION_HEADER, response);
            } else {
                response = new String(ungzip(httpMethodProxyRequest.getResponseBody()));
            }
            httpServletResponse.setContentLength(response.length());
        }

        // Send the content to the client
        debug("Received status code: " + intProxyResponseCode,
              "Response: " + response);

        httpServletResponse.getWriter().write(response);
    }


    /**
     * The response body will be assumed to be gzipped if the GZIP header has been set.
     *
     * @param responseHeaders of response headers
     * @return true if the body is gzipped
     */
    private boolean isBodyParameterGzipped(List<Header> responseHeaders) {
        for (Header header : responseHeaders) {
            if (header.getValue().equals("gzip")) {
                return true;
            }
        }
        return false;
    }

    /**
     * A highly performant ungzip implementation. Do not refactor this without taking new timings.
     * See ElementTest in ehcache for timings
     *
     * @param gzipped the gzipped content
     * @return an ungzipped byte[]
     * @throws java.io.IOException when something bad happens
     */
    private byte[] ungzip(final byte[] gzipped) throws IOException {
        final GZIPInputStream inputStream = new GZIPInputStream(new ByteArrayInputStream(gzipped));
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(gzipped.length);
        final byte[] buffer = new byte[FOUR_KB];
        int bytesRead = 0;
        while (bytesRead != -1) {
            bytesRead = inputStream.read(buffer, 0, FOUR_KB);
            if (bytesRead != -1) {
                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }
        }
        byte[] ungzipped = byteArrayOutputStream.toByteArray();
        inputStream.close();
        byteArrayOutputStream.close();
        return ungzipped;
    }

    public String getServletInfo() {
        return "GWT Proxy Servlet";
    }

    /**
     * Retrieves all of the headers from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestHeaders(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an Enumeration of all of the header names sent by the client
        Enumeration enumerationOfHeaderNames = httpServletRequest.getHeaderNames();
        while (enumerationOfHeaderNames.hasMoreElements()) {
            String stringHeaderName = (String) enumerationOfHeaderNames.nextElement();
            if (stringHeaderName.equalsIgnoreCase(STRING_CONTENT_LENGTH_HEADER_NAME)) {
                continue;
            }
            // As per the Java Servlet API 2.5 documentation:
            //		Some headers, such as Accept-Language can be sent by clients
            //		as several headers each with a different value rather than
            //		sending the header as a comma separated list.
            // Thus, we get an Enumeration of the header values sent by the client
            Enumeration enumerationOfHeaderValues = httpServletRequest.getHeaders(stringHeaderName);
            while (enumerationOfHeaderValues.hasMoreElements()) {
                String stringHeaderValue = (String) enumerationOfHeaderValues.nextElement();
                // In case the proxy host is running multiple virtual servers,
                // rewrite the Host header to ensure that we get content from
                // the correct virtual server
                if (stringHeaderName.equalsIgnoreCase(STRING_HOST_HEADER_NAME)) {
                    stringHeaderValue = getProxyHostAndPort();
                }
                Header header = new Header(stringHeaderName, stringHeaderValue);
                // Set the same header on the proxy request
                httpMethodProxyRequest.setRequestHeader(header);
            }
        }
    }

    /**
     * Retrieves all of the cookies from the servlet request and sets them on
     * the proxy request
     *
     * @param httpServletRequest The request object representing the client's
     *                            request to the servlet engine
     * @param httpMethodProxyRequest The request that we are about to send to
     *                                the proxy host
     */
    @SuppressWarnings("unchecked")
    private void setProxyRequestCookies(HttpServletRequest httpServletRequest, HttpMethod httpMethodProxyRequest) {
        // Get an array of all of all the cookies sent by the client
        Cookie[] cookies = httpServletRequest.getCookies();
        if (cookies == null) {
            return;
        }

        for (Cookie cookie : cookies) {
            cookie.setDomain(stringProxyHost);
            cookie.setPath(httpServletRequest.getServletPath());
            httpMethodProxyRequest.setRequestHeader("Cookie", cookie.getName() + "=" + cookie.getValue() + "; Path=" + cookie.getPath());
        }
    }

    // Accessors
    private String getProxyURL(HttpServletRequest httpServletRequest) {
        // Set the protocol to HTTP
        String protocol = (isSecure) ? "https://" : "http://";
        String stringProxyURL = protocol + this.getProxyHostAndPort();

        // simply use whatever servlet path that was part of the request as opposed to getting a preset/configurable proxy path
        if (!removePrefix) {
            stringProxyURL += httpServletRequest.getServletPath();
        }
        stringProxyURL += "/";
        
        // Handle the path given to the servlet
        String pathInfo = httpServletRequest.getPathInfo();
        if (pathInfo != null && pathInfo.startsWith("/")) {
            if (stringProxyURL != null && stringProxyURL.endsWith("/")) {
                // avoid double '/'
                stringProxyURL += pathInfo.substring(1);
            }
        } else {
            stringProxyURL += httpServletRequest.getPathInfo();
        }
        // Handle the query string
        if (httpServletRequest.getQueryString() != null) {
            stringProxyURL += "?" + httpServletRequest.getQueryString();
        }
        
        return stringProxyURL;
    }

    private String getProxyHostAndPort() {
        if (this.getProxyPort() == 80) {
            return this.getProxyHost();
        } else {
            return this.getProxyHost() + ":" + this.getProxyPort();
        }
    }

    protected String getProxyHost() {
        return this.stringProxyHost;
    }

    protected void setProxyHost(String stringProxyHostNew) {
        this.stringProxyHost = stringProxyHostNew;
    }

    protected int getProxyPort() {
        return this.intProxyPort;
    }

    protected void setSecure(boolean secure) {
        this.isSecure = secure;
    }
    
    protected void setFollowRedirects(boolean followRedirects) {
        this.followRedirects = followRedirects;
    }

    protected void setProxyPort(int intProxyPortNew) {
        this.intProxyPort = intProxyPortNew;
    }

    protected String getProxyPath() {
        return this.stringProxyPath;
    }

    protected void setProxyPath(String stringProxyPathNew) {
        this.stringProxyPath = stringProxyPathNew;
    }

    protected void setRemovePrefix(boolean removePrefix) {
        this.removePrefix = removePrefix;
    }

    protected int getMaxFileUploadSize() {
        return this.intMaxFileUploadSize;
    }

    protected void setMaxFileUploadSize(int intMaxFileUploadSizeNew) {
        this.intMaxFileUploadSize = intMaxFileUploadSizeNew;
    }

    private void debug(String ... msg) {
        for (String m : msg) {
            System.out.println("[DEBUG] " + m);
        }
    }
}

I generally subclass ProxyServlet to provide my own configuration:

public class MyProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {
        setFollowRedirects(true);
        setRemovePrefix(false);
        setProxyPort(8080);
    }
}

Here's another example that reads configuration settings from web.xml and proxies to a different domain name:

public class AlternateHostProxyServlet extends ProxyServlet {

    @Override
    public void init(ServletConfig servletConfig) {

        setProxyHost(servletConfig.getInitParameter("proxyHost"));

        String secure = servletConfig.getInitParameter("secure");
        if (secure != null) {
            setSecure(Boolean.valueOf(secure));
        }

        setFollowRedirects(false);
        setRemovePrefix(true);
        setProxyPort(80);
    }
}

After you've added these to your project, simply map the servlet (and its path) in your *.gwt.xml file (if you're using GWT) and your web.xml.