Search for [jQuery] in weblog rd

Best Practices for using Foundation with AngularJS Revisited

2015-02-19T09:38:42-07:00

A couple weeks ago I wrote about using Foundation with AngularJS. Based on research I'd done, I concluded that it was best to use Foundation for Apps for any webapps my client created and Foundation for Sites for any websites (e.g. a WordPress-based intranet).

After doing my initial research, I did some prototyping with Foundation for Apps (F4A). What I discovered is that F4A does not include all the same components as Foundation for Sites (F5). For example, the top-bar and dropdown functionality are missing. I posted my issues to the Foundation Forums.

The response I received:

It should work. You would need to copy over all the Scss and global mixins that you used in top-bar or at least all the output CSS from it. Otherwise there is no reason the components won't fit into the grid.

I was able to import Foundation for Sites into my project by adding it to bower.json:

  "dependencies": {
    "foundation-apps": "~1.0.2",
    "foundation": "~5.5.1"
  }

After doing this, I added the new path to Gulpfile.js:

var sassPaths = [
  'client/assets/scss',
  'bower_components/foundation/scss',
  'bower_components/foundation-apps/scss'
];

After making this change, the top-bar rendered and my dropdowns worked. Since there was no jQuery in the page, I thought this might be a viable option. However, Jason Demitri quickly pointed out it probably wouldn't work with mobile. He was right.

While using F4A, I noticed that its components, and much of its look-n-feel, was different than F5. If you look at its Email App template, you'll see it looks kinda like a mobile app, even in a desktop browser. After trying F4A myself, I decided that F4A wasn't for us. First of all, it doesn't seem to provide a consistent look and feel with a website that's written using F5. Furthermore, F4A only supports IE10+. In the healthcare industry, there's a lot of older browsers out there, so my client needs to support IE9 as a minimum.

For these reasons, I decided to try Angular directives for Foundation. I took a prototype I'd written with F5, removed its JavaScript, added Angular Foundation + Foundation dependencies to bower.json, added references to the respective scripts in index.html and added 'mm.foundation' as a dependency in app.js. The experiment worked beautifully and I was quite happy with the results. I shared my findings with the team and we decided Angular Foundation is the best way to integrate Foundation and AngularJS.

F4A is pretty new and I imagine it'll add more of F5's features as it evolves. However, I don't know if the two will ever be so similar that they can live side-by-side and allow a seamless experience for users. If you're interested in mixing F4A and F5, you might want to watch Jason Demitri's foundationUltra. This project combines Angular Foundation, Foundation for Sites, Foundation for Apps and Font Awesome. You can see a demo at http://relutiondev.github.io/foundationUltra/.

Getting Started with AngularJS

2015-01-29T11:12:38-07:00

I was hired by my current client in November to help them choose a technology stack for developing modern web applications. In our first sprint, we decided to look at JavaScript MVC frameworks. I suggested AngularJS, Ember.js and React. Since most of the team was new to JavaScript MVC, I decided to create a tutorial for them. I tried to make it easy so they could learn how to write a simple web application with AngularJS. I thought others could benefit from this article as well, so I asked (and received) permission from my client to publish it here.

What you'll build

You'll build a simple web application with AngularJS. You'll also add search and edit features with mock data.

What you'll need

About 15-30 minutes
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and NPM installed.

Create a simple web application

Clone the angular-seed repository using git:

git clone https://github.com/angular/angular-seed.git angular-tutorial
cd angular-tutorial

There are two kinds of dependencies in this project: tools and angular framework code. The tools help manage and test the application.
- To get the tools that depend upon via npm, the node package manager.
- To get the angular code via bower, a client-side code package manager.
The project has preconfigured npm to automatically run bower so you can simply do:
```
npm install
```

Run the application

The project is configured with a simple development web server. The simplest way to start this server is:

npm start

Now browse to the app at http://localhost:8000/app/.

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the directory you cloned angular-seed to.

The Basics

Create an app/search/index.html file with the following HTML:

<form ng-submit="search()">
    <input type="search" name="search" ng-model="term">
    <button>Search</button>
</form>

Create app/search/search.js and define the routes (URLs) and controller for the search feature.

angular.module('myApp.search', ['ngRoute'])

    .config(['$routeProvider', function ($routeProvider) {
        $routeProvider
            .when('/search', {
                templateUrl: 'search/index.html',
                controller: 'SearchController'
            })
    }])

    .controller('SearchController', function () {
        console.log("In Search Controller...");
    });

Modify app/app.js and add the "myApp.search" module you created above.

angular.module('myApp', [
  'ngRoute',
  'myApp.view1',
  'myApp.view2',
  'myApp.version',
  'myApp.search'
])

Modify app/index.html and add a link to the search.js file.

<script src="view2/view2.js"></script>
<script src="search/search.js"></script>
<script src="components/version/version.js"></script>

Refresh your browser and navigate to http://localhost:8000/app/#/search. You should see an input field and search button. You should also see a log message printed in your browser's console. In Chrome, you can view the console using View > Developer > JavaScript Console. You can make it easier to navigate to this page by adding a menu item in app/index.html.
```
<ul class="menu">
  <li><a href="#/view1">view1</a></li>
  <li><a href="#/view2">view2</a></li>
  <li><a href="#/search">search</a></li>
</ul>
```

This section has shown you how to add a new controller and view to a basic AngularJS application. This was fairly simple to create. The next section shows you how to create a fake backend API.

The Backend

To get search results, you're going to create a SearchService that makes HTTP requests. These HTTP requests will be handled by a mock backend using some of Angular's built-in mocking tools. The backend implementation was created using AngularJS Backend-less Development Using a $httpBackend Mock.

Add a SearchService to app/search/search.js. This is done in Angular using its Factory Recipe. Make sure to remove the semicolon from the .controller code block.

.controller('SearchController', function () {
    console.log("In Search Controller...");
})

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        }
    };
    return service;
});

Inject the SearchService into the SearchController and use it to get results from the backend. The form in app/search/index.html calls the search() function. The "term" is defined by the input field in this page with ng-model="term".

.controller('SearchController', function ($scope, SearchService) {
    $scope.search = function () {
        console.log("Search term is: " + $scope.term);
        SearchService.query($scope.term).then(function (response) {
            $scope.searchResults = response.data;
        });
    };
})

If you try to search for a "foo" term now, you'll see the following error in your console.

Search term is: foo
GET http://localhost:8000/search/foo 404 (Not Found)

Create app/search/mock-api.js for the fake backend. Populate it with the following JavaScript.

// We will be using backend-less development
// $http uses $httpBackend to make its calls to the server
// $resource uses $http, so it uses $httpBackend too
// We will mock $httpBackend, capturing routes and returning data
angular.module('myApp')
    .service('ServerDataModel', function ServerDataModel() {
        this.data = [
            {
                id: 1,
                name: "Peyton Manning",
                phone: "(303) 567-8910",
                address: {
                    street: "1234 Main Street",
                    city: "Greenwood Village",
                    state: "CO",
                    zip: "80111"
                }
            },
            {
                id: 2,
                name: "Demaryius Thomas",
                phone: "(720) 213-9876",
                address: {
                    street: "5555 Marion Street",
                    city: "Denver",
                    state: "CO",
                    zip: "80202"
                }
            },
            {
                id: 3,
                name: "Von Miller",
                phone: "(917) 323-2333",
                address: {
                    street: "14 Mountain Way",
                    city: "Vail",
                    state: "CO",
                    zip: "81657"
                }
            }
        ];

        this.getData = function () {
            return this.data;
        };

        this.search = function (term) {
            if (term == "" || term == "*") {
                return this.getData();
            }
            // find the name that matches the term
            var list = $.grep(this.getData(), function (element, index) {
                term = term.toLowerCase();
                return (element.name.toLowerCase().match(term));
            });

            if (list.length === 0) {
                return [];
            } else {
                return list;
            }
        };
    })
    .run(function ($httpBackend, ServerDataModel) {

        $httpBackend.whenGET(/\/search\/\w+/).respond(function (method, url, data) {
            // parse the matching URL to pull out the term (/search/:term)
            var term = url.split('/')[2];

            var results = ServerDataModel.search(term);

            return [200, results, {Location: '/search/' + term}];
        });

        $httpBackend.whenGET(/search\/index.html/).passThrough();
        $httpBackend.whenGET(/view/).passThrough();
    });

This file uses jQuery.grep(), so you'll need to install jQuery. Modify bower.json and add jQuery to the list of dependencies.
```
"dependencies": {
  ...
  "html5-boilerplate": "~4.3.0",
  "jquery": "~1.10.x"
}
```
Stop the npm process, run "npm install" to download and install jQuery, then "npm start" again.

Modify app/app.js and add a dependency on ngMockE2E.

angular.module('myApp', ['ngMockE2E',
  'ngRoute',
  ...

Modify app/index.html to include references to jquery.js, angular-mocks.js and mock-api.js.

<script src="bower_components/angular-route/angular-route.js"></script>
<script src="bower_components/angular-mocks/angular-mocks.js"></script>
<script src="bower_components/jquery/jquery.js"></script>
...
<script src="search/mock-api.js"></script>
<script src="components/version/version.js"></script>

Add the following HTML to app/search/index.html to display the search results.
```
<div>
    <pre>{{ searchResults | json}}</pre>
</div>
```
After making this change, you should be able to search for "p", "d" or "v" and see results as JSON.

To make the results more readable, change the above HTML to use a <table> and Angular's ng-repeat directive.

<table ng-show="searchResults.length" style="width: 100%">
    <thead>
    <tr>
        <th>Name</th>
        <th>Phone</th>
        <th>Address</th>
    </tr>
    </thead>
    <tbody>
    <tr ng-repeat="person in searchResults">
        <td>{{person.name}}</td>
        <td>{{person.phone}}</td>
        <td>{{person.address.street}}<br/>
            {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
        </td>
    </tr>
    </tbody>
</table>

This section has shown you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an Edit Feature

Modify app/search/index.html to add a link for editing a person.

<table ng-show="searchResults.length" style="width: 100%">
    ...
    <tr ng-repeat="person in searchResults">
        <td><a href="" ng-click="edit(person)">{{person.name}}</a></td>
        ...
    </tr>
</table>

Add an edit() function to SearchController. Notice that the $location service dependency has been added to the controller's initialization function.

.controller('SearchController', function ($scope, $location, SearchService) {
    ...

    $scope.edit = function (person) {
        $location.path("/edit/" + person.id);
    }
})

Create a route to handle the edit URL in app/search/search.js.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/edit/:id', {
            templateUrl: 'search/edit.html',
            controller: 'EditController'
        });
}])

Create app/search/edit.html to display an editable form. The HTML below shows how you can use HTML5's data attributes to have valid attributes instead of ng-*.

<form ng-submit="save()">
    <div>
        <label for="name">Name:</label>
        <input type="text" data-ng-model="person.name" id="name">
    </div>
    <div>
        <label for="phone">Phone:</label>
        <input type="text" data-ng-model="person.phone" id="phone">
    </div>
    <fieldset>
        <legend>Address:</legend>
        <address style="margin-left: 50px">
            <input type="text" data-ng-model="person.address.street"><br/>
            <input type="text" data-ng-model="person.address.city">,
            <input type="text" data-ng-model="person.address.state" size="2">
            <input type="text" data-ng-model="person.address.zip" size="5">
        </address>
    </fieldset>
    <div>
        <button type="submit">Save</button>
    </div>
</form>

Modify SearchService to contain functions for finding a person by their id, and saving them.

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        },
        fetch: function (id) {
            return $http.get('/edit/' + id);
        },
        save: function(data) {
            return $http.post('/edit/' + data.id, data);
        }
    };
    return service;
});

Create EditController in app/search/search.js. $routeParams is an Angular service that allows you to grab parameters from a URL.

.controller('EditController', function ($scope, $location, $routeParams, SearchService) {
    SearchService.fetch($routeParams.id).then(function (response) {
        $scope.person = response.data;
    });

    $scope.save = function() {
        SearchService.save($scope.person).then(function(response) {
            $location.path("/search/" + $scope.person.name);
        });
    }
})

At the bottom of app/search/mock-api.js, in its run() function, add the following:

$httpBackend.whenGET(/\/search/).respond(function (method, url, data) {
    var results = ServerDataModel.search("");

    return [200, results];
});

$httpBackend.whenGET(/\/edit\/\d+/).respond(function (method, url, data) {
    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var results = ServerDataModel.find(id);

    return [200, results, {Location: '/edit/' + id}];
});

$httpBackend.whenPOST(/\/edit\/\d+/).respond(function(method, url, data) {
    var params = angular.fromJson(data);

    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var person = ServerDataModel.update(id, params);

    return [201, person, { Location: '/edit/' + id }];
});

$httpBackend.whenGET(/search\/edit.html/).passThrough();

Further up in the same file, add find() and update() methods to ServerDataModel.

this.getData = function () {
    return this.data;
};

this.search = function (term) {
    ...
};

this.find = function (id) {
    // find the game that matches that id
    var list = $.grep(this.getData(), function (element, index) {
        return (element.id == id);
    });
    if (list.length === 0) {
        return {};
    }
    // even if list contains multiple items, just return first one
    return list[0];
};

this.update = function (id, dataItem) {
    // find the game that matches that id
    var people = this.getData();
    var match = null;
    for (var i = 0; i < people.length; i++) {
        if (people[i].id == id) {
            match = people[i];
            break;
        }
    }
    if (!angular.isObject(match)) {
        return {};
    }
    angular.extend(match, dataItem);
    return match;
};

The <form> in app/search/edit.html calls a save() function to update a person's data. You already implemented this above. The function executes the logic below.

$location.path("/search/" + $scope.person.name);

Since the SearchController doesn't execute a search automatically when you execute this URL, add the following logic to do so in app/search/search.js. Note that $routeParams is added to the list of injected dependencies.

.controller('SearchController', function ($scope, $location, $routeParams, SearchService) {
    if ($routeParams.term) {
        SearchService.query($routeParams.term).then(function (response) {
            $scope.term = $routeParams.term;
            $scope.searchResults = response.data;
        });
    }
    ...
})

You'll also need to add a new route so search/term is recognized.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/search/:term', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        ...
}])

After making all these changes, you should be able to refresh your browser and search/edit/update a person's information. If it works - nice job!

Source Code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular-tutorial.

There are three commits that make the changes for the three main steps in this tutorial:

Extra Credit

Deploy your completed app to Heroku. See running version of this tutorial at https://angular-123.herokuapp.com.

Summary

I hope you've enjoyed this quick-and-easy tutorial on how to get started with AngularJS. In a future tutorial, I'll show you how to write unit tests and integration tests for this application. If you're in Denver next Tuesday, I'll be speaking about AngularJS at Denver's Open Source Users Group.

If you're a Java developer that's interested in developing with AngularJS and Spring Boot, you might want to checkout the recently released JHipster 2.0.

AppFuse, Reduced

2014-12-16T06:03:31-07:00

In November, I had some time off between clients. To occupy my time, I exercised my body and brain a bit. I spent a couple hours a day exercising and a few hours a day working on AppFuse. AppFuse isn't used to start projects nearly as much as it once was. This makes sense since there's been a ton of innovation on the JVM and there's lots of get-started-quickly frameworks now. Among my favorites are Spring Boot, JHipster, Grails and Play.

You can see that AppFuse's community activity has decreased quite a bit over the years by looking at its mailing list traffic.

Even though there's not a lot of users talking on the mailing list, it still seems to get quite a few downloads from Maven Central.

I think the biggest value that AppFuse provides now is a learning tool for those who work on it. Also, it's a good place to show other developers how they can evolve with open source frameworks (e.g. Spring, Hibernate, JSF, Tapestry, Struts) over several years. Showing how we migrated to Spring MVC Test, for example, might be useful. The upcoming move to Spring Data instead of our Generic DAO solution might be interesting as well.

Regardless of whether AppFuse is used a lot or not, it should be easy to maintain. Over the several weeks, I made some opinionated changes and achieved some pretty good progress on simplifying things and making the project easier to maintain. The previous structure has a lot of duplicate versions, properties and plugin configurations between different projects. I was able to leverage Maven's inheritance model to make a number of improvements:

Changed AppFuse's parent to be based on the Spring IO Platform. This project is a dependency manager that defines version numbers for open source projects that work well with Spring.
Defined plugins, their versions and configurations in <pluginManagement>.
Defined dependencies, their versions and exclusions in <dependencyManagement>.
Simplified archetypes so new projects have minimal dependencies. For example, here's a basic project's pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.company</groupId>
    <artifactId>springmvc-project</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>AppFuse Spring MVC Application</name>

    <parent>
        <groupId>org.appfuse</groupId>
        <artifactId>appfuse-web</artifactId>
        <version>3.5.0-SNAPSHOT</version>
    </parent>

    <build>
        <plugins>
            <plugin>
                <groupId>de.juplo</groupId>
                <artifactId>hibernate4-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>dbunit-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.appfuse</groupId>
            <artifactId>appfuse-${web.framework}</artifactId>
            <version>${appfuse.version}</version>
            <type>pom</type>
        </dependency>
    </dependencies>

    <properties>
        <amp.genericCore>true</amp.genericCore>
        <amp.fullSource>false</amp.fullSource>
        <dao.framework>hibernate</dao.framework>
        <db.name>mydatabase</db.name>
        <web.framework>spring</web.framework>

        <!-- Framework/Plugin versions -->
        <appfuse.version>3.5.0-SNAPSHOT</appfuse.version>
        <java.version>1.7</java.version>
    </properties>

    <profiles>
        <profile>
            <id>itest</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>webtest-maven-plugin</artifactId>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>webtest-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </reporting>
</project>

The pull request for these changes says it all:

AppFuse +4,822 -14,369
AppFuse Light +776 -4,687

That's right, I was able to eliminate a good chunk of code without affecting any of AppFuse's functionality¹. I think we can all agree that less code == easier maintenance. This theme will continue as we work on future releases.

Other improvements include migrating all tests to use JUnit4, integrating Spring MVC Test, and configuring the surefire plugin to run tests in parallel. I also The build-helper-maven-plugin is now used to find open ports for Cargo to run and a lot of testing was done to ensure you can build/test multiple AppFuse-derived projects at the same time. Finally, I migrated to the hibernate4-maven-plugin and upgraded to Tapestry 5.4.

In the next version of AppFuse, I plan to remove as much XML as possible - moving all of the configuration to Spring's JavaConfig. We'll also be moving to Java 8 as a minimum. I'm even considering getting rid of all the pom.xml files in favor of another build language that requires less code.

In other words, the upcoming 3.5 release will be the last release that supports Java 7 and uses Spring's XML for configuration. AppFuse 4.0 will strive for #NoXML. The project's roadmap has more details on additional hopes and dreams.

We'd love to hear your feedback on these change. Do you like the simplification theme? Are you OK with having AppFuse as a parent in your projects?

1. For project and code stats, see AppFuse on Open Hub.

Developing an iOS Native App with Ionic

2014-03-27T16:38:55-06:00

In my current project, I've been helping a client develop a native iOS app for their customers. It's written mostly in Objective-C and talks to a REST API. I talked about how we documented our REST API a couple days ago. We developed a prototype for this application back in December, using AngularJS and Bootstrap. Rather than using PhoneGap, we loaded our app in a UIWebView.

It all seemed to work well until we needed to read an activation code with the device's camera. Since we didn't know how to do OCR in JavaScript, we figured a mostly-native app was the way to go. We hired an outside company to do iOS development in January and they've been developing the app since the beginning of February. In the last couple weeks, we encountered some screens that seemed fitting for HTML5, so we turned back to our AngularJS prototype.

The prototype used Bootstrap heavily, but we quickly learned it didn't look like an iOS 7 app, which is what our UX Designer requested. A co-worker pointed out Ionic, developed by Drifty. It's basically Bootstrap for Native, so the apps you develop look and behave like a mobile application.

What is Ionic?
Free and open source, Ionic offers a library of mobile-optimized HTML, CSS and JS components for building highly interactive apps. Built with Sass and optimized for AngularJS.

I started developing with Ionic a few weeks ago. Using its CSS classes and AngularJS directives, I was able to create several new screens in a matter of days. Most of the time, I was learning new things: how to override its back button behavior (to launch back into the native app), how to configure routes with ui-router, and how to make the $ionicLoading service look native. Now that I know a lot of the basics, I feel like I can really crank out some code.

Tip: I learned how subviews work with ui-router thanks to a YouTube video of Tim Kindberg on Angular UI-Router. However, subviews never fully made sense until I saw Jared Bell's diagram.

To demonstrate how easy it is to use Ionic, I whipped up a quick example application. You can get the source on GitHub at https://github.com/mraible/boot-ionic. The app is a refactored version of Josh Long's x-auth-security that uses Ionic instead of raw AngularJS and Bootstrap. To keep things simple, I did not develop the native app that wraps the HTML.

Below are the steps I used to convert from AngularJS + Bootstrap to Ionic. If you want to convert a simple AngularJS app to use Ionic, hopefully this will help.

1. Download Ionic and add it to your project.

Ionic 1.0 Beta was released earlier this week. You can download it from here. Add its files to your project. In this example, I added them to src/main/resources/public. In my index.html, I removed Bootstrap's CSS and replaced it with Ionic's.

-    <link href="webjars/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+    <link rel="stylesheet" href="css/ionic.css"/>
  </head>
-<body style="padding-top: 60px">
+<body>

Next, I replaced Angular, Bootstrap and jQuery's JavaScript references.

-    <script src="webjars/jquery/2.0.3/jquery.js"></script>
-    <script src="webjars/bootstrap/3.1.1/js/bootstrap.min.js"></script>
-    <script src="webjars/angularjs/1.2.13/angular.js"></script>
+    <script src="js/ionic.bundle.js"></script>
     <script src="webjars/angularjs/1.2.13/angular-resource.js"></script>
-    <script src="webjars/angularjs/1.2.13/angular-route.js"></script>
     <script src="webjars/angularjs/1.2.13/angular-cookies.js"></script>

What about WebJars?
You might ask - why not use WebJars? You can, once this pull request is accepted and an updated version is deployed to Maven central. Here's how the application would change.

2. Change from Angular's Router to ui-router.

Ionic uses ui-router for matching URLs and loading particular pages. The raw Angular routing looks pretty similar to how it does with ui-router, except it uses a $stateProvider service instead of $routeProvider. You'll notice I also added 'ionic' as a dependency.

-angular.module('exampleApp', ['ngRoute', 'ngCookies', 'exampleApp.services'])
+angular.module('exampleApp', ['ionic', 'ngCookies', 'exampleApp.services'])
 	.config(
-		[ '$routeProvider', '$locationProvider', '$httpProvider', function($routeProvider, $locationProvider, $httpProvider) {
+		[ '$stateProvider', '$urlRouterProvider', '$httpProvider', function($stateProvider, $urlRouterProvider, $httpProvider) {

-	    $routeProvider.when('/create', { templateUrl: 'partials/create.html', controller: CreateController});
+           $stateProvider.state('create', {url: '/create', templateUrl: 'partials/create.html', controller: CreateController})
+               .state('edit', {url: '/edit/:id', templateUrl: 'partials/edit.html', controller: EditController})
+               .state('login', {url: '/login', templateUrl: 'partials/login.html', controller: LoginController})
+               .state('index', {url: '/index', templateUrl: 'partials/index.html', controller: IndexController});

-	    $routeProvider.when('/edit/:id', { templateUrl: 'partials/edit.html', controller: EditController});
-	    $routeProvider.when('/login', { templateUrl: 'partials/login.html', controller: LoginController});
-	    $routeProvider.otherwise({templateUrl: 'partials/index.html', controller: IndexController});
-
-	    $locationProvider.hashPrefix('!');
+	    $urlRouterProvider.otherwise('/index');

3. Add Ionic elements to your index.html.

In contrast to Bootstrap's navbar, Ionic has header and footer elements. Rather than using a ng-view directive, you use an <ion-nav-view>. It's a pretty slick setup once you understand it, especially since they allow you to easily override back-button behavior and nav buttons.

-    <nav class="navbar navbar-fixed-top navbar-default" role="navigation">
-        <!-- lots of HTML here -->
-    </nav>
-
-    <div class="container">
-        <div class="alert alert-danger" ng-show="error">{{error}}</div>
-        <div ng-view></div>
-    </div>
+    <ion-nav-bar class="bar-positive nav-title-slide-ios7"></ion-nav-bar>
+    <ion-nav-view animation="slide-left-right">
+        <div class="alert alert-danger" ng-show="error">{{error}}</div>
+    </ion-nav-view>
+    <ion-footer-bar class="bar-dark" ng-show="user">
+        <button class="button button-assertive" ng-click="logout()">
+            Logout
+        </button>
+    </ion-footer-bar>

4. Change your templates to use <ion-view> and <ion-content>.

After routes are migrated and basic navigation is working, you'll need to modify your templates to use <ion-view> and <ion-content>. Here's a diff from the most complicated page in the app.

-<div style="float: right">
-	<a href="#!/create" class="btn btn-default" ng-show="hasRole('ROLE_ADMIN')">Create</a>
-</div>
-<div class="page-header">
-	<h3>News</h3>
-</div>
+<ion-view title="News">
+    <ion-content>
+        <ion-nav-buttons side="left">
+            <div class="buttons" ng-show="hasRole('ROLE_ADMIN')">
+                <button class="button button-icon icon ion-ios7-minus-outline"
+                        ng-click="data.showDelete = !data.showDelete"></button>
+            </div>
+        </ion-nav-buttons>
+        <ion-nav-buttons side="right">
+            <a href="#/create" class="button button-icon icon ion-ios7-plus-outline"
+               ng-show="hasRole('ROLE_ADMIN')"></a>
+        </ion-nav-buttons>

-<div ng-repeat="newsEntry in newsEntries">
-	<hr />
-	<div class="pull-right">
-		<a ng-click="deleteEntry(newsEntry)" class="btn btn-xs btn-default" ng-show="hasRole('ROLE_ADMIN')">Remove</a>
-		<a href="#!/edit/{{newsEntry.id}}" class="btn btn-xs btn-default" ng-show="hasRole('ROLE_ADMIN')">Edit</a>
-	</div>
-	<h4>{{newsEntry.date | date}}</h4>
-	<p>{{newsEntry.content}}</p>
-</div>
-<hr />
+        <ion-list show-delete="data.showDelete" on-delete="deleteEntry(item)"
+                  option-buttons="itemButtons" can-swipe="hasRole('ROLE_ADMIN')">
+            <ion-item ng-repeat="newsEntry in newsEntries" item="newsEntry">
+                <h4>{{newsEntry.date | date}}</h4>
+                <p>{{newsEntry.content}}</p>
+            </ion-item>
+        </ion-list>
+    </ion-content>
+</ion-view>

I did migrate to use an <ion-list> with delete/options buttons, so some additional JavaScript changes were needed.

-function IndexController($scope, NewsService) {
+function IndexController($scope, $state, NewsService) {

    $scope.newsEntries = NewsService.query();

+     $scope.data = {
+         showDelete: false
+     };
+
    $scope.deleteEntry = function(newsEntry) {
         newsEntry.$remove(function() {
              $scope.newsEntries = NewsService.query();
         });
    };
+
+     $scope.itemButtons = [{
+         text: 'Edit',
+         type: 'button-assertive',
+         onTap: function (item) {
+              $state.go('edit', {id: item.id});
+         }
+     }];
}

Screenshots

After making all these changes, the app looks pretty good in Chrome.

Tips and Tricks

In additional to figuring out how to use Ionic, I discovered a few other tidbits along the way. First of all, we had a different default color for the header. Since Ionic uses generic color names (e.g. light, stable, positive, calm), I found it easy to change the default value for "positive" and then continue to use their class names.

Modifying CSS variable colors
To modify the base color for "positive", I cloned the source, and modified scss/_variables.scss.

$light: #fff !default;
$stable: #f8f8f8 !default;
-$positive: #4a87ee !default;
+$positive: #589199 !default;
$calm: #43cee6 !default;
$balanced: #66cc33 !default;
$energized: #f0b840 !default;

After making this change, I ran "grunt" and copied dist/css/ionic.css into our project.

iOS Native Integration
Our app uses a similar token-based authentication mechanism as x-auth-security, except its backed by Crowd. However, since users won't be logging directly into the Ionic app, we added the "else" clause in app.js to allow a token to be passed in via URL. We also allowed the backend API path to be overridden.

/* Try getting valid user from cookie or go to login page */
var originalPath = $location.path();
$location.path("/login");
var user = $cookieStore.get('user');

if (user !== undefined) {
    $rootScope.user = user;
    $http.defaults.headers.common[xAuthTokenHeaderName] = user.token;
    $location.path(originalPath);
} else {
    // token passed in from native app
    var authToken = $location.search().token;
    if (authToken) {
        $http.defaults.headers.common['X-Auth-Token'] = authToken;
    }
}

// allow overriding the base API path
$rootScope.apiPath = '/api/v1.0';
if ($location.search().apiPath) {
    $rootScope.apiPath = $location.search().apiPath;
}

By adding this logic, the iOS app can pull up any particular page in a webview and let the Ionic app talk to the API. Here's what the Objective-C code looks like:

NSString *versionNumber = @"v1.0";
NSString *apiPath = @"https://server.com/api/";
NSString *authToken = [TemporaryDataStore sharedInstance].authToken;
// webapp is a symbolic link to the Ionic app, created with Angular Seed
NSString *htmlFilePath = [[NSBundle mainBundle] pathForResource:@"index" ofType:@"html" inDirectory:@"webapp/app"];

// Note: We need to do it this way because 'fileURLWithPath:' would encode the '#' to '%23" which breaks the html page
NSURL *htmlFileURL = [NSURL fileURLWithPath:htmlFilePath];

NSString *webappURLPath = [NSString stringWithFormat:@"%@#/news?apiPath=%@%@&token=%@",
                           htmlFileURL.absoluteString, apiPath, versionNumber, authToken];

// Now convert the string to a URL (doesn't seem to encode the '#' this way)
NSURL *webappURL = [NSURL URLWithString:webappURLPath];
[super updateWithURL:webappURL];

We also had to write some logic to navigate back to the native app. We used a custom URL scheme to do this, and the Ionic app simply called it. To override the default back button, I added an "ng-controller" attribute to <ion-nav-bar> and added a custom back button.

<ion-nav-bar class="bar-positive nav-title-slide-ios7" ng-controller="NavController">
    <ion-nav-back-button class="button-icon" ng-click="goBack()">
        <i class="ion-arrow-left-c"></i>
    </ion-nav-back-button>
</ion-nav-bar>

To detect if the app was loaded by iOS (vs. a browser, which we tested in), we used the following logic:

// set native app indicator
if (document.location.toString().indexOf('appName.app') > -1) {
    $rootScope.isNative = true;
}

Our Ionic app has three entry points, defined by "stateName1", "stateName2" and "stateName3" in this example. The code for our NavController handles navigating back normally (when in a browser) or back to the native app. The "appName" reference below is a 3-letter acronym we used for our app.

.controller('NavController', function($scope, $ionicNavBarDelegate, $state) {
    $scope.goBack = function() {
        if ($scope.isNative && backToNative($state)) {
            location.href='appName-ios://back';
        } else {
            $ionicNavBarDelegate.back();
        }
    };

    function backToNative($state) {
        var entryPoints = ['stateName1', 'stateName2', 'stateName3'];
        return entryPoints.some(function (entry) {
            return $state.current === $state.get(entry);
        });
    }
})

Summary

I've enjoyed working with Ionic over the last month. The biggest change I've had to make to our AngularJS app has been to integrate ui-router. Apart from this, the JavaScript didn't change much. However, the HTML had to change quite a bit. As far as CSS is concerned, I found myself tweaking things to fit our designs, but less so than I did with Bootstrap. When I've run into issues with Ionic, the community has been very helpful on their forum. It's the first forum I've used that's powered by Discourse, and I dig it.

You can find the source from this article in my boot-ionic project. Clone it and run "mvn spring-boot:run", then open http://localhost:8080.

If you're looking to create a native app using HTML5 technologies, I highly recommend you take a look at Ionic. We're glad we did. Angular 2.0 will target mobile apps and Ionic is already making them look pretty damn good.

Using Grunt with AngularJS for Front End Optimization

2014-01-15T12:15:52-07:00

I'm passionate about front end optimization and have been for years. My original inspiration was Steve Souders and his Even Faster Web Sites talk at OSCON 2008. Since then, I've optimized this blog, made it even faster with a new design, doubled the speed of several apps for clients and showed how to make AppFuse faster. As part of my Devoxx 2013 presentation, I showed how to do page speed optimization in a Java webapp.

I developed a couple AngularJS apps last year. To concat and minify their stylesheets and scripts, I used mechanisms that already existed in the projects. On one project, it was Ant and its concat task. On the other, it was part of a Grails application, so I used the resources and yui-minify-resources plugins.

The Angular project I'm working on now will be published on a web server, as well as bundled in an iOS native app. Therefore, I turned to Grunt to do the optimization this time. I found it to be quite simple, once I figured out how to make it work with Angular. Based on my findings, I submitted a pull request to add Grunt to angular-seed.

Below are the steps I used to add Grunt to my Angular project.

Install Grunt's command line interface with "sudo npm install -g grunt-cli".
Edit package.json to include a version number (e.g. "version": "1.0.0").

Add Grunt plugins in package.json to do concat/minify/asset versioning:

    "grunt": "~0.4.1",
    "grunt-contrib-concat": "~0.3.0",
    "grunt-contrib-uglify": "~0.2.7",
    "grunt-contrib-cssmin": "~0.7.0",
    "grunt-usemin": "~2.0.2",
    "grunt-contrib-copy": "~0.5.0",
    "grunt-rev": "~0.1.0",
    "grunt-contrib-clean": "~0.5.0"

Run "sudo npm install" to install the project's dependencies.

Create a Gruntfile.js that runs all the plugins.

module.exports = function (grunt) {

    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),

        clean: ["dist", '.tmp'],

        copy: {
            main: {
                expand: true,
                cwd: 'app/',
                src: ['**', '!js/**', '!lib/**', '!**/*.css'],
                dest: 'dist/'
            },
            shims: {
                expand: true,
                cwd: 'app/lib/webshim/shims',
                src: ['**'],
                dest: 'dist/js/shims'
            }
        },

        rev: {
            files: {
                src: ['dist/**/*.{js,css}', '!dist/js/shims/**']
            }
        },

        useminPrepare: {
            html: 'app/index.html'
        },

        usemin: {
            html: ['dist/index.html']
        },

        uglify: {
            options: {
                report: 'min',
                mangle: false
            }
        }
    });

    grunt.loadNpmTasks('grunt-contrib-clean');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-contrib-concat');
    grunt.loadNpmTasks('grunt-contrib-cssmin');
    grunt.loadNpmTasks('grunt-contrib-uglify');
    grunt.loadNpmTasks('grunt-rev');
    grunt.loadNpmTasks('grunt-usemin');

    // Tell Grunt what to do when we type "grunt" into the terminal
    grunt.registerTask('default', [
        'copy', 'useminPrepare', 'concat', 'uglify', 'cssmin', 'rev', 'usemin'
    ]);
};

Add comments to app/index.html so usemin knows what files to process. The comments are the important part, your files will likely be different.

<!-- build:css css/app-name.min.css -->
<link rel="stylesheet" href="lib/bootstrap/bootstrap.min.css"/>
<link rel="stylesheet" href="lib/font-awesome/font-awesome.min.css"/>
<link rel="stylesheet" href="lib/toaster/toaster.css"/>
<link rel="stylesheet" href="css/app.css"/>
<link rel="stylesheet" href="css/custom.css"/>
<link rel="stylesheet" href="css/responsive.css"/>
<!-- endbuild -->
...

<!-- build:js js/app-name.min.js -->
<script src="lib/jquery/jquery-1.10.2.min.js"></script>
<script src="lib/bootstrap/bootstrap.min.js"></script>
<script src="lib/angular/angular.min.js"></script>
<script src="lib/angular/angular-animate.min.js"></script>
<script src="lib/angular/angular-cookies.min.js"></script>
<script src="lib/angular/angular-resource.min.js"></script>
<script src="lib/angular/angular-route.min.js"></script>
<script src="lib/fastclick.min.js"></script>
<script src="lib/toaster/toaster.js"></script>
<script src="lib/webshim/modernizr.min.js"></script>
<script src="lib/webshim/polyfiller.min.js"></script>
<script src="js/app.js"></script>
<script src="js/services.js"></script>
<script src="js/controllers.js"></script>
<script src="js/filters.js"></script>
<script src="js/directives.js"></script>
<!-- endbuild -->

A couple of things to note: 1) the copy task copies the "shims" directory from Webshims lib because it loads files dynamically and 2) setting "mangle: false" on the uglify task is necessary for Angular's dependency injection to work. I tried to use grunt-ngmin with uglify and had no luck.

After making these changes, I'm able to run "grunt" and get an optimized version of my app in the "dist" folder of my project. For development, I continue to run the app from my "app" folder, so I don't currently have a need for watching and processing assets on-the-fly. That could change if I start using LESS or CoffeeScript.

The results speak for themselves: from 27 requests to 5 on initial load, and only 3 requests for less than 2K after that.

	YSlow	Page Speed
No optimization	75 27 HTTP requests / 464K	55/100
Apache optimization (gzip and expires headers)	89 initial load: 26 requests / 166K primed cache: 4 requests / 40K	88/100
Apache + concat/minified/versioned files	98 initial load: 5 requests / 136K primed cache: 3 requests / 1.4K	93/100

Update: Andreas Andreou has a nice tip on how to reduce the LOC in this example.

Add "matchdep" as a dependency in package.json (or run "sudo npm install matchdep --save-dev").
```
"matchdep": "~0.3.0"
```

Replace all the grunt.loadNpmTasks(...) calls with the following:

require('matchdep').filterDev('grunt-*').forEach(grunt.loadNpmTasks);

Thanks Andreas!

Developing with AngularJS - Part IV: Making it Pop

2013-09-12T10:54:29-06:00

Welcome to the final article in a series on my experience developing with AngularJS. I learned its concepts, beat my head against-the-wall with and finally tamed it enough to create a "My Dashboard" feature for a client. For previous articles, please see the following:

The last mile of development for the My Dashboard feature was to spice things up a bit and make it look better. We hired a design company to come up a new look and feel and they went to work. Within a week, we had a meeting with them and they presented a few different options. We picked the one we liked the best and went to work. Below are screenshots that I used to implement the new design.

At first, I thought implementing this design might take quite a bit of effort, since it looked like it used custom fonts. It's true we could use CSS3's @font-face, but I knew it might take awhile to find the right fonts with the appropriate licenses. When I received the screenshot below, I was pleased to see that all fonts were web-safe.

Design Elements

There are a number of elements in this new design that I had to create. For example, if numbers were only 1 digit, we had to add a leading zero to them in the summary band. Other design elements we needed to implement are listed below:

A background image that filled the page
Fade to white on summary widget titles
A responsive grid for summary widgets
Provide a colored background for odd rows in the summary grid
Add a "Show More" band at the bottom of Tasks, Summary and Reports when there's more items to display

In addition to these elements, there was quite a bit of work to conform to the new colors, fonts and drop-shadows. I implemented all of these using CSS3 (border-radius, box-shadow, box-sizing, linear-gradient), and lots of trial-and-error. To use the best fonts across various devices, I used CSS-Trick's Font Stacks.

New Background

The new background shown in the screenshots above has a light source in the middle of it. Therefore, it's impossible to tile/repeat it across the page since it's not uniform. To make it work, I used a 1024 x 768 image and CSS3's background-size: cover. For more information on background-size, see SitePoint's How to Resize Background Images with CSS3. This worked great on smaller screens, but we noticed some issues on 30" monitors. Therefore, we ended up getting a new repeatable background and stopped using background-size.

LeadingZero filter

For the first leading zero feature, I wrote an Angular filter. I put the code for this in filters.js:

filter('leadingZero', function() {
    return function(input) {
        if (input.length === 1) {
            return "0" + input;
        } else if (input.length > 2) {
            return "+99";
        } else {
            return input;
        }
    }
});

This filter is used in the HTML template as follows:

<div class="summary-value">{{widget.value | leadingZero}}</div>

Text Fade Out

To implement the fade-to-white text in summary titles, I started with this tutorial. I quickly discovered that it worked best for vertical text blocks and not for horizontal text. Then I found Text Ellipsis with Gradient Fade in Pure CSS, which uses :after to position a block over the text that fades to white. Since the title is not the right-most element (the numbers are), I had to figure out best positioning that worked cross-browser. Below is the CSS I used to implement this feature.

.dashboard .summary-title:after {
    display: block;
    position: absolute;
    right: 66px;
    top: 5px;
    bottom: 5px;
    width: 30px;
    background: -moz-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* FF3.6+ */
    background: -webkit-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* Chrome10+,Safari5.1+ */
    background: -o-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* Opera 11.10+ */
    background: -ms-linear-gradient(left,  rgba(255,255,255,0) 0%, #fff 20px); /* IE10+ */
    background: linear-gradient(to right,  rgba(255,255,255,0) 0%, #fff 20px); /* W3C */
    filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#00ffffff', endColorstr='#ffffff',GradientType=1 ); /* IE6-9 */
    content: "";
}

Responsive Grid

To implement the responsive grid of summary widgets, I started with Codrops' Responsive Full Width Grid Tutorial. This proved to be a great model and I used the following CSS to position all the <li>'s appropriately. In the code below, .summary-item is the class on the <li> elements.

.dashboard .summary-item {
    border-right: 1px solid #d1d1d1;
    border-bottom: 1px solid #d1d1d1;
    /* put the border on the inside of the box */
    -webkit-box-sizing: border-box;
    -moz-box-sizing: border-box;
    -ms-box-sizing: border-box;
    box-sizing: border-box;
    font-family: Constantia, "Lucida Bright", Lucidabright, "Lucida Serif", Lucida, "DejaVu Serif", "Bitstream Vera Serif", "Liberation Serif", Georgia, serif;
    font-size: 14px;
    color: #666;
    height: 50px;
    box-shadow: inset 0 0 6px rgba(0,0,0, 0.25);
    /* responsive grid */
    position: relative;
    float: left;
    overflow: hidden;
    width: 25% /* Fallback */
    width: -webkit-calc(100% / 4);
    width: calc(100% / 4);
}

@media screen and (max-width: 1400px) {
    .dashboard .summary-item {
        width: 33.33333333333333%; /* Fallback */
        width: -webkit-calc(100% / 3);
        width: calc(100% / 3);
    }
}

@media screen and (max-width: 1000px) {
    .dashboard .summary-item {
        width: 50%; /* Fallback */
        width: -webkit-calc(100% / 2);
        width: calc(100% / 2);
    }
}

This worked great in most browsers, but we did find an issue with IE9. When squishing/expanding the browser window, sometimes there would be a blank column on the right side. To fix this, I changed the width on the default .summary-item to be 25%, and removed the lines with calc.

.dashboard .summary-item {
    ...
    width: 25%
}

Coloring Odd Rows

Coloring odd rows in a table is easy, but when the rows are in a responsive grid, that's a whole different story. For tables, the CSS rules are extremely simple:

tr:nth-child(even) {background: #CCC}
tr:nth-child(odd) {background: #FFF}

Via Twitter, @tomaslin advised me that the nth-child selector could probably be used for this, but it'd likely require some JavaScript to make it responsive. I found the excellent Master of the :nth-child and began trying to figure it out. The following function is what we now use to color odd rows in the Summary Bar.

function colorRows() {
    var lisInRow = 0;
    var items = $('.summary-items li');
    items.each(function() {
        if($(this).prev().length > 0) {
            if($(this).position().top != $(this).prev().position().top) return false;
            lisInRow++;
        }
        else {
            lisInRow++;
        }
    });
    var rows = items.length / lisInRow;
    for (var i = 0; i < rows; i++) {
        var selector = "nth-child(n+{x}):nth-child(-n+{y})";
        var x = (lisInRow * i) + 1;
        var y = x + (lisInRow - 1);
        selector = selector.replace('{x}', '' + x);
        selector = selector.replace('{y}', '' + y);
        if (i % 2) {
            $('.summary-items li:' + selector).addClass('odd');
        } else {
            $('.summary-items li:' + selector).removeClass('odd');
        }
    }
}

The above code is in dashboard.js and is called anytime the browser window is resized (to adapt to the responsive grid).

$(window).resize(colorRows);

It's also called when summary widgets are re-ordered, in the updateOrder() function of WidgetController.

$scope.updateOrder = function(event, ui) {
    ...
    Preferences.saveWidgetOrder(type, {items: items});
    if (type === 'summary') {
        colorRows();
    }
};

I'd like to figure out how to make this more Angular-esque, but all the "how to hook into window.resize" articles I found make it seem harder than this.

Show More

The last feature I had to implement was the "Show More" bar that appears when widgets are hidden. This was the most difficult thing to implement and I tried many different things before arriving at a solution that works. First of all, the widgets bars that can be expanded are put into their original (collapsed) state using max-height and overflow: hidden. From there, I look at the list inside the bar and compare the height's of the two elements. If the list is taller than the bar, the Show More bar is added.

I originally looked at the list's :last-child to see if it was visible, but jQuery's :hidden selector only works on items that are hidden by display: none rather than ones that are hidden by overflow.

As you can see from the code below, there's special logic needed to expand the min-height of the Summary Bar since it doesn't have enough room at the bottom to add the bar in its collapsed state.

function showMore(element) {
    var bar = element.parent().parent();
    var list = element.parent();
    var barId = bar.attr('id');
    var listHeight = list.height();
    var barHeight = bar.height();
    var isSummaryBar = (barId.indexOf('summary') > -1);
    var summaryBarMinHeight = 260;
    var showMoreShouldBeVisible = (isSummaryBar && element.position().top >= 200) ? true : listHeight > barHeight;
    if (showMoreShouldBeVisible) {
        var messages = {};
        // the variables below are defined in the host page, before this file is loaded
        messages.more = showMoreText;
        messages.less = showLessText;

        var showMore = $('<div class="show-more"/>').html(messages.more + " <b class='caret'></b>");
        showMore.appendTo(bar);
        // summary bar doesn't have enough room for the Show More bar in its collapsed state,
        // so change it from 242 to 260
        if (isSummaryBar) {
            bar.css({'min-height': summaryBarMinHeight + 'px', 'max-height': ''});
        }

        showMore.bind('click', function (e) {
            var element = $(this);
            var parent = element.parent();

            if (element.hasClass('less')) {
                parent.css({"max-height": ''});
                if (isSummaryBar) {
                    parent.css({"min-height": summaryBarMinHeight + 'px'}).animate(200);
                } else {
                    parent.css({"min-height": ''}).animate(200);
                }
                element.removeClass('less');
                element.html(messages.more + ' <b class="caret"></b>');
            } else {
                parent.css({
                    "max-height": 9999,
                    "min-height": 'auto'
                }).animate({
                        "min-height": parent.height() + 19
                    }, 200);

                element.addClass('less');
                element.html(messages.less + ' <b class="caret caret-up"></b>');
            }

            // prevent jump-down
            return false;
        });
    } else {
        // Remove show-more in case it was previously added
        if (bar.find('.show-more').length > 0) {
            if (isSummaryBar) {
                bar.css('min-height', summaryBarMinHeight - 18)
            } else {
                bar.attr('style', '');
            }

            bar.find('.show-more').remove();
        }
    }
}

function showMoreOnResize() {
    var dataItems = $('.task-items,.summary-items,.report-items');
    dataItems.each(function() {
        var lastItem = $(this).find('li:last-child');
        if (lastItem.length > 0) {
            showMore(lastItem);
        }
    });
}

At first, I wrote this logic as a directive, but when I needed it for responsiveness, I moved it into dashboard.js. The showMoreOnResize() function is called on window resize.

$(window).resize(showMoreOnResize);

I also found that I had to add it to the Preferences service after widgets were saved (since the number displayed could change).

factory('Preferences', function ($filter) {
    return {
        ...
        // Save hidden and visible (and order) widgets from config dialog
        saveWidgetPreferences: function (type, widgets) {
            ...
            DWRFacade.saveDashboardWidgetPreference(type, preferences, {
                callback: function() {
                    // recalculate show more bar
                    showMoreOnResize();
                },
                errorHandler: function (errorString) {
                    alert(errorString);
                }
            });
        }
    }
});

To implement the .caret-up (the .caret class is from Bootstrap), I found a caret-right howto and used it to create .caret-up:

.caret-up {
    border-left: 4px solid transparent;
    border-right: 4px solid transparent;
    border-top: 4px solid transparent;
    border-bottom: 4px solid black;
}

Summary

The final My Dashboard feature is something that I'm quite proud of. A fellow developer, Vlad, did an excellent job of implementing the backend and admin portions. The Product Team's vision and desire to make it Pop! created something great. The fact that we didn't have to support IE8 helped a lot in the implementation. Below is a screenshot of how My Dashboard looked when we completed the project.

Angular isn't mentioned much in this article. That's because we didn't have to do much to the existing Angular code to implement the new design. It was just a matter of writing/modifying some CSS as well as introducing some JavaScript for colored rows and show more. If you know how these features could be written in a more Angular Way, I'd love to hear about it.

If you'd still like to learn more about Angular and why it's good to integrate it little by little, I encourage you to read 5 reasons to use AngularJS in the corporate app world.

Developing with AngularJS - Part II: Dialogs and Data

2013-06-20T08:45:13-06:00

A couple of days ago, I wrote an article on how I started developing with AngularJS. I used AngularJS for several months to develop a "My Dashboard" feature for a client's product and learned a whole bunch of stuff along the way.

This article provides an overview of how I changed some of My Dashboard's features to use Angular instead of jQuery. After finishing the prototype work in January, we started moving bits and pieces into the main application. We kept the same file names for our Angular-related files and copied them into the project.

All these files are packaged up into a dashboard.js file that's included at the bottom of our Dashboard page. While our prototype used jQuery 1.9 and jQuery UI 1.10, the application's codebase used jQuery 1.7.1 and jQuery UI 1.8.3. Luckily, this didn't present a problem as everything continued to work as expected.

Around this time, we also had many discussions with the Product Team about charts. Since Highcharts required we purchase a license, we took at look at AnyChart, which we were already using. We were able to get AnyChart to work with our existing chart directive with minimal changes. Most changes were in the JSON itself.

We committed the first pass (with sample data still hard-coded) in mid-February.

Angular UI's Carousel

While finishing our initial prototype, I learned about Angular UI's Carousel, an implementation of Bootstrap's Carousel. It required a 70% less HTML and is quite a bit easier to read. Below is the refactored carousel section.

<carousel interval="-1" class="oneup">
    <slide ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'" active="slide.active" class="chart">
        <chart class="widget" value="{{widget}}" type="{{widget.chartType}}"></chart>
    </slide>
</carousel>
<carousel interval="-1" class="twoup">
    <slide ng-repeat="widget in widgets | filter: {type: 'chart'} | chunk: 2 | orderBy: 'order'" active="slide.active" class="chart">
        <chart class="widget" value="{{widget[0]}}" type="{{widget[0].chartType}}"></chart>
        <chart class="widget" value="{{widget[1]}}" type="{{widget[1].chartType}}"></chart>
    </slide>
</carousel>

I was also able to remove the JavaScript that once initialized the carousel.

-    var carousel = $('.carousel');
-    $(carousel).carousel({
-        interval: 0
-    });

While integrating this directive, I found a way to improve it by hiding navigation indicators if there's only one slide. And thus my first contribution to Angular UI was born.

Angular UI's Sortable

Next, I switched from using JavaScript to Angular UI's sortable to initialize drag-and-drop functionality. This was as simple as removing 5 lines of JavaScript and adding "ui-sortable" as an attribute to HTML tags.

<div class="container-widgets" ng-controller="WidgetController" ng-cloak>
     <div class="row-fluid">
         <div class="span9">
-                <ul class="widgets">
+                <ul class="widgets" ui-sortable="{handle:'.heading'}">
                 <li id="summary-bar">
                     <div class="heading">Summary <a href='#' class='configure'></a></div>
-                        <ul class="tiles">
+                        <ul class="tiles" ui-sortable>
                         <li class="span3" ng-repeat="widget in widgets | filter:{type: 'summary'} | orderBy: 'order'">
                             <h3>{{widget.value}}</h3>
@@ -36,7 +36,7 @@
                 </li>
                 <li id="task-bar">
                     <div class="heading">My Tasks <a href='#' class='configure'></a></div>
-                        <ul class="tasks">
+                        <ul class="tasks" ui-sortable>
@@ -151,55 +131,44 @@
-    $('.widgets').sortable({
-        cursor: "move",
-        handle: ".heading"
-    }).disableSelection();
-    $('.tiles,.tasks').sortable();

This directive uses jQuery UI under the covers, so it accepts all the same arguments you'd normally use.

Dialogs with jQuery

The next feature I implemented was a dialog that allowed end users to add/remove widgets from their dashboard. Since I knew how to do this with jQuery, that's the path I started down. I figured it'd be easier to get something working quickly and then refactor to Angular later. I put the HTML for the dialog at the bottom of the page:

<script type="text/javascript" src="js/move.js"></script>
<script type="text/javascript" src="js/upDown.js"></script>
<div id="configure-widgets" class="hidden" data-title="Bar Configuration" data-close="Close">
    <div class="row-fluid center">
        Decide which items you would like to display and set the display order.
        <div class="span5" style="margin-left: 25px">
            <label for="available-widgets" class="bold">Available Widgets</label>
            <select size=10 id="available-widgets" multiple class="width100pr"></select>
        </div>
        <div class="span1 arrows" style="padding-top: 50px">
            <img src="images/arrow_up.png" class="arrow arrow-up" alt="Move Up" onclick="return moveUp($('#assigned-widgets')[0]);"/>
            <img src="images/arrow_right.png" class="arrow arrow-right" alt="Move Right" onclick="return move($('#available-widgets')[0], $('#assigned-widgets')[0]);"/>
            <img src="images/arrow_left.png" class="arrow arrow-left" alt="Move Left" onclick="return move($('#assigned-widgets')[0], $('#available-widgets')[0]);"/>
            <img src="images/arrow_down.png" class="arrow arrow-down" alt="Move Down" onclick="return moveDown($('#assigned-widgets')[0]);"/>
        </div>
        <div class="span5">
            <label for="assigned-widgets" class="bold">Assigned Widgets</label>
            <select size=10 id="assigned-widgets" multiple class="width100pr"></select>
        </div>
    </div>
</div>

Then I changed the WidgetController to split the widgets variable into variables for each widget type.

+    var filter = $filter('filter');
+    var orderBy = $filter('orderBy');
+
+    $scope.summaryWidgets = filterData($scope.widgets, {type: 'summary'}, filter, orderBy);
+    $scope.taskWidgets = filterData($scope.widgets, {type: 'task'}, filter, orderBy);
+    $scope.chartWidgets = filterData($scope.widgets, {type: 'chart'}, filter, orderBy);
+}
+
+function filterData(array, query, filter, orderBy) {
+    var data = filter(array, query);
+    return orderBy(data, 'order');
+}

Finally, I added a new config directive and hooked it into the page by adding config="type" to each heading (e.g. <div class="heading" config="task">My Tasks</div>).


.directive('config', function ($sanitize, $filter) {
    return {
        restrict: 'A',
        link: function (scope, element, attrs) {
            var configBtn = $('<a href="#"/>').addClass('configure');
            configBtn.appendTo(element);
            var widgets = scope[attrs.config + 'Widgets'];
             
            // availableWidgets is defined as a global variable embedded in the page
            var allWidgets = availableWidgets; 
            $(configBtn).on('click', function (e) {
                e.preventDefault();
                var configDialog = $('#configure-widgets');
                var availableWidgets = $('#available-widgets');
                availableWidgets.empty();
                var filter = $filter('filter');
                var orderBy = $filter('orderBy');
                allWidgets = filter(allWidgets, {type: attrs.config});
                allWidgets = orderBy(allWidgets, 'order');
                var unselectedWidgets = jQuery.grep(allWidgets, function(item) {
                    return jQuery.inArray(item, widgets) < 0;
                });
                $.each(unselectedWidgets, function(index, item) {
                    var title = (item.title) ? item.title.replace(/&quot;/g, '"') : 'No Title';
                    availableWidgets.append(new Option(title, item.id));
                });
                var assignedWidgets = $('#assigned-widgets');
                assignedWidgets.empty();
                $.each(widgets, function(index, item) {
                    var title = (item.title) ? item.title.replace(/&quot;/g, '"') : 'No Title';
                    assignedWidgets.append(new Option(title, item.id));
                });
                configDialog.dialog({
                    title: $(element).text() + ' ' + configDialog.attr('data-title'),
                    width: 600,
                    modal: true,
                    buttons: [{
                            text: configDialog.attr('data-close'),
                            'class': 'btn',
                            click: function() {
                                $(this).dialog("close");
                            }
                        }]
                });
            });
        }
    }
})

As you can tell, this is quite a bit of code, and it doesn't even show you the JavaScript in move.js and upDown.js (included at the top of the dialog HTML). While writing this code, I could tell that I was not doing things the Angular Way.

To refactor, I did some research, found the $dialogProvider service and went to work.

Dialogs with Angular

I started by refactoring the config directive to be much shorter.

.directive('config', function() {
    return {
        restrict: 'A',
        link: function (scope, element, attrs) {
            var configBtn = $('<a href=""/>').addClass('configure');
            configBtn.appendTo(element);
            configBtn.bind('click', function(e) {
                e.preventDefault();
                e.stopPropagation();
                scope.$apply(scope.configureDialog(attrs.config, $(element).text()))
            });
        }
    }
})

Line #10 that starts with scope.$apply is what makes the magic happens.This calls the configureDialog() function in WidgetController. The $dialog service you see below is injected by adding it as a parameter to the WidgetController function.


$scope.configureDialog = function (type, title) {
    var dialog = $dialog.dialog({
        dialogClass: 'modal draggable',
        modalFade: false,
        backdropClick: true,
        controller: 'ConfigController',
        template: $('#configure-widgets').html(),
        resolve: {
            title: function() {
                return title;
            },
            hiddenItems: function () {
                return Widget.getHiddenWidgets(type);
            },
            items: function () {
                return angular.copy($scope[type + 'Widgets']);
            }
        }
    });
    dialog.open().then(function(results) {
        if (!angular.isUndefined(results)) {
            $scope[type + 'Widgets'] = results.items;
            Preferences.saveWidgetPreferences(type, results);
        }
    });
};

draggable directive
If you look closely, you'll see this dialog is initialized with a 'draggable' class. I created a draggable directive that gives draggability using jQuery UI to any elements with this class.

.directive('draggable', function() {
    return {
        restrict: 'C',
        link: function(scope, elem, attr, ctrl) {
            elem.draggable({handle: '.modal-header'});
        }
    };
});

The interesting parts of this file are the controller, template, and resolve parameters. The controller is just another function in controllers.js, the template is HTML in the page (so text could be i18n-ized) and resolve has the variables to pass to the controller. Widget and Preferences are services I'll talk about in the Part 3. The reason angular.copy() is used is so the widget arrays aren't modified while the dialog is displayed (we want to wait until the user clicks "Save").

The ConfigController function started quite simply:

function ConfigController($scope, $sanitize, dialog, title, hiddenItems, items) {
    $scope.title = title;
    $scope.hiddenItems = hiddenItems;
    $scope.items = items;
}

The HTML (defined by the aforementioned template parameter) is as follows. You can see that title, hiddenItems and items are the variables defined in $scope and used to render the data.

<div id="configure-widgets" class="modal hidden">
    <div class="modal-header" style="border: 0">
        <button type="button" class="close" ng-click="close(false)" aria-hidden="true" style="margin-top: -3px">&times;</button>
        <h4>{{title}} Bar Configuration</h4>
    </div>
    <div class="modal-body">
        Decide which items you would like to display and set the display order.
        <div class="row-fluid center">
            <div class="span4" style="width: 320px; margin-left: 25px">
                <label for="available-widgets" class="bold">Available Widgets</label>
                <select size=10 id="available-widgets" ng-model="items.available" ng-options="i.title for i in hiddenItems"
                        class="width100pr" multiple="multiple"></select>
            </div>
            <div class="span1 arrows" style="padding-top: 50px">
                <img src="images/arrow_up.png" class="arrow arrow-up" alt="Move Up" ng-click="moveUp(items.selected, items)"/>
                <img src="images/arrow_right.png" class="arrow arrow-right" alt="Move Right" ng-click="moveItem(items.available, hiddenItems, items)"/>
                <img src="images/arrow_left.png" class="arrow arrow-left" alt="Move Left" ng-click="moveItem(items.selected, items, hiddenItems)"/>
                <img src="images/arrow_down.png" class="arrow arrow-down" alt="Move Down" ng-click="moveDown(items.selected, items)"/>
            </div>
            <div class="span4" style="width: 320px">
                <label for="assigned-widgets" class="bold">Assigned Widgets</label>
                <select size=10 id="assigned-widgets" ng-model="items.selected" ng-options="i.title for i in items"
                        class="width100pr" multiple="multiple"></select>
            </div>
        </div>
    </div>
    <div class="modal-footer">
        <button ng-click="close(true)" class="btn btn-primary">Save and Close</button>
    </div>
</div>

There's a couple of new directives introduced by this code: ngOptions and ngClick. The former is used to display options in a <select>, the latter to call functions in the controller. These functions are defined in ConfigController:

$scope.moveUp = function(items, list) {
    angular.forEach(items, function(item) {
        var idx = list.indexOf(item);
        if (idx != -1) {
            list.splice(idx - 1, 0, list.splice(idx, 1)[0]);
        }
    });
};

$scope.moveDown = function(items, list) {
    angular.forEach(items, function(item) {
        var idx = list.indexOf(item);
        if (idx != -1) {
            list.splice(idx + 1, 0, list.splice(idx, 1)[0]);
        }
    });
};

$scope.moveItem = function(items, from, to) {
    angular.forEach(items, function(item) {
        var idx = from.indexOf(item);
        if (idx != -1) {
            from.splice(idx, 1);
            to.push(item);
        }
    });
};

$scope.close = function(save) {
    if (save) {
        dialog.close({
            hidden: $scope.hiddenItems,
            items: $scope.items
        });
    } else {
        dialog.close();
    }
};

As you can see, Angular allows you to easily access and manipulate the data. Its two-way binding feature is great because when you modify the object in JavaScript, it auto-updates the displayed HTML.The only thing you need to do to the HTML is to add the ngModel directive.

<ul class="tasks" ng-model="taskWidgets" ui-sortable>

The dialog's close() method is called in the header (where it passes false) and in the footer (where it passes true). The configureDialog() function handles saving if the user indicates they wanted to do so. The 3rd line (starts with $scope) is all that's needed to update the UI. The Preferences service is covered in the next article.

dialog.open().then(function(results) {
    if (!angular.isUndefined(results)) {
        $scope[type + 'Widgets'] = results.items;
        Preferences.saveWidgetPreferences(type, results);
    }
});

Modals with Data

Displaying the widgets in a consolidated dashboard is a nice product feature, but we wanted to take it a step further and allow users to "click through" to see the data. To do this, I added an "event" directive that could read from our JSON data and act upon it accordingly. We decided on 2 types of events: function and href. The href event type is for report widgets, because we want to allow users to click on the widget and it takes them directly to the report. For function, we simply eval() what's passed in. The function is expected to have a container argument that it can use to render data in a modal window.

Using the event directive, you can attach this behavior to a widget simply by adding a class.

<h3 class="events">{{widget.value}}</h3>

The event directive that attaches click behavior is below:

directive('events', function () {
    // This is necessary because widgets may be chunked when charts are displayed 2-up
    function getWidget(element, widget, scope) {
        if (element.hasClass('chart-title')) {
            if (element.parent().hasClass('first')) {
                widget = scope.widget[0];
            } else if (element.parent().hasClass('second')) {
                widget = scope.widget[1];
            } else {
                widget = scope.widget;
            }
        } else {
            widget = scope.widget;
        }
        return widget;
    }
    return {
        restrict: 'C',
        link: function (scope, element, attrs) {
            var widget = getWidget(element, widget, scope);
            if (angular.isUndefined(widget)) {
                return;
            }
            var events = widget.events;
            if (!angular.isUndefined(events)) {
                for (var e in events) {
                    if (e === 'function') {
                        if ($('#dialog-frame').length === 0) {
                            $('<div id="dialog-frame" class="modal hide"/>').appendTo('body');
                            var header = $('<div class="modal-header">');
                            header.append($('<button type="button" class="close" data-dismiss="modal">×</button>'));
                            header.append($('<h4/>').append(scope.widget.title));
                            header.appendTo('#dialog-frame');
                            $('<div class="modal-body"/>').appendTo('#dialog-frame');
                        }
                        element.bind('click', function(event) {
                            event.preventDefault();
                            event.stopPropagation();
                            var dialog = $('#dialog-frame');
                            var title = widget.title;
                            dialog.find('h4').html(title);
                            var dialogBody = dialog.find('.modal-body');
                            dialogBody.empty();
 
                            // display a checking for new data message when widget's value is 0
                            var checkingMessage = $('#wait-checking').html();
                            if (scope.widget.value === "0") {
                                dialogBody.html(checkingMessage);
                            // otherwise, display a loading message
                            } else {
                                dialogBody.html($('#wait-loading').html());
                            }
 
                            // center the dialog on the page
                            dialog.css({
                                width: '560px',
                                'margin-left': function() {
                                    return -($(this).width() / 2);
                                }
                            });
 
                            dialog.modal('show');
                            var container = dialogBody;
                            eval(events[e]);
                        });
                    } else if (e === "href") {
                        element.bind('click', function() {
                            location.href = events[e];
                        })
                    } else {
                        console.log('Event type "' + e + '" not supported.');
                    }
                }
            }
        }
    }
})

Empty Widgets Message

When there are no widgets for a particular type, we wanted to display a message telling the end user. To do this, I used the ngHide directive and passed in the array's length as an expression. I originally had this on a <li> in the respective widget list, but noticed it causes issues when dragging and dropping.

<div ng-hide="summaryWidgets.length" class="widgets-empty">
    No Summary Bar Widgets currently visible
</div>

Summary

This article has provided an overview of how I changed some of My Dashboard's features to use Angular instead of jQuery. I hope it's helped to show how powerful directives can be and how MVC works in Angular. I particularly enjoyed learning how to use the $dialog service. As a word of warning, its usage might change in future releases since it is currently being rewritten to be more maintainable.

In the next article, I'll talk about how I developed Services and integrated DWR. If you see any code that can be improved upon, or issues with the code/architecture in this article, please leave a comment.

Developing with AngularJS - Part I: The Basics

2013-06-18T09:06:52-06:00

There's many, many different introductions to AngularJS available on the internet. This article is not another introduction, but rather a story about my learning experience. It all started way back in January of this year. I was working as a UI Architecture Consultant at Taleo/Oracle, my client for the last 21 months. My gig there ended last month, but they agreed to let me publish a series of articles about the knowledge I gained.

Project Background

The Director of Product Management had been working on the concepts for a new project - codenamed "Visual MyView". Below is a mockup he created for our kickoff meeting on January 4th.

From his original email about the above mockup:

The intent here is that one of the columns has rows that have a similar width. The rows could be dragged and dropped into a different order – or potentially the two columns could also be reordered. The rows will basically be comprised of similar widgets. You can see in the mockup how the first two rows might look – and sample widgets. The widgets shown can be configured by the end user, as well as the order in which they are displayed. Other requirements given to us were the following.

Row 1 is comprised of 'summary' widgets that are 'todo' items. Reviews needing done – approvals required – etc.
Row 2 will be a graph row – having graphs and charts to display information – larger squares will build this row.
Row 3's content was not determined yet.

I started the initial layout with static HTML and CSS and had a wireframe to show by mid January.

By the end of January, we'd renamed the project to My Dashboard and had a working prototype using CoolClock and moment.js for the clock in the top right, AngularJS to display widget data, jQuery UI for drag-n-drop of rows and widgets, Bootstrap's Carousel for holding charts and Highcharts for rendering charts. For this prototype, we included 4 types of widgets:

Summary
Tasks
Charts
Reports

To create widgets, we had to decide on a common schema for them.

"id": 1, // not necessary for display, but likely needed if we modify and save preferences
"title": "Appointments Today",
"type": "summary", // others include: task, chart, report
"value": 3, 
"description": "10:30 Jim Smith",
"events": "url", // this can have click events
"order": 1 // used to determine order

Below is a screenshot of our wireframe with some sample widgets.

Angular Basics

The decision to use AngularJS came early on in the project, after I read Tyler Renelle's Rant: Backbone, Angular, Meteor, Derby. To learn AngularJS, I briefly looked at its homepage documentation and played with some examples. Then I stumbled upon Misko Hevery and Igor Minar's AngularJS Presentation from Devoxx 2012. At that time, the video wasn't publicly available (it's free now), so I had to buy a Parley's subscription ($79). It was well worth the money because that one hour video greatly contributed to my understanding of how AngularJS works. Another resource I used frequently to figure out how to do things was John Lindquist's egghead.io.

To begin with, we wrote the JSON for a bunch of sample widgets and embedded them into the page as a widgetData JavaScript variable.

var widgetData = [
    {"id": 1, "title": "Appointments Today", "type": "summary", "value": 3, "description": "10:30 Jim Smith", "events": {"click": "alert('foo');"}, "order": 1},
    {"id": 12, "order": 2, "title": "Offer Approvals", "type": "task", "class": "sticky-note", "value": 1},
    {"id": 103, "title": "Browser market shares at a specific website, 2010", "order": 1, "type": "chart", "chartType": "pie", 
         "tooltip": {"pointFormat": "{series.name}: <b>{point.percentage}%</b>", "percentageDecimals": 1}, "series": [
         {"type": "pie", "name": "Browser share", "data": [
             ["Firefox", 45.0],
             ["IE", 26.8],
             {"name": "Chrome", "y": 12.8, "sliced": true, "selected": true},
             ["Safari", 8.5],
             ["Opera", 6.2],
             ["Others", 0.7]
         ]}
    ]},
    ...
];

I used angular-seed to create the initial structure of the prototype, and continued using the same JavaScript file names when we moved it into the product I worked on. Since the application takes a while to login and render the My Dashboard page (when working remotely), I decided not to use the Karma testing framework that ships with Angular. Below is what our directory structure looked like for our prototype.

The JavaScript files in the "js" folder are the most important for Angular. The first file, app.js, loads the other files:

angular.module('dashboard', ['dashboard.filters', 'dashboard.services', 'dashboard.directives']);

The controllers.js file contains the Controllers (functions) that get the data and make it available to the page. Here's the code for our first controller:

'use strict';

/* Controllers */
function WidgetController($scope) {
    $scope.widgets = widgetData;
}

This puts the widgets in scope and then we were able to render them using Angular's ngRepeat directive and the following HTML:

<div ng-app="dashboard" class="dashboard">
    <div class="container-widgets" ng-controller="WidgetController" ng-cloak>
        <div class="row-fluid">
            <div class="span9">
                <ul class="widgets">
                    <li id="summary-bar">
                        <div class="heading">Summary</div>
                        <ul class="tiles">
                            <li class="span3" ng-repeat="widget in widgets | filter:{type: 'summary'} | orderBy: 'order'">
                                <h3 class="events">{{widget.value}}</h3>
                                <div class="title">{{widget.title}}</div>
                                <div class="desc">{{widget.description}}</div>
                            </li>
                        </ul>
                    </li>
                    <li id="task-bar">
                        <div class="heading">My Tasks</div>
                        <ul class="tasks">
                            <li class="task {{widget.class}}" ng-repeat="widget in widgets | filter: {type: 'task'} | orderBy: 'order'">
                                <div class="title events">{{widget.title}}</div>
                                <div class="value">{{widget.value}}</div>
                            </li>
                        </ul>
                    </li>
                    <li id="chart-bar">
                        <div class="heading">Charts</div>
                        <div id="chartCarousel" class="carousel slide">
                            <ol class="carousel-indicators">
                                <li data-target="#chartCarousel"
                                    ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'"
                                    data-slide-to="{{$index}}" ng-class="{active: $index == 0}"></li>
                            </ol>
                            <div class="carousel-inner">
                                <div class="item chart"
                                     ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'"
                                     ng-class="{active: $index == 0}">
                                    <chart class="widget" value="{{widget}}" type="{{widget.chartType}}"></chart>
                                </div>
                            </div>
                            <a class="left carousel-control" href="#chartCarousel" data-slide="prev">‹</a>
                            <a class="right carousel-control" href="#chartCarousel" data-slide="next">›</a>
                        </div>
                    </li>
                </ul>
            </div>
            <div class="span3">
                <!-- clock and reports -->
            </div>
    </div>
</div>

The beginning of this HTML shows how Angular is instantiated: ng-app matches the name defined in app.js, ng-controller instantiates the WidgetController and ng-cloak is used to hide everything until its processed.

<div ng-app="dashboard" class="dashboard">
    <div class="container-widgets" ng-controller="WidgetController" ng-cloak>

If you take a closer look at the way ng-repeat attributes, you'll see how filters are used to filter data. There's filter and orderBy filters that are built in and allow you to filter data. The filter filter allows you to query arrays by strings, objects and even functions. In the following code block, "task" widgets are filtered, ordered and displayed.

<li class="task {{widget.class}}" ng-repeat="widget in widgets | filter: {type: 'task'} | orderBy: 'order'">
    <div class="title events">{{widget.title}}</div>
    <div class="value">{{widget.value}}</div>
</li>

This was pretty straightforward, but we quickly noticed that if a widget had HTML in its title, it didn't display correctly (rendering the raw HTML). To process the HTML, we had to use the ngBindHtml directive (tip: directives are camelCase, but written with dashes in HTML).

<div class="title events" ng-bind-html="widget.title"></div>

After getting this to work, we noticed that some titles weren't fully rendered because they were hidden with overflow: hidden. We tried adding a tooltip with title="{{widget.title}}", but ran into the same issue. I sent an email to the AngularJS Google Group and received a solution: create an htmlTitle directive:

.directive('htmlTitle', function ($sanitize) {
    return {
        restrict: 'A',
        link: function (scope, element, attrs) {
            attrs.$observe('htmlTitle', function (title) {
                // convert &value; to HTML
                var html = angular.element('<div></div>').html($sanitize(title)).text();
                element.attr('title', html);
                element.html(html);
            });
        }
    }
})

Usage:

<div class="title events" ng-bind-html="widget.title" html-title="{{widget.title}}"></div>

Drag-and-Drop

To implement drag-and-drop functionality, I originally used jQuery UI's sortable. At the bottom of the page, the following code initialized sorting for the various lists:

$(document).ready(function() {
    $('.widgets').sortable({
        cursor: "move",
        handle: ".heading"
    }).disableSelection();
    $('.tiles,.tasks').sortable();
    var carousel = $('.carousel');
    $(carousel).carousel({
        interval: 0
    });
};

As you can see, it also initializes the carousel and stops it from cycling automatically.

Carousel Issues

The first problem I ran into with Bootstrap's Carousel was a strange error from Highcharts. If you look in the above HTML, you'll see there's a <chart> element. This is processed by a highcharts directive. When I tried to use this directive for Highcharts in a carousel, it results in the following error:

TypeError: Cannot read property 'length' of undefined at Object.ob.setMaxTicks

This seemed to be caused by the following css in Bootstrap:

.carousel-inner > .item { display: none }

When I added an override with "display: block" to my stylesheet, everything worked, but the charts were stacked instead of in a carousel. To fix this, I modified the directive to show/hide the "item" element so Highcharts was able to write to it. I also logged an issue for this.

if (element.parent().not(':visible')) {
    element.parent().show();
}
var chart = new Highcharts.Chart(newSettings);
element.parent().attr('style', '');

ngRepeat and Grouping

The last thing I accomplished in our end-of-January prototype was rendering 2 charts side-by-side. I got it working with plain HTML, created a "groupBy" filter for Angular and tried to get it to work with the following:

<div id="chartCarousel" class="carousel slide">
    <ol class="carousel-indicators">
        <li data-target="#chartCarousel" ng-repeat="widget in widgets | filter: {type: 'chart'} | groupBy"
            data-slide-to="{{$index}}" ng-class="{active: $index == 0}"></li>
    </ol>
    <div class="carousel-inner">
        <div class="item" ng-repeat="widget in widgets | filter: {type: 'chart'} | groupBy" ng-class="{active: $index == 0}">
            <div class="widget">{{widget[0].title}}</div>
            <div class="widget">{{widget[1].title}}</div>
        </div>
    </div>
    <a class="left carousel-control" href="#chartCarousel" data-slide="prev">‹</a>
    <a class="right carousel-control" href="#chartCarousel" data-slide="next">›</a>
</div>

This all worked like I expected it to in Chrome, but I the following errors showed in my console.

Error: 10 $digest() iterations reached. Aborting! Watchers fired in the last 5 iterations:

I sent an email to the Angular Google Group and received a link to a discussion where I found a "chunk" filter that solved the problem. This worked great, but I wanted to make it more responsive.

If the user has a screen size big enough to fit 2 charts, show 2 charts and paginate by 2.
If the user has a small screen size that only fits 1 chart, show 1 and paginate by 1.

To solve #1 and #2, I ended up rendering two different sections (with classes .oneup and .twoup) and displayed them based on screen size.

<li id="chart-bar">
    <div class="heading">Charts</div>
    <div id="chartCarousel1" class="carousel slide oneup" style="display: none">
        <ol class="carousel-indicators">
            <li data-target="#chartCarousel1"
                ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'"
                data-slide-to="{{$index}}" ng-class="{active: $index == 0}"></li>
        </ol>
        <div class="carousel-inner">
            <div class="item chart"
                 ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'"
                 ng-class="{active: $index == 0}">
                <chart class="widget" value="{{widget}}" type="{{widget.chartType}}"></chart>
            </div>
        </div>
        <a class="left carousel-control" href="#chartCarousel1" data-slide="prev">‹</a>
        <a class="right carousel-control" href="#chartCarousel1" data-slide="next">›</a>
    </div>
    <div id="chartCarousel2" class="carousel slide twoup" style="display: none">
        <ol class="carousel-indicators">
            <li data-target="#chartCarousel2"
                ng-repeat="widget in widgets | filter: {type: 'chart'} | chunk: 2 | orderBy: 'order'"
                data-slide-to="{{$index}}" ng-class="{active: $index == 0}"></li>
        </ol>
        <div class="carousel-inner">
            <div class="item chart"
                 ng-repeat="widget in widgets | filter: {type: 'chart'} | chunk: 2 | orderBy: 'order'"
                 ng-class="{active: $index == 0}">
                <chart class="widget" value="{{widget[0]}}" type="{{widget[0].chartType}}"></chart>
                <chart class="widget" value="{{widget[1]}}" type="{{widget[1].chartType}}"></chart>
            </div>
        </div>
        <a class="left carousel-control" href="#chartCarousel2" data-slide="prev">‹</a>
        <a class="right carousel-control" href="#chartCarousel2" data-slide="next">›</a>
    </div>
</li>

The JavaScript to show the correct number of charts is below:

var chartBar = $('#chart-bar');
function showCharts() {
    if (chartBar.width() < 960) {
        chartBar.find('.oneup').show();
        chartBar.find('.twoup').hide();
    } else {
        chartBar.find('.twoup').show();
        chartBar.find('.oneup').hide();
    }
}

$(document).ready(function () {
    showCharts();
});

$(window).resize(showCharts);

Summary

Even though I got everything to work for our initial prototype using Angular and jQuery, it didn't quite feel like I was taking full advantage of Angular's power. In particular, I learned that Angular UI Bootstrap had their own carousel and Angular UI had a sortable directive. My suspicions were confirmed when I read How do I "think in AngularJS" if I have a jQuery background?

In the next article, I'll talk about how I migrated to use Angular UI's carousel and sortable directives, as well as integrating dialogs.

Happy 10 Year AppFuse!

2013-04-05T08:56:45-06:00

10 years ago yesterday, I released the first version of AppFuse. It started with XDoclet generating ActionForms from POJOs and became very popular for Struts developers that wanted to use Hibernate. The project's popularity peaked in 2006, as you can see from the mailing list traffic below.

It's possible the decrease in traffic is because we re-wrote everything to be based on Maven. It's also possible it was because of more attractive full-stack frameworks like Grails and Rails. However, the real reason is likely that I stopped working on it all the time due to getting a divorce becoming an awesome dad.

Below is a timeline of how the project evolved over its first 4 years.

AppFuse has been a great project for me to work on and it's been a large source of my knowledge about Java, Web Frameworks, Spring, Hibernate - as well as build systems like Ant and Maven. We started with CVS, moved to SVN and now we're on GitHub. We've experienced migrating from Tapestry 4 to Tapestry 5 (thanks Serge Eby!), upgrading to JSF 2 and enjoyed the backwards compatibility of Spring and Struts 2 throughout the years. We've also added REST support, a Web Services archetype and kept up with the latest Spring and Hibernate releases.

Last year, we added Bootstrap and jQuery as foundational front-end frameworks. For our next release, we're switching to PrimeFaces, adding Wicket and changing from jMock to Mockito. Most of these changes are already in source control, we just need to polish them up a bit and add AMP support. I hope to release 3.0 before the bus is done.

Thanks to all the enthusiastic users of and contributors to AppFuse over the years. It's been a great ride!

The Modern Java Web Developer and Java Web Security at Denver JUG

2013-02-14T10:23:18-07:00

Last night, I had the pleasure of delivering two talks at the Denver Java User Group. The first talk, The Modern Java Web Developer, was inspired by the book titled The Well-Grounded Java Developer. Ben Evans and Martijn Verburg mention in the beginning of the book that they wrote it as a training guide to get new Java developers up to speed. For my talk, I wanted to do something similar, but for Java Web Developers.

I mentioned that the first thing you have to do to become modern is to change your title from a Java Web Developer to a JVM Web Developer. After doing that, you have a whole slew of new and wonderful technologies at your disposal. From there, I believe the Modern JVM Web Developer:

Starts with Fast Hardware
Uses IntelliJ IDEA
Leverages jQuery, HTML5, and CSS3
Creates High Performance Web Sites
For Mobile Devices
In the Cloud
And cares about Security

You can also view this presentation on Slideshare or download it from my presentations page.

The second talk was on Java Web Application Security and was largely an updated version of the talk I gave a couple years ago, starting with an appearance at the Utah JUG. It was mostly a live demo session using my Ajax Login application. To prepare the project for this talk, I created branches for each step. This means you can checkout the "baseline" branch and use Git to compare it with the "javaee" branch. You can also compare the "spring-security" branch vs. the "apache-shiro" branch. Finally, you could see what I needed to do to fix many of the vulnerabilities found by Zed Attack Proxy.

You can also view this presentation on Slideshare or download it from my presentations page.

Thanks to the DJUG and Thrive folks for providing good beer (especially the Guinness!) and FullContact for hosting. Also, I'd like to thank Manning for the copies of The Well-Grounded Java Developer they sent and No Starch Press for copies of Michal Zalewsky's The Tangled Web: A Guide to Securing Modern Web Applications. Last, but certainly not least, thanks to all the good people who attended and listened to me ramble on about all this cool technology.

Future speaking engagements include Devoxx France in March and Denver's HTML5 User Group in April.

AppFuse Light 2.2.1 Released!

2013-01-24T19:43:20-07:00

In December, the AppFuse Team released 2.2.1. Right before that release, I decided to wait on enhancing its "light" modules, a.k.a. AppFuse Light. I'm glad I did, because it took some effort to get jQuery and Bootstrap integrated, as well as to make it more secure.

The good news is AppFuse Light 2.2.1 is released and it's sitting out on the Central Repository. This release is a refactoring of all archetypes to be up-to-date with the AppFuse 2.2.1 release. This means Java 7 compatibility, Servlet 3, Bootstrap/jQuery integration, Tapestry 5.3.6 upgrade and security improvements. I integrated Bootstrap and jQuery using WebJars Servlet 3 support since it was simple and straightforward.

You can create projects using AppFuse's light archetypes using a command such as the following:

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-light-spring-freemarker-archetype -DarchetypeVersion=2.2.1 
  -DgroupId=com.mycompany -DartifactId=myproject

The list of archetypes is as follows:

appfuse-light-jsf-archetype
appfuse-light-spring-archetype
appfuse-light-spring-freemarker-archetype
appfuse-light-spring-security-archetype
appfuse-light-stripes-archetype
appfuse-light-struts-archetype
appfuse-light-tapestry-archetype
appfuse-light-wicket-archetype

The QuickStart Guide will help you get setup and demos are available at the following links:

If you have questions about AppFuse, we invite you to ask them on the users mailing list or tweet using #appfuse.

For those enjoying Bootstrap in your apps, I encourage you to check out {wrap}bootstrap and Bootswatch.

What's the best way to compare JVM Web Frameworks?

2013-01-09T08:29:17-07:00

I've been comparing web frameworks ever since 2004. It was the first time I'd ever proposed a talk for a conference. ApacheCon was in Vegas that year and my buddy Bruce suggested I speak at it. I submitted the talk, got accepted and went to work learning the frameworks I was talking about. At the time, I had a lot of Struts experience and I'd made a good living learning it, consulting on it and blogging about it. However, there was a new kid on the block (Spring MVC) that was garnishing attention and some other frameworks (WebWork and Tapestry) that had a lot of high praise from developers. I was inspired to learn why so many people hated Struts.

Fast forward 8 years and I'm still comparing web frameworks. Why? Because there still seems to be a large audience that's interested in the topic. Witness InfoQ's Top 20 JVM Web Frameworks, which was one of their most-read articles for two months in a row. One of the beauties of the Java Community is that it's very diverse. There's tons of folks that are part of this community and, like it or not, several folks that are former Java Developers. However, these developers still seem to maintain an interest in the community and it's still one of the largest pools of talent out there. Java is still quite viable and only seems to be getting better with age.

So the topic of web frameworks on the JVM is still hot, and I still like to write about it. For those of you still enthusiastic about the topic, you're in luck. The two best websites for the Java Community, InfoQ and DZone (formerly Javalobby) are still very interested in the topic too!

Both sites emailed me in November to get my advice for their research on the subject. InfoQ (specifically Dio Synodinos) was mostly interested in 1) having me analyze their recently-gathered data, or 2) helping them create a new version. DZone (specifically Mitch Pronschinske) emailed about doing a similar survey to InfoQ's, but with more relevant data points (include GWT, specifying Struts 2 vs. Struts 1, etc.).

My response to Mitch at DZone:

Interestingly enough, the folks at InfoQ contacted me as well as they're thinking of doing a new survey. One of the things I mentioned to them is it'd be interesting to see what folks are using AND which frameworks they admire. Often, devs don't get to choose their web framework at work. I wonder if it'd be possible to collaborate with InfoQ to gather data from developers so it's not being done on two different sites?

I submitted a talk for ApacheCon NA (in February) called Comparing Apache Web Frameworks. When choosing Web Frameworks, I've often found it helps to eliminate frameworks and narrow the scope. Obviously, this makes sense for an Apache Conference, but not for all developers. However, I do plan on analyzing each framework based on a limited set of criteria. Here's what I have so far:

Community, HTML5, REST, Mobile, Performance, Web Performance Optimization

Obviously, community is important for Apache projects, but might not be for the wider audience. It might be good to limit to these 5 criteria, or expand it to 10, but not more. I think it'd be interesting to get the community to rank the various frameworks on these criteria, and also try to find developer's biases while doing it. For example, I wonder if people would be willing to admit they're biased for/against certain frameworks and then take that into account as part of gathering the data?

Unfortunately, ApacheCon denied my submission. This make sense since their schedule seems to be concentrating on HTTPD, Cloud and Big Data.

For InfoQ, Dio asked for a list of web frameworks to include. Below is a list we started with, followed by my response.

Spring MVC
Play
Grails
JSF
Struts 2
Wicket
Lift
Tapestry 5
Seam
JRuby on Rails
Wicket
GWT
Vaadin

I would add Stripes, vert.x and maybe something like Apache Click. VRaptor is probably a good one to add too. There's always a few less-used frameworks that get a lot of complainers if you don't include them.

Comparing to your previous list, I don't think Seam should be in here since they've split the project into separate bundles and are no longer developing Seam as a whole. JRuby on Rails is a tough one because if you say Ruby on Rails, you'll get a ton of responses, but probably not from the Java community. The Ruby community might chime it quite a bit if you can get in touch with them though.

I believe you should include Clojure web frameworks, but I've only heard of one of them: Compojure.

SiteMesh, Netty, etc. - remove them.

I also offered my advice on instructions:

I actually like the two coordinates you used before, importance and adoptability. However, I don't know that everyone read the instructions this way. Most people didn't rank all frameworks and I believe that's part of the point. I only ranked the ones I'd used, but I think it'd be better if people ranked all of them. I also think having these two criteria opens it up to more than just developers. Project/Product Managers and stakeholders that've been successful with certain frameworks should be able to vote too.

For DZone, they wanted to include a set of criteria for ranking:

Project maturity, community support, and documentation quality (one criteria)
UI Features Capability (maybe some are more graphics driven or form driven?)
Code readability
Flexibility (maybe this could be broken down into what types of projects the framework can handle) or several criteria that ask if it is "Good for 'x' type of project"
Performance/Speed
Cross platform support
Extensibility, Plugins, Community Libraries
Architecture (this may just be information for later, not an opinion question)
Web standards support
REST support (is this something you would just rate a yes or no? In that case it wouldn't need to be an opinion question)
Mobile support

My response:

For web frameworks, I believe the 5 I mentioned (Community, HTML5, REST, Mobile, Performance, Web Performance Optimization) are most important, with Security become more and more of a concern. For web standards support, I'd almost change it to "HTML5" and to see how the various frameworks stack up. I think REST is very important, and I think it's cool that Struts 2, Spring MVC and Grails all have great support for it. It'd be interesting to see how the component-based frameworks think of having REST support in the framework (vs. external like Jersey, CXF, etc.).

Of the list you provided, I don't know about Code readability or Flexibility. Code readability is kinda like Learnability. One of the nice things about Spring MVC and Grails is that you can learn how they work very quickly. Then you can use that knowledge and don't have to look things up much. Tapestry and Wicket might be similar for those writing large apps, but I haven't found that to be as true. The more traditional MVC Frameworks just make more sense to me.

UI Feature Capability is a good one because frameworks with widgets are often popular with developers. Flex, GWT, jQuery UI, Sencha all do this very well.

Summary
The reason for this post is to add some transparency to the process of Comparing JVM Web Frameworks. I like to think that I've tried to do this in the past (especially with my reasons for rankings). Now, we'd like to hear from you, the community that uses these web frameworks.

What's the best way to compare JVM Web Frameworks?

While it's nice to hear from the generous folks that create and maintain JVM Web Frameworks, we're mostly interested in hearing from the developers that are using these things on a day-to-day basis. The Blue Collar Developers, if you will. If you could design a JVM Web Framework comparison that answered your questions, how would it look? What questions would it ask? What conclusions would make you happy? Should commercial frameworks like ZK be included?

Your responses are very greatly appreciated.

AppFuse 2.2.1 Released!

2012-12-11T15:21:44-07:00

The AppFuse Team is pleased to announce the release of AppFuse 2.2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate 4, Struts 2.3.7, Apache CXF 2.7.0 and Spring Security 3.1.3. In addition, we've integrated HTML5, Twitter Bootstrap, jQuery and replaced Compass with Hibernate Search. Last but not least, we've added full support for Java 7 and integrated many security improvements. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is a full-stack framework for building web applications on the JVM. It was originally developed to eliminate the ramp-up time when building new web applications. Over the years, it has matured into a very testable and secure system for creating Java-based webapps.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

A number of blog posts were written about features that went into this release while it was being developed:

If you have questions about AppFuse, please read the FAQ or join the mailing list. If you find any issues, please report them on the users mailing list.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. We highly recommend using the new IntelliJ IDEA 12 for developing web applications.

Improving AppFuse's PageSpeed with Apache

2012-12-04T09:25:05-07:00

One of the most important things when developing webapps is to make them fast. With AppFuse, we've tried to incorporate many of the 14 rules for faster-loading websites. We had a gzip filter before it was cool (2003) and replaced it with the one from EhCache. However, users experienced issues with both of these, both with XFire/CXF and WebWork/Struts 2 and JSPs. Because of these issues, we disabled gzipping a few releases ago.

This article is designed to show you how you can make your AppFuse webapp faster, without modifying any code. The good news is this applies to any webapp that you can deploy behind Apache.

Last Friday, I sent an email to the good folks at Contegix to see if they could install mod_pagespeed on the Apache server that sits in front of *.appfuse.org. My goal was to improve the YSlow and PageSpeed scores of the apps hosted on demo.appfuse.org. I discovered they were getting a dismal score of 24 and figured we could do a lot better. mod_pagespeed speeds up your site and reduces page load time by automatically applying web performance best practices. It seemed like an easy solution.

Unfortunately, we were unable to use mod_pagespeed. From the guys at Contegix:

Attempting to install mod_pagespeed as you requested, we find that it requires Apache httpd 2.2 and libstdc++ 4.1.2, both of which are unsupported in RHEL4. To get mod_pagespeed to work on your present operating system basically means re-rolling the core components, which would make them unsupported. I'm afraid mod_pagespeed is simply not an option on your present configuration.

Since I still wanted to improve performance, I opted for another route instead: using mod_deflate (for gzipping) and mod_expires (for expires headers). I also turned on KeepAlive as recommended by PageSpeed Insights.

mod_deflate
mod_deflate was already installed in Apache (version 2.0.52), so all I had to do was configure it. On RHEL4, Apache is installed at /etc/httpd and there's a conf.d directory that contains all the configuration files. I created a file at /etc/httpd/conf.d/deflate.conf and populated it with the following:

#
# mod_deflate configuration
#
<IfModule mod_deflate.c>
    SetOutputFilter DEFLATE
    
    AddOutputFilterByType DEFLATE text/plain text/html text/xml text/css application/xml application/xhtml+xml application/rss+xml application/javascript application/x-javascript
    
    DeflateCompressionLevel 9
    
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    
    DeflateFilterNote Input instream
    DeflateFilterNote Output outstream
    DeflateFilterNote Ratio ratio
    
    LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
    
</IfModule>

At first, I had separate lines for all the different content types (as recommended by this article). The Contegix support crew figured out the solution (everything needed to be on one line) in 14 minutes, updated the config and verified it worked using an http compression testing page.

mod_expires
mod_expires was already installed, so I added a config file at /etc/httpd/conf.d/expires.conf. I used this howto and asked Contegix for help when it didn't work. Their response took quite a bit longer this time (49 minutes), but they once again figured it out:

It appears that FilesMatch does not like to play will JkMount. It does work using content type.

My final config for expires.conf:

<IfModule mod_expires.c>
    ExpiresActive On
    
    <FilesMatch "\.(jpe?g|png|gif|js|css)$">
        ExpiresDefault "access plus 1 week"
    </FilesMatch>
    
    ExpiresByType image/jpeg "access plus 1 week"
    ExpiresByType image/png "access plus 1 week"
    ExpiresByType image/gif "access plus 1 week"
    ExpiresByType text/css "access plus 1 week"
    ExpiresByType application/javascript "access plus 1 week"
    ExpiresByType application/x-javascript "access plus 1 week"
</IfModule>

I used "1 week" because we're changing things quite a bit right now and we haven't integrated resource fingerprinting yet.

KeepAlive
The last thing I did to improve performance was to turn on KeepAlive by editing /etc/httpd/conf/httpd.conf and changing Off to On.

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

Summary
As a result of these changes, our PageSpeed score went from 24 to 96 and YSlow went from a 90 to a 98. When I started this experiment, I was only trying to fix demo.appfuse.org. However, it also improved the speed of all the other *.appfuse.org sites, including Confluence, Bamboo, JIRA and FishEye. Thanks for all the help Contegix! There's a good chance you've given me back a few minutes in each day.

Originally posted on the AppFuse Blog.

Why the bias against JSF?

2012-11-08T09:24:27-07:00

In my last post about InfoQ's Top 20 Web Frameworks for the JVM, I received a thought-provoking comment from henk53:

There is one little thing that does bother me in those presentations, and that's your fairly obvious bias against JSF.
...
If you are presenting yourself as, more or less, an authority on comparing web frameworks, then having a fairly obvious biased against one of them is just peculiar. I, all of my team, and various clients distrust your ranking of JSF. We do look at your data if the choice is between other frameworks, but as soon as JSF comes into the picture we just look elsewhere.

I'm not really sure where this bias comes from. Yes, JSF 1.0 sucked and 1.2 was only marginally better, but 2.0 is really cool and productive and there are SUPERB component and utility libraries now like PrimeFaces and OmniFaces. As a researcher of this topic I think you should keep up the date and not stick to some old grudge.

This is true, I am biased against JSF. It all started with my first JSF experience back in August 2004. If you remember correctly, 2004 was a big year: JSF 1.0, Spring 1.0 and Flex 1.0 were all released. The "AJAX" term was coined in early 2005.

By 2007 and 2008, JSF still hadn't gotten any better. In late 2009, JSF 2.0 was released and I upgraded in March 2011. As you can see from the aforementioned post, I ran into quite a few issues upgrading. JSF was also the hardest one to get working with extension-less URLs.

Most of my issues with JSF come from having maintained an application built with it since 2004. If I were to start a new application without any legacy migration issues, I imagine it wouldn't be as difficult. However, if you compare it to Struts 2 and Spring MVC, I've had little-to-no issues upgrading those applications over the years.

Also, I'm not just biased against JSF, but most component-based web frameworks. Just ask the Tapestry and Wicket folks. They've felt my criticisms over the years. My reason for preferring request-based frameworks like Struts 2/Spring MVC and Grails/Play has been because I've never seen the appeal in component-based frameworks. Often I've found that their components are just widgets that you can get from any decent JavaScript framework. And chances are that JavaScript framework can work with any web framework. Also, I've worked on a lot of high-traffic web applications that require statelessness for scalability.

I see the value in component-based frameworks, I just don't think components should be authored on the server-side. Most of the Java-based component frameworks require 2+ files for components (one for the component, one for the view, possibly one for the config). I love GWT's component concept in that you can just extract a class and re-use it. With JS frameworks, you can often just include a script. These days, when I think of good component-based frameworks, I think of jQuery UI and Twitter Bootstrap.

All that being said, there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

In other component-based frameworks in AppFuse news, Tapestry 5 has gotten really fast in the last year. I imagine this is because we have a Tapestry expert, Serge Eby, working on it. And we're planning on adding Wicket in the 2.3 release.

So even though I prefer request-based frameworks with REST support and Bootstrap, that doesn't mean everyone does. I'll do my best to be less-biased in the future. However, please remember that my view on web frameworks is as a developer, not an analyst. And aren't developers supposed to be opinionated?

AppFuse News: GitHub, Hibernate Search and The Future

2012-09-25T10:42:14-06:00

It's been a while since I've written anything about AppFuse, but since the project has had quite a bit of activity lately, now seems like a good time.

GitHub
First of all, we moved the source code from java.net to GitHub way back in June. Thanks to Serge for helping with this process and making it quick and painless. For some reason, shortly after moving, we started having quite a few build issues with Bamboo. I was able to diagnose the problem as not enough memory on our server. Thankfully, Contegix was able to add another 2GB of RAM to our box and get everything back up-to-snuff.

New Committer: J. Garcia
J. Garcia has been a regular voice on the users and developers mailing list for several months. He's recently started contributing a lot of patches in JIRA and seems genuinely interested in the success of AppFuse. That's why we voted and added him as a committer. To prove this was a smart move, he recently replaced Compass with Hibernate Search and upgraded to Hibernate 4. As part of this work, he removed iBATIS support, which brings me to my next point.

The Future
In mid-August, I sent an email to the community, asking them "Anyone using iBATIS?"

I'm thinking of replacing AppFuse's Data Tier with Spring Data, especially because it has NoSQL and REST support. There's a good intro on InfoQ today:

http://www.infoq.com/articles/spring-data-intro

Does anyone see an issue with this? The lack of iBATIS support could be an issue, but I doubt it since if we wanted to continue supporting it, we should move to MyBATIS.

Everyone agreed this was a good idea and it seemed like a logical time to remove iBATIS support. In addition, I posted a roadmap I jotted down in early May. Since we've missed all the dates so far, I've removed them from the listing below. We hope to get 1-2 releases done by the end of this year, with 2.2 in the next 2-3 weeks.

2.2
Hibernate 4
Hibernate Search
Bootstrap
H5BP
2.3
AMP for all light modules
Wicket
PrimeFaces
2.4
JSR 303 (might require removing or developing client-side support)
Mockito instead of jMock/EasyMock
2.5
AMP one-to-many
Spring Data
MyBatis (if there's interest in adding it back in)
2.6
wro4j for concatenation and minimizing JS and CSS
pjax - https://github.com/defunkt/jquery-pjax
2.7
Scala example
Gradle example
Article about examples
3.0
GWT

Maven Central Statistics
To prove there's still a fair amount of folks using AppFuse, here's some statistic from Sonatype's OSS Repository Hosting Service.

From this screenshot, you can see that AppFuse artifacts are downloaded around 7,000 times per month. The following graph is even more interesting. Apparently, around 3,000 new projects are created with AppFuse archetypes each month.

The AppFuse Name
Finally, I recently discovered that ShoreTel decided to name a new product AppFuse. I guess this signifies two things: 1) it's a good name for a product and 2) someone didn't do their research before naming it. At this point, I'm not too concerned, but it is an interesting development.

Refreshing Taleo's UI with HTML5, Twitter Bootstrap and CSS3

2012-08-20T12:27:21-06:00

Back in December, I wrote about what I've been working on at Taleo. Shortly after finishing up the Profile Picture, Talent Card and Org Chart features for TBE, I spent two weeks doing page speed optimization. By following Web Performance Best Practices, I was able to make the TBE application twice as fast and improve its score into the low 90s.

Next, I started working on a new project - refreshing the UI. Nick, the Lead UX Designer at Taleo (at the time), had developed a number of mockups and presented it to the developers and product folks in early November. I listened to a WebEx of that meeting and learned that everyone thought it'd take 6-9 months to complete the work. They figured they could release the new design in Q3 2012.

Since I like to provide high-value for my clients, I offered to help with the redesign and do a spike to help estimate. They agreed it'd be a good use of my time and I started working on it the week before Christmas. Since I'd used Twitter Bootstrap for my Play More! app, I recommended we use it as a foundation of the redesign. They agreed and I went to work. By the end of the week, I'd made good progress and told them I thought the redesign was possible in 2-3 months (including QA and cross-browser compatibility).

When I came back to work in January, we decided to split the redesign into two phases. Rather than moving elements around and introducing new features, we decided to do that in the 2nd phase. The 1st phase would entail simply re-skinning the existing UI, with minimal HTML changes. I spent a week refining my spike and integrating it into a branch. The next week, I switched images from individual images to CSS sprites. Next, I implemented a new theming system with different colors/icons and got everything looking good in Chrome, Safari and IE8/9.

The result is something I'm quite proud of. IE8 doesn't have the rounded corners (via border-radius), but it still looks good. Forms look much better thanks to Bootstrap's styling and even jQuery UI's widgets look good thanks to jQuery UI Bootstrap. I did have to override quite a few Bootstrap styles in the process, but the result is something that doesn't look too bootstrappy.

One technique I found to be extremely useful during this process was to pair with Nick (the designer) as mentioned in Building Twitter Bootstrap. At one point, when we were trying to refine slight nuances and spacing in the UI, I paired with the Product Manager and found this to be a real time-saving effort as well.

Taleo's UI Refresh project has been a great experience for me in sharpening my CSS skills. I used quite a bit of child and sibling selectors, which work great in all the browser's we're supporting. Also, by using CSS sprites and colors (vs. images), I was able to get the manual theme-creation process down to around 15 minutes. After getting the manual process greatly reduced, I wrote a Theme Generator (based on Ant, LESS and wro4j) and got it down to mere minutes. I found Sprite Cow to be an invaluable resources for working with CSS sprites.

Below are some before and after shots of what we've been able to accomplish in the first quarter of this year.

I originally wrote this post at the end of January. We ran into some stumbling blocks shorty after its original composition: Nick (the designer) moved onto greener pastures and Oracle bought Taleo. What I didn't expect when I wrote this was to spend the next two months fixing slight bugs that occurred with spacing, alignment and dependent applications I didn't know about at the time. And then there was IE7. We didn't realize we needed to support it until mid-March. Then it took us around a month to make it all work good enough.

The good news is the UI Refresh was released a few months ago and seems to be humming along just fine. Sure, there were slight nuances and customizations we had conflicts with (clashing CSS classes), but overall it seems to have gone well. I can't thank the Bootstrap developers enough for motivating us to move to HTML5 and CSS3. Also, cheers to the excellent co-workers that helped make this happen: Murray Newton (Product Manager) and Vladimir Bazarsky. I couldn't have done it without you guys.

New Look and Feel, Designed by Gillen's Army

2012-08-14T22:58:11-06:00

As part of my 10-year blogiversary, I was hoping to refresh this site with a new look and feel. A few months ago, I contacted my friend Mark Waggoner to see about getting his design help. We promptly worked out a logo/business card/website deal and Gillen's Army went to work.

I picked a logo from numerous choices in late June, finalized a business card for printing in July and received the HTML and CSS for the site on August 2nd. I started converting it to a Roller theme last week and did a whole bunch of other modifications in the process.

Upgraded to Roller 5.0.1.
Upgraded wro4j to the latest version (1.4.8.1) to workaround using a → (\2192) in CSS.
Changed to use jQuery and Lightbox2 for pictures.
Upgraded to the latest version (3.0.83) of SyntaxHighlighter. You might notice there is no longer a toolbar in this version. However, you can still double-click on code and easily copy/paste it.

In addition to these upgrades, I made a few enhancements. I converted to HTML5 (by switching the doctype), added Modernizr and a feature that detects if the sun is up in your location. If you allow your browser to send me your lat and long, I'll give you a dark theme when the sun is down and a light theme when it's daylight. I used Preston's Hunt's JavaScript Class for Sunrise and Sunset Calculations to determine isDaylight. You can also change the theme to light or dark using the small rectangles above the search box on the right. This sets a cookie and overrides the HTML5 Geo check. You can see the implementation of this logic in site.js.

The stylesheet switching doesn't happen as fast as I'd hoped (there's a flash even if using cookies), so I'll likely be converting some theme-setting logic to the server-side. The HTML5 version of the FaceBook Like Button requires you to specify the "data-colorscheme" in markup so this further supports moving to the server.

I have other minor adjustments I'd like to make, but more importantly - I wanted to get it out to you all. Tell me what you like and don't like. Among other things, the form inputs for comments and contact forms have backgrounds that might not be great for those color-impaired. Also, you can see how the iframe on the contact page has a white background instead of one based on the theme.

Here's some stats comparing my old andreas08 theme to the new darklight:

Metric	andreas08	darklight
Size and Speed	167 requests, 3.6MB, 9.89s	148 requests, 3.2MB, 7.34s
YSlow	76	87
PageSpeed	91	96

Sweet! It looks like this site is faster than ever. Cheers to Mark and Gillen's Army for the new design. I dig it!

Upgrading to Play 2: Anorm and Testing

2012-06-05T20:55:40-06:00

This time last year, I decided I wanted to learn Scala. I chose the Play Framework as my vehicle for learning and I added CoffeeScript and Jade to the mix. I packaged it all up, learned a bunch and presented it at Devoxx 2011.

In January, I added SecureSocial, JSON Services and worked a bit on the mobile client. I presented my findings at Jfokus shortly after. As part of my aforementioned post, I wrote:

Right before we left for Jfokus, I was able to get everything to work, but didn't spend as much time as I'd like working on the mobile client. If this talk gets accepted for Devoxx France, I plan on spending most of my time enhancing the mobile client.

I had some complications (a.k.a. too much vacation) with Devoxx France and wasn't able to attend. To make up for it, I submitted the talk to ÜberConf. It got accepted and I started working on my app a couple weeks ago. So far, I've spent about 8 hours upgrading it to Play 2 and I hope to start re-writing the mobile client later this week. I plan on using Cordova, jQTouch and releasing it in the App Store sometime this month.

Upgrading to Play 2
When I heard about Play 2, I thought it was a great thing. The developers were re-writing the framework to use Scala at the core and I was already using Scala in my app. Then I learned they were going to throw backwards compatibility out the window and I groaned. "Really? Another web framework (like Tapestry of old) screwing its users and making them learn everything again?!", I thought. "Maybe they should call it Run instead of Play, leaving the old framework that everyone loves intact."

However, after hearing about it at Devoxx and Jfokus, I figured I should at least try to migrate. I downloaded Play 2.0.1, created a new project and went to work.

The first thing I learned about upgrading from Play 1.x to Play 2.x is there's no such thing. It's like saying you upgraded from Struts 1 to Struts 2 or Tapestry 4 to Tapestry 5. It's a migration, with a whole new project.

Evolutions
I started by looking around to see if anyone had documented a similar migration. I found two very useful resources right off the bat:

From Jan's Blog, I learned to copy my evolutions from my Play 1.x project into conf/evolutions/default. I changed my application.conf to use PostgreSQL and wrote an EvolutionsTest.scala to verify creating the tables worked.

import org.specs2.mutable._

import play.api.db.DB
import play.api.Play.current

import anorm._

import play.api.test._
import play.api.test.Helpers._

class EvolutionsTest extends Specification {

  "Evolutions" should {
    "be applied without errors" in {
      evolutionFor("default")
      running(FakeApplication()) {
        DB.withConnection {
          implicit connection =>
            SQL("select count(1) from athlete").execute()
            SQL("select count(1) from workout").execute()
            SQL("select count(1) from comment").execute()
        }
      }
      success
    }
  }
}

Then I began looking for how to load seed data with Play 2.x. In Play 1.x, you could use a BootStrap job that would load sample data with YAML.

import play.jobs._
import play.Play

@OnApplicationStart
class BootStrap extends Job {

  override def doJob() {

    import models._
    import play.test._

    // Import initial data if the database is empty
    if (Athlete.count().single() == 0) {
      Yaml[List[Any]]("initial-data.yml").foreach {
        _ match {
          case a: Athlete => Athlete.create(a)
          case w: Workout => Workout.create(w)
          case c: Comment => Comment.create(c)
        }
      }
    }
  }
}

This is no longer a recommended practice in Play 2. Instead, they recommend you turn your YAML into code. 10 minutes later, I had a Global.scala that loaded seed data.

import models._
import play.api._
import play.api.Play.current

import anorm._

object Global extends GlobalSettings {

  override def onStart(app: Application) {
    InitialData.insert()
  }
}

/**
 * Initial set of data to be loaded
 */
object InitialData {

  def date(str: String) = new java.text.SimpleDateFormat("yyyy-MM-dd").parse(str)

  def insert() {

    if (Athlete.count() == 0) {

      Seq(
        Athlete(Id(1), "mraible@gmail.com", "beer", "Matt", "Raible"),
        Athlete(Id(2), "trishmcginity@gmail.com", "whiskey", "Trish", "McGinity")
      ).foreach(Athlete.create)

      Seq(
        Workout(Id(1), "Chainsaw Trail",
          """
            A beautiful fall ride: cool breezes, awesome views and yellow leaves.

            Would do it again in a heartbeat.
          """, 7, 90, date("2011-10-13"), 1),
        Workout(Id(2), "Monarch Lake Trail",
          "Awesome morning ride through falling yellow leaves and cool fall breezes.",
          4, 90, date("2011-10-15"), 1),
        Workout(Id(3), "Creekside to Flume to Chainsaw",
          "Awesome morning ride through falling yellow leaves and cool fall breezes.",
          12, 150, date("2011-10-16"), 2)
      ).foreach(Workout.create)

      Seq(
        Comment(1, "Jim", "Nice day for it!"),
        Comment(2, "Joe", "Love that trail."),
        Comment(2, "Jack", "Where there any kittens there?")
      ).foreach(Comment.create)
    }
  }
}

Anorm's Missing Magic
Before starting with Play 2, I knew it had lost some of its magic. After all, the developers had mentioned they wanted to get ride of the magic and moving to Scala allowed them to do that. However, I didn't think I'd miss Magic[T] as much as I do. Like Martin Fowler, I like ORMs and having to use SQL again seems painful. It seems like a strange shift for Play to reduce type-safety on the backend, but increase it in its templates. Oh well, to each their own. I may eventually move to Squery, but I wanted to do a side-by-side comparison as part of my migration.

Using the aforementioned tutorial from James and Jan's blog posts, as well as Guillaume's Play 2.0/Anorm, I set about creating new model objects. I wrote a bunch of SQL, typed up some new finders and migrated my tests from ScalaTest to the new default, specs2. The Mosh Pit's Migrating a Play 1.2 website to Play 2.0 was a great help in migrating tests.

That's when I started having issues with Anorm and figuring out how its parser syntax works. After struggling for a few days, I finally found yabe-play20-scala. Since I'd used the yabe tutorial from Play 1.x, it was familiar and helped me get past my problems. Now, things aren't perfect (Workouts aren't ordered by their posted date), but everything compiles and tests pass.

To illustrate how little code was required for Anorm 1.x, checkout Workout.scala in Play 1.x vs. Play 2.x. The Play 1.x version is 66 lines; Play 2.x requires 193 lines. I don't know about you, but I kinda like a little magic in my frameworks to reduce the amount of code I have to maintain.

I was pleasantly surprised by specs2. First of all, it was an easy migration from ScalaTest. Secondly, Play's FakeApplication made it very easy to write unit tests. The line count on my UnitTests.scala in Play 1.x vs. Play 2.x is almost identical.

Summary
The first few hours of developing with Play 2 were frustrating, mostly because I felt like I had to learn everything over again. However, I was pleased to find good references on migrating from Play 1.x. Last night, I migrated all my Controllers, integrated Scalate and got most of my views rendering. I still have issues rendering validation errors, but I hope to figure that out soon. The last 2 hours have been much more fun and I feel like my Scala skills are coming along. I think if the Play Team could eliminate those first few hours of struggling (and provide almost instant joy like Play 1.x) they'd really be onto something.

As soon as I figure out how to validation and how to add a body class based on the URL, I'll write another post on the rest of my migration. A Play 2-compatible version of SecureSocial just came out this evening, so I may integrate that as well. In the meantime, I'll be working on the iPhone app and finishing up a Grails 2 application for James Ward and my Grails vs. Play Smackdown.

Play Framework 2.0 with Peter Hilton at Jfokus

2012-02-14T07:17:08-07:00

This week, I'm at Jfokus in Stockholm, Sweden. After a fun speaker's dinner last night, I got up this morning and polished up my presentations and demo before attending the conference. The first session I attended was Peter Hilton's Play Framework 2.0 presentation. Below are my notes from this talk.

Peter is a Senior Web Developer, not a Java Developer. His first slide states the following:

"Play brings type safe high-productivity web development to the JVM."

New features in Play 2.0: type-safety, template syntax, compile-time checking and asynchronous HTTP programming. Java, Scala - the language you use is less important than the fact that Play is a web framework. It's a full-stack framework and has everything you need out-of-the-box to build a web application. Play focuses on HTTP and doesn't try to hide it. It's designed by web developers for web developers.

With Play, the Back button just works. Your web framework shouldn't break the first button on your browser's toolbar. The Reload button also works: make a change, hit reload and your changes (even in Scala classes) are shown. You design the URLs and you can use "clean" URLs. DX (Developer eXperience) is Peter's new term. Usability matters: as a developer, you deserve a framework that provides a good experience.

Play doesn't fight HTTP or the browser. It's stateless and HTTP-centric. A few years ago, it seemed like a good idea to try and keep state on the server. It sounded like a good idea, but in practice, it's a really bad idea - especially for things like the back button. Play matches the web's stateless HTTP architecture.

As a Java EE developer, PHP and Rails developers have been laughing at us for years. Like Father Christmas, Peter's heard of class-reloading, but he hasn't actually seen it. Code reloading is the most important part of DX and about achieving high-productivity in web development.

URLs want to be loved too. REST architecture isn't just for web service APIs. When you have clean URLs, you can tweet them, post them and email them.

"You would need to be a super-hero to successfully use some web frameworks." They show you a blank screen in the browser and you have to look at your console's stack trace to figure it out. With Play, the error is shown in your browser and you can see the exact line it happens on.

In Play 1.x, there was a lot of magic and a lot of bytecode enhancement at runtime. This allowed the API to be a lot nicer than traditional Java APIs. However, it caused issues when users viewed the enhanced source and it also caused issues in IDEs. With Play 2.0, the framework itself is implemented in Scala. Scala removes the need for so much bytecode enhancement. There is less 'magic' and strangeness in the API. The code you see in the IDE is the code that runs. Scala source code is not necessarily harder to read. 1.x had some pretty hairy Java code, and you could tell when you dug into it. Especially when you were deep into the source code and saw that a lot of the comments were in French.

Play 2.0's template system is based on Scala. It's similar to the lightweight template syntax in Play 1.x. Templates are compiled into class files for run-time speed. For example:

@(products: Seq[Product])

<ul>
@for(product <- products) {
  <li>@product.name</li>
}
</ul>

@summary(products)

We used to think XML-based templates were great, but it turns out it's a terrible idea. Mostly because you end up having to invent an expression language to create valid XML (to avoid putting XML in your HTML attributes). With Play 2.0's templates, you can define tags in your templates as regular Scala methods.

@display(product: models.Product) ={
 <a href="@routes.Product.details(product.id)">@product.name</a> 
}

@for(product <- products) {
  @display(product)
}

The compile-time checking in Play 2.0 is not just for Java and Scala classes. It also compiles your HTTP routes file (which maps requests to controller actions). Furthermore, it compiles your templates, JavaScript files (using Google Closure Compiler), CoffeeScript files and LESS stylesheets.

Play supports modern web development. It's designed to work with HTML5, but there's no constraints on HTML output. It's front-end developer friendly and has great DX. UI components belong in the client, e.g. jQuery UI. It also has built-in support for improvements to CSS (LESS) and JavaScript (CoffeeScript).

A few years ago, it seemed like a really good idea to hide JavaScript from the web developer. Web frameworks used to say "You don't need to see the JavaScript or the HTML, we'll handle generating your components for you." Now, if you're building a web application and you don't know any jQuery, you doing it the hard way. You should learn how to work with front-end developers or learn how to do it yourself. And make sure your web framework allows this sort of development.

The future of web programming is asynchronous. You'll perform simultaneous web service requests. You'll process streams of data, instead of filling up memory or disk. You'll publish real-time data and have predictable and minimal resource consumption. In the long term, this changes everything. The future of the web is real-time and asynchronous. With Play 2.0, it's not just another feature, it's a fundamental aspect of the architecture. Play's internal architecture uses a reactive model based on Iteratee IO.

In summary, use Play 2, use HTML5, deploy to the Cloud. There's two forthcoming books on Play (both from Manning) and Play 2.0 RC1 will be released today.

I think Peter did a good job of summarizing the new features in Play 2.0, especially how templates work. I enjoyed his emphasis on HTTP and how Play leverages the browser (back, reload and as a console). I liked his humorous speaking style, and agree with his emphasis that client-side development skills are important for modern web applications. I think Play 2.0 is making a big bet on Scala and asynchronous programming, but if they live up to the hype, it should be a very enjoyable web framework to develop with.

Secure JSON Services with Play Scala and SecureSocial

2012-02-12T16:02:43-07:00

Last November, I traveled to Antwerp to speak at Devoxx. After my talk on HTML5 with Play Scala, Mattias Karlsson approached me and we had a chat about doing the same talk at Jfokus in Stockholm. I agreed and we began talking details after Trish and I returned to the US.

I wrote this article on a plane between Denver and Seattle and will be hopping over the North Pole to Stockholm via Iceland tonight. For the past couple of weeks, I've been updating my Play More! HTML5/mobile app to add some new features. Most notably, I wanted to upgrade to Play 2.0, create JSON services and add authentication.

Upgrading to Play 2.0
My attempt to upgrade to Play 2.0 involved checking out the source from GitHub, building and installing the RC1 snapshot. As I tried to upgrade my app and started getting failed imports, I turned to the internet (specifically StackOverflow) to see if it was a good idea. The first answer for that question suggested I stay with 1.x.

If it's a critical project, to be finished before next March 2012, I would go with Play 1.x. If it's a less important project, which could be delayed, and that in any case won't be released before March 2012, try Play 2.0.

While I didn't plan on releasing Play More! before Jfokus, I decided upgrading didn't add a whole lot to the talk. Also, I couldn't find a Play Scala 0.9.1 to Play 2.0 upgrade guide and I didn't have enough time to create one. So I decided to stick with Play 1.2.4 and add some JSON services for my iPhone client.

JSON Servers
I found Manuel Bernhardt's Play! Scala and JSON. This led me to Jerkson, built by the now infamous @coda. I was able to easily get things working fairly quickly and wrote the following WorkoutService.scala:

package controllers.api

import play.mvc.Controller
import models._
import com.codahale.jerkson.Json._

object WorkoutService extends Controller {

  def workouts = {
    response.setContentTypeIfNotSet("application/json")
    generate(Workout.find().list())
  }
  def edit(id: Long) = {
    generate(Workout.byIdWithAthleteAndComments(id))
  }

  def create() = {
    var workout = params.get("workout", classOf[Workout])
    Workout.create(workout)
  }

  def save(id: Option[Long]) = {
    var workout = params.get("workout", classOf[Workout])
    Workout.update(workout)
  }

  def delete(id: Long) = {
    Workout.delete("id={id}").on("id" -> id).executeUpdate()
  }
}

Next, I added routes for my new API to conf/routes:

GET     /api/workouts               api.WorkoutService.workouts
GET     /api/workout/{id}           api.WorkoutService.edit
POST    /api/workout                api.WorkoutService.create
PUT     /api/workout/{id}           api.WorkoutService.save
DELETE  /api/workout/{id}           api.WorkoutService.delete

Then I created an ApiTest.scala class that verifies the first method works as expected.

import play.test.FunctionalTest
import play.test.FunctionalTest._
import org.junit._

class ApiTests extends FunctionalTest {
  
    @Test
    def testGetWorkouts() {
        var response = GET("/api/workouts");
        assertStatus(200, response);
        assertContentType("application/json", response)
        println(response.out)
    }
}

I ran "play test", opened my browser to http://localhost:9000/@tests and clicked ApiTests -> Start to verify it worked. All the green made me happy.

Finally, I wrote some CoffeeScript and jQuery to allow users to delete workouts and make sure delete functionality worked.

$('#delete').click ->
  $.ajax
    type: 'POST'
    url: $(this).attr('rel')
    error: ->
      alert('Delete failed, please try again.')
    success: (data) ->
      location.href = "/more"

I was very impressed with how easy Play made it to create JSON services and I smiled as my CoffeeScript skills got a refresher.

The Friday before we left for Devoxx, I saw the module registration request for SecureSocial.

SecureSocial with Play Scala
From SecureSocial's README:

SecureSocial allows you to add an authentication UI to your app that works with services based on OAuth1, OAuth2, OpenID and OpenID+OAuth hybrid protocols.

It also provides a Username and Password mechanism for users that do not wish to use existing accounts in other networks.

The following services are supported in this release:

Twitter (OAuth1)
Facebook (OAuth2)
Google (OpenID + OAuth Hybrid)
Yahoo (OpenID + OAuth Hybrid)
LinkedIn (OAuth1)
Foursquare (OAuth2)
MyOpenID (OpenID)
Wordpress (OpenID)
Username and Password

In other words, it sounded like a dream come true and I resolved to try it once I found the time. That time found me last Monday evening and I sent a direct message to @jaliss (the module's author) via Twitter.

Does Secure Social work with Play Scala? I'd like to use it in my Play More! project.

Jorge responded 16 minutes later saying that he hadn't used Play Scala and he'd need to do some research. At 8 o'clock that night (1.5 hours after my original DM), Jorge had a sample working and emailed it to me. 10 minutes later I was adding a Secure trait to my project.

package controllers

import play.mvc._
import controllers.securesocial.SecureSocial

/*
 * @author Jorge Aliss <jaliss@gmail.com> of Secure Social fame.
 */
trait Secure {
  self: Controller =>

  @Before def checkAccess() {
    SecureSocial.DeadboltHelper.beforeRoleCheck()
  }

  def currentUser = {
    SecureSocial.getCurrentUser
  }
}

I configured Twitter and Username + Password as my providers by adding the following to conf/application.conf.

securesocial.providers=twitter,userpass

I also had to configure a number of securesocial.twitter.* properties. Next, I made sure my routes were aware of SecureSocial by adding the following to the top of conf/routes:

  *       /auth               module:securesocial

Then I specified it as a dependency in conf/dependencies.yml and ran "play deps".

    - play -> securesocial 0.2.4

After adding "with Secure" to my Profile.scala controller, I tried to access its route and was prompted to login. Right off the bat, I was shown an error about a missing jQuery 1.5.2 file in my "javascripts" folder, so I added it and rejoiced when I was presented with a login screen. I had to add the app on Twitter to use its OAuth servers, but I was pumped when both username/password authentication worked (complete with signup!) as well as Twitter.

The only issue I ran into with SecureSocial was that it didn't find the default implementation of SecureSocial's UserService.Service when running in prod mode. I was able to workaround this by adding a SecureService.scala implementation to my project and coding it to talk to my Athlete model. I didn't bother to hook in creating a new user when they logged in from Twitter, but that's something I'll want to do in the future. I was also pleased to find out customizing SecureSocial's views was a breeze. I simply copied them from the module into my app's views and voila!

package services

import play.db.anorm.NotAssigned
import play.libs.Codec
import collection.mutable.{SynchronizedMap, HashMap}
import models.Athlete
import securesocial.provider.{ProviderType, UserService, SocialUser, UserId}

class SecureService extends UserService.Service {
  val activations = new HashMap[String, SocialUser] with SynchronizedMap[String, SocialUser]

  def find(userId: UserId): SocialUser = {
    val user = Athlete.find("email={email}").on("email" -> userId.id).first()

    user match {
      case Some(user) => {
        val socialUser = new SocialUser
        socialUser.id = userId
        socialUser.displayName = user.firstName
        socialUser.email = user.email
        socialUser.isEmailVerified = true
        socialUser.password = user.password
        socialUser
      }
      case None => {
        if (!userId.provider.eq(ProviderType.userpass)) {
          var socialUser = new SocialUser
          socialUser.id = userId
          socialUser
        } else {
          null
        }
      }
    }
  }

  def save(user: SocialUser) {
    if (find(user.id) == null) {
      val firstName = user.displayName
      val lastName = user.displayName
      Athlete.create(Athlete(NotAssigned, user.email, user.password, firstName, lastName))
    }
  }

  def createActivation(user: SocialUser): String = {
    val uuid: String = Codec.UUID()
    activations.put(uuid, user)
    uuid
  }

  def activate(uuid: String): Boolean = {
    val user: SocialUser = activations.get(uuid).asInstanceOf[SocialUser]
    var result = false

    if (user != null) {
      user.isEmailVerified = true
      save(user)
      activations.remove(uuid)
      result = true
    }

    result
  }

  def deletePendingActivations() {
    activations.clear()
  }
}

Jorge was a great help in getting my authentication needs met and he even wrote a BasicAuth.scala trait to implement Basic Authentication on my JSON services.

package controllers

import _root_.securesocial.provider.{UserService, ProviderType, UserId}
import play._
import play.mvc._
import play.libs.Crypto

import controllers.securesocial.SecureSocial

/*
 * @author Jorge Aliss <jaliss@gmail.com> of Secure Social fame.
 */
trait BasicAuth {
  self: Controller =>

  @Before def checkAccess = {
    if (currentUser != null) {
      // this allows SecureSocial.getCurrentUser() to work.
      renderArgs.put("user", currentUser)
      Continue
    }

    val realm =
      Play.configuration.getProperty("securesocial.basicAuth.realm", "Unauthorized")

    if (request.user == null || request.password == null) {
      Unauthorized(realm)
    } else {
      val userId = new UserId
      userId.id = request.user
      userId.provider = ProviderType.userpass
      val user = UserService.find(userId)

      if (user == null ||
        !Crypto.passwordHash(request.password).equals(user.password)) {
        Unauthorized(realm)
      } else {
        // this allows SecureSocial.getCurrentUser() to work.
        renderArgs.put("user", user)
        Continue
      }
    }
  }

  def currentUser = {
    SecureSocial.getCurrentUser()
  }
}

Summary
My latest pass at developing with Scala and leveraging Play to build my app was a lot of fun. While there were issues with class reloading every-so-often and Scala versions with Scalate, I was able to add the features I wanted. I wasn't able to upgrade to Play 2.0, but I didn't try that hard and figured it's best to wait until its upgrade guide has been published.

I'm excited to describe my latest experience to the developers at Jfokus this week. In addition, the conference has talks on Play 2.0, CoffeeScript, HTML5, Scala and Scalate. I hope to attend many of these and learn some new tricks to improve my skills and my app.

Update: The Delving developers have written an article on Migration to Play 2. While it doesn't provide specific details on what they needed to change, it does have good information on how long it took and things to watch for.

Twitter's Open Source Summit: Bootstrap 2.0 Edition

2012-02-01T11:28:40-07:00

Every few months, Twitter hosts an Open Source Summit to talk about tools they're using. Since I happened to be near San Fransisco, I happily attended their latest #ossummit to learn about Bootstrap 2.0. Below are my notes from last night's event.

95% of Twitter's infrastructure is powered by open source. They hope to contributing back to open source by doing 2-3 summits per year. Without open source, there would be no Twitter. You can find a bunch of Twitter's open source contributions at twitter.github.com. They're also big fans of Apache and commit to a wide variety of projects there.

Bootstrap
Bootstrap is developed by two main guys: @mdo and @fat. Mark (@mdo) has been a designer at Twitter for 2 years. He started on the Revenue Team with ads, but has been working on redesign for last 4 months. Has been doing HTML and CSS for about 11 years. He used Notepad on Windows to build his GeoCities site.

boot·strap: simple and flexible HTML, CSS, and JavaScript for popular user interface components and interactions.

Work on Bootstrap started about a 1.5 years ago. Internal tools didn't get the proper attention they needed. They figured out there was a lot of people that wanted good looking UIs and interactions. It became Twitter Blueprint and was mostly used internally. Jacob (@fat) started shortly after first release and decided to add some JavaScript on top of it. The JavaScript for Bootstrap was originally the "Twitter Internal Toolkit" or "TIT" and was built on Moo Tools. Jacob was like "we gotta open source this, it's gonna be huge!" (he was right).

The 1.0 release supported Chrome, Safari and Firefox (everyone at Twitter was on Macs). 1.3 added cross-browser support and JavaScript plugins.

Now there's Bootstrap 2 (just released!). They rewrote all the documentation and components and removed legacy code.

So, what's new? The biggest thing is the docs. Previously had live examples, now shows live examples and why you would do something, as well as additional options. The "topbar" has been renamed to "navbar", but it's still got all the hotness. It's responsive with CSS media queries for small devices, tablets, small desktops and large desktops. This means the layout breaks at certain points and stacks elements to fit on smaller screens.

CSS: smarter defaults, better classes. In 1.4, all forms were stacked. Now they can flow horizontally. Tables are now namespaced so Bootstrap's styles don't apply to all tables. The available table, form and navigation classes are as follows:

// Tables
.table { ... }
.table-striped { ... }
.table-bordered { ... }
.table-condensed { ... }

// Forms
.form-inline { ... }
.form-search { ... }
.form-horizontal { ... }

// Nav
.nav { ... }
.nav-tabs { ... }
.nav-pills { ... }

The goals with 2.0 are consistency, simplification and future-proofing styles. With 1.4, buttons used "btn primary" and it caused problems if you wanted to have a "primary" class in your project. With 2.0, buttons and all elements are namespaced to avoid collisions (now it's .btn-primary).

After Mark finished talking about the design of Bootstrap, Jacob (@fat) started talking about Bootstrap's JavaScript. Jacob works on The Platform Team at Twitter and claims he made a lot of mistakes with 1.x. However, thanks to semantic versioning, 2.0 is a new version and he got to start over!

The biggest change in 2.0 is the use of data attributes (a.k.a. data-*). They were using them in 1.x, but not to the full potential of what they can be and should be. The first class API for Bootstrap JavaScript is data attributes (or HTML), not JavaScript.

With 1.x, you could add an anchor to close modals and alerts.

// 1.x closing modal/alerts
<a class="close" href="#">×</a>

However, if you put your alerts in your modals, you close them all when you likely only wanted to close one. 2.0 uses a "data-dismiss" attribute.

<a class="close" data-dismiss="model">×</a>

This allows you to target what you want closed (modals or alerts, etc.). You know exactly what's going to happen just by reading the code. Another example is the "href" attribute of an anchor. Rather than using "href", you can now use "data-target".

// 1.x href = target
<a href="#myModal" data-toggle="modal">Launch</a>

// 2.x data-target = target
<a data-target=".fat" data-toggle="modal">Launch</a>

If you'd rather turn off the data attribute API, or just part of it, you can do so by using the following:

// Turn off all data-api
$('body').off('.data-api')

// Turn off alert data-api
$('body).off('.alert.data-api')

2.0's JavaScript API has the same stuff, but better. You can turn off the data-api and do everything with JavaScript. They copied jQuery UI in a lot of ways (defaults, constructors, etc.). Bootstrap's JavaScript has 12 plugins. New ones include collapse, carousel and typeahead.

Customize - a new tab that lets you customize and download Bootstrap. It's basically an alternative to customizing .less files and allows you to choose components, select jQuery plugins, customize variables (colors, font-sizes, backgrounds) and download.

What does the future hold? Internationalization, improving responsiveness, more new features and bug fixes.

After both Mark and Jacob gave their talks, they talked together about Community and how great it's been. Even if you're not into writing CSS and JavaScript, they mentioned they still wanted to hear from you. To give an example of great community contributions, one guy opened 50 issues in the last 2 days.

Someone in the audience asked why they used LESS over SASS. Jacob said the main reason they use LESS is because they're good friends with the guy who invented it (Alexis). SASS turns CSS into a programming language, but they wanted to maintain the approachability of CSS, which LESS does. There's no plans to do an official SASS port, but there is talk of doing one. One advantage of the current LESS compiler is they rewrote it to have better output so it's far more readable.

NASA
After Mark and Jacob finished, there was a 5 minute break to grab beers and snacks. Then Sean Herron (@seanherron) (a.k.a. "NASA Bro") talked about Bootstrap at NASA. He actually didn't talk about Bootstrap much, except that they used it for code.NASA. He talked about NASA and how it's playing a key role in the movement towards open data, open source and open standards in our federal government. He mentioned how data.NASA was launched last August and that they helped develop OpenStack. Finally, he mentioned open.NASA, which is a collaborative approach to open, direct and transparent communication about our space program.

Hogan.js
Next up, Rob Sayre (@sayrer) talked about Hogan.js. Rob has been at Twitter for a few months, before that he wrote JavaScript at other places. Hogan.js is a compiler for Mustache templates.

Why Mustache? Because it's similar to HTML and easy to edit. You can mock data as JSON files and programmers are not required.

At Twitter, designers can do the CSS and Mustache without connecting to the backend. It has cross-language support in Ruby, Java and JavaScript. However, client-side template compilation has performance problems, especially in IE7 on a Windows XP box with 4 viruses.

So they had a few choices: work on mustache.js, or use Dust.js or Handlebars.js. The compilers are very nice for Dust.js and Handlebars.js, but they're huge. Handlebar's parser is 4000 lines. The entire Hogan.js file is 500 lines. They decided they were too large to send to the browser's of their users, so they chose to write a better compiler for Mustache.

Hogan.js's main features:

Compile on the server
Parser API
Performance

Performance is much better with Hogan.js than Mustache.js. On IE7 - Hogan is 5x faster than Mustache. On an iPhone, it's about the same (and an iPhone's browser is faster than IE7 on a decent computer). With modern browsers (Chrome 17, Safari 5 and Firefox 10), it's more than 10x faster.

Hogan.js is currently used at Twitter for Tweet embedding, the Bootstrap build process and soon, Twitter.com.

It's been awhile since I got excited about an open source project. Bootstrap has helped me a lot recently, in my Play More! mobile app, on some client projects and I'm in the process of refreshing AppFuse's UI to use it. I love how you can add a class or two to an element and all of a sudden they pop with good looks. The main problem with Bootstrap at this point is that a lot of Bootstrapped apps look the same. There's talk of adding themes in a future release to help alleviate this problem. In the meantime, there's a lot to get excited about with 2.0.

Thanks to Twitter for hosting this event and kudos to Mark and Jacob (and the community!) for such a fantastic project.

Refreshing AppFuse's UI with Twitter Bootstrap

2012-01-31T17:12:17-07:00

The last time AppFuse had an update done to its look and feel was in way back in 2006. I've done a lot of consulting since then, which has included a fair bit of page speed optimization, HTML5 development and integrating smarter CSS. It was way back in '05 when we first started looking at adding a CSS Framework to AppFuse. It was Mike Stenhouse's CSS Framework that provided the inspiration and my CSS Framework Design Contest that provided its current themes (puzzlewithstyle, andreas01 and simplicity).

Since then, a lot of CSS Frameworks have been invented, including Blueprint in 2007 and Compass in 2008. However, neither has taken the world by storm like Twitter Bootstrap. From Building Twitter Bootstrap:

A year-and-a-half ago, a small group of Twitter employees set out to improve our team’s internal analytical and administrative tools. After some early meetings around this one product, we set out with a higher ambition to create a toolkit for anyone to use within Twitter, and beyond. Thus, we set out to build a system that would help folks like us build new projects on top of it, and Bootstrap was conceived.
...
Today, it has grown to include dozens of components and has become the most popular project on GitHub with more than 13,000 watchers and 2,000 forks.

The fact that Bootstrap has become the most popular project on GitHub says a lot. For AppFuse.next, I'd like to integrate a lot of my learnings over the past few years, as well as support HTML5 and modern browsers as best we can. This means page speed optimizations, getting rid of Prototype and Scriptaculous in favor of jQuery, adding wro4j for resource optimization and integrating HTML5 Boilerplate. I've used Twitter Bootstrap for my Play More! app, as well as some recent client projects. Its excellent documentation has made it easy to use and I love the way you can simply add classes to elements to make them transform into something beautiful.

Last week, I spent a couple late nights integrating Twitter Bootstrap 2.0 into the Struts 2 and Spring MVC versions of AppFuse. The layout was pretty straightforward thanks to Scaffolding. Creating the Struts Menu Velocity template to produce dropdowns wasn't too difficult. I added class="table table-condensed" to the list screen tables, class="well form-horizontal" to forms and class="btn primary" to buttons.

I also added validation errors with the "help-inline" class. This is also where things got tricky with Struts and Spring MVC. For the form elements in Bootstrap, they recommend you use a "control-group" element that contains your label and a "controls" element. The control contains the input/select/textarea and also the error message if there is one. Here's a sample element waiting for data:

<div class="control-group">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name">
    </div>
</div>

Below is what that element should look like to display a validation error:

<div class="control-group error">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name" value="">
        <span class="help-inline">Please enter your name.</span>
    </div>
</div>

You can see this markup is pretty easy, you just need to add an "error" class to control-group and span to show the error message. With Struts 2, this was pretty easy thanks to its customizable templates for its tags. All I had to do was create a "template/css_xhtml" directory in src/main/webapp and modify checkbox.ftl, controlfooter.ftl, controlheader-core.ftl and controlheader.ftl to match Bootstrap's conventions.

Spring MVC was a bit trickier. Since its tags don't have the concept of writing an entire control (label and field), I had to do a bit of finagling to get things to work. In the current implementation, Struts 2 forms have a single line for a control-group and its control-label and controls.

<s:textfield key="user.firstName" required="true"/>

With Spring MVC, it's a bit more complex:

<spring:bind path="user.firstName">
<fieldset class="control-group${(not empty status.errorMessage) ? ' error' : ''}">
</spring:bind>
    <appfuse:label styleClass="control-label" key="user.firstName"/>
    <div class="controls">
        <form:input path="firstName" id="firstName" maxlength="50"/>
        <form:errors path="firstName" cssClass="help-inline"/>
    </div>
</fieldset>

You could probably overcome this verbosity with Tag Files.

Figuring out if a control-group needed an error class before the input tag was rendered was probably the hardest part of this exercise. This was mostly due to Bootstrap's great documentation and useful examples (viewed by inspecting the markup). Below are some screenshots of the old screens and new ones.

Check out the full set on Flickr if you'd like a closer look.

Even though I like the look of the old UI, I can't help but think a lot of the themes are designed for blogs and content sites, not webapps. The old Wufoo forms were a lot better looking though. And if you're going to develop kick-ass webapps, you need to make them look good. Bootstrap goes a long way in doing this, but it certainly doesn't replace a good UX Designer. Bootstap simply helps you get into HTML5-land, start using CSS3 and it takes the pain out of making things work cross-browser. Its fluid layouts and responsive web design seems to work great for business applications, which I'm guessing AppFuse is used for the most.

I can't thank the Bootstrap developers enough for helping me make this all look good. With Bootstrap 2 dropping this week, I can see myself using this more and more on projects. In the near future, I'll be helping integrate Bootstrap into AppFuse's Tapestry 5 and JSF versions.

What do you think of this CSS change? Do you change your CSS and layout a fair bit when starting with AppFuse archetypes? What can we do to make AppFuse apps look better out-of-the-box?

Update: I updated AppFuse to the final Bootstrap 2.0 release. Also, Johannes Geppert wrote a Struts 2 Bootstrap Plugin. I hope to integrate this into AppFuse in the near future.

2011 - A Year in Review

2012-01-11T09:45:20-07:00

2010 was an awesome year, but 2011 rocked the house! At the end of last year, I kept my goals simple:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus. Everything else is gravy.

As predicted, it was a spectacular year, but I only accomplished 50% of my goals. That is, Trish and I had a great time skiing (especially in Utah), moving in together, traveling the world and getting engaged in Versailles. I even satisfied some 5-year goals: building a sauna in my basement and getting a ski shack in the mountains.

However, I didn't get much done with The Bus. Or rather, the guys at MotorWorks Restorations didn't drain me for all I'm worth in 2011. We did make good progress with estimating the final cost and obtaining many hard-to-find parts though. I now have a Porsche 911 Engine (1983 3.0L 6 cylinder), a Porsche 901 5 speed transmission, Porsche "Turbo Twist" wheels and a Custom Air Ride Front Beam from Franklin's VW Works. The thing that slowed our progress the most was the custom beam, as it took almost 6 months from order to delivery. When it arrived in September, I decided to put things on hold. I didn't want to get my bus back in the midst of winter and not be able to drive it.

In January, I started working at Overstock.com and flew out to Salt Lake City the first two weeks to ramp up. In setting up my development environment, I discovered some inefficiencies and wrote about how I fixed them in Making Code Generation Smarter with Maven. I also discovered that JRebel slowed my build process down by almost 50% and stopped using it as part of my daily process. Now I simply run in debug mode or use fancy frameworks that allow instant reload.

At the end of the month, I skied Alta for the first time on a blue bird day. That night, I flew back to Denver, had a gorgeous lunch ride in 70°F weather on Friday afternoon, then picked up my lovely girlfriend, Trish, and drove to Crested Butte for a weekend of skiing with James Ward and friends. It was my first time visiting Crested Butte and I can see why people call it Paradise.

As of February 1st, Trish and I both had 17 ski days. As I'm writing this, we're both up to 14 for this season and very proud to say Abbie and Jack each have 10. Both kids are shredding like I've always dreamed they would.

My parents (both retired) came to visit for the month of February and helped me start building a sauna in my basement. This was something I've wanted to do for years, so it was a lot of fun getting things started. While we didn't finish construction, we did get the plumbing/electrical done, stove installed and walls framed. We also had a fun night with Chef Navarro and Friends.

The rest of my posts in February were purely technical. I wrote about how I implemented extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF. Then I started delving into security-related topics, largely motivated by a new feature I was working on at Overstock. When I wrote about Implementing Ajax Authentication using jQuery, Spring Security and HTTPS, I explained what I was trying to accomplish.

The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience.

I was able to get everything working, posted a demo and wrote about upcoming conferences in Las Vegas and Kraków. In that article, I described a new talk I was going to create: Webapp Security: Develop. Penetrate. Protect. Relax. I ended the month with a post on how to fix XSS in JSP 2.

On February 25th, I experienced The Greatest Snow on Earth with thigh-deep powder and face shots on every run.

That night, we saw Hot Buttered Rum and woke up early for 27" of fresh powder at Alta. The picture of Trish free-heeling at Alta is one of my favorites of the year.

In early March, we experienced a suite night at the Avs game, then I kept it technical and upgraded this blog to Apache Roller 5.0, RC4. I upgraded AppFuse to JSF 2, wrote about JSR 303 and web framework support and added search to AppFuse using Compass. When writing about JSF 2, I mentioned replacing Ajax4JSF with RichFaces. 9 months later, it seems like PrimeFaces has more momentum, so I may try that instead.

For St. Patrick's Day, we flew to Vegas for TSSJS and had a blast. Not only did my talks go well, but we had a bunch of friends in town and had much success gambling the nights away.

After delivering my Comparing JVM Web Frameworks talk, I had a very interesting conversation with James Strachan. He recommended I take a look at CoffeeScript and Jade. A couple months later, when I was thinking of talks for Devoxx, I added these to my list of technologies to learn and submitted a talk about HTML5 with Play Scala, CoffeeScript and Jade.

That Friday, we returned to Denver, got a good night's sleep then headed on a hut trip in Colorado's High Country. The name of the hut was Peter Estin Hut and it was a bit of a hike to get to. My friend, Joe, set my expectations correctly when he warned me it'd be a 5 hour death march. It took us 4 hours, 30 minutes and we skied up 2200 vertical feet of switchbacks.

Trish and I started off April with a bang by moving in together, then embarking on a whirlwind trip of two opening days with a stopover in Kraków. We flew Business Class from Chicago to Munich (releasing AppFuse 2.1 just before departing) and were very impressed with Kraków's Old Town upon arrival.

On that trip, I especially enjoyed meeting Trish's brother for the first time and watching the Red Sox win their first game of the year.

After returning, I realized I lost a lot of data in my Roller 5 upgrade and wrote about what happened and how I recovered it. Then we bid farewell to the 2010-2011 ski season and got ready for mountain bike season.

For next year, I think I'll keep my goal at 30 days. If everything works out as planned, we'll have a place in the mountains this fall and it'll be a bit easier to hit the slopes without sitting in traffic. For now, I'm pumped about the beginning of mountain bike season.

May arrived and Trish and I started looking for a place in the mountains. We found an very cool cabin, but later decided it was too nice and we'd rather have a condo. I finished The Basement Sauna Project and wrote about Java Web Application Security: Java EE 6, Spring Security and Apache Shiro.

On May 21st, we again visited Winter Park and Fraser to look at condos, then drove back to Denver for U2 at Mile High Stadium. I'm not a big fan of U2, but this was quite possibly the best concert I've ever been to. The experience of 70,000+ fans, all screaming, doing the wave, and loving life was epic. We decided that night that we liked one of the condos more than the rest and put in an offer the following week. The kids headed off to Florida for a month with their Mom and we drove to Utah for Desert Rocks and mountain biking in Moab.

I Upgraded to Roller 5.0 in June and showed how to add a Facebook Like button. Next, I transcribed a couple more Java Web Application Security articles: programmatic login APIs and penetrating with Zed Attack Proxy.

The kids returned to us just before the annual Father's Day Camping Trip. We drove to Lake Granby and stayed at Stillwater Campground for the weekend.

From my Father's Day post:

I really enjoyed spending so much time with my family this weekend. The good news is we get to do it again when I meet them at The Cabin in Montana next week. In the meantime, Trish and I are off to Jazoon for a few days of fun in Switzerland.

A few days later, we were in Zurich for the first time, enjoying our accommodations and marveling at Switzerland's excellent transportation system. I experienced an interesting situation while speaking, but quickly forgot when we visited Rhine Falls, celebrated our anniversary at Gruelich and played in the Swiss Alps.

We arrived back in Denver at 6pm on Sunday, June 26th and I flew to Montana at 8am the next morning. I worked from The Cabin that week and wrote a book review for Agile Hiring.

I think Sean [Landis] does a great job in showing how you can apply agile principles (people over process, tracking tools, failing fast, continuous improvement, constant feedback) to improve your hiring process. As a person that interviews often, I think it also gives great insights into how companies interview and what they're looking for. I've often thought that being honest about my skills and what I'm looking for is a good tactic and this book seems to confirm that.

That weekend, it was the "90s Reunion" for the Seeley Swan High School, which I attended my freshman and sophomore year of high school. There were several highlights of the trip, but one of my favorites was our float in the Swan Valley Parade. Abbie came up with the idea (Happy Birthday America) and we decorated an old trailer with streamers, balloons and a birthday cake to celebrate. Abbie made up a song and sang it throughout the entire parade (with a portable microphone and amp).

I went swimming with my iPhone 4 that Sunday and Trish and her Dad showed up a few hours later. On the 4th, we had a blast celebrating the holiday with mine and my parents' old friends. The next day, we adventured to Glacier National Park, where Trish shot an awesome sunrise at Lake McDonald. She also captured a beautiful picture of The Cabin at sunset that week.

In between vacationing in Montana and writing about it, I wrote some technical posts:

We returned to Denver in mid-July and shipped the kids off to Florida just before my annual Birthday Golf Debacle. We teed off at Wellshire Golf Course and had one of the most well-behaved rounds in a few years. The next weekend, we closed on our "Ski Shack" in Fraser and I penned Another Dream Realized: Mountain Views. Trish's October Sunset photo is one of my favorite glimpses of our view.

Next, I blogged about How do you get started in programming? and we headed to the airport a few hours later for a whirlwind trip from Oregon to Cape Cod. That was a wonderful trip and it took me almost a month to write about it. We flew into Portland and spent the weekend hiking in Silver Falls and drinking good beer at the Oregon Brewers Festival and Edgefield. We stayed with my good friends Clint and Autumn (married in Costa Rica in 2008), who had their first child only a few weeks after our visit (August 26th).

We had an awful early flight out of Portland to Boston that Sunday. On Monday afternoon, we picked up Abbie and Jack from the Boston airport. They flew up from West Palm Beach on their first unaccompanied minors flight. We drove them to Cape Cod and settled into Trish's parents house near the water. The rest of the week, we had a blast with our friends Chris and Julie and the entire McGinity clan.

We spent the rest of August going to concerts at Red Rocks, beer festivals in Winter Park, mountain biking, hiking, taking the kids to the Ski Shack and playing with Crazy Sagan. In my only August post, I talked a lot about the Broncos, Hawaii, Devoxx and leaving Overstock.

That pretty much catches you up-to-date with the goings on in my life. As you can tell, it's been a great summer so far. I'm really looking forward to the fall too. Broncos season, a trip to Hawaii, learning new technologies, talking about them at Devoxx and a new gig.
...
Tomorrow is Jack's birthday. I gotta run ... it's time to go buy a bike, play a little golf, hit the pool and go to the BBQ before tonight's CD Release Party.

We watched proudly as Jack turned 7 the next day and loved his excitement when he got his first mountain bike.

September started with mountain breezes and fishing at Lake Granby. Lots of friends, family and delicious food make for a heckuva 3-day weekend. Jack was the only one to catch a fish and you couldn't wipe the smile off his face for the rest of the day.

I described my Digital Diet and wrote my first article related to my Devoxx talk: Integrating Scalate and Jade with Play 1.2.3. The 2nd weekend in September, we enjoyed a September Sports Weekend, going to two Rockies games and the Monday night Broncos Opener before heading to Kauai. While the Broncos didn't win that game, Trish's shot of Brett, Joe and I sums up many-a-game this season.

As expected, Kauai blew our minds. We saw several spectacular sunsets, did some Stand Up Paddling (it's harder than it looks), played golf, took a doorless helicopter ride and kayaked along the Na Pali coast.

We never got bored of the view from our room.

Upon our return, I tried to make CoffeeScript work with Scalate and integrated HTML5 Boilerplate with Play.

October was a quiet month on this blog. I only wrote two articles, one about increasing my developer happiness and another about developing with HTML5, CoffeeScript and Twitter's Bootstrap. Watching the Broncos in October was painful at first, buy many rejoiced when Tebow started against Miami (10/23) and we had the amazing comeback to win. We went to the next game against the Lions and were quickly shown they had a long ways to go. We dressed up as the Scary Family for Halloween and had a ball.

I made up for October's lack of writing by writing 9 articles in November. I started with complaints about Play Scala's Anorm and PostgreSQL, then celebrated Abbie's 9th Birthday with a Magic Show.

The next week, I wrote about my new MacBook Pro and iMac and more Scalate goodness for Play. At this point, I was scrambling to finish up my Play More! application for Devoxx and worked a lot of late nights. That weekend, I used PhoneGap to turn my webapp into a native app and finished everything in the nick of time. We flew to Paris that Monday, took the train to Antwerp on Tuesday and settled in for a memorable week of interesting talks.

That Saturday, we took the train back to Paris and checked into our hotel for a few days of romance. I proposed to Trish at Versailles on Sunday at sunset and we rolled around in our joy for a few days before we told any of our friends.

We flew back to the US the Wednesday before Christmas and spent the weekend in Boston with Trish's family. For full details of that trip, see Our Engaging Trip to Paris and Antwerp.

To end the year, I kept it light and only spun up one blog post in December: What have I been working on at Taleo? A week after writing that post, we moved to the Ski Shack for 3 weeks and enjoyed a wonderful holiday break with many visitors.

Yes, 2011 was a spectacular year. We traveled to four different countries, did a lot of camping and mountain biking and accomplished 2/3 of my 5 year goals (basement sauna and mountain views). I wrote a lot of technical blog posts in the 2nd half of the year; most inspired my extracurricular activities. I plan to continue that this year as I learn more about HTML5, Play Framework, Scala and CoffeeScript. At the end of last year's Year in Review, I wrote:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus.

I think I can safely say I kept it rockin' with Trish - we're living together, engaged and still amazed we found one another. As for The Bus, I think MotorWorks Restorations and I are in a good position to make a final push and finish the sucker. My hope is to have it done by May so I can start driving it and taking it to VW Shows. In the meantime, Trish has started working on McGinity Photo full-time and loving it. We have trips planned to Stockholm (for Jfokus), Madrid (for Spring I/O) and Paris (for Devoxx Paris).

Last weekend, we witnessed one of the greatest playoff games ever when the Broncos beat the Steelers. I think the picture of Trish celebrating the final touchdown pretty much sums up 2011.

I look forward to all the skiing, celebrating, camping, traveling and bus rides in 2012. It's gonna be a fantastic year.

What have I been working on at Taleo?

2011-12-09T12:57:36-07:00

2011 has been a year of great clients for me. I started working with O.co and very much enjoyed my time there, especially on powder days in Utah. The people were great, the contract was great (no end date), but the work was not my forte. I was on a project to modularize the main shopping site's codebase, which involved mostly refactoring. By refactoring, I mean creating new Maven projects, modifying lots of pom.xml files and literally moving files from one directory to another. IntelliJ made this easy, the hard part was refactoring tests, moving from EasyMock to Mockito and splitting classes into interfaces and implementations where appropriate. As a developer who likes developing UIs and visually seeing my accomplishments, the project wasn't that exciting. However, I knew that it was strategically important to O.co, so I didn't complain much.

In mid-May, I received a LinkedIn message from the Director of Software Engineering at Taleo.

This is OB, I am the Director of Software Engineering at Taleo. We are the 2nd largest Software as a Service company. I am building a new specialist UI team that will take the product to the next level. I am looking for someone to lead this initiative. If you are interested to have a chat about it, please let me know.

At that time, I'd never heard of Taleo and quickly recommended they not hire me.

This probably isn't the best position for me. While I am a good leader, I'm not willing to relocate from Denver. I've found that leaders usually do best when face-to-face with their developers.

This conversation continued back-and-forth where I explained how I wasn't willing to go full-time and I didn't want to leave Overstock. In the end, OB was persistent and explained how the position would entail lots of UI work and wouldn't require me to travel much. Our negotiations trailed off in June and resumed in July after I returned from vacation in Montana. Shortly after, we met each other's expectations, agreed on a start date and I started working at Taleo in early September.

When I started, there were three features they wanted to add to to Taleo Business Edition: Profile Pictures, Talent Cards and Org Charts. They knew the schedule was tight (8 weeks), but I was confident I could make it happen. At first, I groaned at the fact that they were using Ant to build the project. Then I smiled when I learned they'd standardized on IntelliJ and set things up so you could do everything from the IDE. After using Maven for many years, this setup has actually become refreshing and I rarely have to restart or long for something like JRebel. Of course, a new kick-ass laptop and awesome IDE make it so I rarely wait for anything to happen.

To give you a taste of how I implemented each of these new features in 8 weeks, I've broken them into sections below.

Profile Pictures
Adding profile pictures was a pretty simple concept, one you see on my social networking sites today. I needed to give users the ability to upload a JPEG or PNG and crop it so it looked good. The uploading was a pretty straightforward process and I used a lot of internal APIs to grab the file from the request and save it to disk. The more difficult part was scaling the image to certain dimensions on upload (to save space) and allowing users to crop it after.

Most of Taleo Business Edition (TBE) is written in good ol' servlets and JSPs, with lots of scriptlets in their JSPs. When I saw the amount of HTML produced from Java, I laughed out loud and cringed. Soon after, I breathed a sigh of relief when I learned that any new features could be written using FreeMarker templates, which IntelliJ has excellent support for.

For image resizing on upload, I used Chris Campbell's Image.getScaledInstance() tutorial. For creating thumbnails, I used a combination of scaling, getSubimage() and the Java Image I/O API. I made sure to write to BufferedOutputStream for scalability. For cropping images client-side, I used jQuery UI's Dialog and Jcrop, the jQuery image cropping plugin. Below is a screenshot of what the cropping UI looks like:

Talent Cards
Talent Cards were a whole different beast. Not only did they need to display profile pictures, they also needed to contain contact information, work history and a number of other data points. Not just for employees, but for candidates as well. They also needed to be rendered with tabs at the bottom that allowed you to navigate between different data sections.

I'll admit, most of the hard work for this feature was done by the server-side developers (Harish and Vlad) on my team. Vlad built the tabbed interface and Harish built the administrative section that allows you to add/remove/sort fields, as well as show and hide certain tabs. I performed most of my magic with jQuery, its clueTip plugin and good ol' CSS. I was particularly thankful for CSS3 and its border-radius, box-shadow and Justin Maxwell's tutorial on CSS String Truncation with Ellipsis. I used DWR to fetch all the data from the server using Ajax.

Talent Cards are a slick feature in TBE 11.5 and I think they're a great way to see a lot of information about someone very quickly. If you enable them for your company, you'll be able to mouse over any employee or candidate's names and see their information.

Org Chart
The last feature I completed in this 8-week sprint was creating an organization chart. For this, I was given a rough prototype based on Caprica Software's JQuery/CSS Organisation Chart. When I received it, it had all kinds of cool CSS 3 transformations (like this one), but they only worked in Safari and Chrome. I ended up removing the transformations and adding the ability to navigate up and down the org tree with Ajax (we currently only show three levels at a time).

The Org Chart feature also allows you to see how many direct/indirect reports an employee has, as well as access their Talent Card by hovering over their name. It's one of my favorite features because it's so visual and because it builds upon all the other features we've built.

Summary
As you might've guessed by now, I've been having a lot of fun doing UI development over the last few months. While I seem to have a knack for backend Java development, I enjoy developing UIs a lot more. The smile you see on people's faces during demos is priceless. I can't help but think this kind of thing contributes greatly to my developer happiness. All these features will be in next week's release of TBE and I couldn't be happier.

If you'd like to work on my team at Taleo (or even take over my current role as UI Architect), please drop me a line. If you live near their headquarters (Dublin, CA), it'd also be great to see you at the next Silicon Valley Spring User Group meetup. I'll be speaking about What's New in Spring 3.1 on February 1st.

Deploying Java and Play Framework Apps to the Cloud with James Ward

2011-11-18T08:14:45-07:00

Yesterday, I attended James Ward's presentation on Deploying Java & Play Framework Apps to the Cloud at Devoxx. I arrived a bit late, but still managed to get there in time to see a lot of demos and learn more about Heroku. Below are my notes from James's talk.

When I arrived, James was doing a demo using Spring Roo. He was using Roo's Petclinic sample app and showed us how you could use Git to create a local repository of the new project and install Heroku's command line tool. From there, he ran the following command to create a new application on Heroku.

heroku create -s cedar

The Cedar Stack is what supports Java, Scala and Play Framework. It's the 3rd generation stack for Heroku. The command above created two endpoints, one for HTTP and one for Git. It picks from a list of randomly generated names, which all seem to have some humor in them. James ended up with "electric-sword-8877" for this demo.

From there, he ran git push heroku master to deploy the project to Heroku. Unfortunately, this resulted in a login error and there was an akward moment where we all thought the Demo Gods were angry. However, James was able to resolve this by using Heroku's sharing feature with the following command.

heroku sharing:add jw@heroku.com

For Java projects, Heroku looks for a pom.xml file in the root directory and runs a Maven build on project. All the dependencies get downloaded on the cloud rather than put them into a WAR and requiring you to upload a large WAR file. You don't have to upload your source code to Heroku; James did it for the sake of the demo because it was faster.

After the build finishes, it creates a slug file. This file contains everything Heroku needs to run your application.

Next, James showed a demo of the running application and added a new Pet through its UI. Then he scaled it to two servers using the following command:

heroku scale web=2

He proved this was working by running heroku ps, which showed there were two running processes. He showed the app again, but noted that the record he added was missing. This is because when it started up a new dyno, Hibernate created the schema again and deleted all records. To fix, James changed Hibernate to only update the schema instead of create a new one. If you're a Hibernate user, you know this is as simple as changing:

hibernate.hbm2ddl.auto=create

to:

hibernate.hbm2ddl.auto=update

After committing this change, James redeployed using Git.

git push heroku master

The slug file got built again and Heroku deployed the new slug onto both dynos, automatically load balancing the app across two servers. James then ran heroku logs to see the logs of his dynos and prove that a request to his app's HTTP endpoint made requests to both dynos. The logging is powered by Logplex and you can read about how it works in the article Heroku Gets Sweet Logging.

James mentioned that Roo has a Heroku plugin, but after watching his talk and searching a bit on the internet, it seems it's just the jetty-runner setup as described in Getting Started with Spring MVC Hibernate on Heroku/Cedar.

What about autoscaling? There are some 3rd party tools that do this. Heroku's Management infrastructure has APIs that these tools talk too. Heroku hasn't built autoscaling into the platform because they don't know where the bottlenecks are in your application.

Heroku = Polyglot + PaaS + Cloud Components. It supports Ruby, node.js, Java, Clojure, Play and Scala and they're working on native Grails and Gradle support. There's currently 534,374 apps running on Heroku.

Heroku is a cloud application platform and there's 5 different components.

Instant deployment
HTTP Routing / Load Balancing
Elastic Polyglot Runtime
Management & Logging
Component as a Service Ecosystem

For instant deployment, it's a pretty simple process:

You add files to a git repo
You provision the app on Heroku (heroku create)
You upload the files to Heroku (git push heroku master)
Heroku runs the build and assembles a "slug" file
Heroku starts a "dyno"
Heroku copies the "slug" to the "dyno"
Heroku starts the web application

Most apps will contain a Procfile that contains information about how to run the web process. For Spring Roo, it has:

web: java $JAVA_OPTS -jar target/dependency/jetty-runner.jar --port $PORT target/*.war

So how does Heroku decide what application server to use? It doesn't, you do. You need to get your application server into the slug file. The easiest way to do this is to specify your application server as a dependency in your pom.xml. In the Roo example, James uses the maven-dependency-plugin to get the jetty-runner dependency and copy it to the target directory. On Heroku, you bring your application server with you.

Heroku gives you 750 free dyno hours per app, per month. For developers, it's very easy to get started and use. Once you extend past one dyno, it's $.05 per dyno hour, which works out to around $30/month. It's only when you want to scale beyond one dyno where you get charged by Heroku, no matter how much data you transfer. Scalatest is running on Heroku. It has one dyno and is doing fine with that. Bill Venners doesn't have to pay anything for it.

java.herokuapp.com is a site James created that allows you to clone example apps and get started quickly with Heroku's Cedar Stack.

For HTTP Routing, Heroku uses an Erlang-based routing system to route all the HTTP requests across your dynos. Heroku doesn't support sticky sessions. Distributed session management does not work well, because it does not scale well. Heroku recommends you use a stateless web architecture or move your state into something like memcached. Jetty has (in the latest version) the ability to automatically serialize your session into a Mongo system. This works fine on Heroku. The problem with this is if you have 2 dynos running, each request can hit a different dyno and get different session state. Hence the recommendation for an external storage mechanism that can synchronize between dynos.

You can also run non-web applications on Heroku. You can have one web process, but as many non-web processes as you want.

Heroku has native support for the Play framework. To detect Play applications, it look for a conf/application.conf file. You don't need to have a Procfile in your root directory because Heroku knows how to start a Play application.

At this point, James created a new Play application, created a new Heroku app (he got "young-night-7104" this time) and pushed it to Heroku. He created a simple model object, a controller to allow adding new data and then wrote some jQuery to show new records via Ajax and JSON. He also showed how to configure the application to talk to Heroku's PostgreSQL database using the DATABASE_URL environment variable. He explained how you can use the heroku config command to see your environment variables.

The reason they use environment variables is so Heroku can update DATABASE_URL (and other variables) without having to call up all their customers and have them change them in their source code.

Play on Heroku supports Scala if you create your app with Scala. Play 2.0 uses Scala, Akka and SBT. Heroku added support for SBT a couple month ago, so everything will work just fine.

Heroku also supports Scala, detecting it by looking for the build.sbt file in the root directory. Heroku supports SBT 0.11.0 and it builds the 'stage' task. It currently does not support Lift because Lift uses an older version of SBT and because it's a very stateful framework that would require sticky sessions. Use Play, BlueEyes or Scalatra if you want Scala on Heroku.

Heroku has addons for adding functionality to your application, including Custom DNS, HTTPS, Amazon RDS, NoSQL and many more. They're also working on making their add-on and management APIs available via Java, so you'll (hopefully) be able to use them from your IDE in the future.

From there, James showed us how Heroku keeps slug files around so you can do rollbacks with heroku rollback. He also showed how you can use:

heroku run "your bash command"

to run any Bash command on the cloud.

Summary
I attended James's talk because he's a good friend, but also because I've been using Heroku to host my latest adventures with Play, Scala, CoffeeScript and Jade. I'm glad I attended because I learned some good tips and tricks and more about how Heroku works.

Heroku seems like a great development tool to me. In my experience, it's been really nice to have instant deployments using Git. In fact, I've created a 'push' alias so I can push to my project's repo and heroku at the same time.

alias push='git push origin master && git push heroku master'

I'd like to see more organizations embrace something like Heroku for developers. It'd be great if everyone had their own sandbox that business owners and product managers could see. I can't help but think this would be awesome for demos, prototyping, etc.

There were some other talks I wanted to attend at the same time, particularly Martin Odersky's What's in store for Scala? and WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks. The WWW talk has posted their presentation but I'm sure it'd be more fun to watch.

It's pretty awesome that all the talks from Devoxx 2011 will be up on Parleys.com soon.

Update: James has posted his slides from this talk.

PhoneGap for Hybrid App Development

2011-11-16T10:22:16-07:00

This afternoon, I attended Brian LeRoux's talk on PhoneGap for Hybrid App Development at Devoxx. You might remember that I tried PhoneGap last week and really enjoyed my experience. Below are my notes from Brian's talk.

PhoneGap is a project for creating native applications using HTML, CSS and JavaScript. PhoneGap started out as a hack. In 2007, Apple shipped the iPhone and Steve Jobs told everyone they should develop webapps. PhoneGap started in 2008 as a lofty summertime hack and gained traction as a concept at Nitobi with Android and Blackberry implementations in the fall. In 2009, people started to pay attention when PhoneGap got rejected by Apple. They added Symbian and webOS support and Sony Ericsson started contributing to the project. They got rejected because all PhoneGap-developed apps were named "PhoneGap". This turned out to be good press for the project and Apple let them in shortly after.

In 2010, IBM began tag-teaming with Nitobi and added 5 developers to the project after meeting them at OSCON. In 2011, RIM started contributing as well as Microsoft. Then Adobe bought the company, so they're obviously contributing.

PhoneGaps Goals: the web is a first class platform, so let people create installable web apps. Their second goal is to cease to exist and get browsers to adopt their model.

PhoneGap is NOT a runtime or a compiler/transpiler. It's not an IDE or predefined framework or proprietary lockin. It's Apache, MIT and BSD licensed to guarantee it's as free as free software gets. You can do whatever you want to do with it. PhoneGap has recently been contributed to the Apache Software Foundation.

As far as Adobe vs. PhoneGap is concerned, the Nitobi team remains contributors to PhoneGap. Adobe is a software tools company and has Apache and WebKit contributors. PhoneGap/Build integration will be added to Creative Cloud.

The biggest issues with contributing PhoneGap to Apache is renaming the project and source control. I'm not sure why it needs to be renamed, but it's likely that Apache Callback is out. There seems to be some consensus on Apache Cordova. Apache likes SVN and the PhoneGap community currently uses Git. They're trying to find a medium road there, but would prefer to stay on Git.

The PhoneGap technique is colloquially called "the bridge". It's a 3 step process: they instantiate a WebView, then they call JavaScript from native code, then they call native code from JavaScript. Apparently, all device APIs are available via JavaScript in a WebView.

The primary platforms supported are iOS >= 3, Android >= 1.5 and BlackBerry >= 5.x. They also support webOS, Symbian, Samsung Bada and Windows Phone. No mobile dev platform supports as many deploy targets as PhoneGap. Primary contributors are Adobe, IBM, RIM and Microsoft.

Documentation for PhoneGap is available at http://docs.phonegap.com. Device APIs for PhoneGap 1.0 included sensors, data and outputs, which all devices have. Examples of sensors are geolocation and camera. Data examples are the filesystem, contacts and media. Outputs are screens, speakers and the speaker jack. All PhoneGap APIs are plugins, but any native API is permitted.

What about security? Brian recommends looking at the HTML5 Security Cheatsheet. PhoneGap has added a lot of security measures since they've found the native API pretty much opens up everything.

PhoneGap doesn't bundle a UI framework, but they support any JavaScript framework that works in the browser. PhoneGap is just a fancy browser, so your code run in less fancy web browsers too. This means you can develop and test your app in your desktop browser and only use PhoneGap to package and distribute your app.

Competition? PhoneGap has no competition.

PhoneGap/Build is for compiling your apps in the cloud and free for open source projects. The biggest reason they did this is because they couldn't redistribute all the SDKs and it was a pain for developers to download and install SDKs in training classes.

For mobile app development, you should have a singular goal. Do one thing really well if you want to be successful. Great UX happens iteratively. You know that the web works and has been widely successfully cross-platform. It's likely you've already invested in the web. Start by building a mobile web client and use PhoneGap as a progressive enhancement technique.

Shipping and unit testing should be a daily activity. Automate everything so you can have one-click builds (test/dev/release). For web client design, constraints are your ally in the battle against complexity and "clients who are not chill". Phones suck and consume a lot: cpu, ram, bandwidth, battery, network... everything! Start with a benchmark of app performance and monitor that benchmark. If you have tons and tons of features, consider splitting into multiple apps.

The mobile web is not WebKit! Opera is huge, Firefox is making strides and IE still happens. For layouts: use flex-box rules (anyone got a link to these?), css media queries and meta tags for viewport. You should try to develop your app without frameworks because they come with a ton of code and can effect the size of your app.

Looks can kill: aesthetics that can hurt performance: border-radius, box-shadow and gradients can slow down your apps. Chances are, you really don't need these features. Design your app for your brand, not for the device manufacturer. An app that looks like an iPhone app on Android doesn't give a positive impression.

For JavaScript libraries, start with your problem, not a generic solution like Sencha or jQuery Mobile. Zepto and its older brother XUI are all you need to start. Jo is a fantastic option. Backbone and Spine are worth watching.

For testing, QUnit and Jasmine are pretty popular. For deployment, concat, minify and obfuscate your JavaScript and CSS. Or you can inline everything into the markup to minimize HTTP chatter. Gmail inlines and comments all their JavaScript and then evals it.

From there, Brian recommended leveraging HTML5's AppCache and and using RESTful JSON endpoints for legacy systems. Next, he tried to show us a demo of a photo sharing application. Unfortunately, the Demo Gods were grumpy and Brian couldn't get his computer to recognize his Android phone. He did show us the client code and it's pretty impressive you can use 1 line of code to take a picture on a phone.

The last thing we looked at was debug.phonegap.com. This is an app that's powered by weinre. It lets you enter a line of JavaScript in your client and then remotely debug it in a tool that looks like Chrome's Web Inspector. Very cool stuff if you ask me.

Summary
I really enjoyed learning more about PhoneGap, particularly because Brain emphasized all my web development skills can be used. I don't have to learn Objective-C or Android to develop native apps and I don't even have to install an SDK if I use PhoneGap/Build. Of course, my mobile developer friends might disagree with this approach. In the meantime, I look forward to using PhoneGap to turn my mobile web clients into native apps and finding out if it's really as good as they say it is.

Play 2.0, A web framework for a new era

2011-11-16T05:58:09-07:00

This week, I'm in Antwerp, Belgium for the annual Devoxx conference. After traveling 21 hours door-to-door yesterday, I woke up and came to the conference to attend some talks on Play and PhoneGap. I just got out of the session on Play 2.0, which was presented by Sadek Drobi and Guillaume Bort. Below are my notes from this presentation.

The Play 2.0 beta is out! You can read more about this release on the mailing list. This beta includes native support for both Scala and Java, meaning you can use both in the same project. The release also bundles Akka and SBT by default.

In other news, Play 2.0 is now part of the Typesafe Stack. Typesafe is the Scala company, started by the founder of Scala (Martin Odersky) and the founder of Akka (Jonas Bonér). Guillaume is also joining the Typesafe Advisory Board.

Sadek and Guillaume both work at zenexity, where Play is the secret weapon for the web applications they've built for the last decade. Play was born in the real world. They kept listening to the market to see what they should add to the project. At some point, they realized they couldn't keep adding to the old model and they needed to create something new.

The web has evolved from static pages to dynamic pages (ASP, PHP). From there, we moved to structured web applications with frameworks and MVC. Then the web moved to Ajax and long-polling to more real-time, live features. And this changes everything.

Now we need to adapt our tools. We need to handle tremendous flows of data. Need to improve expressiveness for concurrent code. We need to pick the appropriate datastore for the problem (not only SQL). We need to integrate with rapidly-evolving client side technologies like JavaScript, CoffeeScript, and Dart. We need to use elastic deployment that allows scaling up and scaling down.

zenexity wanted to integrated all of these modern web needs into Play 2.0. But they also wanted to keep Play approachable. They wanted to maintain fast turnaround so you can change your code and hit reload to see the changes. They wanted to keep it as a full stack framework with support for JSON, XML, Web Services, Jobs, etc. And they wanted to continue to use and conventions over configuration.

At this point, Guillaume did a Play 2.0 Beta demo, show us how it uses SBT and has a console so everything so it runs really fast. You can have both Scala and Java files in the same project. Play 2.0 templates are based on Scala, but you don't need to know Scala to use them. You might have to learn how to write a for loop in Scala, but it's just a subset of Scala for templates and views. SBT is used for the build system, but you don't have to learn or know SBT. All the old play commands still work, they're just powered by a different system.

After the demo, Sadek took over and started discussing the key features of Play 2.0.

To handle tremendous amounts of data, you need to do chunking of data and be able to process a stream of data, not just wait until it's finished. Java's InputStream is outdated and too low level. Its read() method reads the next byte of data from the input and this method can block until input data is available.

To solve this, Play includes a reactive programming feature, which they borrowed from Haskell. It uses Iteratee/Enumerator IO and leverages inversion of control (not like dependency injection, but more like not micro-managing). The feature allows you to have control when you need it so you don't have to wait for the input stream to complete. The Enumerator is the component that sends data and the Iteratee is the component that receives data. The Iteratee does incremental processing and can tell the Enumerator when it's done. The Iteratee can also send back a continuation, where it tells the Enumerator it wants more data and how to give it. With this paradigm, you can do a lot of cool stuff without consuming resources and blocking data flow.

Akka is an actor system that is a great model for doing concurrent code. An Actor could be both an Enumerator and an Iteratee. This vastly improves the expressiveness for concurrent code. For example, here's how you'd use Akka in Play:

def search(keyword: String) = Action {
  AsyncResult {
    // do something with result
  }
}

Play does not try to abstract data access because datastores are different now. You don't want to think of everything as objects if you're using something like MongoDB or navigating a Social Graph. Play 2.0 will provide some default modules for the different datastores, but they also expect a lot of contributed modules. Anorm will be the default SQL implementation for Play Scala and Ebean will be the default ORM implementation for Play Java. The reason they've moved away from Hibernate is because they needed something that was more stateless.

On the client side, there's so many technologies (LESS, CoffeeScript, DART, Backbone.js, jQuery, SASS), they didn't want to bundle any because they move too fast. Instead, there's plugins you can add that help you leverage these technologies. There's a lot of richness you can take advantage of on the client side and you need to have the tools for that.

Lastly, there's a new type of deployment: container-less deployment to the cloud. Akka allows you to distribute your jobs across many servers and Heroku is an implementation of elastic deployment that has built-in support for Play.

They've explained what they tried to design and the results of this new, clean architecture have been suprising. Side effects include: type-safety everywhere for rock-solid applications. There's an awesome performance boost from Scala. There's easier integration with existing projects via SBT. And it only takes 10 lines of code to develop an HTTP Server that responds to web requests.

The memory consumption is amazing: only 2MB of heap is used when a Play 2.0 app is started. Tests on Guillaume's laptop have shown that it can handle up to 40,000 requests per second, without any optimization of the JVM. Not only that, but after the requests subside, garbage collection cleans up everything and reduces the memory consumption back to 2MB.

At this point, Guillaume did another demo, showing how everything is type-safe in 2.0, including the routes file. If you mistype (or comment one out) any routes, the compiler will find it and notify you. Play 2.0 also contains a compiled assets feature. This allows you to use Google's Closure Compiler, CoffeeScript and LESS. If you put your LESS files in app/assets/stylesheets, compilation errors will show up in your browser. If you put JavaScript files in app/assets/javascripts, the Closure compiler will be used and compilation errors will show up in your browser.

Play 2.0 ships with 3 different sample applications, all implemented in both Java and Scala. HelloWorld is more than just text in a browser, it includes a form that shows how validation works. Another app is computer-database. When Guillaume started it, we saw how evolutions were used to create the database schema from the browser. The Play Team has done their best to make the development process a browser-based experience rather than having to look in your console. The computer-database is a nice example of how to do CRUD and leverages Twitter's Bootstrap for its look and feel.

The last sample application is zentasks. It uses Ajax and implements security so you can see how to create a login form. It uses LESS for CSS and CoffeeScript and contains features like in-place editing. If you'd like to see any of these applications in action, you can stop by the Typesafe booth this week at Devoxx.

Unfortunately, there will be no migrating path for Play 1.x applications. The API seems very similar, but there are subtle changes that make this difficult. The biggest thing is how templating has changed from Groovy to Scala. To migrate from 1.2.x would be mostly a copy/paste, modify process. There are folks working on getting Groovy templates working in 2.0. The good news is zenexity has hundreds of 1.x applications in production, so 1.x will likely be maintained for many years.

Summary
This was a great talk on what's new in Play 2.0. I especially like the native support for LESS and CoffeeScript and the emphasis on trying to keep developers using two tools: their editor and the browser. The sample apps look great, but the documentation look sparse. I doubt I'll get a chance to migrate my Play 1.2.3 app to 2.0 this month, but I hope to try migrating sometime before the end of the year.

Developing with HTML5, CoffeeScript and Twitter's Bootstrap

2011-10-20T14:47:36-06:00

This article is the fourth in a series about my adventures developing a Fitness Tracking application with HTML5, Play Scala, CoffeeScript and Jade. Previous articles can be found at:

Developing Features
After getting my desired infrastructure setup, I started coding like a madman. The first feature I needed was a stopwatch to track the duration of a workout, so I started writing one with CoffeeScript. After spending 20 minutes playing with dates and setTimeout, I searched and found a stopwatch jQuery plug-in. I added this to my app, deployed it to Heroku, brought up the app on my iPhone 3G, clicked Start and started riding my bike to work.

When I arrived, I unlocked my phone and discovered that the time had stopped. At first, I thought this was a major setback. My disappointed disappeared when I found a Super Neat JavaScript Stopwatch and Kåre Byberg's version that worked just fine. This stopwatch used setTimeout, so by keeping the start time, the app on the phone would catch up as soon as you unlocked it. I ported Kåre's script to CoffeeScript and rejoiced in my working stopwatch.

# Created by Kåre Byberg © 21.01.2005. Please acknowledge if used 
# on other domains than http://www.timpelen.com.
# Ported to CoffeeScript by Matt Raible. Also added hours support.
flagClock = 0
flagStop = 0
stopTime = 0
refresh = null
clock = null

start = (button, display) ->
  clock = display
  startDate = new Date()
  startTime = startDate.getTime()
  if flagClock == 0
    $(button).html("Stop")
    flagClock = 1
    counter startTime, display
  else
    $(button).html("Start")
    flagClock = 0
    flagStop = 1

counter = (startTime) ->
  currentTime = new Date()
  timeDiff = currentTime.getTime() - startTime
  timeDiff = timeDiff + stopTime  if flagStop == 1
  if flagClock == 1
    $(clock).val formatTime timeDiff, ""
    callback = -> counter startTime
    refresh = setTimeout callback, 10
  else
    window.clearTimeout refresh
    stopTime = timeDiff

formatTime = (rawTime, roundType) ->
  if roundType == "round"
    ds = Math.round(rawTime / 100) + ""
  else
    ds = Math.floor(rawTime / 100) + ""
  sec = Math.floor(rawTime / 1000)
  min = Math.floor(rawTime / 60000)
  hour = Math.floor(rawTime / 3600000)
  ds = ds.charAt(ds.length - 1)
  start() if hour >= 24
  sec = sec - 60 * min + ""
  sec = prependZeroCheck sec
  min = min - 60 * hour + ""
  min = prependZeroCheck min
  hour = prependZeroCheck hour
  hour + ":" + min + ":" + sec + "." + ds

prependZeroCheck = (time) ->
  time = time + "" # convert from int to string
  unless time.charAt(time.length - 2) == ""
    time = time.charAt(time.length - 2) + time.charAt(time.length - 1)
  else
    time = 0 + time.charAt(time.length - 1)

reset = ->
  flagStop = 0
  stopTime = 0
  window.clearTimeout refresh
  if flagClock == 1
    resetDate = new Date()
    resetTime = resetDate.getTime()
    counter resetTime
  else
    $(clock).val "00:00:00.0"

@StopWatch = {
  start: start
  reset: reset
}

The Scalate/Jade template to render this stopwatch looks as follows:

script(type="text/javascript" src={uri("/public/javascripts/stopwatch.coffee")})

#display
  input(id="clock" class="xlarge" type="text" value="00:00:00.0" readonly="readonly")
#controls
  button(id="start" type="button" class="btn primary") Start
  button(id="reset" type="button" class="btn :disabled") Reset

:plain
  <script type="text/coffeescript">
    $(document).ready ->
      $('#start').click ->
        StopWatch.start this, $('#clock')

      $('#reset').click ->
        StopWatch.reset()
  </script>

Next, I wanted to create a map that would show your location. For this, I used Merge Design's HTML 5 Geolocation Demo as a guide. The HTML5 Geo API is pretty simple, containing only three methods:

// Gets the users current position
navigator.geolocation.getCurrentPosition(successCallback,
                                         errorCallback,
                                         options);
// Request repeated updates of position
watchId = navigator.geolocation.watchPosition(successCallback, errorCallback);

// Cancel the updates
navigator.geolocation.clearWatch(watchId);

After rewriting the geolocation example in CoffeeScript, I ended up with the following code in my map.coffee script. You'll notice it uses Google Maps JavaScript API to show an actual map with a marker.

# Geolocation with HTML 5 and Google Maps API based on example from maxheapsize: 
# http://maxheapsize.com/2009/04/11/getting-the-browsers-geolocation-with-html-5/
# This script is by Merge Database and Design, http://merged.ca/ -- if you use some, 
# all, or any of this code, please offer a return link.

map = null
mapCenter = null
geocoder = null
latlng = null
timeoutId = null

initialize = ->
  if Modernizr.geolocation
    navigator.geolocation.getCurrentPosition showMap

showMap = (position) ->
  latitude = position.coords.latitude
  longitude = position.coords.longitude
  mapOptions = {
    zoom: 15,
    mapTypeId: google.maps.MapTypeId.ROADMAP
  }
  map = new google.maps.Map(document.getElementById("map"), mapOptions)
  latlng = new google.maps.LatLng(latitude, longitude)
  map.setCenter(latlng)

  geocoder = new google.maps.Geocoder()
  geocoder.geocode({'latLng': latlng}, addAddressToMap)

addAddressToMap = (results, status) ->
  if (status == google.maps.GeocoderStatus.OK) 
    if (results[1]) 
      marker = new google.maps.Marker({
          position: latlng,
          map: map
      })
      $('#location').html('Your location: ' + results[0].formatted_address)
  else 
    alert "Sorry, we were unable to geocode that address."

start = ->
  timeoutId = setTimeout initialize, 500

reset = ->
  if (timeoutId)
    clearTimeout timeoutId

@Map = {
  start: start
  reset: reset
}

The template to show the map is a mere 20 lines of Jade:

script(type="text/javascript" src="//www.google.com/jsapi")
script(type="text/javascript" src="//maps.googleapis.com/maps/api/js?sensor=false")

:css
  .demo-map {
    border: 1px solid silver;
    height: 200px;
    margin: 10px auto;
    width: 280px;
  }

#map(class="demo-map")

p(id="location")
  span(class="label success") New
  | Fetching your location with HTML 5 geolocation...

script(type="text/javascript" src={uri("/public/javascripts/map.coffee")})
:javascript
    Map.start();

The last two features I wanted were 1) distance traveled and 2) drawing the route taken on the map. For this I learned from A Simple Trip Meter using the Geolocation API. As I was beginning to port the JS to CoffeeScript, I thought, "there's got to be a better way." I searched and found Js2coffee to do most of the conversion for me. If you know JavaScript and you're learning CoffeeScript, this is an invaluable tool.

I tried out the trip meter that evening on a bike ride and noticed it said I'd traveled 3 miles when I'd really gone 6. I quickly figured out it was only calculating start point to end point and not taking into account all the turns in between. To view what was happening, I integrated my odometer.coffee with my map using Google Maps Polylines. Upon finishing the integration, I discovered two things, 1) HTML5 geolocation was highly inaccurate and 2) geolocation doesn't run in the background.

I was able to solve the first problem by passing in {enableHighAccuracy: true} to navigator.geolocation.watchPosition(). Below are two screenshots showing before high accuracy and after. Both screenshots are from the same two-block walk.

The second issue is a slight show-stopper. PhoneGap might be able to solve the problem, but I'm currently using a workaround → turning off auto-lock and keeping Safari in the foreground.

Making it look good
After I got all my desired features developed, I moved onto making the app look good. I started by using SASS for my CSS and installed Play's SASS module. I then switched to LESS when I discovered and added Twitter's Bootstrap to my project. At first I used Play's LESS module (version 0.3), but ran into compilation issues. I then tried Play's GreenScript module, but gave up on it when I found it was incompatible with the CoffeeScript module. Switching back to the LESS module and using the "0.3.compatibility" version solved all remaining issues.

You might remember that I integrated HTML5 Boilerplate and wondering why I have both Bootstrap and Boilerplate in my project. At this point, I don't think Boilerplate is needed, but I've kept it just in case it's doing something for HTML5 cross-browser compatibility. I've renamed its style.css to style.less and added the following so it has access to Bootstrap's variables.

/* Variables from Bootstrap */
@import "libs/variables.less";

Then I made my app look a lot better with layouts, stylish forms, a fixed topbar and alerts. For example, here's the CoffeeScript I wrote to display geolocation errors:

geolocationError = (error) ->
  msg = 'Unable to locate position. '
  switch error.code
    when error.TIMEOUT then msg += 'Timeout.'
    when error.POSITION_UNAVAILABLE then msg += 'Position unavailable.'
    when error.PERMISSION_DENIED then msg += 'Please turn on location services.'
    when error.UNKNOWN_ERROR then msg += error.code
  $('.alert-message').remove()
  alert = $('<div class="alert-message error fade in" data-alert="alert">')
  alert.html('<a class="close" href="#">×</a>' + msg);
  alert.insertBefore($('.span10'))

Then I set about styling up the app so it looked good on a smartphone with CSS3 Media Queries. Below is the LESS code I used to hide elements and squish the widths for smaller devices.

@media all and (max-device-width: 480px) {
  /* hide scrollbar on mobile */
  html { overflow-y:hidden }
  /* hide sidebar on mobile */
  .home .span4, .home .page-header, .topbar form {
    display: none
  }
  .home .container {
    width: 320px;
  } 
  .about {
    .container, .span10 {
      width: 280px;
    }
    .span10 {
      padding-top: 0px;
    }
  }

Tools
In the process of developing a stopwatch, odometer, displaying routes and making everything look good, I used a number of tools. I started out primarily with TextMate and its bundles for LESS, CoffeeScript and Jade. When I started writing more Scala, I installed the Scala TextMate Bundle. When I needed some debugging, I switched to IntelliJ and installed its Scala plugin. CoffeeScript, LESS and HAML plugins (for Jade) were already installed by default. I also used James Ward's Setup Play Framework with Scala in IntelliJ.

Issues
I think it's obvious that my biggest issue so far is the fact that a webapp can't multitask in the background like a native app can. Beyond that, there's accuracy issues with HTML5's geolocation that I haven't seen in native apps.

I also ran into a caching issue when calling getCurrentPosition(). It only worked the first time and I had to refresh my browser to get it to work again. Strangely enough, this only happened on my desktop (in Safari and Firefox) and worked fine on my iPhone. Unfortunately, it looks like PhoneGap has issues similar to this.

My workaround for no webapp multitasking is turning off auto-lock and leaving the browser in the foreground while I exercise. The downside to this is it really drains the battery quickly (~ 3 hours). I constantly have to charge my phone if I'm testing it throughout the day. The testing is a real pain too. I have to deploy to Heroku (which is easy enough), then go on a walk or bike ride. If something's broke, I have to return home, tweak some things, redeploy and go again. Also, there's been a few times where Safari crashes halfway through and I lose all the tracking data. This happens with native apps too, but seemingly not as often.

If you'd like to try the app on your mobile phone and see if you experience these issues, checkout play-more.com.

Summary
Going forward, there's still more HTML5 features I'd like to use. In particular, I'd like to play music while the fitness tracker is running. I'd love it if cloud music services (e.g. Pandora or Spotify) had an API I could use to play music in a webapp. Soundcloud might be an option, but I've also thought of just uploading some MP3s and playing them with the <audio> tag.

I've really enjoyed developing with all these technologies and haven't experienced much frustration so far. The majority has come from integrating Scalate into Play, but I've resolved most problems. Next, I'll talk about how I've improved Play's Scalate support and my experience working with Anorm.

Integrating HTML5 Boilerplate with Scalate and Play

2011-09-28T08:49:35-06:00

HTML5 Boilerplate is a project that provides a number of basic files to help you build an HTML5 application. At its core, it's an HTML template that puts CSS at the top, JavaScript at the bottom, installs Chrome Frame for IE6 users and leverages Modernizr for legacy browser support. It also includes jQuery with the download. One of the major benefits of HTML5 Boilerplate is it ships with a build system (powered by Ant) that concatenates and minimizes CSS and JS for maximum performance. From html5boilerplate.com:

Boilerplate is not a framework, nor does it prescribe any philosophy of development, it's just got some tricks to get your project off the ground quickly and right-footed.

I like the idea of its build system to minify and gzip, but I'd probably only use it if I was working on a project that uses Ant. Since I'm using it in a Play project, the whole Ant build system doesn't help me. Besides, I prefer something like wro4j. Wro4j allows you to specify a group of files and then it compiles, minimizes and gzips them all on-the-fly. As far as I know, Play doesn't have any support for Servlet Filters, so using wro4j in Play is not trivial.

The good news is Play has a GreenScript module that contains much of the wro4j functionality. However, since I'm using Scalate in my project, this goodness is unavailable to me. In the future, the Scalate Team is considering adding better wro4j, JavaScript and CSS integration. In the meantime, I'm going to pretend I don't care about concatenation and minimization and trundle along without this feature.

To add HTML5 Boilerplate to my Play project, I performed the following steps:

Downloaded the 2.0 Zipball.

Copied all the static files to my project. Below are the commands I used (where $boilerplate-download is the expanded download directory and ~/dev/play-more is my project):

cd $boilerplate-download
cp 404.html ~/dev/play-more/app/views/errors/404.html
cp *.png ~/dev/play-more/public/.
cp crossdomain.xml ~/dev/play-more/public/.
cp -r css ~/dev/play-more/public/stylesheets/.
cp favicon.ico ~/dev/play-more/public/.
cp humans.txt ~/dev/play-more/public/.
cp -r js/libs ~/dev/play-more/public/javascripts/.
cp robots.txt ~/dev/play-more/public/.

Copied the index.html to ~/dev/play-more/app/templates/layouts/default.jade and modified it to use Jade syntax. Since I downloaded the comments-heavy version, I modified many of them to be hidden in the final output.

-@ val body: String 
-@ var title: String = "Play More"
-@ var header: String = ""
-@ var footer: String = ""
!!! 5
/ paulirish.com/2008/conditional-stylesheets-vs-css-hacks-answer-neither/ 
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en"> <![endif]-->
-# Consider adding an manifest.appcache: h5bp.com/d/Offline 
<!--[if gt IE 8]><!--> <html class="no-js" lang="en"> <!--<![endif]-->
head
  meta(charset="utf-8")

  -# Use the .htaccess and remove these lines to avoid edge case issues. More info: h5bp.com/b/378 
  meta(http-equiv="X-UA-Compatible" content="IE=edge,chrome=1")

  title=title
  meta(name="description" content="")
  meta(name="author" content="Matt Raible ~ matt@raibledesigns.com")

  -# Mobile viewport optimized: j.mp/bplateviewport 
  meta(name="viewport" content="width=device-width,initial-scale=1")

  -# Place favicon.ico and apple-touch-icon.png in the root directory: mathiasbynens.be/notes/touch-icons

  -# CSS: implied media=all
  link(rel="stylesheet" href={uri("/public/stylesheets/style.css")})
  -# end CSS

  -# More ideas for your <head> here: h5bp.com/d/head-Tips 
  -#
    All JavaScript at the bottom, except for Modernizr / Respond.
    Modernizr enables HTML5 elements & feature detects; Respond is a polyfill for min/max-width CSS3 Media Queries
    For optimal performance, use a custom Modernizr build: www.modernizr.com/download/ 

  script(type="text/javascript" src={uri("/public/javascripts/libs/modernizr-2.0.6.min.js")})
body
  #container
    header = header
    #main(role="main")
      != body
    footer = footer

  -# JavaScript at the bottom for fast page loading 
  
  / Grab Google CDN's jQuery, with a protocol relative URL; fall back to local if offline 
  script(type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js")
  :javascript
    window.jQuery || document.write('<script src={uri("/public/javascripts/libs/jquery-1.6.2.min.js")}><\/script>')

  -# Change UA-XXXXX-X to be your site's ID 
  :javascript
    window._gaq = [['_setAccount','UA-25859875-1'],['_trackPageview'],['_trackPageLoadTime']];
    Modernizr.load({
      load: ('https:' == location.protocol ? '//ssl' : '//www') + '.google-analytics.com/ga.js'
    });

  -# Prompt IE 6 users to install Chrome Frame. Remove this if you want to support IE 6. 
  -# http://chromium.org/developers/how-tos/chrome-frame-getting-started 
  /[if lt IE 7]
    script(src="//ajax.googleapis.com/ajax/libs/chrome-frame/1.0.3/CFInstall.min.js")
    :javascript
      window.attachEvent('onload',function(){CFInstall.check({mode:'overlay'})})
        
!= "</html>"

Next, I had to add support for layouts to my homegrown Scalate support. I did this by specifying a layoutStrategy when initializing the TemplateEngine. From play-more/app/controllers/ScalateTemplate.scala:
```
engine.classLoader = Play.classloader
engine.layoutStrategy = new DefaultLayoutStrategy(engine, 
  Play.getFile("/app/templates/layouts/default" + scalateType).getAbsolutePath)
engine
```

That's it! Now I have HTML5 Boilerplate integrated into my Play/Scalate/Jade application. To set the title and header in my index.jade, I simply added the following lines at the top:

- attributes("title") = "Counting"
- attributes("header") = "HTML5 Rocks!"

CoffeeScript Tip
Yesterday, I mentioned that I was having issues getting CoffeeScript to work with Scalate and that I was going to try and get the in-browser compiler working. First of all, reverting to Scalate 1.4.1 didn't work because there is no CoffeeScript support in 1.4.1. So I stayed with 1.5.2 and used PandaWood's Running CoffeeScript In-Browser Tutorial. I copied coffee-script.js to ~/dev/play-more/public/javascripts/libs and added a reference to it in my default.jade layout:

-# JavaScript at the bottom for fast page loading 
script(type="text/javascript" src={uri("/public/javascripts/libs/coffee-script.js")})

Then I was able to write CoffeeScript in a .jade template using the following syntax:

:plain
  <script type="text/coffeescript">
    alert "hello world"
  </script>

Summary
If you've integrated HTML5 Boilerplate into your Play application, I'd love to hear about it. Now that I have all the infrastructure in place (Jade, CoffeeScript, HTML5 Boilerplate), I'm looking forward to getting some development done. Who knows, maybe I'll even come up with my own Play Un-Features That Really Irk My Inner Geek.

Integrating OAuth with AppFuse and its REST API

2011-07-05T10:56:48-06:00

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
Integrate Spring Security and OAuth
Authenticate and Retrieve Data with Client

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Java Web Application Security - Part IV: Programmatic Login APIs

2011-06-06T21:44:09-06:00

Over the last month, I've posted a number of articles on implementing authentication with Java EE 6, Spring Security and Apache Shiro. One of the things I demonstrated in my live demos (at Utah's JUG Meetings) was programmatic authentication. I left this out of my screencasts and previous tutorials because I thought it'd fit better in a comparison article.

In this article, I'd like to show you how you can programmatically login to an application using the aforementioned security frameworks. To do this, I'll be using my ajax-login application that I wrote for Implementing Ajax Authentication using jQuery, Spring Security and HTTPS.

To begin, I implemented a LoginController as a Spring MVC Controller that returns JSON.

package org.appfuse.examples.webapp.security;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginController {

    @Autowired
    LoginService loginService;

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public LoginStatus getStatus() {
        return loginService.getStatus();
    }

    @RequestMapping(method = RequestMethod.POST)
    @ResponseBody
    public LoginStatus login(@RequestParam("j_username") String username,
                             @RequestParam("j_password") String password) {

        return loginService.login(username, password);
    }
}

This controller delegates its logic to a LoginService interface.

package org.appfuse.examples.webapp.security;

public interface LoginService {

  LoginStatus getStatus();

  LoginStatus login(String username, String password);
}

The Client
The client for this controller is the same as mentioned in my previous article, but I'll post it again for your convenience. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginController.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('Login failed, please try again.');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "${ctx}/api/login.json",
        type: "POST",
        beforeSend: function(xhr) {
            xhr.withCredentials = true;
        },
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href = getHost() + '${ctx}/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

The biggest secret to making this all work (the HTTP -> HTTPS communication, which is considered cross-domain), is the window.name Transport and the jQuery plugin that implements it. To make this plugin work with Firefox 3.6, I had to implement a Filter that adds Access-Control headers.

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "http://" + req.getServerName());
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");
        response.setHeader("Access-Control-Allow-Credentials", "true");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Java EE 6 LoginService
Java EE 6 has a few new methods in HttpServletRequest:

authenticate(response)
login(user, pass)
logout()

In this example, I'll use the new login(username, password) method. The hardest part about getting this working was finding the right Maven dependency. At first, I tried the one that seemed to make the most sense:

<dependency>
    <groupId>javax</groupId>
    <artifactId>javaee-web-api</artifactId>
    <version>6.0</version>
</dependency>

Unfortunately, this resulted in a strange error that means the dependency has the interfaces, but not the implementation classes. I ended up using GlassFish's dependency instead (thanks to Stack Overflow for the tip).

<dependency>
    <groupId>org.glassfish</groupId>
    <artifactId>javax.servlet</artifactId>
    <version>3.0</version>
    <scope>provided</scope>
</dependency>

Since Servlet 3.0 doesn't appear to be in Maven Central, I had to add the GlassFish Repository to my pom.xml's <repositories> element.

<repository>
    <id>glassfish-repo</id>
    <url>http://download.java.net/maven/glassfish</url>
</repository>

After that, it was easy to implement the LoginService interface with a JavaEELoginService class:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;

@Service("javaeeLoginService")
public class JavaEELoginService implements LoginService {
    private Log log = LogFactory.getLog(JavaEELoginService.class);

    @Autowired
    HttpServletRequest request;

    public LoginStatus getStatus() {
        if (request.getRemoteUser() != null) {
            return new LoginStatus(true, request.getRemoteUser());
        } else {
            return new LoginStatus(false, null);
        }
    }

    @Override
    public LoginStatus login(String username, String password) {
        try {
            if (request.getRemoteUser() == null) {
                request.login(username, password);
                log.debug("Login succeeded!");
            }
            return new LoginStatus(true, request.getRemoteUser());
        } catch (ServletException e) {
            e.printStackTrace();
            return new LoginStatus(false, null);
        }
    }
}

I tried to use this with "mvn jetty:run" (with version 8.0.0.M2 of the jetty-maven-plugin), but I got the following error:

javax.servlet.ServletException
        at org.eclipse.jetty.server.Request.login(Request.java:1927)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.beans.factory.support.AutowireUtils$ObjectFactoryDelegatingInvocationHandler.invoke(AutowireUtils.java:178)
        at $Proxy52.login(Unknown Source)
        at org.appfuse.examples.webapp.security.JavaEELoginService.login(JavaEELoginService.java:30)

This lead me to believe that Servlet 3 is not quite implemented, so I tried it with Tomcat 7.0.8. To support SSL and container-managed authentication, I had to create a certificate keystore and uncomment the SSL Connector in $CATALINA_HOME/conf/server.xml. I also had to add an "admin" user with roles="ROLE_ADMIN" to $CATALINA_HOME/conf/tomcat-users.xml.

<user username="admin" password="admin" roles="ROLE_ADMIN"/>

With Tomcat 7, I was able to login successfully, proven by the following logging.

DEBUG - JavaEELoginService.login(31) | Login succeeded!

However, in the UI, I still got a "Login failed, please try again." message. Recalling that I had some issues with ports previous, I configured Apache to proxy the default http/https ports to 8080/8443 and tried again. This time it worked!

Spring Security LoginService
Spring Security offers a programmatic API and I was able to implement its LoginService as follows:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Service;

@Service("springLoginService")
public class SpringSecurityLoginService implements LoginService {
    private Log log = LogFactory.getLog(SpringSecurityLoginService.class);

    @Autowired(required = false)
    @Qualifier("authenticationManager")
    AuthenticationManager authenticationManager;

    public LoginStatus getStatus() {
        Authentication auth = SecurityContextHolder.getContext().getAuthentication();
        if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
            return new LoginStatus(true, auth.getName());
        } else {
            return new LoginStatus(false, null);
        }
    }

    public LoginStatus login(String username, String password) {
        UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
        User details = new User(username);
        token.setDetails(details);

        try {
            Authentication auth = authenticationManager.authenticate(token);
            log.debug("Login succeeded!");
            SecurityContextHolder.getContext().setAuthentication(auth);
            return new LoginStatus(auth.isAuthenticated(), auth.getName());
        } catch (BadCredentialsException e) {
            return new LoginStatus(false, null);
        }
    }
}

I then modified the LoginService dependency in LoginController so this implementation would be used.

@Autowired
@Qualifier("springLoginService")
LoginService loginService;

Since Spring's API doesn't depend on Servlet 3, I tried it in Jetty using "mvn jetty:run". Of course, I modified my web.xml accordingly for Spring Security before doing so. Interestingly enough, I found that the my SpringSecurityLoginService seemed to work:

DEBUG - SpringSecurityLoginService.login(39) | Login succeeded!

But in the UI, the login failed with a "Login failed, please try again." message. Using the standard ports with Apache in front of Jetty solved this issue.

Apache Shiro LoginService
Apache Shiro is nice enough to offer a programmatic API as well. I was able to implement a ShiroLoginService as follows:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.shiro.SecurityUtils;
import org.apache.shiro.authc.AuthenticationException;
import org.apache.shiro.authc.UsernamePasswordToken;
import org.apache.shiro.subject.Subject;
import org.springframework.stereotype.Service;

@Service("shiroLoginService")
public class ShiroLoginService implements LoginService {
    private Log log = LogFactory.getLog(ShiroLoginService.class);

    public LoginStatus getStatus() {
        Subject currentUser = SecurityUtils.getSubject();
        if (currentUser.isAuthenticated()) {
            return new LoginStatus(true, currentUser.getPrincipal().toString());
        } else {
            return new LoginStatus(false, null);
        }
    }

    public LoginStatus login(String username, String password) {
        if (!getStatus().isLoggedIn()) {
            UsernamePasswordToken token = new UsernamePasswordToken(username, password);
            Subject currentUser = SecurityUtils.getSubject();
            try {
                currentUser.login(token);
                log.debug("Login succeeded!");
                return new LoginStatus(currentUser.isAuthenticated(),
                        currentUser.getPrincipal().toString());
            } catch (AuthenticationException e) {
                return new LoginStatus(false, null);
            }

        } else {
            return getStatus();
        }
    }
}

Then I modified the LoginService dependency in LoginController so this implementation would be used.

@Autowired
@Qualifier("shiroLoginService")
LoginService loginService;

Next, I modified my web.xml for Apache Shiro and tried "mvn jetty:run". Again, the login appeared to succeed (based on log messages) on the server, but failed in the UI. When using http://localhost instead of http://localhost:8080, everything worked fine.

Summary
This article has shown you how you can programmatically login using Java EE 6, Spring Security and Apache Shiro. Before Java EE 6 (and Servlet 3), there was no API to programmatically login, so this is a welcome addition. The fact that my Ajax login example didn't work when ports differed is because of browsers' same origin policy, which specifies the ports have to be the same. Specifying no ports (the defaults), seems to be the loophole.

On a related note, I've discovered some interesting articles recently from the AppSec Blog.

The 2nd article has an interesting paragraph:

... there's Apache Shiro (FKA JSecurity and then later as Apache Ki), another secure framework for Java apps. Although it looks simpler to use and understand than ESAPI and covers most of the main security bases (authentication, authorization, session management and encryption), it doesn't help take care of important functions like input validation and output encoding. And Spring users have Spring Security (Acegi) a comprehensive, but heavyweight authorization and authentication framework.

So according to this blog, the security frameworks discussed here aren't the best.

The most comprehensive, up-to-date choice for Java developers is OWASP's ESAPI Enterprise Security API especially now that the 2.0 release has just come out.

I haven't heard of many organizations adopting ESAPI over Java EE 6, Spring Security or Apache Shiro, but maybe I'm wrong. Is ESAPI something that's being used out there by companies?

Java Web Application Security - Part III: Apache Shiro Login Demo

2011-05-26T16:43:22-06:00

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Store Users in a Database
Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

Java Web Application Security - Part II: Spring Security Login Demo

2011-05-13T09:20:51-06:00

Last week, I wrote a tutorial on how to implement Security in Java EE 6. This week, I'd like to show you how to implement the same features using Spring Security. Before I begin, I'd like to explain my reason for writing this article.

Last month, I presented a talk on Java Web Application Security at the Utah JUG (UJUG). As part of that presentation, I did a number of demos about how to implement security with Java EE 6, Spring Security and Apache Shiro. I told the audience that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense.

Today, I've finished the second screencast showing how to implement security with Spring Security. Below is the presentation (with the screencast embedded on slide 16) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Add Remember Me
Store Users in a Database
Summary

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a Spring context file that contains Spring Security's configuration. Create src/main/webapp/WEB-INF/security.xml and populate it with the contents below:

  <?xml version="1.0" encoding="UTF-8"?>
  <beans:beans xmlns="http://www.springframework.org/schema/security"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:beans="http://www.springframework.org/schema/beans"
               xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">

      <!-- New in Spring Security 3.1 -->
      <!-- <http pattern="/css/**" security="none"/> -->

      <http auto-config="true">
          <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN"/>
          <http-basic/>
      </http>

      <authentication-manager alias="authenticationManager">
          <authentication-provider>
              <password-encoder hash="sha"/>
              <user-service>
                  <user name="user" password="12dea96fec20593566ab75692c9949596833adc9" authorities="ROLE_USER"/>
                  <user name="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" authorities="ROLE_ADMIN"/>
              </user-service>
          </authentication-provider>
      </authentication-manager>

      <!-- Override userSecurityAdvice bean in appfuse-service to allow any role to update a user. -->
      <beans:bean id="userSecurityAdvice" class="org.appfuse.examples.webapp.security.UserSecurityAdvice"/>
  </beans:beans>

The last bean, userSecurityAdvice, is an aspect that's needed to override some behavior in AppFuse. You won't need this normally when implementing Spring Security.

Next, open src/main/webapp/WEB-INF/web.xml and add Spring's DelegatingFilterProxy:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

You don't need to add any dependencies in your pom.xml is because this project depends on AppFuse, which already contains these dependencies.

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Spring Security is a bit easier to configure than Java EE 6 out-of-the-box, mostly because it doesn't require you to configure your container.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Spring Security has a way to configure "logout" to match a URL and get rid of a class like LogoutController. Since it was already in the project, I don't cover that in this tutorial.

Force SSL
Spring Security allows you to switch between secure (https) and non-secure (http) protocols using a simple requires-channel attribute on the <intercept-url> element. Possible values are "http", "https" and "any". Add requires-channel="https" to your security.xml file:

<intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

You can switch between http and https protocols. With Java EE, you can only force https. You have to write a custom filter to switch back to http.
Redirecting to https actually works. With Java EE (on Jetty at least), a 403 is returned instead of redirecting the request.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a <form-login> element in security.xml's <http> element:

<http auto-config="true">
    <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>
    <form-login login-page="/login" authentication-failure-url="/login?error=true"
                login-processing-url="/j_security_check"/>
    <http-basic/>
</http>

You can leave the <http-basic> element since Spring Security is smart enough to serve up the form for browsers and use Basic Authentication for clients such as web services. The login.jsp page (that /login forwards to) already exists in the project, in the src/main/webapp directory. The forwarding is done by the UrlRewriteFilter with the following configuration in src/main/webapp/WEB-INF/urlrewrite.xml.

<rule>
    <from>/login</from>
    <to>/login.jsp</to>
</rule>

This JSP has 3 important elements: 1) a form that submits to "/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Add Remember Me
Remember Me is a feature you see in many web applications today. It's usually a checkbox on the login form that allows you to auto-login the next time you visit a site. This feature doesn't exist in Java EE security, but it does exist in Spring Security. To enable it, add the following just below <form-login> in security.xml:

<remember-me user-service-ref="userDao" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"/>

Next, open src/main/webapp/login.jsp and change the name of the "remember me" checkbox to be _spring_security_remember_me:

<input type="checkbox" name="_spring_security_remember_me" id="rememberMe"/>

After making these changes, you should be able to restart Jetty, go to http://localhost:8080/users, enter admin/adminjdbc, check the Remember Me checkbox and login. Then close your browser, and repeat the process. This time, you won't be prompted to login. For more information on this feature, see Spring Security's Remember Me documentation.

While storing usernames and passwords in a file is convenient for demos, it's not very real-world-ish. The next section shows you how to configure Spring Security to use a database for its user store.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a user-service-ref attribute to the <authentication-provider> element. You can also delete the <user-service> element.

<authentication-manager alias="authenticationManager">
    <authentication-provider user-service-ref="userDao">
        <password-encoder hash="sha"/>
    </authentication-provider>
</authentication-manager>

The "userDao" bean is provided by AppFuse and its UserDaoHibernate.java class. This class implements Spring Security's UserDetailsService interface. With Java EE, I had to configure a database connection and make sure the JDBC Driver was in my container's classpath. With Spring Security, you can talk to the database you already have configured in your application.

Of course, you could do this with Java EE too. One thing I neglected to show in my last tutorial was that 1) the app uses H2 and 2) I had to configure Java EE's database to be MySQL. This was because when I tried to access my H2 instance, I got an error about two threads trying to access it at once.

2011-05-13 08:47:29.081:WARN::UserRealm Java EE Login could not connect to database; will try later
org.h2.jdbc.JdbcSQLException: Database may be already in use: "Locked by another process". 
        Possible solutions: close all other connection(s); use the server mode [90020-154]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:167)
	at org.h2.message.DbException.get(DbException.java:144)
	at org.h2.store.FileLock.getExceptionAlreadyInUse(FileLock.java:443)
	at org.h2.store.FileLock.lockFile(FileLock.java:338)
	at org.h2.store.FileLock.lock(FileLock.java:134)
	at org.h2.engine.Database.open(Database.java:535)
	at org.h2.engine.Database.openDatabase(Database.java:218)

The password for the "admin" user is configured in src/test/resources/sample-data.xml and it's loaded by DbUnit before the application starts. You can view your pom.xml and the dbunit-maven-plugin's configuration if you're interested in learning how this is done. The password is currently configured to "adminjdbc", but you can reset it by generating a new password and modifying sample-data.xml.

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Spring Security 3.0.5. In addition to the basic XML configuration, Spring Security also provides a AOP support and annotations you can use to secure methods. It also has many more features than standard Java EE Security. In my opinion, it's the most mature security framework we have in Java today. Currently, I think its reference documentation is the best place to learn more.

There are a few limitations I found with Spring Security:

The authentication mechanism (file, database, ldap, etc.) is contained in the WAR
Securing methods only works on Spring beans
Remember Me doesn't work in my screencast (because I forgot to rename the checkbox in login.jsp)

Of course, you can configure Spring to load its configuration from outside the WAR (e.g. a file or JNDI), but it's not as easy as including the configuration in your app.

In the next couple weeks, I'll post Part III of this series, where I'll show you how to implement this same set of features using Apache Shiro. In the meantime, please let me know if you have any questions.

I created the screencasts with Camtasia. For small screens, and embedding in the presentation, I created it at 50% and used the SmartFocus feature to zoom in and out during the demo. For larger screens, I published another screencast at 100%, in HD. If you have a preference for which screencast is better, I'd love to hear about it.

Java Web Application Security - Part I: Java EE 6 Login Demo

2011-05-05T16:58:00-06:00

Back in February, I wrote about my upcoming conferences:

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks:

Implement Basic Authentication
Implement Form-based Authentication
Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs)
Force SSL for certain URLs
Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone)
Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone)
Encrypt Passwords
Secure methods with annotations

For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities.

I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml:

<security-constraint>
    <web-resource-collection>
        <web-resource-name>users</web-resource-name>
        <url-pattern>/users</url-pattern>
        <http-method>GET</http-method>
        <http-method>POST</http-method>
    </web-resource-collection>
    <auth-constraint>
        <role-name>ROLE_ADMIN</role-name>
    </auth-constraint>
</security-constraint>

<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Java EE Login</realm-name>
</login-config>

<security-role>
    <role-name>ROLE_ADMIN</role-name>
</security-role>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after </webAppConfig> in the Jetty plugin's configuration.

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.HashLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/realm.properties</config>
    </loginService>
</loginServices>

The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath());
}

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to replace the <login-config> in your web.xml with the following:

<login-config>
    <auth-method>FORM</auth-method>
    <form-login-config>
        <form-login-page>/login.jsp</form-login-page>
        <form-error-page>/login.jsp?error=true</form-error-page>
    </form-login-config>
</login-config>

The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Force SSL
Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same <security-constraint> you already have in web.xml, add the following after </auth-constraint>:

<user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>

To configure Jetty to listen on an SSL port, add the following just after </loginServices> in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you.

Store Users in a Database
Finally, to store your users in a database instead of file, you'll need to change the <loginService> in the Jetty plugin's configuration. Replace the existing <loginService> element with the following:

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.JDBCLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/jdbc-realm.properties</config>
    </loginService>
</loginServices>

The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information.

jdbcdriver = com.mysql.jdbc.Driver
url = jdbc:mysql://localhost/appfuse
username = root
password =
usertable = app_user
usertablekey = id
usertableuserfield = username
usertablepasswordfield = password
roletable = role
roletablekey = id
roletablerolefield = name
userroletable = user_role
userroletableuserkey = user_id
userroletablerolekey = role_id
cachetime = 300

Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands:

mysql -u root -p -e 'create database appfuse'
curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql
mysql -u root -p appfuse < create-appfuse.sql

Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the <configuration> element (before <executions>) in pom.xml:

<dependencies>
    <!-- MySQL for JDBC Realm -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.14</version>
    </dependency>
</dependencies>

Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example:

$ sh jetty-password.sh javaeelogin
javaeelogin
OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur
MD5:53b176e6ce1b5183bc970ef1ebaffd44

The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement.

UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin';

Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0:

authenticate(response)
login(user, pass)
logout()

This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely:

No error messages for failed logins
No Remember Me
No auto-redirect from HTTP to HTTPS
Container has to be configured
Doesn’t support regular expressions for URLs

Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain.

In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions.

Upgraded to Apache Roller 5.0, RC4

2011-03-03T11:39:37-07:00

Last Sunday, Dave Johnson released Apache Roller 5.0, RC4. Since I'm an Apache Roller committer, and I've been seeing issues with comments throwing exceptions on this site, I decided to upgrade. In doing so, I discovered a number of issues. Hopefully by documenting them here, you'll be able to upgrade from Roller 4 to Roller 5 without any issues.

To upgrade, I took a snapshot of my existing site and database and copied them locally. After getting everything setup locally (importing existing database and upgrading it), I started Tomcat and began solving problems.

Database settings - JNDI vs. Properties
With Roller 4, I configured by database settings in a ROOT.xml so they'd be read using JNDI. With Roller 5, I got the following error when I tried to do this.

<openjpa-2.0.1-r422266:989424 fatal user error> org.apache.openjpa.persistence.ArgumentException:
A JDBC Driver or DataSource class name must be specified in the ConnectionDriverName property.
       at org.apache.openjpa.jdbc.schema.DataSourceFactory.newDataSource(DataSourceFactory.java:76)

I was able to get around this issue by adding the following to my roller-custom.properties:

database.configurationType=jdbc
database.jdbc.driverClass=com.mysql.jdbc.Driver
database.jdbc.connectionURL=jdbc:mysql://localhost/rollerdb
database.jdbc.username=root
database.jdbc.password=
mail.configurationType=properties
mail.hostname=localhost

After making this change, I received an error when Planet tried to startup:

ERROR 2011-03-02 09:56:08,502 DatabaseProvider:errorMessage - ERROR: unable to obtain database connection. 
Likely problem: bad connection parameters or database unavailable.
FATAL 2011-03-02 09:56:08,502 RollerContext:contextInitialized - Roller Planet startup failed during app preparation
org.apache.roller.planet.business.startup.StartupException: ERROR: unable to obtain database connection. 
Likely problem: bad connection parameters or database unavailable.

I don't remember why I enabled planet, but turning it off in roller-custom.properties seemed to solve the problem.

planet.aggregator.enabled=false

Password Encyrption
The next thing I tried to do was login. When this didn't work, I figured it must be related to password encryption. With Roller 4, I had to have "passwds.encryption.enabled=true" in roller-custom.properties. In Roller 5, I also had to add the encryption algorithm.

passwds.encryption.algorithm=SHA

GZip Compression
In November 2009, I optimized this site and used Roller's CompressionFilter and wro4j to gzip and concatenate JavaScript and CSS. With Roller 4, I used the CompressionFilter to compress *.css and *.js instead of using Wro4J's built-in gzip compression. The Roller 5 CompressionFilter seems to have issues with wro4j, so I had to disable it for *.css and *.js and use wro4j instead.

At this point, I figured I was good to go, so I zipped up my local WAR and scp'ed it to raibledesigns.com. I stopped Tomcat and attempted to upgrade my production MySQL database (version 3.23.56). Below is the error I received.

$ mysql -u raible -p raible < 400-to-500-migration.sql
Enter password:
ERROR 1064 (00000) at line 42: You have an error in your SQL syntax near 'as w set
   lastmodified = lastmodified,
   datecreated = datecreated,
   cr' at line 1

At this point, I figured my database might be slightly hosed, but since it was simply creating tables, I was probably OK. I restarted Tomcat and left the old version in place while I waited for a MySQL 5 database instance from my hosting provider, KGB Internet. Once I got the new instance, I imported my backed-up database, ran the upgrade script and everything worked just peachy.

I generally upgrade Roller by coping the new codebase over my old one. This is because I have a lot of symlinks and other files in my "ROOT" directory and like to keep those. In doing this, I found I had to do a couple things after copying everything over:

Delete WEB-INF/lib and recopy from RC4's WEB-INF/lib.
Delete WEB-INF/classes and recopy from RC4's WEB-INF/classes.

I then experienced some issues with JARs not being present for Roller's JSPWikiPlugin. I enabled this long ago, but don't use it anymore. However, to keep old posts still working, I wanted to enable it. The downloads for the plugin seem to be gone, but luckily I found a copy and put all the JARs into my WEB-INF/lib directory.

After starting Tomcat and browsing around a bit, I discovered two more issues:

Search doesn't seem to work. For example, there are no results for jQuery.
My Archives page's calendar didn't work. It showed the following:
```
$calendarModel.showWeblogEntryCalendarBig($weblog, $cat)
```

I was able to fix issue #2 by changing #showBigWeblogCalendar() to the following.

#showWeblogEntryCalendarBig($model.weblog "nil")

The first issue with search seems to remain.

If you notice any other issues on this site, please let me know. I'll try to get them fixed asap.

Update: I entered an issue for my search problem in Roller's JIRA. I also managed to figure out that the problem is due to the old version of oscache that's needed by the JSPWiki plugin. Hopefully we can get the plugin upgraded to avoid this issue for other users.

Implementing Ajax Authentication using jQuery, Spring Security and HTTPS

2011-02-23T16:55:55-07:00

I've always had a keen interest in implementing security in webapps. I implemented container-managed authentication (CMA) in AppFuse in 2002, watched Tomcat improve it's implementation in 2003 and implemented Remember Me with CMA in 2004. In 2005, I switched from CMA to Acegi Security (now Spring Security) and never looked back. I've been very happy with Spring Security over the years, but also hope to learn more about Apache Shiro and implementing OAuth to protect JavaScript APIs in the near future.

I was recently re-inspired to learn more about security when working on a new feature at Overstock.com. The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience. User experience aside, I think it's interesting to see what's required to implement such a feature.

To demonstrate how we did it, I whipped up an example using AppFuse Light, jQuery and Spring Security. The source is available in my ajax-login project on GitHub. To begin, I wanted to accomplish a number of things to replicate the Overstock environment:

Force HTTPS for authentication.
Allow testing HTTPS without installing a certificate locally.
Implement a RESTful LoginService that allows users to login.
Implement login with Ajax, with the request coming from an insecure page.

Forcing HTTPS with Spring Security
The first feature was fairly easy to implement thanks to Spring Security. Its configuration supports a requires-channel attribute that can be used for this. I used this to force HTTPS on the "users" page and it subsequently causes the login to be secure.

<intercept-url pattern="/app/users" access="ROLE_ADMIN" requires-channel="https"/>

Testing HTTPS without adding a certificate locally
After making the above change in security.xml, I had to modify my jWebUnit test to work with SSL. In reality, I didn't have to modify the test, I just had to modify the configuration that ran the test. In my last post, I wrote about adding my 'untrusted' cert to my JVM keystore. For some reason, this works for HttpClient, but not for jWebUnit/HtmlUnit. The good news is I figured out an easier solution - adding the trustStore and trustStore password as system properties to the maven-failsafe-plugin configuration.

<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.2</version>
<configuration>
    <includes>
        <include>**/*WebTest.java</include>
    </includes>
    <systemPropertyVariables>
      <javax.net.ssl.trustStore>${project.build.directory}/ssl.keystore</javax.net.ssl.trustStore>
      <javax.net.ssl.trustStorePassword>appfuse</javax.net.ssl.trustStorePassword>
    </systemPropertyVariables>
</configuration>

The disadvantage to doing things this way is you'll have to pass these in as arguments when running unit tests in your IDE.

package org.appfuse.examples.web;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginService {

  @Autowired
  @Qualifier("authenticationManager")
  AuthenticationManager authenticationManager;

  @RequestMapping(method = RequestMethod.GET)
  @ResponseBody
  public LoginStatus getStatus() {
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
      return new LoginStatus(true, auth.getName());
    } else {
      return new LoginStatus(false, null);
    }
  }

  @RequestMapping(method = RequestMethod.POST)
  @ResponseBody
  public LoginStatus login(@RequestParam("j_username") String username,
                           @RequestParam("j_password") String password) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    User details = new User(username);
    token.setDetails(details);

    try {
      Authentication auth = authenticationManager.authenticate(token);
      SecurityContextHolder.getContext().setAuthentication(auth);
      return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
      return new LoginStatus(false, null);
    }
  }

  public class LoginStatus {

    private final boolean loggedIn;
    private final String username;

    public LoginStatus(boolean loggedIn, String username) {
      this.loggedIn = loggedIn;
      this.username = username;
    }

    public boolean isLoggedIn() {
      return loggedIn;
    }

    public String getUsername() {
      return username;
    }
  }
}

To verify this class worked as expected, I wrote a unit test using JUnit and Mockito. I used Mockito because Overstock is transitioning to it from EasyMock and I've found it very simple to use.

package org.appfuse.examples.web;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Matchers;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.TestingAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.context.SecurityContextImpl;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class LoginServiceTest {

  LoginService loginService;
  AuthenticationManager authenticationManager;

  @Before
  public void before() {
    loginService = new LoginService();
    authenticationManager = mock(AuthenticationManager.class);
    loginService.authenticationManager = authenticationManager;
  }

  @After
  public void after() {
    SecurityContextHolder.clearContext();
  }

  @Test
  public void testLoginStatusSuccess() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    SecurityContext context = new SecurityContextImpl();
    context.setAuthentication(auth);
    SecurityContextHolder.setContext(context);

    LoginService.LoginStatus status = loginService.getStatus();
    assertTrue(status.isLoggedIn());
  }

  @Test
  public void testLoginStatusFailure() {
    LoginService.LoginStatus status = loginService.getStatus();
    assertFalse(status.isLoggedIn());
  }

  @Test
  public void testGoodLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    when(authenticationManager.authenticate(Matchers.<Authentication>anyObject())).thenReturn(auth);
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertTrue(status.isLoggedIn());
    assertEquals("foo", status.getUsername());
  }

  @Test
  public void testBadLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(false);
    when(authenticationManager.authenticate(Matchers.anyObject()))
        .thenThrow(new BadCredentialsException("Bad Credentials"));
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertFalse(status.isLoggedIn());
    assertEquals(null, status.getUsername());
  }
}

Implement login with Ajax
The last feature was the hardest to implement and still isn't fully working as I'd hoped. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

Passing in the "ajax=true" parameter disables SiteMesh decoration on the login page, something that's described in my Ajaxified Body article.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

Instead of adding a click handler to a specific id, it's probably better to use a CSS class that indicates authentication is required for a link, or -- even better -- use Ajax to see if the link is secured.

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginService.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('<div class="error">Login failed, please try again.</div>');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "/api/login.json",
        type: "POST",
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href= getHost() + '/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Issues
I encountered a number of issues when implementing this in the ajax-login project.

If you try to run this with ports (e.g. 8080 and 8443) in your URLs, you'll get a 501 (Not Implemented) response. Removing the ports by fronting with Apache and mod_proxy solves this problem.
If you haven't accepted the certificate in your browser, the Ajax request will fail. In the example, I solved this by clicking on the "Users" tab to make a secure request, then going back to the homepage to try and login.
The jQuery window.name version 0.9.1 doesn't work with jQuery 1.5.0. The error is "$.httpSuccess function not found."

Finally, even though I was able to authenticate successfully, I was unable to make the authentication persist. I tried adding the following to persist the updated SecurityContext to the session, but it doesn't work. I expect the solution is to create a secure JSESSIONID cookie somehow.

@Autowired
SecurityContextRepository repository;

@RequestMapping(method = RequestMethod.POST)
@ResponseBody
public LoginStatus login(@RequestParam("j_username") String username,
                         @RequestParam("j_password") String password,
                         HttpServletRequest request, HttpServletResponse response) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    ...

    try {
        Authentication auth = authenticationManager.authenticate(token);
        SecurityContextHolder.getContext().setAuthentication(auth);
        // save the updated context to the session
        repository.saveContext(SecurityContextHolder.getContext(), request, response);
        return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
        return new LoginStatus(false, null);
    }
}

Conclusion
This article has shown you how to force HTTPS for login, how to do integration testing with a self-generated certificate, how to implement a LoginService with Spring MVC and Spring Security, as well as how to use jQuery to talk to a service cross-domain with the window.name Transport. While I don't have everything working as much as I'd like, I hope this helps you implement a similar feature in your applications.

One thing to be aware of is with lightbox/dialog logins and HTTP -> HTTPS is that users won't see a secure icon in their address bar. If your app has sensitive data, you might want to force https for your entire app. OWASP's Secure Login Pages has a lot of good tips in this area.

Update: I've posted a demo of the ajax-login webapp. Thanks to Contegix for hosting the demo and helping obtain/install an SSL certificate so quickly.

2010 - A Year in Review

2011-01-10T11:42:01-07:00

2010 was a heckuva year, possibly one of my all-time favorites. It started with lots of anticipated fun and ended with lots of travel, skiing and relaxing and I'm only just now finding time to write this post. I had a lot of goals when I started 2010; the top two being my favorite:

Happiness
Girl

The rest of my goals involved running, skiing, mountain biking and finishing The Bus. I also had some professional goals that involved open source, conferences, publishing articles and learning new technologies. I accomplished about 75% of my personal goals and 50% of my professional goals. I'm pretty happy with these percentages considering how much fun I had last year.

In December 2009, I started working with Time Warner Cable as their Chief Architect of Web Development. I was hired to help them build a team of hard-working developers that could quickly build their online video presence. In January, we started getting our ducks in a row with some Agile Training in Virginia, followed by hiring some Agile Coaches. The only technical post I had during this process was about development infrastructure stacks. Both at TWC and my current gig, I've found myself enjoying the following stack:

Source Control: SVN
Source Viewer: FishEye
Wiki: Confluence
Bug Tracker: JIRA
Continuous Integration: Hudson

We tried Git for a couple months at TWC, but ended up reverting to SVN after we had "code deleted" issues during one of our most intense development cycles. When the kids and I weren't skiing, we worked on documenting How to be a Super Hero and The Adventures of Upside Down Man.

In February, I refreshed my Grails knowledge, later learning that it was tough to teach Grails to developers that didn't know Java. First of all, a lot of Grails and Groovy books are targeted at Java Developers. Secondly, the developers I was trying to sell it to had more interest in learning Java. Since I failed to sell Grails, we ended up using Spring + jQuery to build our app. I don't think was a bad decision as most of our development ended up being UI: ActionScript/Flex, Objective-C and jQuery/HTML.

My laptop was stolen from my living room in early February. It worked out nicely in the end since I didn't lose any data (thanks to good backups) and my business insurance covered the loss. My parents came out and helped me finish remodeling my guest room. I posted about My Future of Web Frameworks Presentations and became a proud father when "Jack was flying past both Abbie and I and giggling while doing it".

By the beginning of March, we'd hired a team of 10 at TWC and I took a trip to Jackson Hole to celebrate.

The thing I enjoyed the most about this trip was how well the group jelled. Kudos to Chris for assembling such an awesome group and putting such a spectacular trip together. Can't wait for next year.

The next week, I flew to Las Vegas for The ServerSide Java Symposium and enjoyed sessions on Cloud, Web Service APIs, Flex, Spring 3 and Cameron's Lessons Learned from the Internet Age. I posted my Future of Web Frameworks and GWT vs. Flex Smackdown presentations after the conference.

I ended March with The Trifecta.

For Easter, I purchased an iPad and reviewed it a few days later. Seven months later, I'm not enthralled with the iPad, but I do think it's a heckuva lot better than the Galaxy Tab. I don't use it much, except for movies when traveling. My kids are its primary users, mostly using it for games and Netflix.

We visited my parents in Oregon and celebrated my Mom's retirement.

The picture I took of my Mom and Abbie that weekend is one of my all-time favorites.

At the end of April, I said farewell to the ski season, getting in 25 days; a personal best. I'm happy to report I have 10 days in so far this year, so my goal of 30 days looks to be well on track. No helicopter attempts yet, but hopefully soon.

I ended the month with a work trip to Seattle and painting the town red with my sister and Mya.

I returned to Denver and turned off my TV for a month. I wrote about my experience in early June. For Memorial Day Weekend, I went mountain biking in Moab and had a blast at the Desert Rocks Music Festival. The Porcupine Rim ride took us 4.5 hours and we tracked 26.75 miles. The several points in the trail with "death on the right" were truly epic.

June started with our annual Ride to Red Rocks. I slept six hours and did the 25-mile off-road Elephant Rock ride the next morning. The next morning I hopped on a plane to Ireland for the Irish Software Show. My sister joined me and we had an incredible time with Jamie, Rob, John and Josh.

Upon returning to work, I got to have an awkward conversation with my client about all the vacation I was taking.

For Father's Day, my parents drove to Denver and we packed up the camping gear for a fun weekend at The Great Sand Dunes. After The Dunes, my parents and kids drove to The Cabin, camping out and touring Yellowstone along the way. That Thursday, June 24th, I attended my good friend Jason's birthday party in Lodo.

That's the night I met Trish.

I saw her switch from a Martini to a Guinness and I knew I had to talk to her. I introduced myself and quickly found myself conflicted with having a soon-to-expire iPhone 4 reservation at the Apple Store. I rushed out to grab the phone, returning because there was something special about Trish. We talked for a couple hours that night ... me mesmerized, her smiling a lot. I was dumbfounded to find such a cool person existed in the world. At the end of the night, I got her number and drove home with a feeling that my life was about to change.

The next day, I began a road trip to The Last Best Place for a Montana Summer Vacation.

During that trip, Trish and I exchanged occasional text messages and I told many friends about the kick-ass girl I met. It was another great Montana vacation.

My favorite part of this year's trip to The Cabin was seeing it as a home again. My Mom retired in April and my parents moved back to Montana shortly after. Seeing how happy they are there is truly magical. I especially enjoy the thought of visiting them and all the wonderful folks in the Swan Valley many, many times in the future.

The kids and I drove like bats out of hell and made it back to Denver (950 miles) in 14.25 hours. I quickly scheduled a first date with Trish that Wednesday, and went back to work at TWC with a renewed energy. After our first date, I formed a sort of perma-grin. That Friday was my birthday. I've been having a blast with Trish ever since.

July ended with a trip to the Lost Coast for Jess and Lili's Wedding.

The reception afterwards was a truly spectacular party that lasted well into the evening. Clint and I vowed to go to bed early, but we ended up having so much fun we closed the place down. Jess and Lili were an instrumental part in creating a spectacular night, especially with their wedding dance and infectious happiness.

In August, I celebrated this blog's 8th birthday and we started our "60-Day Push" at work. The goal of our 60-Day Push was to re-write our Video Portal, iPad and Sony Blu-ray apps from scratch, without politics dictating their features. We hired Method for design, chose our own features and went to work. I wrote about the success of this effort in October.

I neglected to write about the Denver Cruisers Saints and Sinners ride or how much fun we had listening to B.B. King at Red Rocks.

Abbie and Jack's first day of school was on August 16th.

At the end of August, we celebrated Jack's 6th Birthday and attended my Cousin Amy's Wedding in Missouri.

September rolled in, we finished the majority of the work in our 60-Day Push and Trish and I drove up to Estes Park for an epic weekend at the Scottish-Irish Festival. We rode our bikes in the parade (by accident), enjoyed a few pints and even did a bit of fly-fishing near our riverside accommodations.

We listened to the 1st 2010 Broncos game while driving back from Estes and slipped into a Rockies game shortly after losing. Our sadness over the Broncos loss was erased within a couple hours as we celebrated Jason Giambi's walk-off home run. Unfortunately, the Rockies didn't make it to the post-season, but the Broncos looked good at their home opener.

October came quickly and I wrote How's the ol' Team Doing and defended the Age of Frameworks. At the end of October, we moved into TWC's Lodo Office on Wynkoop. That weekend, we dressed up as superheros for Halloween.

November started off with Abbie's birthday and Trish got to meet the kids for the first time. From there, we went into traveling-like-madmen mode. We enjoyed suite seats at the Broncos vs. Kansas City game, then flew to Amsterdam for Devoxx the next morning (performing an AppFuse release along the way). I presented on Thursday and posted my Comparing JVM Web Frameworks presentation shortly after. I wrote about our trip while at The Cabin for Thanksgiving.

I flew back from Montana, stopped in at the Lodo office for a couple days, then hopped on a plane to the Rich Web Experience with Trish. My presentations went well, sparked some controversy, and we raced to Key West to celebrate the end of the conference season.

I returned to Denver for my last week at TWC, enjoyed a couple days of skiing, then headed to Utah for an interview with Overstock.com. While I enjoyed my time at TWC, my contract duration was up and being a full-time employee didn't give me the vacation time I tend to enjoy. My interview with Overstock.com was two days, with the 2nd day on the slopes at Snowbird. I was very impressed by the company, people and interview process and agreed to work there on the way to the airport.

I returned to Denver for Trish's Birthday Weekend at Breckenridge, then flew down to Naples on Wednesday to spend Christmas with her family. It was my first time meeting her parents, but that didn't stop us from having a great time talking, beach-going, golfing and relaxing.

We returned to Denver, I tried to get AppFuse 2.1 finished and then we celebrated New Years with friends in Steamboat.

As I reminisce about last year, I can't help but smile. While I've been a happy person for a while, having someone to share your life with is a special thing. I feel like I dreamed up Trish a couple years ago. I was looking for someone that liked to do my favorite activities: mountain biking, skiing, traveling the world and enjoying good beer. I found that and more in Trish and couldn't be happier.

I didn't write as many technical posts on this blog as I'd like to, but I attribute that mostly to the lack of learning new things at TWC. At Overstock.com, I expect that to change and hope to have more technical articles in the coming year.

At the end of last year's Year in Review, I wrote:

I hope to speak at (or attend) 3 conferences, finish up The Bus and do a whole bunch of skiing and mountain biking.

I accomplished all but one goal: finishing The Bus. In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus. Everything else is gravy.

It's gonna be a spectacular year.

How's the ol' Team Doing?

2010-10-21T09:00:41-06:00

Back in March, I wrote about How We Hired a Team of 10 in 2 Months:

This week, we on-boarded 3 of our final 4 developers. I breathed a big sigh of relief that the hiring was over and we could get back to slinging code and making things happen. As luck would have it, I received an e-mail from my boss on Tuesday that the hiring engine is starting up again and we need to hire 6 more developers. While I'm not anxious to start the Hiring Engine again, I am glad to know it works well and it has helped us build a great team.

We never ended up hiring those additional 6 developers, but we've had quite a ride since March. One of the first commenters on my original post wrote:

I hope that whirlwind of hiring works out for you Matt. But, don't you think it's a bit early to be declaring success?

At that time, we still had a lot of work to do to become a successful team. However, by the end of March, we'd finished our first deliverable - converting a somewhat slow ColdFusion/jQuery/Video webapp into a fast Java/jQuery/Video webapp. The slowness wasn't due to ColdFusion, but mostly performance, caching and YSlow-type optimizations. At that time, we surprised the folks that were in charge of our app. They didn't think we'd finish so fast and it took them awhile to decide what to do with our work.

While we were waiting for the product folks to launch our app in April/May, we decided to experiment with developing some new clients. So we wrote an iPhone app, an Android App, an iPhone/Android/iPad/HTML5 version of our webapp and a Blu-ray client. All of these applications used our webapp's backend RESTful services and we were able to learn the SDKs and implement all the apps in a matter of weeks. In May, we demoed all our clients and got rave reviews from executives. We celebrated that afternoon with a big sigh of relief.

The glowing from our many-clients demo was short-lived. A week later, we were asked to enhance our iPad app to include TV-Remote type features, namely channel-changing and DVR functionality. After freaking out and trying to figure out how to deliver such an app in a week, the demo was rescheduled and we were afforded 2 weeks to build the app. After much frantic development, we were able to complete the app in time and the demo was published to YouTube a couple months later.

In June, if you asked me if we were a successful team, I would've definitely said "Yes!" We'd been asked to develop apps for 3 different demos and we delivered on-time.

The remainder of June and July we slipped into a bit of limbo where we weren't asked to develop anything, but simply maintain and enhance the stuff we'd already developed. After a few weeks of doing this, several of us began to wonder if the apps we'd developed would ever see the light of day. We expressed this concern to our VP and a new idea was hatched: The 60-Day Push.

The 60-Day Push was designed to eliminate politics and meetings and allow us to develop a 3-screen (PC, iPad, TV) experience for our video content. We decided to aim high and try to complete most of the work in 30-days, so we could do executive demos of our progress. We started all our applications from scratch, split into Services, Portal, TV and iPad teams and worked with Method to implement a slick design for all our apps. I'm proud to say we delivered yet again and there were many proud moments as we demoed to the top executives of the company.

In early September, after doing several demos, we were approved to launch and we've been working towards that goal ever since. We feel we have several weeks of work to coax our DemoWare into RealWorldWare, but the momentum is there and the end of the tunnel is in sight.

As a further sign of our success, we're moving into a new office in LoDo next week. This also means the End of an Era, where the Raible Designs' office across from Forest Room 5 will cease to exist. We (Goodwill, Scotty, Country Bry and I) first moved into this office while working for Evite in April 2009. It's been a fantastic location, a facilitator of extensive collaboration and host to many dart games, FACs, bird-dog spottings and way too many Mom Jokes.

As part of our transition, I'll be looking for renters to fill out the rest of my lease (through March 2010). If you're looking for a sweet location for 5-6 people in Denver's Highlands (near the REI store downtown), please let me know.

Also, there's a good chance my team will continue to grow as we move our apps into production and start ramping up for millions of users. If you're a strong Web or Java developer with social skills and a Ditchdigger attitude, I'd love to hear from you. We probably won't be hiring until January, but that doesn't mean we can't start talking now.

My Presentations from The Irish Software Show 2010

2010-06-10T07:11:35-06:00

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

JavaScript and CSS Concatenation with wro4j

2009-11-09T10:44:44-07:00

This past weekend, I decided it was about time to fix my YSlow score on this site. I did the easiest thing first by moving all my JavaScript files to the bottom of each page. Then I turned on GZip compression using Roller's built-in CompressionFilter. These changes helped, but the most glaring problem continued to be too many requests. To solve this, I turned to wro4j (as recommended on Twitter) to concatenate my JS and CSS files into one.

I have to say, I'm very happy with the results. I'm now sitting at a YSlow (V2) score of 75; 90 if I use the "Small Site or Blog" ruleset. I believe I can improve this by adding expires headers to my images, js and css. More than anything, I'm impressed with wro4j, its great support and easy setup. I was looking for a runtime solution (b/c I didn't want to have to rebuild Roller) and it seems to be perfect for the job. Furthermore, wro4j minifies everything on the fly and they'll have an expires header filter in the next release.

JAWR and the YUI Compressor are other alternatives to this filter, but I'm currently sold on wro4j. First of all, it passed the 10-minute test. Secondly, it didn't require me to modify Roller's build system.

At this point, if I'm going to implement JS/CSS concatenation and minification in AppFuse and Roller, wro4j seems like the best option. If you disagree, I'd love to hear your reasoning.

TIP: See Javascript Compression in Nexus for information on using YUI Compressor with Maven.

Consulting, SOFEA, Grails and GWT at next week's Denver JUG

2009-11-05T22:52:37-07:00

Next Wednesday, I'll be at Denver's JUG meeting to talk about Independent Consulting and Building SOFEA Applications with Grails and GWT. The first talk will be a a panel discussion among local independent consultants, including James Goodwill, Matthew McCullough, Tim Berglund and myself.

This session explores the trials and tribulations of an independent consultant. How do you find contracts? Should you setup an LLC, an S-Corp or just be a sole proprietorship? What about health insurance and benefits? Are recruiters helpful or hurtful? Learn lots of tips and tricks to get your dream job and your ideal lifestyle.

The Grails and GWT talk is a preview of a talk I'll be doing at the Rich Web Experience in December. Below is a rewrite of the abstract in first-person.

Earlier this year, I participated in a major enhancement of a high-traffic well-known internet site. The company wanted us to quickly re-architect their site and use a modern Ajax framework to do it with. An Ajax Framework evaluation was done to help the team choose the best framework for their skillset. The application was built with a SOFEA architecture using GWT on the frontend and Grails/REST on the backend.

This talk will cover how Bryan Noll, Scott Nicholls, James Goodwill and I came to choose GWT and Grails, as well as stumbling blocks we encountered along the way. In addition, we'll explore many topics such as raw GWT vs. GXT/SmartGWT, the Maven GWT Plugin, modularizing your code, multiple EntryPoints, MVP, integration testing and JSON parsing with Overlay Types.

If you're in Denver next Wednesday night (November 11th), you should stop by the Denver JUG meeting. It'll be a fun night and there's sure to be a few beers afterward.

What would you like to see at TSSJS 2010?

2009-10-12T11:28:21-06:00

A couple months ago, I was asked by TheServerSide to speak at next year's TheServerSide Java Symposium in Las Vegas. In addition, they asked me to help them evaluate presentation proposals and suggest topics/speakers.

First of all, I think the biggest thing that TSSJS could do to improve is to host more networking events. With the JavaOne Party being over, I think there's a tremendous opportunity to fill a gap in the networking needs of the Java Community. When I first attended TSSJS in 2006, there were a fair amount of parties and everyone got to interact quite a bit. In 2008, there were no networking events. I believe having a strong networking story would attract a lot more attendees, companies and sponsors.

Secondly, I think it's possible that TSSJS has too many server-side related sessions. IMO, the server-side (and middleware in general) isn't that exciting. TechTarget appears to own TheClientSide, so why not add some more client-side stuff to the mix? For example, I'd love to see a Struts 1 app-makeover using different technologies (for example, Flex, GWT and jQuery). I think HTML5 and Google Wave's Architecture sessions would be interesting too. If adding client-side sessions is too far away from TheServerSide, maybe it should be renamed to TheServerSide JVM Symposium and there can be all kinds of sessions on JVM languages (e.g. Scala, JRuby, Groovy) and all the great things those languages can accomplish.

Lastly, I've been asked to send a couple session proposals. Currently, I'm thinking about a doing GWT vs. Flex Smackdown with James Ward, but I'm open to other ideas. It's been quite awhile since I did a "Comparing Web Frameworks" talk. Maybe "Hot Web Frameworks for 2010" is more appropriate? I also think it'd be interesting to do a somewhat philosophical talk on "The State of Web Frameworks" and where we're headed in the next year.

What would make you want to attend TSSJS next year? Let me know your thoughts and I'll do my best to make them a reality.

Update October 22, 2009: Whoo hoo! It looks like TheClientSide will be a part of TSSJS Vegas next year. Should be a great show.

Creating a Facebook-style Autocomplete with GWT

2009-06-05T07:05:10-06:00

Have you used the "To:" widget on on Facebook or LinkedIn when composing a message? It's an autocompleter that looks up contact names and displays them as you type. It looks like a normal textbox (a.k.a. <input type="text">), but wraps the contact name to allow you to easily delete it. Here's a screenshot of what Facebook's widget looks like.

Last week, I was asked to create a similar widget with GWT. After searching the web and not finding much, I decided to try writing my own. The best example I found on how to create this widget was from James Smith's Tokenizing Autocomplete jQuery Plugin. I used its demo to help me learn how the DOM changed after you selected a contact.

GWT's SelectBox allows you to easily create an autocompleter. However, it doesn't have support for multiple values (for example, a comma-delimited list). The good news is it's not difficult to add this functionality using Viktor Zaprudnev's HowTo. Another feature you might want in a SelectBox is to populate it with POJOs. GWT SuggestBox backed by DTO Model is a good blog post that shows how to do this.

Back to the Facebook Autocompleter. To demonstrate how to create this widget in GWT, I put together a simple application. You can view the demo or download it. The meat of this example is in an InputListWidget. After looking at the jQuery example, I learned the widget was a <div> with a unordered list (<ul>). It starts out looking like this:

<ul class="token-input-list-facebook">
    <li class="token-input-input-token-facebook">
        <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
    </li>
</ul>

I did this in GWT using custom BulletList and ListItem widgets (contained in the download).

final BulletList list = new BulletList();
list.setStyleName("token-input-list-facebook");

final ListItem item = new ListItem();
item.setStyleName("token-input-input-token-facebook");

final TextBox itemBox = new TextBox();
itemBox.getElement().setAttribute("style", 
        "outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;");

final SuggestBox box = new SuggestBox(getSuggestions(), itemBox);
box.getElement().setId("suggestion_box");

item.add(box);
list.add(item);

After tabbing off the input, I noticed that it was removed and replaced with a <p> around the value and a <span> to show the "x" to delete it. After adding a couple items, the HTML is as follows:

<ul class="token-input-list-facebook">
    <li class="token-input-token-facebook">
        <p>What's New Scooby-Doo?</p>
        <span class="token-input-delete-token-facebook">x</span>
    </li>
    <li class="token-input-token-facebook">
        <p>Fear Factor</p>
        <span class="token-input-delete-token-facebook">x</span>
     </li>
     <li class="token-input-input-token-facebook">
         <input type="text" style="outline-color: -moz-use-text-color; outline-style: none; outline-width: medium;"/>
     </li>
</ul>

To do this, I created a deselectItem() method that triggers the DOM transformation.

private void deselectItem(final TextBox itemBox, final BulletList list) {
    if (itemBox.getValue() != null && !"".equals(itemBox.getValue().trim())) {
        /** Change to the following structure:
         * <li class="token-input-token-facebook">
         * <p>What's New Scooby-Doo?</p>
         * <span class="token-input-delete-token-facebook">x</span>
         * </li>
         */

        final ListItem displayItem = new ListItem();
        displayItem.setStyleName("token-input-token-facebook");
        Paragraph p = new Paragraph(itemBox.getValue());

        displayItem.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                displayItem.addStyleName("token-input-selected-token-facebook");
            }
        });

        Span span = new Span("x");
        span.addClickHandler(new ClickHandler() {
            public void onClick(ClickEvent clickEvent) {
                list.remove(displayItem);
            }
        });

        displayItem.add(p);
        displayItem.add(span);
        
        list.insert(displayItem, list.getWidgetCount() - 1);
        itemBox.setValue("");
        itemBox.setFocus(true);
    }
}

This method is called after selecting a new item from the SuggestBox:

box.addSelectionHandler(new SelectionHandler<SuggestOracle.Suggestion>() {
    public void onSelection(SelectionEvent selectionEvent) {
        deselectItem(itemBox, list);
    }
});

I also added the ability for you to type in an e-mail address manually and to delete the previous item when you backspace from the input field. Here's the handler that calls deselectItem() and allows deleting with backspace:

// this needs to be on the itemBox rather than box, or backspace will get executed twice
itemBox.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
            // only allow manual entries with @ signs (assumed email addresses)
            if (itemBox.getValue().contains("@"))
                deselectItem(itemBox, list);
        }
        // handle backspace
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            if ("".equals(itemBox.getValue().trim())) {
                ListItem li = (ListItem) list.getWidget(list.getWidgetCount() - 2);
                Paragraph p = (Paragraph) li.getWidget(0);

                list.remove(li);
                itemBox.setFocus(true);
            }
        }
    }
});

I'm happy with the results, and grateful for the jQuery plugin's CSS. However, it still has one issue that I haven't been able to solve: I'm unable to click on a list item (to select it) and then delete it (with the backspace key). I believe this is because I'm unable to give focus to the list item. Here's the code that highlights the item and you can see the commented-out code that doesn't work.

displayItem.addClickHandler(new ClickHandler() {
    public void onClick(ClickEvent clickEvent) {
        displayItem.addStyleName("token-input-selected-token-facebook");
    }
});

/** TODO: Figure out how to select item and allow deleting with backspace key
displayItem.addKeyDownHandler(new KeyDownHandler() {
    public void onKeyDown(KeyDownEvent event) {
        if (event.getNativeKeyCode() == KeyCodes.KEY_BACKSPACE) {
            list.remove(displayItem);
        }
    }
});
displayItem.addBlurHandler(new BlurHandler() {
    public void onBlur(BlurEvent blurEvent) {
        displayItem.removeStyleName("token-input-selected-token-facebook");
    }
});
*/

If you know of a solution to this issue, please let me know. Feel free to use this widget and improve it as you see fit. I'd love to see this as a native widget in GWT. In the meantime, here's the GWT Facebook-style Autocomplete demo and code.

Ajax Framework Analysis Results

2009-04-23T20:34:44-06:00

Way back in January, I wrote about how my colleagues and I were evaluating Ajax frameworks to build a SOFEA-style architecture. To make our choice, we used the following process:

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation and recommendation.

When I wrote that entry, we had just finished step 2 and were starting step 3. I first wrote this blog post a week later, when we delivered step 5. Here is the comparison and conclusion sections of the analysis document we composed.

Framework Comparison
In order to evaluate the different frameworks against important criteria, we created a matrix with weights and ranks for each framework. This matrix shows how our weighting and rankings lead us to the winner for our project. You can view this matrix online or see below for a summary.

Note: Criteria whose values were identical across all candidates were weighted at zero. Charting capability was weighted at zero b/c we decided to use Flash for this.

This matrix indicates that GWT is the best candidate for our team to develop SOFEA-style applications with. In addition to the matrix, below are graphs that illustrate interesting (and possibly meaningless) statistics about each project.

Conclusion
After working with the various frameworks, we believe that all the frameworks were very good and could be used to write applications with. If all weights are equal, these frameworks were almost even when compared against our evaluation criteria. The graph below illustrates this.

Even after applying the weighted criteria, the evenness doesn't change a whole lot.

Without considering the even or weighted criteria, we believe the decision all comes down to what the developers on the project feel they will be most comfortable with. If you're developing with Dojo or YUI, chances are you're dressing up existing HTML and possibly using progressive enhancement to add more rich functionality. On the other hand, Ext JS and GWT are similar to Swing programming where you build the UI with code (JavaScript for Ext JS, Java for GWT).

The tools available for JavaScript development have gotten increasingly better in recent years. IntelliJ IDEA has a JavaScript Editor that provides many of the same features as its Java editor. Aptana Studio also has excellent support for authoring and debugging JavaScript. However, we believe the Java debugging and authoring support in IDEs is much better. Furthermore, we are more familiar with organizing code in Java projects and feel more comfortable in this development environment.

Based on this evaluation, we believe that GWT is the best framework for our team to develop SOFEA-style applications with.

Flash Forward to Today...
The core GWT library from Google doesn't have a whole lot of widgets, nor do they look good out-of-the-box. So early on, we experimented with two alternative implementations that continue to leverage GWT concepts and tools:

GXT: a GWT version of Ext JS
SmartGWT: a GWT version of SmartClient

Unfortunately, over the past few months, we've found that both of these implementations are too heavy for our requirements, mostly because of the file size of the generated JavaScript code. For example, a feature I wrote generated a 275K *.cache.html file using GXT. After determining that was too slow to give users the initial "pop", I re-wrote it without GXT. After a day, we had an application with *.cache.html files of 133K. Yes, that's over a 50% reduction in size!*

Because of these findings, we are proceeding with the core GWT library from Google and adding in new components as needed. It is cool to know you can make a UI "pop" with GWT, as long as you stick to the core - close-to-the-metal - components. For those applications that can afford an initial "loading..." state, I'd definitely recommend looking at GXT and SmartGWT.

* To make refactoring easier, I copied GXT MVC into our source tree and modified all imports.

My Drunk on Software Interview

2009-04-05T22:23:57-06:00

Back in February, I met up with James Ward and Jon Rose for a Drunk on Software interview. We enjoyed some good beer and had a great conversation about SOFEA, open source and RIA. See larger video here.

Optimizing a GWT Application with Multiple EntryPoints

2009-03-25T16:00:37-06:00

Building a GWT application is an easy way for Java Developers to write Ajax applications. However, it can be difficult to release a GWT application to production before it's finished. One of the most important things I've learned in Software Development is to get a new application into production as soon as possible. Not only does getting it from dev → qa → prod verify your process works, it also can do a lot to test the viability of the new application.

One of the biggest issues with GWT applications is size. The project I'm working on compiles Java to JavaScript and creates ~570K *.cache.html files (one for each modern browser). These files end up being around 180K gzipped. I believe this is an OK size for an entire application. However, if you're going to release early, release often with GWT, chances are you'll just want to release one feature at a time.

When the first feature was completed on my project, the *.cache.html files were around 300K. Rather than using branches to release to QA and UAT, bug fixes and new features were developed on trunk. Unfortunately, the QA and UAT process took several weeks longer than expected so by the time the feature was ready to release, the *.cache.html files had grown to around ~570K. The reason the file had grown so much was because it included all of the other features.

Earlier this week, while running to a dentist appointment, I thought of a solution to this problem. The basic idea was to optimize the compilation process so only the to-be-released feature was included. Even better, the solution didn't require more modularization. The results:

Before: *.cache.html -> 569K, gzipped 175K
After: *.cache.html -> 314K, gzipped 100K

According to my calculations, that's a 56% reduction in size. How did I do it?

Created a new FeatureName.java EntryPoint with only the to-be-released features imported.
Created a new FeatureName.gwt.xml that references the new EntryPoint.
Copied old (kitchen-sink) EntryPoint.html to FeatureName.html and changed the reference to the nocache.js file.
Created a Maven profile that allows using -PFeatureName to build a FeatureName-only module.

One downside to doing things this way is it's possible to create a WAR that has the same name and different features. Surely the Maven Overlords would frown upon this. Since this is just a temporary solution to release features incrementally, I'm not too worried about it. A possible workaround is to create different WAR names when a feature's profile is activated. I believe the true "Maven way" would be to make the "kitchen sink" application into a JAR and have several WAR modules with the different EntryPoints. Seems a bit complicated to me.

Other than this Maven publishing issue, the only other issue I can foresee is keeping the two EntryPoints and HTML files in synch. Then again, the separate files allow a feature to be customized for the release and can be deleted when its no longer needed.

What do you think? Do you know of a better way to compile a GWT application so it only contains certain features?

Ajax: The State of the Art with Dion and Ben

2009-02-05T11:03:10-07:00

This morning, I added Dion and Ben's talk titled Ajax: The State of the Art. Below are my notes from the event.

Ajax started out as a bunch of hacks. It showed that we could take our web interfaces and do a lot more with them. A hack isn't necessarily a bad thing. Often, they turn into something much more elegant over time. The new browsers have many amazing capabilities that we haven't taken advantage of yet. We've seen discussions on Ajax go from how to do XHR to frameworks and how rich and mature they are. Dojo is great for Enterprise Development (packing system, namespaces). jQuery is well-suited for lightweight developers (PHP). Prototype is fantastic for people who do a lot of JavaScript development and take it very seriously.

Today's Ajax landscape is mature, really rich, and really exciting. Today, Dion and Ben are going to talk about technologies they're really excited about for the future.

Canvas
The building blocks of the web are text, boxes and images. With canvas, it really makes a lot more things possible. You can do bitmap rendering and image manipulation. They're showing a slide with Doom and Mario Kart running. Canvas 3D does true 3D rendering. Firefox and Opera have done prototypes of this. Can you do canvas-type things today in a browser? Yes, if you use Flash or Curl. Dion and Ben are excited about canvas over plugins for the following reasons:

No start-up delay
Available on mobile devices today
Rendering fidelity with browser (especially important for typography)
No bridges necessary (no marshalling/unmarshalling)
Not a plug-in

The <canvas> tag originally came from Apple's Dashboard. Dashboard's programming model was in HTML and JavaScript. Dashboard is using WebKit under the covers. Today, canvas support exists in every major browser except for IE. The good news is there are Flash and Silverlight bridges to add support to IE. There's also an ActiveX component that wraps the Firefox implementation and allows it to run in IE.

SVG
Dion and Ben aren't that excited about SVG because it's such a huge spec. We've been struggling with the HTML standard for the last 10 years and the thought of another huge spec for the next 10 years isn't that appealing.

Fast JavaScript
Almost all major browsers have a Fast JavaScript implementation. Chrome has V8, Safari has SquirrelFish Extreme, Firefox has TraceMonkey and Opera has Carakan. This is exciting because of industry trends and how companies are trying to reduce computation cycles in data centers. The more computing that can be put on the client, the better. IE doesn't have anything, but Dion and Ben believe they are working on something.

Web Workers
Interface latency is awful for applications. Jakob Nielsen once said:

0.1 second is about the limit for having the user feel that the system is reacting instantaneously. 1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay.

Anything that takes longer than a tenth of a second should be pushed to a background thread. Unfortunately, there are no threads in the web. Maybe we can add threads to JavaScript? Brendan Eich has said that "Threads suck" and there's very little chance for threads getting into JavaScript. Gears brought Worker Pools and this is going into HTML 5 as Web Workers. You could also use Java applets to do this. With the latest Java Plugin, many of applets' long-standing issues have been solved.

Desktop Integration
The ability to build desktop apps as web apps is very exciting. There's a few technologies that demonstrate this: Fluid, Mozilla Prism, Adobe AIR, Appcelerator Titanium and Gears. The Palm Pre demonstrates the logical extension of this. The Palm Pre uses the web stack as its developer SDK. It's very cool that web developers don't have to learn anything new to become a Palm developer. Desktop integration is exciting especially if we can access desktop applications like email and address book.

The Ajax frameworks that are out there have done a lot to make web development simpler. However, there's still a lot of pain with CSS and cross-browser issues. What if you took canvas and combined it with a sophisticated grid-based layout in JavaScript?

There's a lot of platforms out there: Microsoft Silverlight, Adobe Flash, Apple Cocoa and Sun's JavaFX. The web often isn't considered a platform. Dion and Ben believe there should be an Open Web Platform. The problem right now is there is no central location to find out how to get stuff done. You have to search and find resources from many different locations. Mozilla is putting it's resources into creating an Open Web Platform. This site will consist of 4 different areas:

Home
Documentation (for different frameworks, browsers, quirks)
Dashboard (state of the open web)
Roadmap (what's going on)

This is not just Mozilla, it's very much a community effort. This is something that Ben and Dion have been working on. But there's something else they've been working on too. They've been talking about all these cool things, but what about an interesting application to test all these technologies?

Bespin
As they looked at code editors, most of them provide awful user experiences. Bespin is the Editor of Your Dreams and contains the following features:

Accessible from anywhere - any device in any location
Simple to use, like Textmate (not heavyweight like Eclipse) - an editor, not an IDE
Wicked Fast - performance, performance, performance
Rock-solid real-time collaboration, like SubEthaEdit - it just works
Integrated command-line, like vi - Fun like Quicksilver, social like Ubiquity
"Self-hosted" environment, like Emacs - For extreme extensibility, but with JavaScript!

Dion and Ben are showed a screen shot of Bespin and now they're doing a demo. The core editor has what you'd expect with syntax highlighting and line numbers. Canvas doesn't have text-selection by default, so they had to write it from scratch. The command line allows you to get help, run core command and also to subscribe to commands that others write. You can change your keybindings to emacs or vi as well as many other settings. Much of Bespin is event-driven, so you can easily plugin new behavior for different events.

For viewing files, they couldn't bring themselves to use a tree. Instead, they developed a file-browsing interface that looks very much like Apple's Finder. Personally, I like Finder, but wish it had Windows Explorer's path bar that allows you to simply type in the path without mouse clicks. Back to the command line. They've done a lot to make things more discoverable so users can easily find the power of the editor.

Bespin could be used to engage developers more with open source projects. Checking out projects, modifying code and creating patches can be a real pain. Bespin could be used to interface with open source projects in the cloud. You could login, modify code and easily patch/build with the click of a button. One other thing they want to do is to have the server do code-analysis as you're developing.

Is it OK to love a software tool? You must love your software tools. What we do as Software Developers is one of the most difficult jobs on the planet. Programmers, like poets, start with a blank slate and create something from nothing. If you don't love your tools, you'll start resenting what you do. If you don't love your tools, it shows in your work. -- Dave Thomas at RubyConf08

Thunderhead
A GUI Toolkit written with canvas and JavaScript. Allows you to do layouts with very little thought. It's a lab experiment that's in progress, stay tuned for more information.

All users care about is the user interface. Dion and Ben believe there's a key to creating compelling user experiences. It all has to do with managing expectations. It's not that different from how you manage relationships in your life. Expectations for movies and games have changes drastically over the years. What used to be the web (animated gifs and awful web pages) has also changed drastically (video of Apple's online store). What was cool with MapQuest got changed drastically with Google Maps. What we have today isn't the end of the game - expectations will continue to change. However, users have different expectations for software.

Alan Cooper has done some interesting work in this area. The software designer needs to focus in on a user's goals. There are basic things you can apply to all users, for instance "sex sells". An example of this is Delicious Library. This application allows you to keep track of things in your home such as books, movies, music and games. They made $500K in 3 months and made $54K the first day, with no advertising.

The quality of any software is determined by the interaction. If the interaction isn't good, it will poison the entire experience. Donald Norman has a good quote: "Attractive things work better". In society, this is often called "Dress for Success".

The Open Web is hear to stay because it has:

An Easy Programming Model
Easy Remoting
Extensive Customization Vectors (e.g. GreaseMonkey)
Easy Deployment
Great Widgets
Great Visual Effects
Great Mobile Story
Desktop Integration
State-of-the-Art Plug-ins

Bespin is a tech preview that they hope to release next week. Thunderhead will be released at the same time.

Conclusion
This was a great talk and easily the most inspiring of the conference. Dion and Ben always do a great job and the sexiness of their presentation made it all the more appealing.

The State of the Web 2009

2009-02-04T18:05:38-07:00

This afternoon, I attended The State of the Web 2009 at Web Directions North. Below are my notes from this session.

This panel has quite the list of superstars:

Chris Wilson (Microsoft)
Dan Connolly (W3C HTML Working Group)
Mike (TM) Smith (W3C HTML Working Group)
Lars Erik Bolstad (Opera)
Scott Fegette (Adobe)
John Allsopp (Organizer and Style Master Developer)

John is moderating this session and is starting by asking each panelist to speak about what they believe the state of the web is.

Chris Wilson
The future of the web may be as ubiquitous as electricity. Chris has a desktop, two laptops (one 10" NetBook, one is a 13" MacBook) and an iPhone. There's a lot of difference between these devices, especially when it comes to screen size. Chris uses a number of different browsers throughout the day. The web isn't just one browser, it isn't just one platform. He's showing a slide with a browser market share graph from http://www.netapplications.com.

Many different browsers are a reality. Many different devices are a reality. Web builders need to learn to write scalable applications that run across multiple browsers, devices and environments. They need to use progressive functionality and learn the tools they have in CSS and HTML. Semantic structuring helps.

Developers need to learn to live with multiple browsers. Cross-browser interoperability will get better, but it's likely to be an issue forever. Test suites with new specifications are helping. Developers should build for browsers of today and tomorrow.

You should build the applications you want to build and then figure out how how to make them degrade gracefully on the web.

Lars Erik Bolstad
Opera Software has 600 employees in 10 countries. They've been developing Opera and other browsers since 1995. The bulk of what Opera does is based on commercial browser deliveries to OEMs around the world.

Devices, platforms and networks: they come in all shapes and sizes. Only one thing unifies them: The Web. The browser is becoming more and more important on these devices. Users are not satisfied with WAP-based content anymore, they want the same content no matter which device they're using.

Opera Mini does its processing on the server-side. This allows Opera to gather statistics. These stats show that users around the world hit the same top sites on their mobile devices as they do on their desktops. It's a one-to-one match. Opera is seeing tremendous growth in the usage of Opera Mini, both in developed countries and emerging markets.

The point: don't just develop for desktop browsers. The mobile market seems to be growing much faster. The problem is actually more in the hands of browser developers since they have to satisfy the user's desire to see the same content on mobile vs. desktop.

Opera is focusing on advancing core browser technology in three areas:

Web standards: CSS (webfonts, backgrounds/borders, transitions, transforms), HTML 5 (video/audio, persistent storage, drag and drop) and W3C Geolocation API and "Mobile DOM" API (access to camera, address book, calendars).
Performance: VEGA (vector graphics-based rendering, hardware acceleration) and CARAKAN (new JavaScript engine, native compilation). For more details, see http://my.opera.com/core.
Web applications: Standalone web apps, RIAs, Widgets. Gears support and Developer Tools.

Dan Connolly
The web is kinda important these days. It's a big deal. Make a mistake and 300 million dollars go away (see end of last entry about United news). One of the beauties of the web is you can easily participate as an individual. You can report bugs, write articles and be a part of many web standards groups. Most of the other systems in the world don't provide this kind of access.

Dan has been under a rock for the last 5 years working on Semantic Web stuff. Now that he's back in the game, it's incredible how much stuff is going on. He's glad there's JavaScript frameworks so he doesn't have to learn everything. The default security policies in browsers are a little rickety at this point. They allow you to download and run JavaScript from virtually any site. Caja might help to solve this. Dan believes that security will become more important and stricter to protect web users.

Scott Fegette
Scott is a Product Manager in the Web Group of Adobe. At the beginning of each year, they do heavy user research. Adobe wants people that develop content for the web to be as expressive as possible. Scott is going to give us a peak into the conversations he's had with the web community.

One of the biggest topics on people's minds is The Economy, but it's not negative as you might think. Small web designers are actually getting more business in the downturn, likely because companies are polishing their presence on the web. People are working much more distributed these days. There's a few areas that Adobe generally asks about: CSS, JavaScript, HTML (both statically and dynamically).

Frameworks are becoming more important to developers, as well as with clients. They've even seen some clients demand certain frameworks. Two years ago, when Adobe talked to small design shops and agencies, most web sites were built statically. Now they're developing with frameworks like WordPress. Out of 60 folks they talked to, only 2 were using static systems and not CMSes.

In the JavaScript frameworks arena, jQuery is the dominant leader. Shops are starting to use CSS frameworks as well. The only one Scott mentioned was Reset. Design is becoming a technical discipline and Adobe is calling this Stateful Design. WYSIWIG is definitely dying and designers aren't developing with visual tools.

The kind of projects that people are working on has changed a lot. Many shops are being asked to do work on mobile development. The iPhone has done wonders for the industry in raising the awareness of what a mobile device can do.

The other big investments for Adobe is RIAs and AIR. Ajax has matured enough that it can now compete with proprietary plugins like Flash. The reason for AIR is to allow web developers to use their skills to develop desktop applications. Flash and Flex are often overkill for browser-based applications, but they do often handle video and audio better than Ajax applications.

Mike (TM) Smith
Mike is also known as the "W3C HTML jackass". Mike thinks the state of the web is that it's a mess in a lot of ways. If you don't believe him, ask Doug Crockford. Most of this stuff is going to remain a mess for the next 20 years, unless another genius like Tim Berners-Lee comes along and invents something new. However, the good part about it being a mess is that we all have jobs.

One of the biggest things they're trying to do with HTML 5 is not breaking backward compatibility. Other working groups at the W3C don't share this philosophy, hence the reason they don't have browser vendors participating. Many of the ideas for HTML 5 game from Gears and Ajax Framework developers like John Resig. All this will make things less messy, especially with the help of browser vendors.

Developers like the ubiquitous web and are pushing the mobile web. Mike thinks everyone just needs to get a life (big applause). For mobile, SVG has already been a big success. You will see significant great things with SVN happen in major browsers by next fall. If you're a web developer, you should spend some time experiment with SVG. It will payoff for you. If it doesn't pay off for you and you see Mike next year at Web Directions North, you can punch him in the face.

Location-aware applications will be big as well. Browser vendors are implementing the Geo Location API. It's implemented in Opera, Firefox, WebKit and Gears. Video on the web will be significant as well. The SVG working group pioneered video support into standards, before HTML 5. Many of the problems they face are related to video codecs. The only way to solve the problems with video on the web is with money and lawyers. Very specifically, there's no royalty-free codec for video. This is nothing that standards bodies can solve. The most promising is that Sun Microsystems is developing an open codec and spending money to make sure they're not infringing on patents.

Not only is the HTML Working Group improving markup in HTML, they're also working on coming up with new APIs that give you access to features. If you have ideas that aren't included in HTML 5, the group is definitely interested in hearing about them.

After each panelist talked, John asked them questions about what's the biggest thing they'd like to see implemented by everyone (open video codec, geo location api were the winners). Mike also did some complaining about XML and how broken it is because there's no failure mechanism. There was some audience banter with Chris about SVG in IE.

Conclusion
This was a very interesting session, especially to hear from the people who are building/supporting the future of the web. I liked Scott's talk on what Adobe's hearing from their users. I also liked hearing Mike (TM)'s opinionated thoughts on XML and his non-marketing approach to most everything related to the web. Lars from Opera had a marketing-ish presentation, but it was nevertheless interesting to hear what Opera's working on. Good stuff.

Choosing an Ajax Framework

2009-01-08T21:36:22-07:00

This past week, my colleagues and I have been researching Ajax Frameworks. We're working on a project that's following SOFEA-style architecture principles and we want the best framework for our needs. I'm writing this post to see 1) if you, the community, agree with our selection process and 2) to learn about your experiences with the frameworks we're evaluating. Below is the process we're following to make our choice.

Choose a short list of frameworks to prototype with.
Create an application prototype with each framework.
Document findings and create a matrix with important criteria.
Create presentation to summarize document.
Deliver document, presentation (with demos) and recommendation.

For #1, we chose Ext JS, Dojo, YUI and GWT because we feel these Ajax libraries offer the most UI widgets. We also considered Prototype/Scriptaculous, jQuery and MooTools, but decided against them because of their lack of UI widgets.

For #2, we time-boxed ourselves to 3 days of development. In addition to basic functionality, we added several features (i.e. edit in place, drag and drop, calendar widgets, transitions, charts, grid) that might be used in the production application. We all were able to complete most of the functionality of the application. Of course, there's still some code cleanup as well as styling to make each app look good for the demo. The nice thing about doing this is we're able to look at each others code and see how the same thing is done in each framework. None of us are experts in any of the frameworks, so it's possible we could do things better. However, I think it's good we all started somewhat green because it shows what's possible for someone relatively new to the frameworks.

For #3, we're creating a document with the following outline:

Introduction

Ajax Framework Candidates
(intro and explanation)

  Project Information
  (history)
  (license / cost)
  (number of committers)
  (support options)
  (mailing list traffic (nov/dec 2008))

Matrix and Notes

Conclusion

For the Matrix referenced in the outline above, we're using a table with weights and ranks:

Weight	Criteria	Dojo	YUI	GWT	Ext JS	Notes
#	Important Criteria for Customer	0..1	0..1	0..1	0..1	Notes about rankings

Our strategy for filling in this matrix:

Customer adjusts the weight for each criteria (removing/adding as needed) so all weights add up to 1.
We rank each framework with 0, .5 or 1 where 0 = doesn't satisfy criteria, .5 = partially satisfies, 1 = satisfies.

The list of criteria provided to us by our client is as follows (in no particular order).

Quality of Documentation/Tutorials/Self Help
Browser support (most important browsers/versions based on web stats)
Testability (esp. Selenium compatibility)
Licensing
Project health/adoption
Performance
Scalability
Flexibility/extensibility
Productivity (app dev, web dev)
Richness of widget/component library
Charting capability
Ability to create new widgets
Match to existing Java team skill-set
Ease of deployment (on Ops, QA, Users)
Degree of risk generally
Ability to integrate with existing site (which includes Prototype)
Easy to style with CSS
Validation (esp. marking form elements invalid)
Component Theme-ing/Decoration
CDN Availability (i.e. Google's Ajax Libraries API or Ext CDN)

What do you think? How could this process be improved? Of course, if you have framework answers (0, .5 or 1) for our matrix, we'd love to hear your opinions.