Search for [mvc] in weblog rd

2018 - A Year in Review

2019-01-29T13:25:24-07:00

In 2018, I spoke at several fantastic meetups, traveled a bit overseas, and enjoyed some fun family vacations. We had the time of our lives driving Hefe in the Denver St. Paddy's Day Parade!

We traveled to Crete for JCrete and had a blast with the Java community. We journeyed to Ireland with my folks, and I thoroughly enjoyed a speaking tour of Ireland JUGs and the Dublin JHipster Meetup. Our classic VWs had a great year with only minor repairs needed.

I'm going to look back on 2018 using the following categories.

Professional

For those stumbling upon this post with no context, I'm a veteran open source developer that works at Okta as a developer advocate.

This year I focused on blogging more than speaking. I still spoke a fair bit, but I tried to focus on meetups more than conferences. The DevEx Team at Okta released a slew of SDKs at the end of 2017, so I had plenty to write about. Over the year, I found more and more developers had heard of Okta during my talks. I even found users in most audiences! This was quite a change from 2017, so it seems my team's advocacy efforts might be working.

I wrote 29 blog posts throughout the year on the Okta developer blog. We publish two types of blog posts: thought leadership and conversion. Thought leadership posts tend to be about a hot topic while conversion posts are generally tutorials. I wrote a series of posts on PWAs with Ionic + Angular and Spring Boot. The series started in January and ended in July.

I also wrote a number of posts on JHipster, starting with Ionic for JHipster. Then I transitioned to microservices and showed how to build a Spring microservices architecture with Spring Security and OAuth 2.0. I felt this was important, as it's the same basic architecture that JHipster uses.

I published Develop a Microservices Architecture with OAuth 2.0 and JHipster, and had two more JHipster posts about its React and React Native support.

I learned about React and React Native from research I did while writing previous posts.

I stayed current on the latest releases of Angular and Spring Boot because I ❤ them both!

I heard from several developers that they wanted to use Okta with their Java apps, but they weren't using Spring. Or they were using Spring MVC, but not Spring Boot. I wrote Build a Java REST API with Java EE and OIDC to show a few authn/authz techniques: JWT validation, using Spring Security, and using Pac4j.

I had a blast in August and September collaborating with Josh Long about going Full Stack Reactive with Spring WebFlux and React. Josh authored his parts with AsciiDoc and forced me to integrate Asciidoctor support in our blog. It took awhile, but I was thrilled with the result. Now I get to author all my blog posts with AsciiDoc! 🎉

I played with other new technologies in 2018, namely GraphQL and TypeORM, Electron, and Vue with TypeScript.

The Super Epic Advocacy Battle!

I was super motivated to write a bunch of blog posts in June and July because I was having a "most views" contest with Randall Degges. The contest started after I mouthed off that I thought I could outperform him in the waning hours of our Oktane 2018 conference. He accepted my challenge and we competed for two months. He wrote mostly thought leadership posts, I wrote mostly tutorials. Long story short: he won. I had to wear gold pants the week of CodeOne for my lack of winning.

Hanging out at Oracle Code One conf with my buddy @mraible and his amazing MC Hammer pants.

I wonder why he's wearing them? :D

He's about to get onstage to talk about micro services <33333 pic.twitter.com/syoiIeSDMc
— Randall Degges (@rdegges) October 22, 2018

If only I'd published 10 Excellent Ways to Secure Your Spring Boot Application earlier in our contest. I collaborated on it with Simon Maple, and it seemed to be popular.

I wrote a few other thought leadership posts throughout 2018:

Speaking

According to TripIt, I took 26 trips, to 37 cities, in 6 countries. That's three fewer trips than 2017, and I only had 119 days on the road (versus 141). By my count, I spoke at 20 events.

JHipsterConf was especially fun because I took my son, Jack, with me.

I had a wonderful time this week in Paris and at @jhipsterconf. Many thanks to the @java_hipster developers, community, @juliendubois, @deepu105, and @ippontech for making it all possible! #JHipsterConf pic.twitter.com/61ZfRZxg8G
— Matt Raible (@mraible) June 23, 2018

I also attended a few conferences that I did not speak at: Iterate, JCrete, and Monktoberfest. All of these were special in their own ways. JCrete was an unconference in an exotic location (Crete, Greece) and was a fantastic experience. I learned that 1) it's important to rent a car if you bring your family and 2) an unconference is what you make of it. Put a lot into it, and you'll get a lot out of it. As a speaker, I think it's a fantastic conference format because it's an excellent networking atmosphere. You can also learn a lot if there are peers with expertise in the topic you're interested in. Monktoberfest also provided next-level networking.

Community

I was involved in many community activities in 2018: Devoxx4Kids Denver, Denver JUG, Denver Microservices Meetup, and I attended a few DOSUG meetups too.

We had three Devoxx4Kids Denver workshops in 2018:

Thanks to Kaitlyn Hornbuckle, James Sablatura, and Cassandra Chin for teaching these classes.

Finished training Denver Pokémon Trainers at #Devoxx4Kids. Gotta catch them all! @mraible @melissajmckay pic.twitter.com/zQSbIdH7Zb
— Cassandra Chin (@tingsterchin) December 9, 2018

We announced the 2018 Denver JUG Rock Stars at our December meetup. Venkat, David, and Jeff were voted as the group's favorite speakers. Y'all were excellent! Thanks for adding DJUG to your schedules.

At this week's @denverjug, we announced the Denver JUG Rock Stars of 2018. If you want to be entertained, intrigued, & educated: @venkat_s, @dblevins, and @jeffscottbrown are three of our favorite speakers. #denverjug #rockstars #java

Thanks to @oktadev for sponsoring trophies! pic.twitter.com/uTNcDZVCFi
— Matt Raible (@mraible) December 14, 2018

Projects

My projects these days are mostly around JHipster. I supported its OAuth 2.0 and OIDC support as best I could, and released five versions of its Ionic module. I wrote about how to use Ionic with JHipster in late January. I recorded a screencast showing how to get started with JHipster 5 in June.

We released the JHipster Mini-Book version 4.5 in April and version 5.0 in November. I published a Pluralsight course on developing microservices and mobile apps with JHipster in April and released 21-Points Health 5.0 in October.

Personal

This year was a mix of work your ass off and try to have fun at the same time. There were many times when we wished we were home, but the times on the road were grand. I suspect this will be a constant battle for Trish and I. We love to be home, but we love to explore new places. If old friends gather with us at home or in new places, it's bound to be a fantastic evening!

The biggest change was Abbie became a driver. For those parents with kids that have to chaperone endlessly throughout the weeks and weekends, you know this is a BFD! As of December 27, 2018, Abbie's now the captain of the ranch-to-school ship. She even drove in the blowing snow yesterday!

There were many highlights in 2019. My early favorite (that makes me tear up as I write this) is Hefe 3.5.

My daughter observed the other day that most people are frowning when they’re driving. I’m not one of this people. In fact, I have to wear sunglasses to hide my tears of joy! 🤣 #HefeTheBus #21windows pic.twitter.com/0LWaQkjZlx
— Matt Raible (@mraible) March 14, 2018

We had a blast with Trish's parents during Spring Break in Florida and squeezed in a day trip to Disney World.

The 2017-18 ski season didn't bring a lot of snow, but our ski trip to Utah was wonderful!

Trish and I celebrated our 5th anniversary at JCrete. I hope the next five years are as adventurous as the last five!

Happy Friday! We’re having a blast celebrating our 5th anniversary in Greece. 😃 #JCrete pic.twitter.com/yMzphpNhnj
— Matt Raible (@mraible) July 27, 2018

Jack turned 14, and Abbie turned 16. I'm proud to say they're both fabulous humans. 😊

If you want a glimpse of my backwoods childhood (without electricity) in Montana, see my Life Update: Thanksgiving in Montana, RWX2018, Devoxx4Kids, DJUG, and Trish's Birthday post.

2019

Professionally, I'm planning to travel a bit more to exotic locations. Atlanta, San Francisco, Paris, Bangalore, Barcelona, and Kansas City are on my list (before August). The Developer Relations team at Okta is a fantastic place to be. As many of y'all know, the best thing about a job is often your boss. Mine is Randall Degges. He's a legitimate badass hacker. Don't try to win a popularity contest with him. There's a good chance I'll be at Okta as long as he's my boss.

On the developer relations team, we set our own schedules, plan our own interests, schedule our own conference appearances, and sponsor our favorite local events. Being a developer advocate is a gratifying job. You can write about your favorite open source frameworks, attend/sponsor local meetups, and learn new things every day! If you happen to be #devrel folk (or a like-minded awesomeperson), you might like my Pro Tips for Developer Relations.

Personally, I'd like to ski, raft, bike, and have a good time in 2019 a lot more! Y'all know I won't have any problem with the last one, but its predecessors are the ones that really make me smile. Skiing down a chute, rafting through a class IV, and biking in Moab are some of the most exhilarating activities I've ever experienced. I'm getting a bit older (44) these days, but I still love playing in the outdoors. Luckily, I married a magical woman that loves the outdoors as much as I do. Even better, she photographs its excellence!

Abbie is the school-commute driver, but Jack will be 15 in August, and he's determined to get his learners permit as soon as he can. That means he'll need a family member in the car that's 21+. If he wants to drive, the 20-minute-each-way commute is back on in the fall. For those of you with children of similar ages, you know that this is a precious time in our lives. I don't have a work commute. Driving my kids to school takes an hour round-trip. If it were a commute to work, I'd hate it. However, it's precious time with my kids, and it's really quite fabulous. Even if they're staring at their phones, you have their full attention. Ask questions and enjoy their responses.

Last year, I said one of my goals was to simply be content. This year, I'm taking it up a notch. If you don't have goals, how can you measure how you improved your life? I'm a big believer in self-improvement, so here goes (in order of priority):

Personal

Good Blood Pressure
More 2 and 3 point days
Ride/ski/hoops 4x week
Walk daily (preferably at home with Sagan and Daisy)
More salads, less red meat
Love is a verb, embrace 1:1 time with everyone (Trish, kids, parents, friends)
20 days camping
25 days skiing
15 days rafting

I'm a big fan of the last three. Those activities typically involve a break from technology, the great outdoors, and exquisite views. Follow me on Instagram if you want a highlight reel.

I have some work-related goals too, because metrics seem to motivate me.

Work

Author/publish two blog posts per month
Review/mentor four blog posts per month
Record/publish two videos per month
One performance/speaking gig per month
Three "no travel" months
Stand more while working

I believe 2019 will be fabulous, and I hope to reflect my favorite moments on this blog. If you're a long-time follower, you'll notice I'm mostly active @mraible. Y'all use Twitter, right? That's my modern day notification channel. When I'm super giddy, I'll post pictures of my favorite moments on Instagram. When I want to save them forever, I publish them to Flickr.

I hope to see you at a conference or meetup in 2019!

View this post on Instagram

Fun family photo in Crested Butte. #lastbestskitown

A post shared by Matt Raible (@vwsforlife) on Jan 6, 2018 at 4:27pm PST

Angular Summit 2015

2015-10-01T10:29:31-06:00

I was in Boston this week, speaking and attending the very first Angular Summit. I had the privilege of delivering the opening keynote on Monday. I spoke about the Art of Angular and used a slide deck similar to last time. I did update the presentation to show the astronomical growth of AngularJS in terms of candidate skills (on LinkedIn) and job opportunities (on Dice.com)¹.

I mentioned the recently announced good news for Angular 2:

We're enabling mixing of Angular 1 and Angular 2 in the same application.
You can mix Angular 1 and Angular 2 components in the same view.
Angular 1 and Angular 2 can inject services across frameworks.
Data binding works across frameworks.

In related news, Craig Doremus recently posted a state-geo-angular project that shows how you can develop an Angular 1.x application that will be easy to upgrade to Angular 2.x. Thanks Craig!

Download | SlideShare

After my keynote, I attended Pratik Patel's session on High Performance JavaScript Web Apps. Pratik pointed out mobitest.akamai.com for testing an app's performance and seeing its blocking resources. He also mentioned speedgun.io (currently unavailable) for capturing performance numbers as part of a continuous integration process. Finally, he recommended Addy Somani's JavaScript Memory Management Masterclass.

My second presentation was about JHipster. Near the end of the presentation, I mentioned that I hope to finish the JHipster Book this month. Writing presentations for SpringOne 2GX and the Angular Summit occupied a lot of my free time in September. Now that it's October, I'll be dedicating my free time to finishing the book. In fact, I think I can finish the rough draft this week!

Download | SlideShare

For the last session of the day, I attended John Lindquist's session on Angular 2 Components. John showed us how everything is a component in Angular 2. He also said "now is the time to learn ES6" and built an Angular 2 ToDo App using ES6 and a bit of TypeScript. You might recognize John's name; he's the founder of egghead.io, an excellent site for learning Angular with bite-sized videos.

Tuesday morning started with a Angular 2.0 keynote from Peter Pavlovich. I really enjoyed this session and received lots of good tips about getting ready for Angular 2. The tweet below from Ksenia Dmitrieva shows his advice.

Best Practices for #angularjs 1.X if you plan to switch to 2.0 by @ppavlovich #AngularSummit pic.twitter.com/9nobqDc9G9
— Ksenia Dmitrieva (@KseniaDmitrieva) September 29, 2015

My biggest takeaway was to start following John Papa's Angular Style Guide ASAP.

The first session I attended on Tuesday was Judd Flamm's Google Material Design & Angular. I'm using Material Design for Bootstrap on a side project, so I was interested in learning more about its inspiration. We learned that Google Design has everything you need to know about why Material Design exists. We also learned about Angular Material and spent most of the session looking at its components. Judd recommended Angular Material-Start for those looking to get started quickly with both frameworks. Judd was a very entertaining speaker; I highly recommend you attend one of his talks if you get the opportunity.

After being dazzled by Peter's knowledge of Angular 2 in Tuesday's keynote, I attended two more of his talks: one on Meteor and another on Aurelia. I've known about Meteor for a while, but have become more intrigued by it lately with its 1.2 release and Angular support. Meteor's command line tools that auto-inject CSS and JS demoed very well, as did it's installable features like a LESS support and Facebook authentication.

After hearing all the good things about Angular 2 from Peter, it was interesting to hear him downplay it in his Aurelia talk later that day. When he started showing code, it was pretty obvious that Aurelia is doing a great job of simplifying JavaScript MVC syntax for developers. You can develop components with almost half the code that Angular 2 requires, and it uses ES6, jspm and SystemJS. If you're developing JavaScript, learning these tools will help prepare you for the future. It's cool that Aurelia encourages learning things you should learn anyway.

Aurelia and Angular 2 are both still in Alpha, so I'm not sure it makes sense to use them on a project this year. However, I do think it's important to track them both. I especially think it's interesting that the founder of Aurelia, Rob Eisenberg, left the Angular Team in November 2014 and announced Aurelia in January 2015 (Hacker News thread). Peter mentioned several times that Aurelia wants to help developers write apps, while AngularJS is more tied to helping Google write apps.

There were around 400 people at Angular Summit, which I think is pretty good for a first-run conference. As with most No Fluff Just Stuff shows, it ran smoothly, had plenty of time between sessions and was filled with knowledgeable, entertaining speakers. It was fun doing my first keynote and I look forward to speaking again in November (at Devoxx) and December (at The Rich Web Experience).

1. I know Dice.com is probably not a great site, but it makes sense to use it since I've been tracking JavaScript MVC framework job stats on it since February 2014.

SpringOne 2GX 2015: My Presentations on Comparing Hot JavaScript Frameworks and NoXML

2015-09-20T12:29:00-06:00

Last week, I had the pleasure of traveling to Washington, DC to speak at the annual SpringOne 2GX conference. I was pretty stressed for the last few weeks because I had to create two new presentations from scratch, and both had to be 90 minutes long. I was also hoping to finish the JHipster Book before the conference started. I was able to finish both presentations in the nick of time, but did not find the time to write the last chapter in the JHipster Book.

The first presentation was titled Comparing Hot JavaScript Frameworks: AngularJS, Ember.js and React.js. I started by revisiting the Comparing JVM Web Frameworks talk I did at vJUG last February. I explained how I think traditional web frameworks are no longer relevant in 2015, but I do believe server-side rendering is still very relevant. From there, I used Yevgeniy Brikman’s framework scorecard (from his Node.js vs. Play Framework presentation) to rank each framework by a number of different criteria. You can see the final results on slide 160. Since the scores were so close, I believe you could tweak some scores a bit (or add weights to the different criteria) and make any of the frameworks come out on top.

You can click through the presentation below, download it from my presentations page, or see it on SlideShare.

I started writing the second presentation a week before I had to deliver it. On Thursday, September 10th, I stayed up late, trying to figure out how to create a good presentation on NoXML and finish the last part of the JHipster Book. Then it came to me, I needed to parallelize and do them both at the same time. I decided to compare AppFuse (which is similar to a legacy Spring application with lots of XML) to JHipster (which hardly contains any XML).

I wrote a 10-page Google Doc on how I planned to do this, then went rafting and camping with my family for the weekend. I finished most of the presentation on Monday night, but then realized the presentation wouldn't be long enough to fill 90 minutes. So I hunkered down at midnight, created a new AppFuse application and removed a bunch of its XML. This took me until 3:30am, and I was able to accomplish the following tasks:

Spring XML to Java
Spring Security Configuration to Java
web.xml to WebApplicationInitializer
Spring MVC to Java
Migrated to Spring Boot
Maven to Groovy

I was pretty pumped when I completed my final goal: converting to Spring Boot and getting a test to pass. I made commits to an appfuse-noxml project on GitHub as I accomplished each step. You can see all the changes in the project's commit log. While I'd figured everything out, I still needed to complete the presentation. Luckily, I found time to do this the night before, the morning of, and in the final hour before I had to deliver the talk. You can imagine my relief when I was done delivering both talks.

You can click through the presentation below, download it from my presentations page, or view it on SlideShare.

While I didn't get to spend much time at the conference, I did have a lot of fun while I was there. I got to meet some new folks, reconnect with old friends, and enjoy beers and dinner with a smiling crew on Thursday night. The Broncos victory late that night was the icing on the cake.

Getting Hip with JHipster at Denver's Java User Group

2015-04-09T08:31:54-06:00

Last night, I had the pleasure of speaking at Denver's Java User Group Meetup about JHipster. I've been a big fan of JHipster ever since I started using it last fall. I developed a quick prototype for a client and wrote about solving some issues I had with it on OS X. I like the project because it encapsulates the primary open source tools I've been using for the last couple of years: Spring Boot, AngularJS and Bootstrap. I also wrote about its 2.0 release on InfoQ in January.

To add some humor to my talk, I showed up as a well-dressed Java Developer. Like a mature gentleman might do, I started the evening with a glass of scotch (Glenlivet 12). Throughout the talk I became more hip and adjusted my attire, and beverage, accordingly. As you might expect, my demos had failures. The initial project creation stalled during Bower's download all JavaScript dependencies. Luckily, I had a backup and was able to proceed. Towards the end, when I tried to deploy to Heroku, I was presented with a lovely message that "Heroku toolbelt updating, please try again later". I guess auto-updating has its downsides.

After finishing the demo, I cracked open a cold PBR to ease my frustration.

I did two live coding sessions during this presentation; standing on the shoulders of giants to do so. I modeled Josh Long's Getting Started with Spring Boot to create a quick introduction to Spring Boot. IntelliJ IDEA 14.1 has a nice way to create Spring Boot projects, so that came in handy. For the JHipster portion, I created a blogging app and used relationships and business logic similar to what Julien Dubois did in his JHipster for Spring Boot Webinar. Watching Josh and Julien's demos will give you a similar experience to what DJUG attendees experienced last night, without the download/deployment failures.

You can click through my presentation below, download it from my presentations page, or view it on SlideShare.

You might notice my announcement on slide #32 that I've signed up to write a book on JHipster.

I haven't started writing the book yet, but I have been talking with InfoQ and other folks about it for several months. I plan to use Asciidoctor and Gradle as my authoring tools. If you have experience writing a book with these tools, I'd love to hear about it. If you've developed an application with JHipster and have some experience in the trenches, I'd love to hear your stories too.

As I told DJUG last night, I plan to be done with the book in a few months. However, if you've been a reader of this blog, you'll know I've been planning to be done with my '66 VW Bus in just a few more months for quite some time, so that phrase has an interesting meaning for me.

The Art of AngularJS in 2015

2015-02-04T09:14:57-07:00

I've been tracking statistics on jobs and skills for JavaScript MVC frameworks ever since I Compared JVM Web Frameworks at Devoxx France in 2013. At that time, Backbone was the dominant framework.

Last year, I updated those statistics for a presentation on AngularJS at Denver's Derailed. Angular had a similar amount of jobs as Backbone and a lot of people added it to their LinkedIn profiles. I found that Ember had grown around 300%, Backbone 200% and Angular 1000%!

Before presenting on AngularJS at last night's Denver Open Source Users Group, I updated these statistics once again. The charts below show how the number of jobs for Angular has doubled in the last year, while jobs for Ember and Backbone have fallen slightly. As far as skills, developers learning Ember and Backbone has increased 200%, while skilled Angular folks has risen 400%.

Yes, AngularJS has experienced huge growth in the last couple of years. You might even say it's the Struts of the JavaScript world. I like to say that Struts 1.x was the 'Killer App' for J2EE back in the day.

For the presentation I delivered last night, I made a number of improvements over last year's. I added a live coding demo based on my Getting Started with AngularJS tutorial. I used IntelliJ's live templates to make it look easy. However, since the audience was quiet, and some were falling asleep, I skipped over the testing demo.

I added a few slides on Foundation for Apps (FA). We've selected AngularJS and Foundation on my current project, and I've been researching how to integrate the two lately. FA is one solution I've found, Angular Foundation is another. If you know of others, please let me know.

For Java developers getting started with Angular, I recommended JHipster. I talked about its foundational frameworks and project options when creating your project. I included screenshots of its slick metrics UI and code generation features.

I also added a slide for Dave Syer's excellent five-part series on Spring and AngularJS:

Finally, I added a number of slides on Angular 2.0. I encouraged folks to checkout The Twelve-Factor App and James Ward's Java Doesn’t Suck – You’re Just Using it Wrong.

The discussion with the audience was great, particularly around HTTP/2 and minification/concatenation of assets. Thanks to all who attended, I really enjoyed having the opportunity to share what I've learned.

You can click through my presentation below, download it from my presentations page, or view it on SlideShare.

If you live in Denver, there's a number of interesting meetups happening in the next couple months.

Monday, February 16: Introduction to ReactJS at HTML5 Denver
Tuesday & Wednesday, March 3rd and 4th: ThingMonk at a distillery!
Wednesday, April 8: I'll be speaking about JHipster at Denver Java User Group

I'm also looking for speakers to teach programming to kids at Devoxx4Kids Denver. Let me know if you have a fun topic you'd like to present.

2014 - A Year in Review

2015-01-31T09:37:43-07:00

2014 was destined to be a spectacular year. When I wrote my thoughts down last January, I thought the Broncos would win the Super Bowl and my VW Bus restoration project would be finished by summer. To focus on finishing the bus project, I didn't submit any talks to conferences. Instead of traveling to exotic locations, we opted to visit a bunch in our own backyard instead.

I should've known it'd be an interesting year when the Broncos flopped in the Super Bowl.

For this Year in Review post, I'll use the format I've used the last couple of years.

Professional

I had three different clients throughout the year, all in the health care industry. I continued working on a project with John Muir Health where we developed a hybrid mobile app. I wrote about what I learned in Documenting your Spring API with Swagger and Developing an iOS Native App with Ionic. Since I wrote the article about Swagger's Spring MVC support, it has become much easier to integrate; now it only requires an @EnableSwagger annotation.

In April, I started consulting with a small company in Alabama. I helped them modernize their tech stack and implemented a number of web services with Apache Camel. I wrote about this in a four-part series in September and October.

I continued leveraging Spring Boot in my client projects and wrote about Building a REST API with JAXB, Spring Boot and Spring Data at the end of that engagement.

In November, I had time off between clients. I visited my parents and their awesome retirement home in Montana. I also took some time to refactor AppFuse to be a bit leaner and meaner. I signed up with a new health care client in December. It's been an enjoyable experience so far, especially since I'm only working 20 hours per week this quarter.

I'm currently booked with clients through the end of March, but hoping to find one with a challenging project in April.

Speaking

To optimize cash flow towards the bus project, I didn't submit any talks to conferences in 2014. However, I did talk about Comparing JVM Web Frameworks at the Denver Open Source Users Group and vJUG in February. I also spoke at local user groups (DeRailed and HTML5 Users) about The Art of AngularJS in the first part of the year. In the fall, I participated in panels on unit testing at Denver Startup Week and building a personal brand at AirConf.

I hope to attend and speak at more conferences in 2015. To help facilitate this, I've published a list of talks I'd like to present.

Devoxx4Kids Denver

2014 was the inaugural year for Devoxx4Kids Denver. In April, I announced our first meetup. It was an Introduction to Server-Side Minecraft Programming, taught by Denver's own Scott Davis. It was a wild success and the kids had a great time hacking.

I didn't plan any events during the summer, but we held a couple workshops in the fall: one on littleBits and a second on Greenfoot. The kids had a lot of fun with littleBits thanks to Juan Sanchez of Tack Mobile.

Unfortunately, I wasn't able to attend the Greenfoot workshop with Steve Pousty, but I heard it was awesome. Thanks again to Dan Allen for assisting with this class.

I haven't scheduled any Devoxx4Kids Denver meetups for 2015 yet. If you're interested in speaking, please let me know!

Projects

AppFuse: I didn't have many posts about AppFuse in 2014, but I did work on it from time to time. Upgrading its dependencies was a good way to stay up-to-date with open source frameworks, in addition to lots of reading. InfoQ covered the AppFuse 3.0 release and I continue to write for them.

In December, I removed 10,000 lines of XML from the project without affecting any functionality. In the past month, I've added support for generating CRUD for all its web frameworks (including AppFuse Light's). This includes Spring + FreeMarker, Stripes and Wicket. GWT support is still in progress. I hope to do a release soon; there's only 10 issues left as of this writing.

Roller: I didn't contribute any code to Apache Roller in 2014, but I upgraded this site to the latest release, upgraded to Java 8 + Tomcat 7, and integrated Wufoo Forms. I hope to make some scalability improvements to how I've deployed Roller in the near future.

Personal

I started off the year with a trip of a lifetime: Heli Skiing in British Columbia with CMH Gothics. An excerpt from my blog post:

Everyday was awesome, but Wednesday was incredible. It snowed 10 cm on Tuesday night, which turned into knee-deep powder and face shots. The hoots and hollers from everyone floating down forced my face into a sorta perma-grin. For many of us, it was the best skiing day of our entire lives. The video below tries to capture how much fun we had throughout the week, but ultimately fails. It's one of those things you have to experience to really appreciate. I ended up skiing around 80,000 vertical feet in four days.

After returning from British Columbia, Trish and I skied Breckenridge and Copper before taking the kids to Aspen for three feet of powder on Super Bowl weekend.

We skied all over Colorado in February and March. We even did some winter camping in the Syncro for Valentine's Day. It snowed a lot that night.

I wrote about all our skiing adventures in Farewell to the 2013-2014 Ski Season.

On April 10th, I reminisced that 10 years ago today, I bought a VW Bus. In that post, I explained some body-work quality issues the current restoration shop found. Nevertheless, I still thought it would be done in just a few more months.

We journeyed to Florida with Abbie and Jack for Spring Break. We relished in a super-fun family vacation where we golfed, swam and even spent a day frolicking with their Mom and fiancé (now husband), Dave. We're pretty proud that we're all still a family even though Julie and I are divorced. I wrote about our vacation in Spring Break in Florida: Golf, Beaches and Boats!

The hospitality from Trish's parents was like no other. I especially enjoyed a fishing trip with her Dad Joe in the Everglades. I love these people!

Our winter activities in the Syncro were incredible. Trish sold her Nissan Xterra (the one with the sick bass) in December. Yes, that's right - we only had one vehicle during ski season last year. The fact that it worked so well is amazing. In May, we kicked it up a knotch and drove it to Moab for Syncro Solstice. My parents went with us and we experienced fabulous weather and met a bunch of super-cool Vanagon owners.

We made record time from Moab back to Denver (5 hours). The next day is when things started to go downhill. I washed the engine for the VWs on the Green show. On that Monday, the engine started "lurching" where it seemed like it wasn't getting gas (it turned out to be caused by a lack of air). We drove the lurcher for several weeks, journeying to Campout for the Cause and a exhilarating raft trip down the Yampa River.

We had the lurching examined in mid-June by Rocky Mountain Westy; we also had them install some bumpers (for a better trailer hitch) and refreshed the tires.

The ol' van looked pretty good when our Dads arrived in Denver for some whitewater on the Arkansas.

In July, we skipped the annual road trip to The Cabin in favor of a romantic road trip to see old friends in Santa Fe. The Syncro's engine seized on that trip and I experienced my favorite breakdown ever. The engine froze at a gas station near our Ski Shack and I was able to walk home. Luckily, Trish's Dad had recently (over Father's Day weekend) gifted his Nissan Maxima to us. After a train ride to fetch our other car, our road trip continued and we celebrated our first anniversary at The Broadmoor.

After we returned from Santa Fe, I turned 40 years old. We rocked out a Journey concert to celebrate and Trish surprised me with a birthday visit from my kick-ass sister. The following evening, a bunch of friends took me on a Craft Brewery Tour in a school/prison bus. Both nights were epic.

At the end of July, Trish had one of her favorite photos featured on the cover of Whisky Magazine.

In August, we flew to Boston to celebrate Trish's Dad's 75th birthday. My parents joined us, we visited my Dad's family in New York and took the kids to the Big Apple. They loved the Statue of Liberty.

Abbie started middle school (6th grade) in mid-August and Jack started 4th grade. They looked great on their first day of school.

We were pumped about our new engine in the Syncro when we left Denver for Rafting the Green River through Desolation Canyon. We celebrated Jack's 10th birthday on that trip and had a very memorable expedition with some really cool people.

Trish's photo from our last night on the river makes me want do it all again.

In October, I started considering health over wealth when we embarked on a 21-day Sugar Detox. I'm proud to say that, as a result, I'm off blood pressure medication and continue to lose weight and feel better. We're cooking at home more than ever, exercising daily and I've lost almost 20 pounds in the process.

Medallion, the horse that Trish and Abbie were leasing (and almost ready to buy) got sick with colic on Friday, October 10. I remembered the day without looking at a calendar because Trish called me to tell me about his illness. She was out of town on a business trip and was absolutely distraught. She told me about the situation and asked me to visit Medallion while he endured extreme pain. I worked out of my van at the stables that day, checking on him every couple hours. They had to put him down that evening. Abbie took it pretty well when I told her a couple hours later, but there were heaps of tears later that night.

Our little girl turned 12 a few weeks later. The kids put on some dark costumes for Halloween.

A week later, I visited my parents in Montana. We played cribbage, ate scrumptious meals and trounced around the woods looking for bucks. I marveled at the beauty of my parents' retirement home.

I neglected to blog or tweet about my self-caused Syncro accidents in Q4 of 2014. Long story short: it's been getting body work done for quite some time (two accidents!) and I look forward to getting it back in tip-top-shape next week. Thank goodness for USAA insurance.

The biggest event of the year happened just before Christmas: THE BUS IS PAINTED!! HOLY CHRISTMAS PRESENT BATMAN!

2015

It's hard to believe that 2015 could possibly top 2014. I started the year heli-skiing and ended it with a painted bus. However, the bus isn't done yet. Last year, I wrote:

My goal for 2014 is singular: finish The Bus.

I continue that mission this year. I've negotiated a deal with ReinCARnation to have it done by April 1st. Yes, it seems strange that April Fools is the finished bus delivery date. However, I'm confident they'll make it happen. This could be the year I've dreamed about since I was in high school. It was in 1991 that I first fantasized about putting a 911 engine in a VW Bus.

This year, I hope to be cruise around Colorado in two running VW Busses. From there, I plan to surround myself with the laughter of our children, the enthusiasm of our border collies and the warmness of Trish's love. It's gonna be a really great year!

Getting Started with AngularJS

2015-01-29T11:12:38-07:00

I was hired by my current client in November to help them choose a technology stack for developing modern web applications. In our first sprint, we decided to look at JavaScript MVC frameworks. I suggested AngularJS, Ember.js and React. Since most of the team was new to JavaScript MVC, I decided to create a tutorial for them. I tried to make it easy so they could learn how to write a simple web application with AngularJS. I thought others could benefit from this article as well, so I asked (and received) permission from my client to publish it here.

What you'll build

You'll build a simple web application with AngularJS. You'll also add search and edit features with mock data.

What you'll need

About 15-30 minutes
A favorite text editor or IDE. I recommend IntelliJ IDEA.
Git installed.
Node.js and NPM installed.

Create a simple web application

Clone the angular-seed repository using git:

git clone https://github.com/angular/angular-seed.git angular-tutorial
cd angular-tutorial

There are two kinds of dependencies in this project: tools and angular framework code. The tools help manage and test the application.
- To get the tools that depend upon via npm, the node package manager.
- To get the angular code via bower, a client-side code package manager.
The project has preconfigured npm to automatically run bower so you can simply do:
```
npm install
```

Run the application

The project is configured with a simple development web server. The simplest way to start this server is:

npm start

Now browse to the app at http://localhost:8000/app/.

Add a search feature

To add a search feature, open the project in an IDE or your favorite text editor. For IntelliJ IDEA, use File > New Project > Static Web and point to the directory you cloned angular-seed to.

The Basics

Create an app/search/index.html file with the following HTML:

<form ng-submit="search()">
    <input type="search" name="search" ng-model="term">
    <button>Search</button>
</form>

Create app/search/search.js and define the routes (URLs) and controller for the search feature.

angular.module('myApp.search', ['ngRoute'])

    .config(['$routeProvider', function ($routeProvider) {
        $routeProvider
            .when('/search', {
                templateUrl: 'search/index.html',
                controller: 'SearchController'
            })
    }])

    .controller('SearchController', function () {
        console.log("In Search Controller...");
    });

Modify app/app.js and add the "myApp.search" module you created above.

angular.module('myApp', [
  'ngRoute',
  'myApp.view1',
  'myApp.view2',
  'myApp.version',
  'myApp.search'
])

Modify app/index.html and add a link to the search.js file.

<script src="view2/view2.js"></script>
<script src="search/search.js"></script>
<script src="components/version/version.js"></script>

Refresh your browser and navigate to http://localhost:8000/app/#/search. You should see an input field and search button. You should also see a log message printed in your browser's console. In Chrome, you can view the console using View > Developer > JavaScript Console. You can make it easier to navigate to this page by adding a menu item in app/index.html.
```
<ul class="menu">
  <li><a href="#/view1">view1</a></li>
  <li><a href="#/view2">view2</a></li>
  <li><a href="#/search">search</a></li>
</ul>
```

This section has shown you how to add a new controller and view to a basic AngularJS application. This was fairly simple to create. The next section shows you how to create a fake backend API.

The Backend

To get search results, you're going to create a SearchService that makes HTTP requests. These HTTP requests will be handled by a mock backend using some of Angular's built-in mocking tools. The backend implementation was created using AngularJS Backend-less Development Using a $httpBackend Mock.

Add a SearchService to app/search/search.js. This is done in Angular using its Factory Recipe. Make sure to remove the semicolon from the .controller code block.

.controller('SearchController', function () {
    console.log("In Search Controller...");
})

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        }
    };
    return service;
});

Inject the SearchService into the SearchController and use it to get results from the backend. The form in app/search/index.html calls the search() function. The "term" is defined by the input field in this page with ng-model="term".

.controller('SearchController', function ($scope, SearchService) {
    $scope.search = function () {
        console.log("Search term is: " + $scope.term);
        SearchService.query($scope.term).then(function (response) {
            $scope.searchResults = response.data;
        });
    };
})

If you try to search for a "foo" term now, you'll see the following error in your console.

Search term is: foo
GET http://localhost:8000/search/foo 404 (Not Found)

Create app/search/mock-api.js for the fake backend. Populate it with the following JavaScript.

// We will be using backend-less development
// $http uses $httpBackend to make its calls to the server
// $resource uses $http, so it uses $httpBackend too
// We will mock $httpBackend, capturing routes and returning data
angular.module('myApp')
    .service('ServerDataModel', function ServerDataModel() {
        this.data = [
            {
                id: 1,
                name: "Peyton Manning",
                phone: "(303) 567-8910",
                address: {
                    street: "1234 Main Street",
                    city: "Greenwood Village",
                    state: "CO",
                    zip: "80111"
                }
            },
            {
                id: 2,
                name: "Demaryius Thomas",
                phone: "(720) 213-9876",
                address: {
                    street: "5555 Marion Street",
                    city: "Denver",
                    state: "CO",
                    zip: "80202"
                }
            },
            {
                id: 3,
                name: "Von Miller",
                phone: "(917) 323-2333",
                address: {
                    street: "14 Mountain Way",
                    city: "Vail",
                    state: "CO",
                    zip: "81657"
                }
            }
        ];

        this.getData = function () {
            return this.data;
        };

        this.search = function (term) {
            if (term == "" || term == "*") {
                return this.getData();
            }
            // find the name that matches the term
            var list = $.grep(this.getData(), function (element, index) {
                term = term.toLowerCase();
                return (element.name.toLowerCase().match(term));
            });

            if (list.length === 0) {
                return [];
            } else {
                return list;
            }
        };
    })
    .run(function ($httpBackend, ServerDataModel) {

        $httpBackend.whenGET(/\/search\/\w+/).respond(function (method, url, data) {
            // parse the matching URL to pull out the term (/search/:term)
            var term = url.split('/')[2];

            var results = ServerDataModel.search(term);

            return [200, results, {Location: '/search/' + term}];
        });

        $httpBackend.whenGET(/search\/index.html/).passThrough();
        $httpBackend.whenGET(/view/).passThrough();
    });

This file uses jQuery.grep(), so you'll need to install jQuery. Modify bower.json and add jQuery to the list of dependencies.
```
"dependencies": {
  ...
  "html5-boilerplate": "~4.3.0",
  "jquery": "~1.10.x"
}
```
Stop the npm process, run "npm install" to download and install jQuery, then "npm start" again.

Modify app/app.js and add a dependency on ngMockE2E.

angular.module('myApp', ['ngMockE2E',
  'ngRoute',
  ...

Modify app/index.html to include references to jquery.js, angular-mocks.js and mock-api.js.

<script src="bower_components/angular-route/angular-route.js"></script>
<script src="bower_components/angular-mocks/angular-mocks.js"></script>
<script src="bower_components/jquery/jquery.js"></script>
...
<script src="search/mock-api.js"></script>
<script src="components/version/version.js"></script>

Add the following HTML to app/search/index.html to display the search results.
```
<div>
    <pre>{{ searchResults | json}}</pre>
</div>
```
After making this change, you should be able to search for "p", "d" or "v" and see results as JSON.

To make the results more readable, change the above HTML to use a <table> and Angular's ng-repeat directive.

<table ng-show="searchResults.length" style="width: 100%">
    <thead>
    <tr>
        <th>Name</th>
        <th>Phone</th>
        <th>Address</th>
    </tr>
    </thead>
    <tbody>
    <tr ng-repeat="person in searchResults">
        <td>{{person.name}}</td>
        <td>{{person.phone}}</td>
        <td>{{person.address.street}}<br/>
            {{person.address.city}}, {{person.address.state}} {{person.address.zip}}
        </td>
    </tr>
    </tbody>
</table>

This section has shown you how to fetch and display search results. The next section builds on this and shows how to edit and save a record.

Add an Edit Feature

Modify app/search/index.html to add a link for editing a person.

<table ng-show="searchResults.length" style="width: 100%">
    ...
    <tr ng-repeat="person in searchResults">
        <td><a href="" ng-click="edit(person)">{{person.name}}</a></td>
        ...
    </tr>
</table>

Add an edit() function to SearchController. Notice that the $location service dependency has been added to the controller's initialization function.

.controller('SearchController', function ($scope, $location, SearchService) {
    ...

    $scope.edit = function (person) {
        $location.path("/edit/" + person.id);
    }
})

Create a route to handle the edit URL in app/search/search.js.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/edit/:id', {
            templateUrl: 'search/edit.html',
            controller: 'EditController'
        });
}])

Create app/search/edit.html to display an editable form. The HTML below shows how you can use HTML5's data attributes to have valid attributes instead of ng-*.

<form ng-submit="save()">
    <div>
        <label for="name">Name:</label>
        <input type="text" data-ng-model="person.name" id="name">
    </div>
    <div>
        <label for="phone">Phone:</label>
        <input type="text" data-ng-model="person.phone" id="phone">
    </div>
    <fieldset>
        <legend>Address:</legend>
        <address style="margin-left: 50px">
            <input type="text" data-ng-model="person.address.street"><br/>
            <input type="text" data-ng-model="person.address.city">,
            <input type="text" data-ng-model="person.address.state" size="2">
            <input type="text" data-ng-model="person.address.zip" size="5">
        </address>
    </fieldset>
    <div>
        <button type="submit">Save</button>
    </div>
</form>

Modify SearchService to contain functions for finding a person by their id, and saving them.

.factory('SearchService', function ($http) {
    var service = {
        query: function (term) {
            return $http.get('/search/' + term);
        },
        fetch: function (id) {
            return $http.get('/edit/' + id);
        },
        save: function(data) {
            return $http.post('/edit/' + data.id, data);
        }
    };
    return service;
});

Create EditController in app/search/search.js. $routeParams is an Angular service that allows you to grab parameters from a URL.

.controller('EditController', function ($scope, $location, $routeParams, SearchService) {
    SearchService.fetch($routeParams.id).then(function (response) {
        $scope.person = response.data;
    });

    $scope.save = function() {
        SearchService.save($scope.person).then(function(response) {
            $location.path("/search/" + $scope.person.name);
        });
    }
})

At the bottom of app/search/mock-api.js, in its run() function, add the following:

$httpBackend.whenGET(/\/search/).respond(function (method, url, data) {
    var results = ServerDataModel.search("");

    return [200, results];
});

$httpBackend.whenGET(/\/edit\/\d+/).respond(function (method, url, data) {
    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var results = ServerDataModel.find(id);

    return [200, results, {Location: '/edit/' + id}];
});

$httpBackend.whenPOST(/\/edit\/\d+/).respond(function(method, url, data) {
    var params = angular.fromJson(data);

    // parse the matching URL to pull out the id (/edit/:id)
    var id = url.split('/')[2];

    var person = ServerDataModel.update(id, params);

    return [201, person, { Location: '/edit/' + id }];
});

$httpBackend.whenGET(/search\/edit.html/).passThrough();

Further up in the same file, add find() and update() methods to ServerDataModel.

this.getData = function () {
    return this.data;
};

this.search = function (term) {
    ...
};

this.find = function (id) {
    // find the game that matches that id
    var list = $.grep(this.getData(), function (element, index) {
        return (element.id == id);
    });
    if (list.length === 0) {
        return {};
    }
    // even if list contains multiple items, just return first one
    return list[0];
};

this.update = function (id, dataItem) {
    // find the game that matches that id
    var people = this.getData();
    var match = null;
    for (var i = 0; i < people.length; i++) {
        if (people[i].id == id) {
            match = people[i];
            break;
        }
    }
    if (!angular.isObject(match)) {
        return {};
    }
    angular.extend(match, dataItem);
    return match;
};

The <form> in app/search/edit.html calls a save() function to update a person's data. You already implemented this above. The function executes the logic below.

$location.path("/search/" + $scope.person.name);

Since the SearchController doesn't execute a search automatically when you execute this URL, add the following logic to do so in app/search/search.js. Note that $routeParams is added to the list of injected dependencies.

.controller('SearchController', function ($scope, $location, $routeParams, SearchService) {
    if ($routeParams.term) {
        SearchService.query($routeParams.term).then(function (response) {
            $scope.term = $routeParams.term;
            $scope.searchResults = response.data;
        });
    }
    ...
})

You'll also need to add a new route so search/term is recognized.

.config(['$routeProvider', function ($routeProvider) {
    $routeProvider
        .when('/search', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        .when('/search/:term', {
            templateUrl: 'search/index.html',
            controller: 'SearchController'
        })
        ...
}])

After making all these changes, you should be able to refresh your browser and search/edit/update a person's information. If it works - nice job!

Source Code

A completed project with this code in it is available on GitHub at https://github.com/mraible/angular-tutorial.

There are three commits that make the changes for the three main steps in this tutorial:

Extra Credit

Deploy your completed app to Heroku. See running version of this tutorial at https://angular-123.herokuapp.com.

Summary

I hope you've enjoyed this quick-and-easy tutorial on how to get started with AngularJS. In a future tutorial, I'll show you how to write unit tests and integration tests for this application. If you're in Denver next Tuesday, I'll be speaking about AngularJS at Denver's Open Source Users Group.

If you're a Java developer that's interested in developing with AngularJS and Spring Boot, you might want to checkout the recently released JHipster 2.0.

AppFuse, Reduced

2014-12-16T06:03:31-07:00

In November, I had some time off between clients. To occupy my time, I exercised my body and brain a bit. I spent a couple hours a day exercising and a few hours a day working on AppFuse. AppFuse isn't used to start projects nearly as much as it once was. This makes sense since there's been a ton of innovation on the JVM and there's lots of get-started-quickly frameworks now. Among my favorites are Spring Boot, JHipster, Grails and Play.

You can see that AppFuse's community activity has decreased quite a bit over the years by looking at its mailing list traffic.

Even though there's not a lot of users talking on the mailing list, it still seems to get quite a few downloads from Maven Central.

I think the biggest value that AppFuse provides now is a learning tool for those who work on it. Also, it's a good place to show other developers how they can evolve with open source frameworks (e.g. Spring, Hibernate, JSF, Tapestry, Struts) over several years. Showing how we migrated to Spring MVC Test, for example, might be useful. The upcoming move to Spring Data instead of our Generic DAO solution might be interesting as well.

Regardless of whether AppFuse is used a lot or not, it should be easy to maintain. Over the several weeks, I made some opinionated changes and achieved some pretty good progress on simplifying things and making the project easier to maintain. The previous structure has a lot of duplicate versions, properties and plugin configurations between different projects. I was able to leverage Maven's inheritance model to make a number of improvements:

Changed AppFuse's parent to be based on the Spring IO Platform. This project is a dependency manager that defines version numbers for open source projects that work well with Spring.
Defined plugins, their versions and configurations in <pluginManagement>.
Defined dependencies, their versions and exclusions in <dependencyManagement>.
Simplified archetypes so new projects have minimal dependencies. For example, here's a basic project's pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.company</groupId>
    <artifactId>springmvc-project</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>AppFuse Spring MVC Application</name>

    <parent>
        <groupId>org.appfuse</groupId>
        <artifactId>appfuse-web</artifactId>
        <version>3.5.0-SNAPSHOT</version>
    </parent>

    <build>
        <plugins>
            <plugin>
                <groupId>de.juplo</groupId>
                <artifactId>hibernate4-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>dbunit-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.appfuse</groupId>
            <artifactId>appfuse-${web.framework}</artifactId>
            <version>${appfuse.version}</version>
            <type>pom</type>
        </dependency>
    </dependencies>

    <properties>
        <amp.genericCore>true</amp.genericCore>
        <amp.fullSource>false</amp.fullSource>
        <dao.framework>hibernate</dao.framework>
        <db.name>mydatabase</db.name>
        <web.framework>spring</web.framework>

        <!-- Framework/Plugin versions -->
        <appfuse.version>3.5.0-SNAPSHOT</appfuse.version>
        <java.version>1.7</java.version>
    </properties>

    <profiles>
        <profile>
            <id>itest</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>webtest-maven-plugin</artifactId>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>webtest-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </reporting>
</project>

The pull request for these changes says it all:

AppFuse +4,822 -14,369
AppFuse Light +776 -4,687

That's right, I was able to eliminate a good chunk of code without affecting any of AppFuse's functionality¹. I think we can all agree that less code == easier maintenance. This theme will continue as we work on future releases.

Other improvements include migrating all tests to use JUnit4, integrating Spring MVC Test, and configuring the surefire plugin to run tests in parallel. I also The build-helper-maven-plugin is now used to find open ports for Cargo to run and a lot of testing was done to ensure you can build/test multiple AppFuse-derived projects at the same time. Finally, I migrated to the hibernate4-maven-plugin and upgraded to Tapestry 5.4.

In the next version of AppFuse, I plan to remove as much XML as possible - moving all of the configuration to Spring's JavaConfig. We'll also be moving to Java 8 as a minimum. I'm even considering getting rid of all the pom.xml files in favor of another build language that requires less code.

In other words, the upcoming 3.5 release will be the last release that supports Java 7 and uses Spring's XML for configuration. AppFuse 4.0 will strive for #NoXML. The project's roadmap has more details on additional hopes and dreams.

We'd love to hear your feedback on these change. Do you like the simplification theme? Are you OK with having AppFuse as a parent in your projects?

1. For project and code stats, see AppFuse on Open Hub.

Building a REST API with JAXB, Spring Boot and Spring Data

2014-10-29T05:52:37-06:00

If someone asked you to develop a REST API on the JVM, which frameworks would you use? I was recently tasked with such a project. My client asked me to implement a REST API to ingest requests from a 3rd party. The project entailed consuming XML requests, storing the data in a database, then exposing the data to internal application with a JSON endpoint. Finally, it would allow taking in a JSON request and turning it into an XML request back to the 3rd party.

With the recent release of Apache Camel 2.14 and my success using it, I started by copying my Apache Camel / CXF / Spring Boot project and trimming it down to the bare essentials. I whipped together a simple Hello World service using Camel and Spring MVC. I also integrated Swagger into both. Both implementations were pretty easy to create (sample code), but I decided to use Spring MVC. My reasons were simple: its REST support was more mature, I knew it well, and Spring MVC Test makes it easy to test APIs.

Camel's Swagger support without web.xml
As part of the aforementioned spike, I learned out how to configure Camel's REST and Swagger support using Spring's JavaConfig and no web.xml. I made this into a sample project and put it on GitHub as camel-rest-swagger.

This article shows how I built a REST API with Java 8, Spring Boot/MVC, JAXB and Spring Data (JPA and REST components). I stumbled a few times while developing this project, but figured out how to get over all the hurdles. I hope this helps the team that's now maintaining this project (my last day was Friday) and those that are trying to do something similar.

XML to Java with JAXB

The data we needed to ingest from a 3rd party was based on the NCPDP Standards. As a member, we were able to download a number of XSD files, put them in our project and generate Java classes to handle the incoming/outgoing requests. I used the maven-jaxb2-plugin to generate the Java classes.

<plugin>
    <groupId>org.jvnet.jaxb2.maven2</groupId>
    <artifactId>maven-jaxb2-plugin</artifactId>
    <version>0.8.3</version>
    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <args>
                    <arg>-XtoString</arg>
                    <arg>-Xequals</arg>
                    <arg>-XhashCode</arg>
                    <arg>-Xcopyable</arg>
                </args>
                <plugins>
                    <plugin>
                        <groupId>org.jvnet.jaxb2_commons</groupId>
                        <artifactId>jaxb2-basics</artifactId>
                        <version>0.6.4</version>
                    </plugin>
                </plugins>
                <schemaDirectory>src/main/resources/schemas/ncpdp</schemaDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

The first error I ran into was about a property already being defined.

[INFO] --- maven-jaxb2-plugin:0.8.3:generate (default) @ spring-app ---
[ERROR] Error while parsing schema(s).Location [ file:/Users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd{1811,48}].
com.sun.istack.SAXParseException2; systemId: file:/Users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd;
    lineNumber: 1811; columnNumber: 48; Property "MultipleTimingModifierAndTimingAndDuration" is already defined.
    Use <jaxb:property> to resolve this conflict.
at com.sun.tools.xjc.ErrorReceiver.error(ErrorReceiver.java:86)

I was able to workaround this by upgrading to maven-jaxb2-plugin version 0.9.1. I created a controller and stubbed out a response with hard-coded data. I confirmed the incoming XML-to-Java marshalling worked by testing with a sample request provided by our 3rd party customer. I started with a curl command, because it was easy to use and could be run by anyone with the file and curl installed.

curl -X POST -H 'Accept: application/xml' -H 'Content-type: application/xml' \
--data-binary @sample-request.xml http://localhost:8080/api/message -v

This is when I ran into another stumbling block: the response wasn't getting marshalled back to XML correctly. After some research, I found out this was caused by the lack of @XmlRootElement annotations on my generated classes. I posted a question to Stack Overflow titled Returning JAXB-generated elements from Spring Boot Controller. After banging my head against the wall for a couple days, I figured out the solution.

I created a bindings.xjb file in the same directory as my schemas. This causes JAXB to generate @XmlRootElement on classes.

<?xml version="1.0"?>
<jxb:bindings version="1.0"
              xmlns:xsd="http://www.w3.org/2001/XMLSchema"
              xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
              xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://java.sun.com/xml/ns/jaxb http://java.sun.com/xml/ns/jaxb/bindingschema_2_0.xsd">

    <jxb:bindings schemaLocation="transport.xsd" node="/xsd:schema">
        <jxb:globalBindings>
            <xjc:simple/>
        </jxb:globalBindings>
    </jxb:bindings>
</jxb:bindings>

To add namespaces prefixes to the returned XML, I had to modify the maven-jaxb2-plugin to add a couple arguments.

<arg>-extension</arg>
<arg>-Xnamespace-prefix</arg>

And add a dependency:

<dependencies>
    <dependency>
        <groupId>org.jvnet.jaxb2_commons</groupId>
        <artifactId>jaxb2-namespace-prefix</artifactId>
        <version>1.1</version>
    </dependency>
</dependencies>

Then I modified bindings.xjb to include the package and prefix settings. I also moved <xjc:simple/> into a global setting. I eventually had to add prefixes for all schemas and their packages.

<?xml version="1.0"?>
<bindings version="2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://java.sun.com/xml/ns/jaxb"
          xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:namespace="http://jaxb2-commons.dev.java.net/namespace-prefix"
          xsi:schemaLocation="http://java.sun.com/xml/ns/jaxb http://java.sun.com/xml/ns/jaxb/bindingschema_2_0.xsd
              http://jaxb2-commons.dev.java.net/namespace-prefix http://java.net/projects/jaxb2-commons/sources/svn/content/namespace-prefix/trunk/src/main/resources/prefix-namespace-schema.xsd">

    <globalBindings>
        <xjc:simple/>
    </globalBindings>

    <bindings schemaLocation="transport.xsd" node="/xsd:schema">
        <schemaBindings>
            <package name="org.ncpdp.schema.transport"/>
        </schemaBindings>
        <bindings>
            <namespace:prefix name="transport"/>
        </bindings>
    </bindings>
</bindings>

I learned how to add prefixes from the namespace-prefix plugins page.

Finally, I customized the code-generation process to generate Joda Time's DateTime instead of the default XMLGregorianCalendar. This involved a couple custom XmlAdapters and a couple additional lines in bindings.xjb. You can see the adapters and bindings.xjb with all necessary prefixes in this gist. Nicolas Fränkel's Customize your JAXB bindings was a great resource for making all this work.

I wrote a test to prove that the ingest API worked as desired.

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
public class InitiateRequestControllerTest {

    @Inject
    private InitiateRequestController controller;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        MockitoAnnotations.initMocks(this);
        this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build();
    }

    @Test
    public void testGetNotAllowedOnMessagesAPI() throws Exception {
        mockMvc.perform(get("/api/initiate")
                .accept(MediaType.APPLICATION_XML))
                .andExpect(status().isMethodNotAllowed());
    }

    @Test
    public void testPostPaInitiationRequest() throws Exception {
        String request = new Scanner(new ClassPathResource("sample-request.xml").getFile()).useDelimiter("\\Z").next();

        mockMvc.perform(post("/api/initiate")
                .accept(MediaType.APPLICATION_XML)
                .contentType(MediaType.APPLICATION_XML)
                .content(request))
                .andExpect(status().isOk())
                .andExpect(content().contentType(MediaType.APPLICATION_XML))
                .andExpect(xpath("/Message/Header/To").string("3rdParty"))
                .andExpect(xpath("/Message/Header/SenderSoftware/SenderSoftwareDeveloper").string("HID"))
                .andExpect(xpath("/Message/Body/Status/Code").string("010"));
    }
}

Spring Data for JPA and REST

With JAXB out of the way, I turned to creating an internal API that could be used by another application. Spring Data was fresh in my mind after reading about it last summer. I created classes for entities I wanted to persist, using Lombok's @Data to reduce boilerplate.

I read the Accessing Data with JPA guide, created a couple repositories and wrote some tests to prove they worked. I ran into an issue trying to persist Joda's DateTime and found Jadira provided a solution.

I added its usertype.core as a dependency to my pom.xml:

<dependency>
    <groupId>org.jadira.usertype</groupId>
    <artifactId>usertype.core</artifactId>
    <version>3.2.0.GA</version>
</dependency>

... and annotated DateTime variables accordingly.

@Column(name = "last_modified", nullable = false)
@Type(type="org.jadira.usertype.dateandtime.joda.PersistentDateTime")
private DateTime lastModified;

With JPA working, I turned to exposing REST endpoints. I used Accessing JPA Data with REST as a guide and was looking at JSON in my browser in a matter of minutes. I was surprised to see a "profile" service listed next to mine, and posted a question to the Spring Boot team. Oliver Gierke provided an excellent answer.

Swagger

Spring MVC's integration for Swagger has greatly improved since I last wrote about it. Now you can enable it with a @EnableSwagger annotation. Below is the SwaggerConfig class I used to configure Swagger and read properties from application.yml.

@Configuration
@EnableSwagger
public class SwaggerConfig implements EnvironmentAware {
	public static final String DEFAULT_INCLUDE_PATTERN = "/api/.*";

	private RelaxedPropertyResolver propertyResolver;

	@Override
	public void setEnvironment(Environment environment) {
		this.propertyResolver = new RelaxedPropertyResolver(environment, "swagger.");
	}

	/**
	 * Swagger Spring MVC configuration
	 */
	@Bean
	public SwaggerSpringMvcPlugin swaggerSpringMvcPlugin(SpringSwaggerConfig springSwaggerConfig) {
		return new SwaggerSpringMvcPlugin(springSwaggerConfig)
				.apiInfo(apiInfo())
				.genericModelSubstitutes(ResponseEntity.class)
				.includePatterns(DEFAULT_INCLUDE_PATTERN);
	}

	/**
	 * API Info as it appears on the swagger-ui page
	 */
	private ApiInfo apiInfo() {
		return new ApiInfo(
				propertyResolver.getProperty("title"),
				propertyResolver.getProperty("description"),
				propertyResolver.getProperty("termsOfServiceUrl"),
				propertyResolver.getProperty("contact"),
				propertyResolver.getProperty("license"),
				propertyResolver.getProperty("licenseUrl"));
	}
}

After getting Swagger to work, I discovered that endpoints published with @RepositoryRestResource aren't picked up by Swagger. There is an open issue for Spring Data support in the swagger-springmvc project.

Liquibase Integration

I configured this project to use H2 in development and PostgreSQL in production. I used Spring profiles to do this and copied XML/YAML (for Maven and application*.yml files) from a previously created JHipster project.

Next, I needed to create a database. I decided to use Liquibase to create tables, rather than Hibernate's schema-export. I chose Liquibase over Flyway based of discussions in the JHipster project. To use Liquibase with Spring Boot is dead simple: add the following dependency to pom.xml, then place changelog files in src/main/resources/db/changelog.

<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
</dependency>

I started by using Hibernate's schema-export and changing hibernate.ddl-auto to "create-drop" in application-dev.yml. I also commented out the liquibase-core dependency. Then I setup a PostgreSQL database and started the app with "mvn spring-boot:run -Pprod".

I generated the liquibase changelog from an existing schema using the following command (after downloading and installing Liquibase).

liquibase --driver=org.postgresql.Driver --classpath="/Users/mraible/.m2/repository/org/postgresql/postgresql/9.3-1102-jdbc41/postgresql-9.3-1102-jdbc41.jar:/Users/mraible/snakeyaml-1.11.jar" --changeLogFile=/Users/mraible/dev/spring-app/src/main/resources/db/changelog/db.changelog-02.yaml --url="jdbc:postgresql://localhost:5432/mydb" --username=user --password=pass generateChangeLog

I did find one bug - the generateChangeLog command generates too many constraints in version 3.2.2. I was able to fix this by manually editing the generated YAML file.

Tip: If you want to drop all tables in your database to verify Liquibase creation is working in PostgeSQL, run the following commands:

psql -d mydb
drop schema public cascade;
create schema public;

After writing minimal code for Spring Data and configuring Liquibase to create tables/relationships, I relaxed a bit, documented how everything worked and added a LoggingFilter. The LoggingFilter was handy for viewing API requests and responses.

@Bean
public FilterRegistrationBean loggingFilter() {
    LoggingFilter filter = new LoggingFilter();
    FilterRegistrationBean registrationBean = new FilterRegistrationBean();
    registrationBean.setFilter(filter);
    registrationBean.setUrlPatterns(Arrays.asList("/api/*"));
    return registrationBean;
}

Accessing API with RestTemplate

The final step I needed to do was figure out how to access my new and fancy API with RestTemplate. At first, I thought it would be easy. Then I realized that Spring Data produces a HAL-compliant API, so its content is embedded inside an "_embedded" JSON key.

After much trial and error, I discovered I needed to create a RestTemplate with HAL and Joda-Time awareness.

@Bean
public RestTemplate restTemplate() {
	ObjectMapper mapper = new ObjectMapper();
	mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
	mapper.registerModule(new Jackson2HalModule());
	mapper.registerModule(new JodaModule());

	MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
	converter.setSupportedMediaTypes(MediaType.parseMediaTypes("application/hal+json"));
	converter.setObjectMapper(mapper);
	StringHttpMessageConverter stringConverter = new StringHttpMessageConverter();
	stringConverter.setSupportedMediaTypes(MediaType.parseMediaTypes("application/xml"));

	List<HttpMessageConverter<?>> converters = new ArrayList<>();
	converters.add(converter);
	converters.add(stringConverter);

	return new RestTemplate(converters);
}

The JodaModule was provided by the following dependency:

<dependency>
    <groupId>com.fasterxml.jackson.datatype</groupId>
    <artifactId>jackson-datatype-joda</artifactId>
</dependency>

With the configuration complete, I was able to write a MessagesApiITest integration test that posts a request and retrieves it using the API. The API was secured using basic authentication, so it took me a bit to figure out how to make that work with RestTemplate. Willie Wheeler's Basic Authentication With Spring RestTemplate was a big help.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = IntegrationTestConfig.class)
public class MessagesApiITest {
    private final static Log log = LogFactory.getLog(MessagesApiITest.class);
    @Value("http://${app.host}/api/initiate")
    private String initiateAPI;
    @Value("http://${app.host}/api/messages")
    private String messagesAPI;
    @Value("${app.host}")
    private String host;
    @Inject
    private RestTemplate restTemplate;

    @Before
    public void setup() throws Exception {
        String request = new Scanner(new ClassPathResource("sample-request.xml").getFile()).useDelimiter("\\Z").next();

        ResponseEntity<org.ncpdp.schema.transport.Message> response = restTemplate.exchange(getTestUrl(initiateAPI),
                HttpMethod.POST, getBasicAuthHeaders(request), org.ncpdp.schema.transport.Message.class,
                Collections.emptyMap());
        assertEquals(HttpStatus.OK, response.getStatusCode());
    }

    @Test
    public void testGetMessages() {
        HttpEntity<String> request = getBasicAuthHeaders(null);

        ResponseEntity<PagedResources<Message>> result = restTemplate.exchange(getTestUrl(messagesAPI), HttpMethod.GET,
                request, new ParameterizedTypeReference<PagedResources<Message>>() {});
        HttpStatus status = result.getStatusCode();
        Collection<Message> messages = result.getBody().getContent();

        log.debug("messages found: " + messages.size());
        assertEquals(HttpStatus.OK, status);
        for (Message message : messages) {
            log.debug("message.id: " + message.getId());
            log.debug("message.dateCreated: " + message.getDateCreated());
        }
    }

    private HttpEntity<String> getBasicAuthHeaders(String body) {
        String plainCreds = "user:pass";
        byte[] plainCredsBytes = plainCreds.getBytes();
        byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
        String base64Creds = new String(base64CredsBytes);

        HttpHeaders headers = new HttpHeaders();
        headers.add("Authorization", "Basic " + base64Creds);
        headers.add("Content-type", "application/xml");

        if (body == null) {
            return new HttpEntity<>(headers);
        } else {
            return new HttpEntity<>(body, headers);
        }
    }
}

To get Spring Data to populate the message id, I created a custom RestConfig class to expose it. I learned how to do this from Tommy Ziegler.

/**
 * Used to expose ids for resources.
 */
@Configuration
public class RestConfig extends RepositoryRestMvcConfiguration {

    @Override
    protected void configureRepositoryRestConfiguration(RepositoryRestConfiguration config) {
        config.exposeIdsFor(Message.class);
        config.setBaseUri("/api");
    }
}

Summary

This article explains how I built a REST API using JAXB, Spring Boot, Spring Data and Liquibase. It was relatively easy to build, but required some tricks to access it with Spring's RestTemplate. Figuring out how to customize JAXB's code generation was also essential to make things work.

I started developing the project with Spring Boot 1.1.7, but upgraded to 1.2.0.M2 after I found it supported Log4j2 and configuring Spring Data REST's base URI in application.yml. When I handed the project off to my client last week, it was using 1.2.0.BUILD-SNAPSHOT because of a bug when running in Tomcat.

This was an enjoyable project to work on. I especially liked how easy Spring Data makes it to expose JPA entities in an API. Spring Boot made things easy to configure once again and Liquibase seems like a nice tool for database migrations.

If someone asked me to develop a REST API on the JVM, which frameworks would I use? Spring Boot, Spring Data, Jackson, Joda-Time, Lombok and Liquibase. These frameworks worked really well for me on this particular project.

Getting Started with JHipster on OS X

2014-09-08T11:30:33-06:00

Last week I was tasked with developing a quick prototype that used AngularJS for its client and Spring MVC for its server. A colleague developed the same application using Backbone.js and Spring MVC. At first, I considered using my boot-ionic project as a starting point. Then I realized I didn't need to develop a native mobile app, but rather a responsive web app.

My colleague mentioned he was going to use RESThub as his starting point, so I figured I'd use JHipster as mine. We allocated a day to get our environments setup with the tools we needed, then timeboxed our first feature spike to four hours.

My first experience with JHipster failed the 10-minute test. I spent a lot of time flailing about with various "npm" and "yo" commands, getting permissions issues along the way. After getting thinks to work with some sudo action, I figured I'd try its Docker development environment. This experience was no better.

JHipster seems like a nice project, so I figured I'd try to find the causes of my issues. This article is designed to save you the pain I had. If you'd rather just see the steps to get up and running quickly, skip to the summary.

The "npm" and "yo" issues I had seemed to be caused by a bad node/npm installation. To fix this, I removed node and installed nvm. Here's the commands I needed to remove node and npm:

sudo rm -rf /usr/local/lib/node_modules
sudo rm -rf /usr/local/include/node
sudo rm /usr/local/bin/node

sudo rm -rf /usr/local/bin/npm
sudo rm /usr/local/share/man/man1/node.1
sudo rm -rf /usr/local/lib/dtrace/node.d
sudo rm -rf ~/.npm

Next, I ran "brew doctor" to make sure Homebrew was still happy. It told me some things were broken:

$ brew doctor
Warning: Broken symlinks were found. Remove them with `brew prune`:
  /usr/local/bin/yo
  /usr/local/bin/ionic
  /usr/local/bin/grunt
  /usr/local/bin/bower

I ran brew update && brew prune, followed by brew install nvm. Next, I added the following to my ~/.profile:

source $(brew --prefix nvm)/nvm.sh

To install the latest version of node, I ran the commands below and set the latest version as the default:

nvm ls-remote
nvm install v0.11.13
nvm alias default v0.11.13

Once I had a fresh version of Node.js, I was able to run JHipster's local installation instructions.

npm install -g yo
npm install -g generator-jhipster

Then I created my project:

yo jhipster

I was disappointed to find this created all the project files in my current directory, rather than in a subdirectory. I'd recommend you do the following instead:

mkdir ~/projectname && cd ~/projectname && yo jhipster

Before creating your project, JHipster asks you a number of questions. To see what they are, see its documentation on creating an application. Two things to be aware of:

Hot reloading Java code doesn't work well (yet) with Java 8
Its OAuth2 implementation doesn't work with WebSockets

In other words, I'd recommend using Java 7 + (cookie-based authentication with websockets) or (oauth2 authentication w/o websockets).

After creating my project, I was able to run it using "mvn spring-boot:run" and view it at http://localhost:8080. To get hot-reloading for the client, I ran "grunt server" and opened my browser to http://localhost:9000.

JHipster + Docker on OS X

I had no luck getting the Docker instructions to work initially. I spent a couple hours on it, then gave up. A couple of days ago, I decided to give it another good ol' college-try. To make sure I figured out everything from scratch, I started by removing Docker.

I re-installed Docker and pulled the JHipster image using the following:

sudo docker pull jdubois/jhipster-docker

The error I got from this was the following:

2014/09/05 19:43:38 Post http:///var/run/docker.sock/images/create?fromImage=jdubois%2Fjhipster-docker&tag=:
dial unix /var/run/docker.sock: no such file or directory

After doing some research, I learned I needed to run boot2docker init first. Next I ran boot2docker up to start the Docker daemon. Then I copied/pasted "export DOCKER_HOST=tcp://192.168.59.103:2375" into my console and tried to run docker pull again.

It failed with the same error. The solution was simpler than you might think: don't use sudo.

$ docker pull jdubois/jhipster-docker
Pulling repository jdubois/jhipster-docker
01bdc74025db: Pulling dependent layers
511136ea3c5a: Download complete
...

The next command that JHipster's documentation recommends is to run the Docker image, forward ports and share folders. When you run it, the terminal seems to hang and trying to ssh into it doesn't work. Others have recently reported a similar issue. I discovered the hanging is caused by a missing "-d" parameter and ssh doesn't work because you need to add a portmap to the VM to expose the port to your host. You can fix this by running the following:

boot2docker down
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containerssh,tcp,,4022,,4022"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containertomcat,tcp,,8080,,8080"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containergruntserver,tcp,,9000,,9000"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containergruntreload,tcp,,35729,,35729"
boot2docker start

After making these changes, I was able to start the image and ssh into it.

docker run -d -v ~/jhipster:/jhipster -p 8080:8080 -p 9000:9000 -p 35729:35729 -p 4022:22 -t jdubois/jhipster-docker
ssh -p 4022 jhipster@localhost

I tried creating a new project within the VM (cd /jhipster && yo jhipster), but it failed with the following error:

/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:89
                    throw err0;
                          ^
Error: EACCES, permission denied '/jhipster/src'

The fix was giving the "jhipster" user ownership of the directory.

sudo chown jhipster /jhipster

After doing this, I was able to generate an app and run it using "mvn spring-boot:run" and access it from my Mac at http://localhost:8080. I was also able to run "grunt server" and see it at http://localhost:9000

However, I was puzzled to see that there was nothing in my ~/jhipster directory. After doing some searching, I found that the docker run -v /host/path:/container/path doesn't work on OS X.

David Gageot's A Better Boot2Docker on OSX led me to svendowideit/samba, which solved this problem. The specifics are documented in boot2docker's folder sharing section.

I shutdown my docker container by running "docker ps", grabbing the first two characters of the id and then running:

docker stop [2chars]

I started the JHipster container without the -v parameter, used "docker ps" to find its name (backstabbing_galileo in this case), then used that to add samba support.

docker run -d -p 8080:8080 -p 9000:9000 -p 35729:35729 -p 4022:22 -t jdubois/jhipster-docker
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba backstabbing_galileo

Then I was able to connect using Finder > Go > Connect to Server, using the following for the server address:

cifs://192.168.59.103/jhipster

To make this volume appear in my regular development area, I created a symlink:

ln -s /Volumes/jhipster ~/dev/jhipster

After doing this, all the files were marked as read-only. To fix, I ran "chmod -R 777 ." in the directory on the server. I noticed that this also worked if I ran it from my Mac's terminal, but it took quite a while to traverse all the files. I noticed a similar delay when loading the project into IntelliJ.

Summary

Phew! That's a lot of information that can be condensed down into four JHipster + Docker on OS X tips.

Make sure your npm installation doesn't require sudo rights. If it does, reinstall using nvm.
Add portmaps to your VM to expose ports 4022, 8080, 9000 and 35729 to your host.
Change ownership on the /jhipster in the Docker image: sudo chown jhipster /jhipster.
Use svendowideit/samba to share your VM's directories with OS X.

Documenting your Spring API with Swagger

2014-03-25T13:07:18-06:00

Over the last several months, I've been developing a REST API using Spring Boot. My client hired an outside company to develop a native iOS app, and my development team was responsible for developing its API. Our main task involved integrating with Epic, a popular software system used in Health care. We also developed a Crowd-backed authentication system, based loosely on Philip Sorst's Angular REST Security.

To document our API, we used Spring MVC integration for Swagger (a.k.a. swagger-springmvc). I briefly looked into swagger4spring-web, but gave up quickly when it didn't recognize Spring's @RestController. We started with swagger-springmvc 0.6.5 and found it fairly easy to integrate. Unfortunately, it didn't allow us to annotate our model objects and tell clients which fields were required. We were quite pleased when a new version (0.8.2) was released that supports Swagger 1.3 and its @ApiModelProperty.

What is Swagger?
The goal of Swagger is to define a standard, language-agnostic interface to REST APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.

To demonstrate how Swagger works, I integrated it into Josh Long's x-auth-security project. If you have a Boot-powered project, you should be able to use the same steps.

1. Add swagger-springmvc dependency to your project.

<dependency>
    <groupId>com.mangofactory</groupId>
    <artifactId>swagger-springmvc</artifactId>
    <version>0.8.2</version>
</dependency>

Note: on my client's project, we had to exclude "org.slf4j:slf4j-log4j12" and add "jackson-module-scala_2.10:2.3.1" as a dependency. I did not need to do either of these in this project.

2. Add a SwaggerConfig class to configure Swagger.

The swagger-springmvc documentation has an example of this with a bit more XML.

package example.config;

import com.mangofactory.swagger.configuration.JacksonScalaSupport;
import com.mangofactory.swagger.configuration.SpringSwaggerConfig;
import com.mangofactory.swagger.configuration.SpringSwaggerModelConfig;
import com.mangofactory.swagger.configuration.SwaggerGlobalSettings;
import com.mangofactory.swagger.core.DefaultSwaggerPathProvider;
import com.mangofactory.swagger.core.SwaggerApiResourceListing;
import com.mangofactory.swagger.core.SwaggerPathProvider;
import com.mangofactory.swagger.scanners.ApiListingReferenceScanner;
import com.wordnik.swagger.model.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import static com.google.common.collect.Lists.newArrayList;

@Configuration
@ComponentScan(basePackages = "com.mangofactory.swagger")
public class SwaggerConfig {

    public static final List<String> DEFAULT_INCLUDE_PATTERNS = Arrays.asList("/news/.*");
    public static final String SWAGGER_GROUP = "mobile-api";

    @Value("${app.docs}")
    private String docsLocation;

    @Autowired
    private SpringSwaggerConfig springSwaggerConfig;
    @Autowired
    private SpringSwaggerModelConfig springSwaggerModelConfig;

    /**
     * Adds the jackson scala module to the MappingJackson2HttpMessageConverter registered with spring
     * Swagger core models are scala so we need to be able to convert to JSON
     * Also registers some custom serializers needed to transform swagger models to swagger-ui required json format
     */
    @Bean
    public JacksonScalaSupport jacksonScalaSupport() {
        JacksonScalaSupport jacksonScalaSupport = new JacksonScalaSupport();
        //Set to false to disable
        jacksonScalaSupport.setRegisterScalaModule(true);
        return jacksonScalaSupport;
    }


    /**
     * Global swagger settings
     */
    @Bean
    public SwaggerGlobalSettings swaggerGlobalSettings() {
        SwaggerGlobalSettings swaggerGlobalSettings = new SwaggerGlobalSettings();
        swaggerGlobalSettings.setGlobalResponseMessages(springSwaggerConfig.defaultResponseMessages());
        swaggerGlobalSettings.setIgnorableParameterTypes(springSwaggerConfig.defaultIgnorableParameterTypes());
        swaggerGlobalSettings.setParameterDataTypes(springSwaggerModelConfig.defaultParameterDataTypes());
        return swaggerGlobalSettings;
    }

    /**
     * API Info as it appears on the swagger-ui page
     */
    private ApiInfo apiInfo() {
        ApiInfo apiInfo = new ApiInfo(
                "News API",
                "Mobile applications and beyond!",
                "https://helloreverb.com/terms/",
                "matt@raibledesigns.com",
                "Apache 2.0",
                "http://www.apache.org/licenses/LICENSE-2.0.html"
        );
        return apiInfo;
    }

    /**
     * Configure a SwaggerApiResourceListing for each swagger instance within your app. e.g. 1. private  2. external apis
     * Required to be a spring bean as spring will call the postConstruct method to bootstrap swagger scanning.
     *
     * @return
     */
    @Bean
    public SwaggerApiResourceListing swaggerApiResourceListing() {
        //The group name is important and should match the group set on ApiListingReferenceScanner
        //Note that swaggerCache() is by DefaultSwaggerController to serve the swagger json
        SwaggerApiResourceListing swaggerApiResourceListing = new SwaggerApiResourceListing(springSwaggerConfig.swaggerCache(), SWAGGER_GROUP);

        //Set the required swagger settings
        swaggerApiResourceListing.setSwaggerGlobalSettings(swaggerGlobalSettings());

        //Use a custom path provider or springSwaggerConfig.defaultSwaggerPathProvider()
        swaggerApiResourceListing.setSwaggerPathProvider(apiPathProvider());

        //Supply the API Info as it should appear on swagger-ui web page
        swaggerApiResourceListing.setApiInfo(apiInfo());

        //Global authorization - see the swagger documentation
        swaggerApiResourceListing.setAuthorizationTypes(authorizationTypes());

        //Every SwaggerApiResourceListing needs an ApiListingReferenceScanner to scan the spring request mappings
        swaggerApiResourceListing.setApiListingReferenceScanner(apiListingReferenceScanner());
        return swaggerApiResourceListing;
    }

    @Bean
    /**
     * The ApiListingReferenceScanner does most of the work.
     * Scans the appropriate spring RequestMappingHandlerMappings
     * Applies the correct absolute paths to the generated swagger resources
     */
    public ApiListingReferenceScanner apiListingReferenceScanner() {
        ApiListingReferenceScanner apiListingReferenceScanner = new ApiListingReferenceScanner();

        //Picks up all of the registered spring RequestMappingHandlerMappings for scanning
        apiListingReferenceScanner.setRequestMappingHandlerMapping(springSwaggerConfig.swaggerRequestMappingHandlerMappings());

        //Excludes any controllers with the supplied annotations
        apiListingReferenceScanner.setExcludeAnnotations(springSwaggerConfig.defaultExcludeAnnotations());

        //
        apiListingReferenceScanner.setResourceGroupingStrategy(springSwaggerConfig.defaultResourceGroupingStrategy());

        //Path provider used to generate the appropriate uri's
        apiListingReferenceScanner.setSwaggerPathProvider(apiPathProvider());

        //Must match the swagger group set on the SwaggerApiResourceListing
        apiListingReferenceScanner.setSwaggerGroup(SWAGGER_GROUP);

        //Only include paths that match the supplied regular expressions
        apiListingReferenceScanner.setIncludePatterns(DEFAULT_INCLUDE_PATTERNS);

        return apiListingReferenceScanner;
    }

    /**
     * Example of a custom path provider
     */
    @Bean
    public ApiPathProvider apiPathProvider() {
        ApiPathProvider apiPathProvider = new ApiPathProvider(docsLocation);
        apiPathProvider.setDefaultSwaggerPathProvider(springSwaggerConfig.defaultSwaggerPathProvider());
        return apiPathProvider;
    }


    private List<AuthorizationType> authorizationTypes() {
        ArrayList<AuthorizationType> authorizationTypes = new ArrayList<>();

        List<AuthorizationScope> authorizationScopeList = newArrayList();
        authorizationScopeList.add(new AuthorizationScope("global", "access all"));

        List<GrantType> grantTypes = newArrayList();

        LoginEndpoint loginEndpoint = new LoginEndpoint(apiPathProvider().getAppBasePath() + "/user/authenticate");
        grantTypes.add(new ImplicitGrant(loginEndpoint, "access_token"));

        return authorizationTypes;
    }

    @Bean
    public SwaggerPathProvider relativeSwaggerPathProvider() {
        return new ApiRelativeSwaggerPathProvider();
    }

    private class ApiRelativeSwaggerPathProvider extends DefaultSwaggerPathProvider {
        @Override
        public String getAppBasePath() {
            return "/";
        }

        @Override
        public String getSwaggerDocumentationBasePath() {
            return "/api-docs";
        }
    }
}

The ApiPathProvider class referenced above is as follows:

package example.config;

import com.mangofactory.swagger.core.SwaggerPathProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.util.UriComponentsBuilder;

import javax.servlet.ServletContext;

public class ApiPathProvider implements SwaggerPathProvider {
    private SwaggerPathProvider defaultSwaggerPathProvider;
    @Autowired
    private ServletContext servletContext;

    private String docsLocation;

    public ApiPathProvider(String docsLocation) {
        this.docsLocation = docsLocation;
    }

    @Override
    public String getApiResourcePrefix() {
        return defaultSwaggerPathProvider.getApiResourcePrefix();
    }

    public String getAppBasePath() {
        return UriComponentsBuilder
                .fromHttpUrl(docsLocation)
                .path(servletContext.getContextPath())
                .build()
                .toString();
    }

    @Override
    public String getSwaggerDocumentationBasePath() {
        return UriComponentsBuilder
                .fromHttpUrl(getAppBasePath())
                .pathSegment("api-docs/")
                .build()
                .toString();
    }

    @Override
    public String getRequestMappingEndpoint(String requestMappingPattern) {
        return defaultSwaggerPathProvider.getRequestMappingEndpoint(requestMappingPattern);
    }

    public void setDefaultSwaggerPathProvider(SwaggerPathProvider defaultSwaggerPathProvider) {
        this.defaultSwaggerPathProvider = defaultSwaggerPathProvider;
    }
}

In src/main/resources/application.properties, add an "app.docs" property. This will need to be changed as you move your application from local -> test -> staging -> production. Spring Boot's externalized configuration makes this fairly simple.

app.docs=http://localhost:8080

3. Verify Swagger produces JSON.

After completing the above steps, you should be able to see the JSON Swagger generates for your API. Open http://localhost:8080/api-docs in your browser or curl http://localhost:8080/api-docs.

{
    "apiVersion": "1",
    "swaggerVersion": "1.2",
    "apis": [
        {
            "path": "http://localhost:8080/api-docs/mobile-api/example_NewsController",
            "description": "example.NewsController"
        }
    ],
    "info": {
        "title": "News API",
        "description": "Mobile applications and beyond!",
        "termsOfServiceUrl": "https://helloreverb.com/terms/",
        "contact": "matt@raibledesigns.com",
        "license": "Apache 2.0",
        "licenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.html"
    }
}

4. Copy Swagger UI into your project.

Swagger UI is a good-looking JavaScript client for Swagger's JSON. I integrated it using the following steps:

git clone https://github.com/wordnik/swagger-ui
cp -r swagger-ui/dist ~/dev/x-auth-security/src/main/resources/public/docs

I modified docs/index.html, deleting its header (<div id='header'>) element, as well as made its url dynamic.

...
$(function () {
  var apiUrl = window.location.protocol + "//" + window.location.host;
  if (window.location.pathname.indexOf('/api') > 0) {
    apiUrl += window.location.pathname.substring(0, window.location.pathname.indexOf('/api'))
  }
  apiUrl += "/api-docs";
  log('API URL: ' + apiUrl);
  window.swaggerUi = new SwaggerUi({
    url: apiUrl,
    dom_id: "swagger-ui-container",
...

After making these changes, I was able to open fire up the app with "mvn spring-boot:run" and view http://localhost:8080/docs/index.html in my browser.

5. Annotate your API.

There are two services in x-auth-security: one for authentication and one for news. To provide more information to the "news" service's documentation, add @Api and @ApiOperation annotations. These annotations aren't necessary to get a service to show up in Swagger UI, but if you don't specify the @Api("user"), you'll end up with an ugly-looking class name instead (e.g. example_xauth_UserXAuthTokenController).

@RestController
@Api(value = "news", description = "News API")
class NewsController {

    Map<Long, NewsEntry> entries = new ConcurrentHashMap<Long, NewsEntry>();

    @RequestMapping(value = "/news", method = RequestMethod.GET)
    @ApiOperation(value = "Get News", notes = "Returns news items")
    Collection<NewsEntry> entries() {
        return this.entries.values();
    }

    @RequestMapping(value = "/news/{id}", method = RequestMethod.DELETE)
    @ApiOperation(value = "Delete News item", notes = "Deletes news item by id")
    NewsEntry remove(@PathVariable Long id) {
        return this.entries.remove(id);
    }

    @RequestMapping(value = "/news/{id}", method = RequestMethod.GET)
    @ApiOperation(value = "Get a news item", notes = "Returns a news item")
    NewsEntry entry(@PathVariable Long id) {
        return this.entries.get(id);
    }

    @RequestMapping(value = "/news/{id}", method = RequestMethod.POST)
    @ApiOperation(value = "Update News", notes = "Updates a news item")
    NewsEntry update(@RequestBody NewsEntry news) {
        this.entries.put(news.getId(), news);
        return news;
    }
...
}

You might notice the screenshot above only shows news. This is because SwaggerConfig.DEFAULT_INCLUDE_PATTERNS only specifies news. The following will include all APIs.

public static final List<String> DEFAULT_INCLUDE_PATTERNS = Arrays.asList("/.*");

After adding these annotations and modifying SwaggerConfig, you should see all available services.

In swagger-springmvc 0.8.x, the ability to use @ApiModel and @ApiModelProperty annotations was added. This means you can annotate NewsEntry to specify which fields are required.

@ApiModel("News Entry")
public static class NewsEntry {
    @ApiModelProperty(value = "the id of the item", required = true)
    private long id;
    @ApiModelProperty(value = "content", required = true)
    private String content;

    // getters and setters
}

This results in the model's documentation showing up in Swagger UI. If "required" isn't specified, a property shows up as optional.

Parting Thoughts
The QA Engineers and 3rd Party iOS Developers have been very pleased with our API documentation. I believe this is largely due to Swagger and its nice-looking UI. The Swagger UI also provides an interface to test the endpoints by entering parameters (or JSON) into HTML forms and clicking buttons. This could benefit those QA folks that prefer using Selenium to test HTML (vs. raw REST endpoints).

I've been quite pleased with swagger-springmvc, so kudos to its developers. They've been very responsive in fixing issues I've reported. The only thing I'd like is support for recognizing JSR303 annotations (e.g. @NotNull) as required fields.

To see everything running locally, checkout my modified x-auth-security project on GitHub and the associated commits for this article.

Comparing JVM Web Frameworks at vJUG

2014-02-06T10:54:17-07:00

A couple months ago, I was invited to speak at Virtual JUG - an online-only Java User Group organized by the ZeroTurnaround folks. They chose my Comparing JVM Web Frameworks presentation and we agreed I'd speak yesterday morning. They used a combination of Google Hangouts, live streaming on YouTube and IRC to facilitate the meeting. It all went pretty smoothly and produced a comfortable speaking environment. To practice for vJUG, I delivered the same talk on Tuesday night at the Denver Open Source Users Group.

The last time I delivered this talk was at Devoxx France in March 2013. I didn't change any of the format this time, keeping with referencing the Paradox of Choice and encouraging people to define constraints to help them make their decision. I did add a few new slides regarding RebelLabs' Curious Coder’s Java Web Frameworks Comparison: Spring MVC, Grails, Vaadin, GWT, Wicket, Play, Struts and JSF and The 2014 Decision Maker’s Guide to Java Web Frameworks.

I also updated all the pretty graphs (which may or may not have any significance) with the latest stats from Dice.com, LinkedIn, StackOverflow and respective mailing lists. Significant changes I found compared to one year ago:

Job Listings on Dice.com
- Play Framework job listings increased almost 4x
- Tapestry jobs are 1/3 of what they were a year ago
- Wicket jobs are 1/2 of what they were a year ago
- JavaScript framework jobs are up quite a bit: Ember.js up ~300%, AngularJS up 900%, Backbone up 160%
LinkedIn Skills
- Rails down ~30%
- Grails up 25%
- Play Framework up 200%
- Spring Roo up 40%
- Ember.js up 300%
- AngularJS up 840%
- Backbone up 200%

As you can tell from these findings, AngularJS has gained quite a bit of mindshare in the last year. There's a lot of companies looking for JavaScript skills and quite a few folks have added JavaScript frameworks to their LinkedIn profiles.

You can watch the recording on YouTube or click play in the embedded video below.

You can also quickly browse the slide deck below, download the PDF or view it on SlideShare.

Thanks to all the folks who attended these talks. And thanks to @dosug and @virtualjug for giving me the opportunity to speak.

A Webapp Makeover with Spring 4 and Spring Boot

2013-12-11T12:47:15-07:00

A typical Maven and Spring web application has a fair amount of XML and verbosity to it. Add in Jersey and Spring Security and you can have hundreds of lines of XML before you even start to write your Java code. As part of a recent project, I was tasked with upgrading a webapp like this to use Spring 4 and Spring Boot. I also figured I'd try to minimize the XML.

This is my story on how I upgraded to Spring 4, Jersey 2, Java 8 and Spring Boot 0.5.0 M6.

When I started, the app was using Spring 3.2.5, Spring Security 3.1.4 and Jersey 1.18. The pom.xml had four Jersey dependencies, three Spring dependencies and three Spring Security dependencies, along with a number of exclusions for "jersey-spring".

Upgrading to Spring 4
Upgrading to Spring 4 was easy, I changed the version property to 4.0.0.RC2 and added the new Spring bill of materials to my pom.xml. I also add the Spring milestone repo since Spring 4 won't be released to Maven central until tomorrow.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-framework-bom</artifactId>
            <version>${spring.framework.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<repositories>
    <repository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Next, I removed all the references to ${spring.framework.version} in dependencies since it'd be controlled by Maven's dependency management feature.

     <dependency>
         <groupId>org.springframework</groupId>
         <artifactId>spring-web</artifactId>
-        <version>${spring.framework.version}</version>
     </dependency>

I also changed to use Maven 3's wildcard syntax to exclude multiple dependencies.

    <dependency>
        <groupId>com.sun.jersey.contribs</groupId>
        <artifactId>jersey-spring</artifactId>
        <exclusions>
             <exclusion>
                 <groupId>org.springframework</groupId>
-                    <artifactId>spring</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-core</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-web</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-beans</artifactId>
-                </exclusion>
-                <exclusion>
-                    <groupId>org.springframework</groupId>
-                    <artifactId>spring-context</artifactId>
+                    <artifactId>*</artifactId>
             </exclusion>
         </exclusions>
     </dependency>

I confirmed the upgrade worked by running "mvn dependency:tree | grep spring", followed by "mvn jetty:run" and viewing the app in my browser.

Upgrading to Jersey 2
The next item I tackled was upgrading to Jersey 2.4.1. I changed the version number in my pom.xml, then added the Jersey BOM.

<dependency>
    <groupId>org.glassfish.jersey</groupId>
    <artifactId>jersey-bom</artifactId>
    <version>${jersey.version}</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

You might ask "why Jersey?" if we already have Spring MVC and its REST support? You might also ask why not Play or Grails instead of a Java + Spring stack? For this particular project, I recommended technology options, and these were certainly among them. However, the team chose differently and I support their decision. The project is creating an iOS app, as well as a responsive HTML5 mobile/desktop app. We figured we had enough risk with new technologies on the front-end that we should play it a bit safer on the backend. To make the backend work a bit sexier, we've decided to allow Spring 4, Java 8 and possibly some reactive principles.

Next, I changed from the old com.sun.jersey dependencies to org.glassfish.jersey and removed jersey-spring.

<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet</artifactId>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-json-jackson</artifactId>
</dependency>

The last thing I needed to do was change the servlet-class and param-name in web.xml:

<servlet>
    <servlet-name>jersey-servlet</servlet-name>
    <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
    <init-param>
        <param-name>jersey.config.server.provider.packages</param-name>
        <param-value>com.raibledesigns.boot.service</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>

Requiring Java 8
Requiring Java 8 to compile was easy enough. I added the maven-compiler-plugin to enforce a minimum version.

<plugin>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
    </configuration>
</plugin>

I downloaded the latest Java 8 SDK and installed it. Then I set my JAVA_HOME to use it.

export JAVA_HOME=`/usr/libexec/java_home -v 1.8`

Integrating Spring Boot
I learned about Spring Boot a few weeks ago at Devoxx. Josh Long gave me a 3-minute demo at the speaker's dinner and showed me enough to pique my interest. To integrate it into my project, I started with the Quick Start. I added the boot-parent, dependencies for web, security and actuator (logging, metrics, etc.) and the Maven plugin. I removed all the Spring and Spring Security dependencies.

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>0.5.0.M6</version>
</parent>
...
<pluginRepositories>
    <pluginRepository>
        <id>spring-milestones</id>
        <url>http://repo.spring.io/milestone</url>
    </pluginRepository>
</pluginRepositories>
...
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
...
<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
</plugin>

Upon restarting my app, I got an error about spring-security.xml using a 3.1 XSD. I fixed it by changing to 3.2. Next, I wanted to eliminate web.xml. First of all, I created an ApplicationInitializer so the WAR could be started from the command line.

package com.raibledesigns.boot.config;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.SpringBootServletInitializer;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableAutoConfiguration
@ComponentScan
public class ApplicationInitializer extends SpringBootServletInitializer {

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(ApplicationInitializer.class);
    }

    public static void main(String[] args) {
        SpringApplication.run(ApplicationInitializer.class, args);
    }
}

However, after adding this, I received the following error on startup:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 
'org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor': 
Invocation of init method failed; nested exception is 
java.lang.AbstractMethodError: org.hibernate.validator.internal.engine.ConfigurationImpl
.getDefaultParameterNameProvider()Ljavax/validation/ParameterNameProvider;

Adding hibernate-validator as a dependency solved this problem:

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
</dependency>

To configure Spring Security without web.xml and spring-security.xml, I created WebSecurityConfig.java:

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@EnableWebSecurity
@Order(Ordered.LOWEST_PRECEDENCE - 6)
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .antMatchers("/", "/home").permitAll()
                .antMatchers("/v1.0/**").hasRole("USER")
                .anyRequest().authenticated();
        http.httpBasic().realmName("My API");
    }

    @Override
    protected void configure(AuthenticationManagerBuilder authManagerBuilder) throws Exception {
        authManagerBuilder.inMemoryAuthentication()
                .withUser("test").password("test123").roles("USER");
    }
}

To configure Jersey without web.xml, I created a JerseyConfig class:

package com.raibledesigns.boot.config;

import org.glassfish.jersey.filter.LoggingFilter;
import org.glassfish.jersey.jackson.JacksonFeature;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.server.ServerProperties;

import javax.ws.rs.ApplicationPath;

@ApplicationPath("/v1.0")
public class JerseyConfig extends ResourceConfig {

    public JerseyConfig() {
        packages("com.raibledesigns.boot.service");
        property(ServerProperties.BV_SEND_ERROR_IN_RESPONSE, true);
        property(ServerProperties.JSON_PROCESSING_FEATURE_DISABLE, false);
        property(ServerProperties.MOXY_JSON_FEATURE_DISABLE, true);
        property(ServerProperties.WADL_FEATURE_DISABLE, true);
        register(LoggingFilter.class);
        register(JacksonFeature.class);
    }
}

Finally, I created MvcConfig.java to set the welcome page.

package com.raibledesigns.boot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.ViewControllerRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;

@Configuration
public class MvcConfig extends WebMvcConfigurerAdapter {

    @Override
    public void addViewControllers(ViewControllerRegistry registry) {
        registry.addViewController("/").setViewName("index");
    }
}

To cleanup, I deleted src/main/webapp/WEB-INF and created src/main/resources/logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>
    <logger name="org.springframework.boot" level="INFO"/>
    <logger name="org.springframework.security" level="ERROR"/>
</configuration>

Since Boot doesn't support JSP out-of-the-box, I renamed my index.jsp file to index.html and changed the URL in it to point to "/v1.0/hello". I was pleased to see that everything worked nicely. I learned shortly after that I could remove the Spring BOM since Spring Boot uses a <spring.version> property to control its Spring version.

The only issue I found is when started the app with "mvn package && java -jar target/app.war", it failed to initialize Jersey. I tried adding a @Bean for the servlet:

@Bean
public ServletRegistrationBean jerseyServlet() {
    ServletRegistrationBean registration = new ServletRegistrationBean(new ServletContainer(), "/v1.0/*");
    registration.addInitParameter(ServletProperties.JAXRS_APPLICATION_CLASS, JerseyConfig.class.getName());
    return registration;
}

Unfortunately, when running it using "java -jar", I get the following error:

org.glassfish.hk2.api.MultiException: A MultiException has 1 exceptions.  They are:
1. org.glassfish.jersey.server.internal.scanning.ResourceFinderException: 
java.io.FileNotFoundException: /.../target/app.war!/WEB-INF/classes (No such file or directory)
	at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:869)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:814)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:906)
	at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:898)
	at org.glassfish.jersey.server.ApplicationHandler.createApplication(ApplicationHandler.java:300)
	at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:279)
	at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:302)

This seems strange since there is a WEB-INF/classes in my WAR. Regardless, this is not a Boot problem per se, but more of a Jersey issue. From one of the Boot developers:

The whole idea with Boot is that servlets are just a transport - they are a means to an end, and hopefully not the only one - the "container" is Spring, not the servlet container. We probably could add some form of support for SCI but only by hacking the containers since the spec really doesn't allow for much control of their lifecycle. It hasn't been a priority so far.

Summary
I hope this article is useful to see how you to upgrade your Java webapps to use Spring 4 and Spring Boot. I've created a boot-makeover project on GitHub with all the code mentioned. You can also view the commits for each step.

The Modern Java Web Developer Bootcamp at Devoxx

2013-10-29T10:21:49-06:00

At this year's Devoxx, I'll be delivering my first University session. University talks are in depth presentations of 3 hours (= 75m + 30m break + 75m). I'm calling it The Modern Java Web Developer Bootcamp and my goal is to teach people some new concepts and techniques that'll make them more valuable developers. My session's hashtag is #dv13-javaweb$ to exemplify the important takeaways: Java is back, web development is fun and you can make more money.

Three hours is quite a bit longer than I'm used to, but I'm confident I can fill the time with lots of knowledge. My plan is to enhance my presentation from JavaOne and add a few demos. Currently, I'm thinking of developing the following additional content:

HTTP Overview (with SPDY)
Polymer and Web Components
Bootstrap 3 Overview
HTML5 Storage
API Framework Comparison (Play, Grails, Dropwizard)
Load Testing
Performance Monitoring (including RUM)
Internal Cloud Options

For demos, I'd like to show a few that provide real value to attendees and teach them how to do something they haven't done before. The ones below are candidates I'm thinking of, and I'd like to pick three for the final presentation.

Browser Tools Demo
Developing with Bootstrap Demo
AngularJS Demo
Refactor an app from Spring to Java EE, no XML, all Java 8
Page Speed Improvement Demo
Security Demo (add LDAP to Angular app + OWASP ZAP)

If you could pick three real-time tutorials from the choices above, which ones would you choose?

I'm also thinking of adding some stories about impressive loads served with very little hardware and real-time dashboard development. If you have a story about either of these, please let me know. I'd be happy to credit you (or your company) and talk about any technical implementation details you're willing to provide.

Video of Comparing JVM Web Frameworks from Devoxx France

2013-06-24T09:10:45-06:00

Whenever I do a talk, I get requests for a recording of it. It's rare that recordings are made, but when they are, I like to share them. In March of this year, I traveled to Devoxx France and had a great time. One of the talks I delivered was Comparing JVM Web Frameworks, with a bit of a twist from prior versions.

I started reading The Paradox of Choice and found many parallels to the agony that developers experience with choosing a web framework. I described how I didn't think good framework decisions were based on the many, many features that frameworks have, but often on pre-defined constraints. There's those lucky developers that get to choose a Full Stack Framework because they're doing greenfield development. Then there's those that want a better Pure Web Framework that replaces something (e.g. Struts) that's not satisfying their needs. And lastly, there's those that've found it possible to leverage a SOFEA and use a JavaScript MVC framework with an API Framework on the backend. I don't think it makes sense to compare all web frameworks and I tried to use these pre-defined constraints (language, platform and application type) argument to separate into categories and help make choosing easier.

The good folks at Parleys have published the video of this talk. If you haven't heard of Parleys, it's an awesome platform for watching conference talks. As their Mission Statement says: If YouTube and Slideshare would make a baby then it would be named Parleys.

Below is an embedded video of this presentation - I hope you enjoy watching it as much as I did delivering it!

<br />

Developing with AngularJS - Part II: Dialogs and Data

2013-06-20T08:45:13-06:00

A couple of days ago, I wrote an article on how I started developing with AngularJS. I used AngularJS for several months to develop a "My Dashboard" feature for a client's product and learned a whole bunch of stuff along the way.

This article provides an overview of how I changed some of My Dashboard's features to use Angular instead of jQuery. After finishing the prototype work in January, we started moving bits and pieces into the main application. We kept the same file names for our Angular-related files and copied them into the project.

All these files are packaged up into a dashboard.js file that's included at the bottom of our Dashboard page. While our prototype used jQuery 1.9 and jQuery UI 1.10, the application's codebase used jQuery 1.7.1 and jQuery UI 1.8.3. Luckily, this didn't present a problem as everything continued to work as expected.

Around this time, we also had many discussions with the Product Team about charts. Since Highcharts required we purchase a license, we took at look at AnyChart, which we were already using. We were able to get AnyChart to work with our existing chart directive with minimal changes. Most changes were in the JSON itself.

We committed the first pass (with sample data still hard-coded) in mid-February.

Angular UI's Carousel

While finishing our initial prototype, I learned about Angular UI's Carousel, an implementation of Bootstrap's Carousel. It required a 70% less HTML and is quite a bit easier to read. Below is the refactored carousel section.

<carousel interval="-1" class="oneup">
    <slide ng-repeat="widget in widgets | filter: {type: 'chart'} | orderBy: 'order'" active="slide.active" class="chart">
        <chart class="widget" value="{{widget}}" type="{{widget.chartType}}"></chart>
    </slide>
</carousel>
<carousel interval="-1" class="twoup">
    <slide ng-repeat="widget in widgets | filter: {type: 'chart'} | chunk: 2 | orderBy: 'order'" active="slide.active" class="chart">
        <chart class="widget" value="{{widget[0]}}" type="{{widget[0].chartType}}"></chart>
        <chart class="widget" value="{{widget[1]}}" type="{{widget[1].chartType}}"></chart>
    </slide>
</carousel>

I was also able to remove the JavaScript that once initialized the carousel.

-    var carousel = $('.carousel');
-    $(carousel).carousel({
-        interval: 0
-    });

While integrating this directive, I found a way to improve it by hiding navigation indicators if there's only one slide. And thus my first contribution to Angular UI was born.

Angular UI's Sortable

Next, I switched from using JavaScript to Angular UI's sortable to initialize drag-and-drop functionality. This was as simple as removing 5 lines of JavaScript and adding "ui-sortable" as an attribute to HTML tags.

<div class="container-widgets" ng-controller="WidgetController" ng-cloak>
     <div class="row-fluid">
         <div class="span9">
-                <ul class="widgets">
+                <ul class="widgets" ui-sortable="{handle:'.heading'}">
                 <li id="summary-bar">
                     <div class="heading">Summary <a href='#' class='configure'></a></div>
-                        <ul class="tiles">
+                        <ul class="tiles" ui-sortable>
                         <li class="span3" ng-repeat="widget in widgets | filter:{type: 'summary'} | orderBy: 'order'">
                             <h3>{{widget.value}}</h3>
@@ -36,7 +36,7 @@
                 </li>
                 <li id="task-bar">
                     <div class="heading">My Tasks <a href='#' class='configure'></a></div>
-                        <ul class="tasks">
+                        <ul class="tasks" ui-sortable>
@@ -151,55 +131,44 @@
-    $('.widgets').sortable({
-        cursor: "move",
-        handle: ".heading"
-    }).disableSelection();
-    $('.tiles,.tasks').sortable();

This directive uses jQuery UI under the covers, so it accepts all the same arguments you'd normally use.

Dialogs with jQuery

The next feature I implemented was a dialog that allowed end users to add/remove widgets from their dashboard. Since I knew how to do this with jQuery, that's the path I started down. I figured it'd be easier to get something working quickly and then refactor to Angular later. I put the HTML for the dialog at the bottom of the page:

<script type="text/javascript" src="js/move.js"></script>
<script type="text/javascript" src="js/upDown.js"></script>
<div id="configure-widgets" class="hidden" data-title="Bar Configuration" data-close="Close">
    <div class="row-fluid center">
        Decide which items you would like to display and set the display order.
        <div class="span5" style="margin-left: 25px">
            <label for="available-widgets" class="bold">Available Widgets</label>
            <select size=10 id="available-widgets" multiple class="width100pr"></select>
        </div>
        <div class="span1 arrows" style="padding-top: 50px">
            <img src="images/arrow_up.png" class="arrow arrow-up" alt="Move Up" onclick="return moveUp($('#assigned-widgets')[0]);"/>
            <img src="images/arrow_right.png" class="arrow arrow-right" alt="Move Right" onclick="return move($('#available-widgets')[0], $('#assigned-widgets')[0]);"/>
            <img src="images/arrow_left.png" class="arrow arrow-left" alt="Move Left" onclick="return move($('#assigned-widgets')[0], $('#available-widgets')[0]);"/>
            <img src="images/arrow_down.png" class="arrow arrow-down" alt="Move Down" onclick="return moveDown($('#assigned-widgets')[0]);"/>
        </div>
        <div class="span5">
            <label for="assigned-widgets" class="bold">Assigned Widgets</label>
            <select size=10 id="assigned-widgets" multiple class="width100pr"></select>
        </div>
    </div>
</div>

Then I changed the WidgetController to split the widgets variable into variables for each widget type.

+    var filter = $filter('filter');
+    var orderBy = $filter('orderBy');
+
+    $scope.summaryWidgets = filterData($scope.widgets, {type: 'summary'}, filter, orderBy);
+    $scope.taskWidgets = filterData($scope.widgets, {type: 'task'}, filter, orderBy);
+    $scope.chartWidgets = filterData($scope.widgets, {type: 'chart'}, filter, orderBy);
+}
+
+function filterData(array, query, filter, orderBy) {
+    var data = filter(array, query);
+    return orderBy(data, 'order');
+}

Finally, I added a new config directive and hooked it into the page by adding config="type" to each heading (e.g. <div class="heading" config="task">My Tasks</div>).


.directive('config', function ($sanitize, $filter) {
    return {
        restrict: 'A',
        link: function (scope, element, attrs) {
            var configBtn = $('<a href="#"/>').addClass('configure');
            configBtn.appendTo(element);
            var widgets = scope[attrs.config + 'Widgets'];
             
            // availableWidgets is defined as a global variable embedded in the page
            var allWidgets = availableWidgets; 
            $(configBtn).on('click', function (e) {
                e.preventDefault();
                var configDialog = $('#configure-widgets');
                var availableWidgets = $('#available-widgets');
                availableWidgets.empty();
                var filter = $filter('filter');
                var orderBy = $filter('orderBy');
                allWidgets = filter(allWidgets, {type: attrs.config});
                allWidgets = orderBy(allWidgets, 'order');
                var unselectedWidgets = jQuery.grep(allWidgets, function(item) {
                    return jQuery.inArray(item, widgets) < 0;
                });
                $.each(unselectedWidgets, function(index, item) {
                    var title = (item.title) ? item.title.replace(/&quot;/g, '"') : 'No Title';
                    availableWidgets.append(new Option(title, item.id));
                });
                var assignedWidgets = $('#assigned-widgets');
                assignedWidgets.empty();
                $.each(widgets, function(index, item) {
                    var title = (item.title) ? item.title.replace(/&quot;/g, '"') : 'No Title';
                    assignedWidgets.append(new Option(title, item.id));
                });
                configDialog.dialog({
                    title: $(element).text() + ' ' + configDialog.attr('data-title'),
                    width: 600,
                    modal: true,
                    buttons: [{
                            text: configDialog.attr('data-close'),
                            'class': 'btn',
                            click: function() {
                                $(this).dialog("close");
                            }
                        }]
                });
            });
        }
    }
})

As you can tell, this is quite a bit of code, and it doesn't even show you the JavaScript in move.js and upDown.js (included at the top of the dialog HTML). While writing this code, I could tell that I was not doing things the Angular Way.

To refactor, I did some research, found the $dialogProvider service and went to work.

Dialogs with Angular

I started by refactoring the config directive to be much shorter.

.directive('config', function() {
    return {
        restrict: 'A',
        link: function (scope, element, attrs) {
            var configBtn = $('<a href=""/>').addClass('configure');
            configBtn.appendTo(element);
            configBtn.bind('click', function(e) {
                e.preventDefault();
                e.stopPropagation();
                scope.$apply(scope.configureDialog(attrs.config, $(element).text()))
            });
        }
    }
})

Line #10 that starts with scope.$apply is what makes the magic happens.This calls the configureDialog() function in WidgetController. The $dialog service you see below is injected by adding it as a parameter to the WidgetController function.


$scope.configureDialog = function (type, title) {
    var dialog = $dialog.dialog({
        dialogClass: 'modal draggable',
        modalFade: false,
        backdropClick: true,
        controller: 'ConfigController',
        template: $('#configure-widgets').html(),
        resolve: {
            title: function() {
                return title;
            },
            hiddenItems: function () {
                return Widget.getHiddenWidgets(type);
            },
            items: function () {
                return angular.copy($scope[type + 'Widgets']);
            }
        }
    });
    dialog.open().then(function(results) {
        if (!angular.isUndefined(results)) {
            $scope[type + 'Widgets'] = results.items;
            Preferences.saveWidgetPreferences(type, results);
        }
    });
};

draggable directive
If you look closely, you'll see this dialog is initialized with a 'draggable' class. I created a draggable directive that gives draggability using jQuery UI to any elements with this class.

.directive('draggable', function() {
    return {
        restrict: 'C',
        link: function(scope, elem, attr, ctrl) {
            elem.draggable({handle: '.modal-header'});
        }
    };
});

The interesting parts of this file are the controller, template, and resolve parameters. The controller is just another function in controllers.js, the template is HTML in the page (so text could be i18n-ized) and resolve has the variables to pass to the controller. Widget and Preferences are services I'll talk about in the Part 3. The reason angular.copy() is used is so the widget arrays aren't modified while the dialog is displayed (we want to wait until the user clicks "Save").

The ConfigController function started quite simply:

function ConfigController($scope, $sanitize, dialog, title, hiddenItems, items) {
    $scope.title = title;
    $scope.hiddenItems = hiddenItems;
    $scope.items = items;
}

The HTML (defined by the aforementioned template parameter) is as follows. You can see that title, hiddenItems and items are the variables defined in $scope and used to render the data.

<div id="configure-widgets" class="modal hidden">
    <div class="modal-header" style="border: 0">
        <button type="button" class="close" ng-click="close(false)" aria-hidden="true" style="margin-top: -3px">&times;</button>
        <h4>{{title}} Bar Configuration</h4>
    </div>
    <div class="modal-body">
        Decide which items you would like to display and set the display order.
        <div class="row-fluid center">
            <div class="span4" style="width: 320px; margin-left: 25px">
                <label for="available-widgets" class="bold">Available Widgets</label>
                <select size=10 id="available-widgets" ng-model="items.available" ng-options="i.title for i in hiddenItems"
                        class="width100pr" multiple="multiple"></select>
            </div>
            <div class="span1 arrows" style="padding-top: 50px">
                <img src="images/arrow_up.png" class="arrow arrow-up" alt="Move Up" ng-click="moveUp(items.selected, items)"/>
                <img src="images/arrow_right.png" class="arrow arrow-right" alt="Move Right" ng-click="moveItem(items.available, hiddenItems, items)"/>
                <img src="images/arrow_left.png" class="arrow arrow-left" alt="Move Left" ng-click="moveItem(items.selected, items, hiddenItems)"/>
                <img src="images/arrow_down.png" class="arrow arrow-down" alt="Move Down" ng-click="moveDown(items.selected, items)"/>
            </div>
            <div class="span4" style="width: 320px">
                <label for="assigned-widgets" class="bold">Assigned Widgets</label>
                <select size=10 id="assigned-widgets" ng-model="items.selected" ng-options="i.title for i in items"
                        class="width100pr" multiple="multiple"></select>
            </div>
        </div>
    </div>
    <div class="modal-footer">
        <button ng-click="close(true)" class="btn btn-primary">Save and Close</button>
    </div>
</div>

There's a couple of new directives introduced by this code: ngOptions and ngClick. The former is used to display options in a <select>, the latter to call functions in the controller. These functions are defined in ConfigController:

$scope.moveUp = function(items, list) {
    angular.forEach(items, function(item) {
        var idx = list.indexOf(item);
        if (idx != -1) {
            list.splice(idx - 1, 0, list.splice(idx, 1)[0]);
        }
    });
};

$scope.moveDown = function(items, list) {
    angular.forEach(items, function(item) {
        var idx = list.indexOf(item);
        if (idx != -1) {
            list.splice(idx + 1, 0, list.splice(idx, 1)[0]);
        }
    });
};

$scope.moveItem = function(items, from, to) {
    angular.forEach(items, function(item) {
        var idx = from.indexOf(item);
        if (idx != -1) {
            from.splice(idx, 1);
            to.push(item);
        }
    });
};

$scope.close = function(save) {
    if (save) {
        dialog.close({
            hidden: $scope.hiddenItems,
            items: $scope.items
        });
    } else {
        dialog.close();
    }
};

As you can see, Angular allows you to easily access and manipulate the data. Its two-way binding feature is great because when you modify the object in JavaScript, it auto-updates the displayed HTML.The only thing you need to do to the HTML is to add the ngModel directive.

<ul class="tasks" ng-model="taskWidgets" ui-sortable>

The dialog's close() method is called in the header (where it passes false) and in the footer (where it passes true). The configureDialog() function handles saving if the user indicates they wanted to do so. The 3rd line (starts with $scope) is all that's needed to update the UI. The Preferences service is covered in the next article.

dialog.open().then(function(results) {
    if (!angular.isUndefined(results)) {
        $scope[type + 'Widgets'] = results.items;
        Preferences.saveWidgetPreferences(type, results);
    }
});

Modals with Data

Displaying the widgets in a consolidated dashboard is a nice product feature, but we wanted to take it a step further and allow users to "click through" to see the data. To do this, I added an "event" directive that could read from our JSON data and act upon it accordingly. We decided on 2 types of events: function and href. The href event type is for report widgets, because we want to allow users to click on the widget and it takes them directly to the report. For function, we simply eval() what's passed in. The function is expected to have a container argument that it can use to render data in a modal window.

Using the event directive, you can attach this behavior to a widget simply by adding a class.

<h3 class="events">{{widget.value}}</h3>

The event directive that attaches click behavior is below:

directive('events', function () {
    // This is necessary because widgets may be chunked when charts are displayed 2-up
    function getWidget(element, widget, scope) {
        if (element.hasClass('chart-title')) {
            if (element.parent().hasClass('first')) {
                widget = scope.widget[0];
            } else if (element.parent().hasClass('second')) {
                widget = scope.widget[1];
            } else {
                widget = scope.widget;
            }
        } else {
            widget = scope.widget;
        }
        return widget;
    }
    return {
        restrict: 'C',
        link: function (scope, element, attrs) {
            var widget = getWidget(element, widget, scope);
            if (angular.isUndefined(widget)) {
                return;
            }
            var events = widget.events;
            if (!angular.isUndefined(events)) {
                for (var e in events) {
                    if (e === 'function') {
                        if ($('#dialog-frame').length === 0) {
                            $('<div id="dialog-frame" class="modal hide"/>').appendTo('body');
                            var header = $('<div class="modal-header">');
                            header.append($('<button type="button" class="close" data-dismiss="modal">×</button>'));
                            header.append($('<h4/>').append(scope.widget.title));
                            header.appendTo('#dialog-frame');
                            $('<div class="modal-body"/>').appendTo('#dialog-frame');
                        }
                        element.bind('click', function(event) {
                            event.preventDefault();
                            event.stopPropagation();
                            var dialog = $('#dialog-frame');
                            var title = widget.title;
                            dialog.find('h4').html(title);
                            var dialogBody = dialog.find('.modal-body');
                            dialogBody.empty();
 
                            // display a checking for new data message when widget's value is 0
                            var checkingMessage = $('#wait-checking').html();
                            if (scope.widget.value === "0") {
                                dialogBody.html(checkingMessage);
                            // otherwise, display a loading message
                            } else {
                                dialogBody.html($('#wait-loading').html());
                            }
 
                            // center the dialog on the page
                            dialog.css({
                                width: '560px',
                                'margin-left': function() {
                                    return -($(this).width() / 2);
                                }
                            });
 
                            dialog.modal('show');
                            var container = dialogBody;
                            eval(events[e]);
                        });
                    } else if (e === "href") {
                        element.bind('click', function() {
                            location.href = events[e];
                        })
                    } else {
                        console.log('Event type "' + e + '" not supported.');
                    }
                }
            }
        }
    }
})

Empty Widgets Message

When there are no widgets for a particular type, we wanted to display a message telling the end user. To do this, I used the ngHide directive and passed in the array's length as an expression. I originally had this on a <li> in the respective widget list, but noticed it causes issues when dragging and dropping.

<div ng-hide="summaryWidgets.length" class="widgets-empty">
    No Summary Bar Widgets currently visible
</div>

Summary

This article has provided an overview of how I changed some of My Dashboard's features to use Angular instead of jQuery. I hope it's helped to show how powerful directives can be and how MVC works in Angular. I particularly enjoyed learning how to use the $dialog service. As a word of warning, its usage might change in future releases since it is currently being rewritten to be more maintainable.

In the next article, I'll talk about how I developed Services and integrated DWR. If you see any code that can be improved upon, or issues with the code/architecture in this article, please leave a comment.

Devoxx France: A Great Conference in a Magnificent City

2013-03-29T13:14:30-06:00

This week, my lovely fiancé and I traveled to the City of Light. Our journey was designed around some speaking engagements at Devoxx France. Devoxx is one of my favorite conference franchises and Devoxx France has been special to me ever since the Devoxx (Belgium) I spoke at in 2011.

2011 was the year I spoke about my experience with Play, Scala, CoffeeScript and Jade. I wrote the presentation on my flight over, composed the demo video the night before and made it all happen in the nick of time. Of course, this was after 120 hours of research and preparation, so the presentation composition process had all the data I needed. You can imagine my sense of relief after pulling off that talk and getting an enthusiastic applause from the audience for my efforts.

One of the first audience questions I received was from Nicolas Martignole, asking if I'd speak at Devoxx France the following year. I whole-heartedly agreed to do it and was excited for the opportunity. It was with great disappointment that I later found out I couldn't attend Devoxx France in 2012. My client didn't like me taking so much time off and I agreed to scale my two week vacation back to 1 week. This year, I was determined to go, so I submitted some of my favorite talks: Comparing JVM Web Frameworks and The Play vs. Grails Smackdown with James Ward. I was extremely pleased when they both got accepted.

Side Story: I met Martin Odersky shortly when he sat down next to me for the Java Posse presentation in Belgium in 2011. After shaking his hand and introducing myself, I had to politely ask him to leave because it was Trish's seat. Talk about awkward; but Martin was very gracious and promptly found a new seat close by.

Comparing JVM Web Frameworks
Both talks required a bit of updating. For Comparing JVM Web Frameworks, I started reading The Paradox of Choice and found many parallels to the agony that developers experience with choosing a web framework. I described how I didn't think good framework decisions were based on the many, many features that frameworks have, but often on pre-defined constraints. There's those lucky developers that get to choose a Full Stack Framework because they're doing greenfield development. Then there's those that want a better Pure Web Framework that replaces something (e.g. Struts) that's not satisfying their needs. And lastly, there's those that've found it possible to leverage a SOFEA and use a JavaScript MVC framework with an API Framework on the backend. I don't think it makes sense to compare all web frameworks and I tried to use these pre-defined constraints (language, platform and application type) argument to separate into categories and help make choosing easier.

I took out the parts of the presentation that've pissed people off in the past - particular the JSF bashing by James Gosling, the Rails gushing from Craig McClanahan and the Pros and Cons sections of each framework. I added the history of web frameworks and research from InfoQ and devrates.com.

The best part of the JVM Web Frameworks talk was the audience's reaction and enthusiasm. Devoxx always seems to attract passionate developers and Devoxx France was no different. Developers packing the room, clapping after your intro, laughing at your jokes, signifying that they agree with you about JSF. As a speaker, it's an unbelievable experience.

You can view my Comparing JVM Web Frameworks presentation below or on Slideshare.net.

Play Frameworks vs. Grails Smackdown
To prepare for James Ward and my Play vs. Grails Smackdown, we had a number of goals. First of all, we wanted to update our apps to use the latest versions of each framework. I documented what it took for Grails, James just checked in his code to GitHub. It was interesting to see that Grails 2.0.3 -> 2.2.1 caused a number of issues with testing, while Play 2.0.3 -> Play 2.1.0 required API changes, but nothing for tests. Secondly, we updated all the stats for our pretty graphs and ran load tests again.

This is where the fun started. On Tuesday evening, I decided to challenge the notion that Play was twice as fast as Grails. James had proven this with Apache Bench tests. With Play 2.0 and Grails 2.0 (last summer), we clocked Play at 251/requests per second and 198 for Grails. After upgrading each app to the latest releases, we found the numbers to be 233/second for Play and 118 for Grails.

However, Apache Bench only tests until the first byte is received. Since I've done a lot of browser optimizations recently, I fired up whichloadsfaster.com, captured a screenshot and added it to our presentation. The next day, James added a CDN and a bunch of caching to his app and re-ran his AB tests.

Now he was smoking Grails, so I added a CDN and caching as well. However, the best I could do was just over 1000/requests per second, while he was around 2200/second. When he ran live tests during our talk, Play was around 2800/sec and Grails was around 900.

It was great to see how much better performance we could get with caching and a CDN. The best part is this should be available to most applications, not just these frameworks. By adding a CDN (we used Amazon CloudFront) and caching, we were both able to 10x the performance of our apps. You can find our presentation here or view it below.

Summary
This was a very enjoyable conference to attend as a speaker. First of all, it was in one of the most beautiful cities in the world, but it's also a very special place for Trish and I. We got engaged just outside of Paris in Versailles after the last Devoxx conference I spoke at. Trish has some amazing photos from that trip. Secondly, the Devoxx conference attracts a special kind of developer - one that is passionate about and eager for knowledge. Lastly, speaking with my good friend James, in an exotic city about something we love - that was special. Asking for beers and having them brought to us at the start of our Smackdown. That was magical (thanks Nicolas!).

To all the Devoxx organizers and crew - well done on a great show!

The Modern Java Web Developer and Java Web Security at Denver JUG

2013-02-14T10:23:18-07:00

Last night, I had the pleasure of delivering two talks at the Denver Java User Group. The first talk, The Modern Java Web Developer, was inspired by the book titled The Well-Grounded Java Developer. Ben Evans and Martijn Verburg mention in the beginning of the book that they wrote it as a training guide to get new Java developers up to speed. For my talk, I wanted to do something similar, but for Java Web Developers.

I mentioned that the first thing you have to do to become modern is to change your title from a Java Web Developer to a JVM Web Developer. After doing that, you have a whole slew of new and wonderful technologies at your disposal. From there, I believe the Modern JVM Web Developer:

Starts with Fast Hardware
Uses IntelliJ IDEA
Leverages jQuery, HTML5, and CSS3
Creates High Performance Web Sites
For Mobile Devices
In the Cloud
And cares about Security

You can also view this presentation on Slideshare or download it from my presentations page.

The second talk was on Java Web Application Security and was largely an updated version of the talk I gave a couple years ago, starting with an appearance at the Utah JUG. It was mostly a live demo session using my Ajax Login application. To prepare the project for this talk, I created branches for each step. This means you can checkout the "baseline" branch and use Git to compare it with the "javaee" branch. You can also compare the "spring-security" branch vs. the "apache-shiro" branch. Finally, you could see what I needed to do to fix many of the vulnerabilities found by Zed Attack Proxy.

You can also view this presentation on Slideshare or download it from my presentations page.

Thanks to the DJUG and Thrive folks for providing good beer (especially the Guinness!) and FullContact for hosting. Also, I'd like to thank Manning for the copies of The Well-Grounded Java Developer they sent and No Starch Press for copies of Michal Zalewsky's The Tangled Web: A Guide to Securing Modern Web Applications. Last, but certainly not least, thanks to all the good people who attended and listened to me ramble on about all this cool technology.

Future speaking engagements include Devoxx France in March and Denver's HTML5 User Group in April.

AppFuse Light 2.2.1 Released!

2013-01-24T19:43:20-07:00

In December, the AppFuse Team released 2.2.1. Right before that release, I decided to wait on enhancing its "light" modules, a.k.a. AppFuse Light. I'm glad I did, because it took some effort to get jQuery and Bootstrap integrated, as well as to make it more secure.

The good news is AppFuse Light 2.2.1 is released and it's sitting out on the Central Repository. This release is a refactoring of all archetypes to be up-to-date with the AppFuse 2.2.1 release. This means Java 7 compatibility, Servlet 3, Bootstrap/jQuery integration, Tapestry 5.3.6 upgrade and security improvements. I integrated Bootstrap and jQuery using WebJars Servlet 3 support since it was simple and straightforward.

You can create projects using AppFuse's light archetypes using a command such as the following:

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-light-spring-freemarker-archetype -DarchetypeVersion=2.2.1 
  -DgroupId=com.mycompany -DartifactId=myproject

The list of archetypes is as follows:

appfuse-light-jsf-archetype
appfuse-light-spring-archetype
appfuse-light-spring-freemarker-archetype
appfuse-light-spring-security-archetype
appfuse-light-stripes-archetype
appfuse-light-struts-archetype
appfuse-light-tapestry-archetype
appfuse-light-wicket-archetype

The QuickStart Guide will help you get setup and demos are available at the following links:

If you have questions about AppFuse, we invite you to ask them on the users mailing list or tweet using #appfuse.

For those enjoying Bootstrap in your apps, I encourage you to check out {wrap}bootstrap and Bootswatch.

What's the best way to compare JVM Web Frameworks?

2013-01-09T08:29:17-07:00

I've been comparing web frameworks ever since 2004. It was the first time I'd ever proposed a talk for a conference. ApacheCon was in Vegas that year and my buddy Bruce suggested I speak at it. I submitted the talk, got accepted and went to work learning the frameworks I was talking about. At the time, I had a lot of Struts experience and I'd made a good living learning it, consulting on it and blogging about it. However, there was a new kid on the block (Spring MVC) that was garnishing attention and some other frameworks (WebWork and Tapestry) that had a lot of high praise from developers. I was inspired to learn why so many people hated Struts.

Fast forward 8 years and I'm still comparing web frameworks. Why? Because there still seems to be a large audience that's interested in the topic. Witness InfoQ's Top 20 JVM Web Frameworks, which was one of their most-read articles for two months in a row. One of the beauties of the Java Community is that it's very diverse. There's tons of folks that are part of this community and, like it or not, several folks that are former Java Developers. However, these developers still seem to maintain an interest in the community and it's still one of the largest pools of talent out there. Java is still quite viable and only seems to be getting better with age.

So the topic of web frameworks on the JVM is still hot, and I still like to write about it. For those of you still enthusiastic about the topic, you're in luck. The two best websites for the Java Community, InfoQ and DZone (formerly Javalobby) are still very interested in the topic too!

Both sites emailed me in November to get my advice for their research on the subject. InfoQ (specifically Dio Synodinos) was mostly interested in 1) having me analyze their recently-gathered data, or 2) helping them create a new version. DZone (specifically Mitch Pronschinske) emailed about doing a similar survey to InfoQ's, but with more relevant data points (include GWT, specifying Struts 2 vs. Struts 1, etc.).

My response to Mitch at DZone:

Interestingly enough, the folks at InfoQ contacted me as well as they're thinking of doing a new survey. One of the things I mentioned to them is it'd be interesting to see what folks are using AND which frameworks they admire. Often, devs don't get to choose their web framework at work. I wonder if it'd be possible to collaborate with InfoQ to gather data from developers so it's not being done on two different sites?

I submitted a talk for ApacheCon NA (in February) called Comparing Apache Web Frameworks. When choosing Web Frameworks, I've often found it helps to eliminate frameworks and narrow the scope. Obviously, this makes sense for an Apache Conference, but not for all developers. However, I do plan on analyzing each framework based on a limited set of criteria. Here's what I have so far:

Community, HTML5, REST, Mobile, Performance, Web Performance Optimization

Obviously, community is important for Apache projects, but might not be for the wider audience. It might be good to limit to these 5 criteria, or expand it to 10, but not more. I think it'd be interesting to get the community to rank the various frameworks on these criteria, and also try to find developer's biases while doing it. For example, I wonder if people would be willing to admit they're biased for/against certain frameworks and then take that into account as part of gathering the data?

Unfortunately, ApacheCon denied my submission. This make sense since their schedule seems to be concentrating on HTTPD, Cloud and Big Data.

For InfoQ, Dio asked for a list of web frameworks to include. Below is a list we started with, followed by my response.

Spring MVC
Play
Grails
JSF
Struts 2
Wicket
Lift
Tapestry 5
Seam
JRuby on Rails
Wicket
GWT
Vaadin

I would add Stripes, vert.x and maybe something like Apache Click. VRaptor is probably a good one to add too. There's always a few less-used frameworks that get a lot of complainers if you don't include them.

Comparing to your previous list, I don't think Seam should be in here since they've split the project into separate bundles and are no longer developing Seam as a whole. JRuby on Rails is a tough one because if you say Ruby on Rails, you'll get a ton of responses, but probably not from the Java community. The Ruby community might chime it quite a bit if you can get in touch with them though.

I believe you should include Clojure web frameworks, but I've only heard of one of them: Compojure.

SiteMesh, Netty, etc. - remove them.

I also offered my advice on instructions:

I actually like the two coordinates you used before, importance and adoptability. However, I don't know that everyone read the instructions this way. Most people didn't rank all frameworks and I believe that's part of the point. I only ranked the ones I'd used, but I think it'd be better if people ranked all of them. I also think having these two criteria opens it up to more than just developers. Project/Product Managers and stakeholders that've been successful with certain frameworks should be able to vote too.

For DZone, they wanted to include a set of criteria for ranking:

Project maturity, community support, and documentation quality (one criteria)
UI Features Capability (maybe some are more graphics driven or form driven?)
Code readability
Flexibility (maybe this could be broken down into what types of projects the framework can handle) or several criteria that ask if it is "Good for 'x' type of project"
Performance/Speed
Cross platform support
Extensibility, Plugins, Community Libraries
Architecture (this may just be information for later, not an opinion question)
Web standards support
REST support (is this something you would just rate a yes or no? In that case it wouldn't need to be an opinion question)
Mobile support

My response:

For web frameworks, I believe the 5 I mentioned (Community, HTML5, REST, Mobile, Performance, Web Performance Optimization) are most important, with Security become more and more of a concern. For web standards support, I'd almost change it to "HTML5" and to see how the various frameworks stack up. I think REST is very important, and I think it's cool that Struts 2, Spring MVC and Grails all have great support for it. It'd be interesting to see how the component-based frameworks think of having REST support in the framework (vs. external like Jersey, CXF, etc.).

Of the list you provided, I don't know about Code readability or Flexibility. Code readability is kinda like Learnability. One of the nice things about Spring MVC and Grails is that you can learn how they work very quickly. Then you can use that knowledge and don't have to look things up much. Tapestry and Wicket might be similar for those writing large apps, but I haven't found that to be as true. The more traditional MVC Frameworks just make more sense to me.

UI Feature Capability is a good one because frameworks with widgets are often popular with developers. Flex, GWT, jQuery UI, Sencha all do this very well.

Summary
The reason for this post is to add some transparency to the process of Comparing JVM Web Frameworks. I like to think that I've tried to do this in the past (especially with my reasons for rankings). Now, we'd like to hear from you, the community that uses these web frameworks.

What's the best way to compare JVM Web Frameworks?

While it's nice to hear from the generous folks that create and maintain JVM Web Frameworks, we're mostly interested in hearing from the developers that are using these things on a day-to-day basis. The Blue Collar Developers, if you will. If you could design a JVM Web Framework comparison that answered your questions, how would it look? What questions would it ask? What conclusions would make you happy? Should commercial frameworks like ZK be included?

Your responses are very greatly appreciated.

Why the bias against JSF?

2012-11-08T09:24:27-07:00

In my last post about InfoQ's Top 20 Web Frameworks for the JVM, I received a thought-provoking comment from henk53:

There is one little thing that does bother me in those presentations, and that's your fairly obvious bias against JSF.
...
If you are presenting yourself as, more or less, an authority on comparing web frameworks, then having a fairly obvious biased against one of them is just peculiar. I, all of my team, and various clients distrust your ranking of JSF. We do look at your data if the choice is between other frameworks, but as soon as JSF comes into the picture we just look elsewhere.

I'm not really sure where this bias comes from. Yes, JSF 1.0 sucked and 1.2 was only marginally better, but 2.0 is really cool and productive and there are SUPERB component and utility libraries now like PrimeFaces and OmniFaces. As a researcher of this topic I think you should keep up the date and not stick to some old grudge.

This is true, I am biased against JSF. It all started with my first JSF experience back in August 2004. If you remember correctly, 2004 was a big year: JSF 1.0, Spring 1.0 and Flex 1.0 were all released. The "AJAX" term was coined in early 2005.

By 2007 and 2008, JSF still hadn't gotten any better. In late 2009, JSF 2.0 was released and I upgraded in March 2011. As you can see from the aforementioned post, I ran into quite a few issues upgrading. JSF was also the hardest one to get working with extension-less URLs.

Most of my issues with JSF come from having maintained an application built with it since 2004. If I were to start a new application without any legacy migration issues, I imagine it wouldn't be as difficult. However, if you compare it to Struts 2 and Spring MVC, I've had little-to-no issues upgrading those applications over the years.

Also, I'm not just biased against JSF, but most component-based web frameworks. Just ask the Tapestry and Wicket folks. They've felt my criticisms over the years. My reason for preferring request-based frameworks like Struts 2/Spring MVC and Grails/Play has been because I've never seen the appeal in component-based frameworks. Often I've found that their components are just widgets that you can get from any decent JavaScript framework. And chances are that JavaScript framework can work with any web framework. Also, I've worked on a lot of high-traffic web applications that require statelessness for scalability.

I see the value in component-based frameworks, I just don't think components should be authored on the server-side. Most of the Java-based component frameworks require 2+ files for components (one for the component, one for the view, possibly one for the config). I love GWT's component concept in that you can just extract a class and re-use it. With JS frameworks, you can often just include a script. These days, when I think of good component-based frameworks, I think of jQuery UI and Twitter Bootstrap.

All that being said, there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

In other component-based frameworks in AppFuse news, Tapestry 5 has gotten really fast in the last year. I imagine this is because we have a Tapestry expert, Serge Eby, working on it. And we're planning on adding Wicket in the 2.3 release.

So even though I prefer request-based frameworks with REST support and Bootstrap, that doesn't mean everyone does. I'll do my best to be less-biased in the future. However, please remember that my view on web frameworks is as a developer, not an analyst. And aren't developers supposed to be opinionated?

InfoQ's Top 20 Web Frameworks for the JVM

2012-11-06T12:04:28-07:00

Back in early October, InfoQ.com published a community research article titled Top 20 Web Frameworks for the JVM. Their goal seemed to be fairly simple:

Using the new community research tool, we at InfoQ want to get YOUR opinions on the relative importance and maturity of a variety of web frameworks that are targeted for the JVM. Please vote by dragging each practice across two dimensions – how important is the framework relative to the other frameworks, and how much is it actually used in real teams and projects.

When I first saw this article, I noticed some strange web frameworks listed. Namely, Netty, SiteMesh and Spark. I haven't heard of many folks using Netty for a web framework, but I'm sure it's possible. SiteMesh is certainly not a web framework and I've never even heard of Spark. And where is GWT and Vaadin? Regardless of the choices, I went ahead and voted.

Last week, InfoQ posted their top content for October on Facebook.

Top 20 Web Frameworks for the JVM 25,992 PV
Ruby on Rails vs. Node.js at LinkedIn 11,904 PV
Mobile Webdev: The Horror 11,150 PV
A Brief Introduction to REST 8,872 PV
Kanban and Scrum - making the most of both 7,665 PV

First of all, it's interesting to see that JVM Web Frameworks is still a hot topic for developers. Whenever I do my Comparing JVM Web Frameworks talk at conferences, I always see a few jabs about "he's still doing that talk!?" Yes, it seems strange that a talk I first did in 2004 is still in high demand.

Secondly, I think InfoQ does good in showing how the frameworks ranked and showing their heatmaps. Below are their rankings from 1109 participants.

According to this research, the top 5 web frameworks for the JVM are Spring MVC, Play, Grails, JSF and Struts (I hope those surveyed meant Struts 2, not Struts 1).

In my research from last February (slide 21), I ranked them (with no particular weightings) as Grails, GWT, JRuby on Rails, Spring MVC and Vaadin. So I guess you could say I got 2 out of 5 right (Grails and Spring MVC). Not bad considering InfoQ didn't even consider GWT and Vaadin.

Another intriguing data point in this study is each frameworks' heatmap. For example, below are heatmaps for the top 4 frameworks.

Notice how Grails and Spring MVC are both hotter in the bottom right corner? It seems the community's overall opinions of these two frameworks are more aligned than JSF and Play, which a fair amount of folks rank as hyped and unimportant.

What I really like about this research is it's the community's opinions, visualized. It also confirms that some of my favorite frameworks are still on top. I don't know if JSF belongs as a top framework, however it seems a lot of folks do. I recently thought about removing it from AppFuse, but decided to keep it (at least for the next release). I hope InfoQ does more research projects like this, especially if they get their list of web frameworks right.

Play vs. Grails Smackdown at ÜberConf

2012-06-25T07:10:57-06:00

Play and Grails have been hyped as the most productive JVM Web Frameworks for the last couple of years. That hype has recently grown thanks to both frameworks' 2.0 releases. That's why James Ward and I decided to do a presentation at ÜberConf comparing the two. In April, we proposed the talk to Jay Zimmerman, got accepted and went to work.

How we did it
In the beginning of May, we met at a brewery in LoDo and sketched out the app we wanted to build. We also came up with a schedule for development and a plan for the presentation. We decided to build two different webapps, each with little-to-no Ajax functionality and a few features that we could use to load test and compare the applications.

We started out with the name “Happy Trails” since we both liked trails and happy hours. Later, James found that www.ubertracks.com was available and purchased the domain. We setup the Grails app to be on bike.ubertracks.com and Play/Java to be on hike.ubertracks.com. We managed our source code on GitHub, continuously tested on CloudBees and deployed to Heroku. Two weeks ago, when we were finishing up our apps, we hired a friend (Linsay Shirley) to do QA.

After fixing bugs, I emailed Patrick Lightbody, got some “cloud dollars” for Neustar Web Performance and started running load tests. The Wednesday before last, at 2 in the morning, I recorded a simple browsing regions and routes script and set it to go to 50 users over a 5 minute period and then sustain 50 for another 5 minutes. It was fun to watch the log messages whiz through my console so fast they got blurry. About halfway through testing the Grails app, there was an OOM issue, but it eventually recovered. Limiting db connections to 4 and scaling to 5 Dynos in future tests helped alleviate any issues.

We took our development experience, the load/performance testing data, and a bunch of ecosystem stats and built our smackdown presentation. We used reveal.js, GitHub Files and Google Charts to make things more dynamic.

What we found
We arrived at a number of conclusions after doing our research:

Code

From a code perspective, Play 2 and Grails 2 are very similar frameworks.
Code authoring was good in both, but lacking IDE support for Play 2's Scala Templates.
Grails Plugin Ecosystem is excellent.
TDD-Style Development is easy with both.
Type-safety in Play 2 was really useful, especially routes.

Statistical Analysis

Grails has better support for FEO (YSlow, PageSpeed)
Grails has less LOC! (6 lines less, but 40% more files)
1 Dyno - Grails had 2x transactions!
- Grails experienced OOM about halfway through.
Apache Benchmark with 10K requests:
- Play: ~10% failed requests, Grails: 0
- Requests per second: {Play: 170, Grails: 198}
- Requests per second: {Play: 251, Grails: 198}
Load Test with 100 Real Users:
- Grails: 10% more transactions, 0 errors

Ecosystem Analysis

"Play" is difficult to search for.
Grails is more mature.
Play has momentum issues.
LinkedIn: more people know Grails than Spring MVC.
Play has 3x user mailing list traffic.
We had similar experiences with documentation and questions.
Outdated documentation is a problem for both.
Play has way more hype!

We figured we spent around 100 hours developing the apps, gathering data and creating the presentation. The good news is it's all open source! This means you can clone the project on GitHub (Grails is in the grails2 branch, Play is in the play2_java branch) and help us improve it. The presentation is in the master branch in the preso directory.

All the data we gathered is open for debate and we’d love to tune our apps to handle more requests per second. In fact, we already had a contributor discover an issue and provide a fix for Play that increases its throughput from 170 req/second to 252 req/second!

Regardless of what the stats and pretty graphs say, we both enjoyed our experiences with Play 2 and Grails 2. If you haven't tried them yourself, we encourage you to do so.

Comparing Web Frameworks and HTML5 with Play Scala at Jfokus 2012

2012-02-16T00:01:05-07:00

Stockholm seems a lot like Denver this time of year. Cold, snowy and beautiful. Trish and I arrived in Stockholm (Sweden) on Monday for the Jfokus conference and we're traveling to Madrid today for the Spring I/O conference. I was invited to Jfokus within minutes of delivering my HTML5 with Play Scala talk at Devoxx.

Both the Jfokus and Spring I/O Organizers were interested in my Comparing JVM Web Frameworks talk, so I updated it to reflect my latest thoughts. First of all, I mentioned that there's a lot of great frameworks out there and I think the reason people are so apprehensive to choose one is because they've chosen badly at one point. This might've been Struts back in the day (even thought it was one of the best frameworks at the time) or it might be because a vendor talked them into it. However, if you look at the modern JVM frameworks today, you should be able to see that they're all pretty awesome.

I mentioned how I think Web developers should know JavaScript and CSS. If you're a Java developer and you call yourself a web developer, you're letting your framework do too much of the work for you. I mentioned Rich Manalang's Modern Principles in Web Development, where he talks about his core web development principles.

Designing for mobile first (even if you’re not building a mobile app)
Build only single page apps
Create and use your own REST API
“Sex sells” applies to web apps

I've found these principles to be true in my own experience and suggested that if you want to be a web developer, the frameworks you might want to learn are not traditional JVM web frameworks, but rather client-side MVC frameworks. For those Java developers that don't want to be web developers, I suggest they strengthen their services development knowledge by reading Hot to GET a Cup of Coffee.

You can see my updated presentation below, on Slideshare or as a downloadable PDF. You can also watch the video.

I delivered my 2nd presentation on HTML5 with Play Scala, CoffeeScript and Jade on Wednesday morning. This talk is one of my favorites and I prepared for it over the last several weeks by adding JSON CRUD Services and SecureSocial to my HTML5 Fitness Tracking application. Right before we left for Jfokus, I was able to get everything to work, but didn't spend as much time as I'd like working on the mobile client. If this talk gets accepted for Devoxx France, I plan on spending most of my time enhancing the mobile client. After my latest experience developing, I can see how Rich's first principle (above) makes a lot of sense.

Below is my presentation for this talk. Of course, it's on Slideshare and downloadable as a PDF.

I also updated the Developing Play More demo video to show my latest efforts.

Delivering these talks at Jfokus was a lot of fun. Yes, it was a lot of work and stress to prepare them. However, I also learned a lot creating them and I hope the audience benefitted from that.

The conference itself was incredible. I got to meet Peter Hilton and Helena Hjertén as I was registering. The speaker's dinner at F12 was off-the-hook good and I had the pleasure of finally meeting Rickard Öberg.

I also attended some fantastic presentations, including Peter Hilton's Play Framework 2.0, Bodil Stokke's CoffeeScript: JavaScript without the Fail, Pamela Fox's Client-side Storage and Heiko Seeberger's Scala in Action. I don't know if Heiko has published any slides, but I'm guessing not since most of his presentation was live coding.

I have lots of good memories from Jfokus. Many thanks to Mattias for inviting me!

Refreshing AppFuse's UI with Twitter Bootstrap

2012-01-31T17:12:17-07:00

The last time AppFuse had an update done to its look and feel was in way back in 2006. I've done a lot of consulting since then, which has included a fair bit of page speed optimization, HTML5 development and integrating smarter CSS. It was way back in '05 when we first started looking at adding a CSS Framework to AppFuse. It was Mike Stenhouse's CSS Framework that provided the inspiration and my CSS Framework Design Contest that provided its current themes (puzzlewithstyle, andreas01 and simplicity).

Since then, a lot of CSS Frameworks have been invented, including Blueprint in 2007 and Compass in 2008. However, neither has taken the world by storm like Twitter Bootstrap. From Building Twitter Bootstrap:

A year-and-a-half ago, a small group of Twitter employees set out to improve our team’s internal analytical and administrative tools. After some early meetings around this one product, we set out with a higher ambition to create a toolkit for anyone to use within Twitter, and beyond. Thus, we set out to build a system that would help folks like us build new projects on top of it, and Bootstrap was conceived.
...
Today, it has grown to include dozens of components and has become the most popular project on GitHub with more than 13,000 watchers and 2,000 forks.

The fact that Bootstrap has become the most popular project on GitHub says a lot. For AppFuse.next, I'd like to integrate a lot of my learnings over the past few years, as well as support HTML5 and modern browsers as best we can. This means page speed optimizations, getting rid of Prototype and Scriptaculous in favor of jQuery, adding wro4j for resource optimization and integrating HTML5 Boilerplate. I've used Twitter Bootstrap for my Play More! app, as well as some recent client projects. Its excellent documentation has made it easy to use and I love the way you can simply add classes to elements to make them transform into something beautiful.

Last week, I spent a couple late nights integrating Twitter Bootstrap 2.0 into the Struts 2 and Spring MVC versions of AppFuse. The layout was pretty straightforward thanks to Scaffolding. Creating the Struts Menu Velocity template to produce dropdowns wasn't too difficult. I added class="table table-condensed" to the list screen tables, class="well form-horizontal" to forms and class="btn primary" to buttons.

I also added validation errors with the "help-inline" class. This is also where things got tricky with Struts and Spring MVC. For the form elements in Bootstrap, they recommend you use a "control-group" element that contains your label and a "controls" element. The control contains the input/select/textarea and also the error message if there is one. Here's a sample element waiting for data:

<div class="control-group">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name">
    </div>
</div>

Below is what that element should look like to display a validation error:

<div class="control-group error">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name" value="">
        <span class="help-inline">Please enter your name.</span>
    </div>
</div>

You can see this markup is pretty easy, you just need to add an "error" class to control-group and span to show the error message. With Struts 2, this was pretty easy thanks to its customizable templates for its tags. All I had to do was create a "template/css_xhtml" directory in src/main/webapp and modify checkbox.ftl, controlfooter.ftl, controlheader-core.ftl and controlheader.ftl to match Bootstrap's conventions.

Spring MVC was a bit trickier. Since its tags don't have the concept of writing an entire control (label and field), I had to do a bit of finagling to get things to work. In the current implementation, Struts 2 forms have a single line for a control-group and its control-label and controls.

<s:textfield key="user.firstName" required="true"/>

With Spring MVC, it's a bit more complex:

<spring:bind path="user.firstName">
<fieldset class="control-group${(not empty status.errorMessage) ? ' error' : ''}">
</spring:bind>
    <appfuse:label styleClass="control-label" key="user.firstName"/>
    <div class="controls">
        <form:input path="firstName" id="firstName" maxlength="50"/>
        <form:errors path="firstName" cssClass="help-inline"/>
    </div>
</fieldset>

You could probably overcome this verbosity with Tag Files.

Figuring out if a control-group needed an error class before the input tag was rendered was probably the hardest part of this exercise. This was mostly due to Bootstrap's great documentation and useful examples (viewed by inspecting the markup). Below are some screenshots of the old screens and new ones.

Check out the full set on Flickr if you'd like a closer look.

Even though I like the look of the old UI, I can't help but think a lot of the themes are designed for blogs and content sites, not webapps. The old Wufoo forms were a lot better looking though. And if you're going to develop kick-ass webapps, you need to make them look good. Bootstrap goes a long way in doing this, but it certainly doesn't replace a good UX Designer. Bootstap simply helps you get into HTML5-land, start using CSS3 and it takes the pain out of making things work cross-browser. Its fluid layouts and responsive web design seems to work great for business applications, which I'm guessing AppFuse is used for the most.

I can't thank the Bootstrap developers enough for helping me make this all look good. With Bootstrap 2 dropping this week, I can see myself using this more and more on projects. In the near future, I'll be helping integrate Bootstrap into AppFuse's Tapestry 5 and JSF versions.

What do you think of this CSS change? Do you change your CSS and layout a fair bit when starting with AppFuse archetypes? What can we do to make AppFuse apps look better out-of-the-box?

Update: I updated AppFuse to the final Bootstrap 2.0 release. Also, Johannes Geppert wrote a Struts 2 Bootstrap Plugin. I hope to integrate this into AppFuse in the near future.

2011 - A Year in Review

2012-01-11T09:45:20-07:00

2010 was an awesome year, but 2011 rocked the house! At the end of last year, I kept my goals simple:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus. Everything else is gravy.

As predicted, it was a spectacular year, but I only accomplished 50% of my goals. That is, Trish and I had a great time skiing (especially in Utah), moving in together, traveling the world and getting engaged in Versailles. I even satisfied some 5-year goals: building a sauna in my basement and getting a ski shack in the mountains.

However, I didn't get much done with The Bus. Or rather, the guys at MotorWorks Restorations didn't drain me for all I'm worth in 2011. We did make good progress with estimating the final cost and obtaining many hard-to-find parts though. I now have a Porsche 911 Engine (1983 3.0L 6 cylinder), a Porsche 901 5 speed transmission, Porsche "Turbo Twist" wheels and a Custom Air Ride Front Beam from Franklin's VW Works. The thing that slowed our progress the most was the custom beam, as it took almost 6 months from order to delivery. When it arrived in September, I decided to put things on hold. I didn't want to get my bus back in the midst of winter and not be able to drive it.

In January, I started working at Overstock.com and flew out to Salt Lake City the first two weeks to ramp up. In setting up my development environment, I discovered some inefficiencies and wrote about how I fixed them in Making Code Generation Smarter with Maven. I also discovered that JRebel slowed my build process down by almost 50% and stopped using it as part of my daily process. Now I simply run in debug mode or use fancy frameworks that allow instant reload.

At the end of the month, I skied Alta for the first time on a blue bird day. That night, I flew back to Denver, had a gorgeous lunch ride in 70°F weather on Friday afternoon, then picked up my lovely girlfriend, Trish, and drove to Crested Butte for a weekend of skiing with James Ward and friends. It was my first time visiting Crested Butte and I can see why people call it Paradise.

As of February 1st, Trish and I both had 17 ski days. As I'm writing this, we're both up to 14 for this season and very proud to say Abbie and Jack each have 10. Both kids are shredding like I've always dreamed they would.

My parents (both retired) came to visit for the month of February and helped me start building a sauna in my basement. This was something I've wanted to do for years, so it was a lot of fun getting things started. While we didn't finish construction, we did get the plumbing/electrical done, stove installed and walls framed. We also had a fun night with Chef Navarro and Friends.

The rest of my posts in February were purely technical. I wrote about how I implemented extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF. Then I started delving into security-related topics, largely motivated by a new feature I was working on at Overstock. When I wrote about Implementing Ajax Authentication using jQuery, Spring Security and HTTPS, I explained what I was trying to accomplish.

The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience.

I was able to get everything working, posted a demo and wrote about upcoming conferences in Las Vegas and Kraków. In that article, I described a new talk I was going to create: Webapp Security: Develop. Penetrate. Protect. Relax. I ended the month with a post on how to fix XSS in JSP 2.

On February 25th, I experienced The Greatest Snow on Earth with thigh-deep powder and face shots on every run.

That night, we saw Hot Buttered Rum and woke up early for 27" of fresh powder at Alta. The picture of Trish free-heeling at Alta is one of my favorites of the year.

In early March, we experienced a suite night at the Avs game, then I kept it technical and upgraded this blog to Apache Roller 5.0, RC4. I upgraded AppFuse to JSF 2, wrote about JSR 303 and web framework support and added search to AppFuse using Compass. When writing about JSF 2, I mentioned replacing Ajax4JSF with RichFaces. 9 months later, it seems like PrimeFaces has more momentum, so I may try that instead.

For St. Patrick's Day, we flew to Vegas for TSSJS and had a blast. Not only did my talks go well, but we had a bunch of friends in town and had much success gambling the nights away.

After delivering my Comparing JVM Web Frameworks talk, I had a very interesting conversation with James Strachan. He recommended I take a look at CoffeeScript and Jade. A couple months later, when I was thinking of talks for Devoxx, I added these to my list of technologies to learn and submitted a talk about HTML5 with Play Scala, CoffeeScript and Jade.

That Friday, we returned to Denver, got a good night's sleep then headed on a hut trip in Colorado's High Country. The name of the hut was Peter Estin Hut and it was a bit of a hike to get to. My friend, Joe, set my expectations correctly when he warned me it'd be a 5 hour death march. It took us 4 hours, 30 minutes and we skied up 2200 vertical feet of switchbacks.

Trish and I started off April with a bang by moving in together, then embarking on a whirlwind trip of two opening days with a stopover in Kraków. We flew Business Class from Chicago to Munich (releasing AppFuse 2.1 just before departing) and were very impressed with Kraków's Old Town upon arrival.

On that trip, I especially enjoyed meeting Trish's brother for the first time and watching the Red Sox win their first game of the year.

After returning, I realized I lost a lot of data in my Roller 5 upgrade and wrote about what happened and how I recovered it. Then we bid farewell to the 2010-2011 ski season and got ready for mountain bike season.

For next year, I think I'll keep my goal at 30 days. If everything works out as planned, we'll have a place in the mountains this fall and it'll be a bit easier to hit the slopes without sitting in traffic. For now, I'm pumped about the beginning of mountain bike season.

May arrived and Trish and I started looking for a place in the mountains. We found an very cool cabin, but later decided it was too nice and we'd rather have a condo. I finished The Basement Sauna Project and wrote about Java Web Application Security: Java EE 6, Spring Security and Apache Shiro.

On May 21st, we again visited Winter Park and Fraser to look at condos, then drove back to Denver for U2 at Mile High Stadium. I'm not a big fan of U2, but this was quite possibly the best concert I've ever been to. The experience of 70,000+ fans, all screaming, doing the wave, and loving life was epic. We decided that night that we liked one of the condos more than the rest and put in an offer the following week. The kids headed off to Florida for a month with their Mom and we drove to Utah for Desert Rocks and mountain biking in Moab.

I Upgraded to Roller 5.0 in June and showed how to add a Facebook Like button. Next, I transcribed a couple more Java Web Application Security articles: programmatic login APIs and penetrating with Zed Attack Proxy.

The kids returned to us just before the annual Father's Day Camping Trip. We drove to Lake Granby and stayed at Stillwater Campground for the weekend.

From my Father's Day post:

I really enjoyed spending so much time with my family this weekend. The good news is we get to do it again when I meet them at The Cabin in Montana next week. In the meantime, Trish and I are off to Jazoon for a few days of fun in Switzerland.

A few days later, we were in Zurich for the first time, enjoying our accommodations and marveling at Switzerland's excellent transportation system. I experienced an interesting situation while speaking, but quickly forgot when we visited Rhine Falls, celebrated our anniversary at Gruelich and played in the Swiss Alps.

We arrived back in Denver at 6pm on Sunday, June 26th and I flew to Montana at 8am the next morning. I worked from The Cabin that week and wrote a book review for Agile Hiring.

I think Sean [Landis] does a great job in showing how you can apply agile principles (people over process, tracking tools, failing fast, continuous improvement, constant feedback) to improve your hiring process. As a person that interviews often, I think it also gives great insights into how companies interview and what they're looking for. I've often thought that being honest about my skills and what I'm looking for is a good tactic and this book seems to confirm that.

That weekend, it was the "90s Reunion" for the Seeley Swan High School, which I attended my freshman and sophomore year of high school. There were several highlights of the trip, but one of my favorites was our float in the Swan Valley Parade. Abbie came up with the idea (Happy Birthday America) and we decorated an old trailer with streamers, balloons and a birthday cake to celebrate. Abbie made up a song and sang it throughout the entire parade (with a portable microphone and amp).

I went swimming with my iPhone 4 that Sunday and Trish and her Dad showed up a few hours later. On the 4th, we had a blast celebrating the holiday with mine and my parents' old friends. The next day, we adventured to Glacier National Park, where Trish shot an awesome sunrise at Lake McDonald. She also captured a beautiful picture of The Cabin at sunset that week.

In between vacationing in Montana and writing about it, I wrote some technical posts:

We returned to Denver in mid-July and shipped the kids off to Florida just before my annual Birthday Golf Debacle. We teed off at Wellshire Golf Course and had one of the most well-behaved rounds in a few years. The next weekend, we closed on our "Ski Shack" in Fraser and I penned Another Dream Realized: Mountain Views. Trish's October Sunset photo is one of my favorite glimpses of our view.

Next, I blogged about How do you get started in programming? and we headed to the airport a few hours later for a whirlwind trip from Oregon to Cape Cod. That was a wonderful trip and it took me almost a month to write about it. We flew into Portland and spent the weekend hiking in Silver Falls and drinking good beer at the Oregon Brewers Festival and Edgefield. We stayed with my good friends Clint and Autumn (married in Costa Rica in 2008), who had their first child only a few weeks after our visit (August 26th).

We had an awful early flight out of Portland to Boston that Sunday. On Monday afternoon, we picked up Abbie and Jack from the Boston airport. They flew up from West Palm Beach on their first unaccompanied minors flight. We drove them to Cape Cod and settled into Trish's parents house near the water. The rest of the week, we had a blast with our friends Chris and Julie and the entire McGinity clan.

We spent the rest of August going to concerts at Red Rocks, beer festivals in Winter Park, mountain biking, hiking, taking the kids to the Ski Shack and playing with Crazy Sagan. In my only August post, I talked a lot about the Broncos, Hawaii, Devoxx and leaving Overstock.

That pretty much catches you up-to-date with the goings on in my life. As you can tell, it's been a great summer so far. I'm really looking forward to the fall too. Broncos season, a trip to Hawaii, learning new technologies, talking about them at Devoxx and a new gig.
...
Tomorrow is Jack's birthday. I gotta run ... it's time to go buy a bike, play a little golf, hit the pool and go to the BBQ before tonight's CD Release Party.

We watched proudly as Jack turned 7 the next day and loved his excitement when he got his first mountain bike.

September started with mountain breezes and fishing at Lake Granby. Lots of friends, family and delicious food make for a heckuva 3-day weekend. Jack was the only one to catch a fish and you couldn't wipe the smile off his face for the rest of the day.

I described my Digital Diet and wrote my first article related to my Devoxx talk: Integrating Scalate and Jade with Play 1.2.3. The 2nd weekend in September, we enjoyed a September Sports Weekend, going to two Rockies games and the Monday night Broncos Opener before heading to Kauai. While the Broncos didn't win that game, Trish's shot of Brett, Joe and I sums up many-a-game this season.

As expected, Kauai blew our minds. We saw several spectacular sunsets, did some Stand Up Paddling (it's harder than it looks), played golf, took a doorless helicopter ride and kayaked along the Na Pali coast.

We never got bored of the view from our room.

Upon our return, I tried to make CoffeeScript work with Scalate and integrated HTML5 Boilerplate with Play.

October was a quiet month on this blog. I only wrote two articles, one about increasing my developer happiness and another about developing with HTML5, CoffeeScript and Twitter's Bootstrap. Watching the Broncos in October was painful at first, buy many rejoiced when Tebow started against Miami (10/23) and we had the amazing comeback to win. We went to the next game against the Lions and were quickly shown they had a long ways to go. We dressed up as the Scary Family for Halloween and had a ball.

I made up for October's lack of writing by writing 9 articles in November. I started with complaints about Play Scala's Anorm and PostgreSQL, then celebrated Abbie's 9th Birthday with a Magic Show.

The next week, I wrote about my new MacBook Pro and iMac and more Scalate goodness for Play. At this point, I was scrambling to finish up my Play More! application for Devoxx and worked a lot of late nights. That weekend, I used PhoneGap to turn my webapp into a native app and finished everything in the nick of time. We flew to Paris that Monday, took the train to Antwerp on Tuesday and settled in for a memorable week of interesting talks.

That Saturday, we took the train back to Paris and checked into our hotel for a few days of romance. I proposed to Trish at Versailles on Sunday at sunset and we rolled around in our joy for a few days before we told any of our friends.

We flew back to the US the Wednesday before Christmas and spent the weekend in Boston with Trish's family. For full details of that trip, see Our Engaging Trip to Paris and Antwerp.

To end the year, I kept it light and only spun up one blog post in December: What have I been working on at Taleo? A week after writing that post, we moved to the Ski Shack for 3 weeks and enjoyed a wonderful holiday break with many visitors.

Yes, 2011 was a spectacular year. We traveled to four different countries, did a lot of camping and mountain biking and accomplished 2/3 of my 5 year goals (basement sauna and mountain views). I wrote a lot of technical blog posts in the 2nd half of the year; most inspired my extracurricular activities. I plan to continue that this year as I learn more about HTML5, Play Framework, Scala and CoffeeScript. At the end of last year's Year in Review, I wrote:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus.

I think I can safely say I kept it rockin' with Trish - we're living together, engaged and still amazed we found one another. As for The Bus, I think MotorWorks Restorations and I are in a good position to make a final push and finish the sucker. My hope is to have it done by May so I can start driving it and taking it to VW Shows. In the meantime, Trish has started working on McGinity Photo full-time and loving it. We have trips planned to Stockholm (for Jfokus), Madrid (for Spring I/O) and Paris (for Devoxx Paris).

Last weekend, we witnessed one of the greatest playoff games ever when the Broncos beat the Steelers. I think the picture of Trish celebrating the final touchdown pretty much sums up 2011.

I look forward to all the skiing, celebrating, camping, traveling and bus rides in 2012. It's gonna be a fantastic year.

Deploying Java and Play Framework Apps to the Cloud with James Ward

2011-11-18T08:14:45-07:00

Yesterday, I attended James Ward's presentation on Deploying Java & Play Framework Apps to the Cloud at Devoxx. I arrived a bit late, but still managed to get there in time to see a lot of demos and learn more about Heroku. Below are my notes from James's talk.

When I arrived, James was doing a demo using Spring Roo. He was using Roo's Petclinic sample app and showed us how you could use Git to create a local repository of the new project and install Heroku's command line tool. From there, he ran the following command to create a new application on Heroku.

heroku create -s cedar

The Cedar Stack is what supports Java, Scala and Play Framework. It's the 3rd generation stack for Heroku. The command above created two endpoints, one for HTTP and one for Git. It picks from a list of randomly generated names, which all seem to have some humor in them. James ended up with "electric-sword-8877" for this demo.

From there, he ran git push heroku master to deploy the project to Heroku. Unfortunately, this resulted in a login error and there was an akward moment where we all thought the Demo Gods were angry. However, James was able to resolve this by using Heroku's sharing feature with the following command.

heroku sharing:add jw@heroku.com

For Java projects, Heroku looks for a pom.xml file in the root directory and runs a Maven build on project. All the dependencies get downloaded on the cloud rather than put them into a WAR and requiring you to upload a large WAR file. You don't have to upload your source code to Heroku; James did it for the sake of the demo because it was faster.

After the build finishes, it creates a slug file. This file contains everything Heroku needs to run your application.

Next, James showed a demo of the running application and added a new Pet through its UI. Then he scaled it to two servers using the following command:

heroku scale web=2

He proved this was working by running heroku ps, which showed there were two running processes. He showed the app again, but noted that the record he added was missing. This is because when it started up a new dyno, Hibernate created the schema again and deleted all records. To fix, James changed Hibernate to only update the schema instead of create a new one. If you're a Hibernate user, you know this is as simple as changing:

hibernate.hbm2ddl.auto=create

to:

hibernate.hbm2ddl.auto=update

After committing this change, James redeployed using Git.

git push heroku master

The slug file got built again and Heroku deployed the new slug onto both dynos, automatically load balancing the app across two servers. James then ran heroku logs to see the logs of his dynos and prove that a request to his app's HTTP endpoint made requests to both dynos. The logging is powered by Logplex and you can read about how it works in the article Heroku Gets Sweet Logging.

James mentioned that Roo has a Heroku plugin, but after watching his talk and searching a bit on the internet, it seems it's just the jetty-runner setup as described in Getting Started with Spring MVC Hibernate on Heroku/Cedar.

What about autoscaling? There are some 3rd party tools that do this. Heroku's Management infrastructure has APIs that these tools talk too. Heroku hasn't built autoscaling into the platform because they don't know where the bottlenecks are in your application.

Heroku = Polyglot + PaaS + Cloud Components. It supports Ruby, node.js, Java, Clojure, Play and Scala and they're working on native Grails and Gradle support. There's currently 534,374 apps running on Heroku.

Heroku is a cloud application platform and there's 5 different components.

Instant deployment
HTTP Routing / Load Balancing
Elastic Polyglot Runtime
Management & Logging
Component as a Service Ecosystem

For instant deployment, it's a pretty simple process:

You add files to a git repo
You provision the app on Heroku (heroku create)
You upload the files to Heroku (git push heroku master)
Heroku runs the build and assembles a "slug" file
Heroku starts a "dyno"
Heroku copies the "slug" to the "dyno"
Heroku starts the web application

Most apps will contain a Procfile that contains information about how to run the web process. For Spring Roo, it has:

web: java $JAVA_OPTS -jar target/dependency/jetty-runner.jar --port $PORT target/*.war

So how does Heroku decide what application server to use? It doesn't, you do. You need to get your application server into the slug file. The easiest way to do this is to specify your application server as a dependency in your pom.xml. In the Roo example, James uses the maven-dependency-plugin to get the jetty-runner dependency and copy it to the target directory. On Heroku, you bring your application server with you.

Heroku gives you 750 free dyno hours per app, per month. For developers, it's very easy to get started and use. Once you extend past one dyno, it's $.05 per dyno hour, which works out to around $30/month. It's only when you want to scale beyond one dyno where you get charged by Heroku, no matter how much data you transfer. Scalatest is running on Heroku. It has one dyno and is doing fine with that. Bill Venners doesn't have to pay anything for it.

java.herokuapp.com is a site James created that allows you to clone example apps and get started quickly with Heroku's Cedar Stack.

For HTTP Routing, Heroku uses an Erlang-based routing system to route all the HTTP requests across your dynos. Heroku doesn't support sticky sessions. Distributed session management does not work well, because it does not scale well. Heroku recommends you use a stateless web architecture or move your state into something like memcached. Jetty has (in the latest version) the ability to automatically serialize your session into a Mongo system. This works fine on Heroku. The problem with this is if you have 2 dynos running, each request can hit a different dyno and get different session state. Hence the recommendation for an external storage mechanism that can synchronize between dynos.

You can also run non-web applications on Heroku. You can have one web process, but as many non-web processes as you want.

Heroku has native support for the Play framework. To detect Play applications, it look for a conf/application.conf file. You don't need to have a Procfile in your root directory because Heroku knows how to start a Play application.

At this point, James created a new Play application, created a new Heroku app (he got "young-night-7104" this time) and pushed it to Heroku. He created a simple model object, a controller to allow adding new data and then wrote some jQuery to show new records via Ajax and JSON. He also showed how to configure the application to talk to Heroku's PostgreSQL database using the DATABASE_URL environment variable. He explained how you can use the heroku config command to see your environment variables.

The reason they use environment variables is so Heroku can update DATABASE_URL (and other variables) without having to call up all their customers and have them change them in their source code.

Play on Heroku supports Scala if you create your app with Scala. Play 2.0 uses Scala, Akka and SBT. Heroku added support for SBT a couple month ago, so everything will work just fine.

Heroku also supports Scala, detecting it by looking for the build.sbt file in the root directory. Heroku supports SBT 0.11.0 and it builds the 'stage' task. It currently does not support Lift because Lift uses an older version of SBT and because it's a very stateful framework that would require sticky sessions. Use Play, BlueEyes or Scalatra if you want Scala on Heroku.

Heroku has addons for adding functionality to your application, including Custom DNS, HTTPS, Amazon RDS, NoSQL and many more. They're also working on making their add-on and management APIs available via Java, so you'll (hopefully) be able to use them from your IDE in the future.

From there, James showed us how Heroku keeps slug files around so you can do rollbacks with heroku rollback. He also showed how you can use:

heroku run "your bash command"

to run any Bash command on the cloud.

Summary
I attended James's talk because he's a good friend, but also because I've been using Heroku to host my latest adventures with Play, Scala, CoffeeScript and Jade. I'm glad I attended because I learned some good tips and tricks and more about how Heroku works.

Heroku seems like a great development tool to me. In my experience, it's been really nice to have instant deployments using Git. In fact, I've created a 'push' alias so I can push to my project's repo and heroku at the same time.

alias push='git push origin master && git push heroku master'

I'd like to see more organizations embrace something like Heroku for developers. It'd be great if everyone had their own sandbox that business owners and product managers could see. I can't help but think this would be awesome for demos, prototyping, etc.

There were some other talks I wanted to attend at the same time, particularly Martin Odersky's What's in store for Scala? and WWW: World Wide Wait? A Performance Comparison of Java Web Frameworks. The WWW talk has posted their presentation but I'm sure it'd be more fun to watch.

It's pretty awesome that all the talks from Devoxx 2011 will be up on Parleys.com soon.

Update: James has posted his slides from this talk.

Play 2.0, A web framework for a new era

2011-11-16T05:58:09-07:00

This week, I'm in Antwerp, Belgium for the annual Devoxx conference. After traveling 21 hours door-to-door yesterday, I woke up and came to the conference to attend some talks on Play and PhoneGap. I just got out of the session on Play 2.0, which was presented by Sadek Drobi and Guillaume Bort. Below are my notes from this presentation.

The Play 2.0 beta is out! You can read more about this release on the mailing list. This beta includes native support for both Scala and Java, meaning you can use both in the same project. The release also bundles Akka and SBT by default.

In other news, Play 2.0 is now part of the Typesafe Stack. Typesafe is the Scala company, started by the founder of Scala (Martin Odersky) and the founder of Akka (Jonas Bonér). Guillaume is also joining the Typesafe Advisory Board.

Sadek and Guillaume both work at zenexity, where Play is the secret weapon for the web applications they've built for the last decade. Play was born in the real world. They kept listening to the market to see what they should add to the project. At some point, they realized they couldn't keep adding to the old model and they needed to create something new.

The web has evolved from static pages to dynamic pages (ASP, PHP). From there, we moved to structured web applications with frameworks and MVC. Then the web moved to Ajax and long-polling to more real-time, live features. And this changes everything.

Now we need to adapt our tools. We need to handle tremendous flows of data. Need to improve expressiveness for concurrent code. We need to pick the appropriate datastore for the problem (not only SQL). We need to integrate with rapidly-evolving client side technologies like JavaScript, CoffeeScript, and Dart. We need to use elastic deployment that allows scaling up and scaling down.

zenexity wanted to integrated all of these modern web needs into Play 2.0. But they also wanted to keep Play approachable. They wanted to maintain fast turnaround so you can change your code and hit reload to see the changes. They wanted to keep it as a full stack framework with support for JSON, XML, Web Services, Jobs, etc. And they wanted to continue to use and conventions over configuration.

At this point, Guillaume did a Play 2.0 Beta demo, show us how it uses SBT and has a console so everything so it runs really fast. You can have both Scala and Java files in the same project. Play 2.0 templates are based on Scala, but you don't need to know Scala to use them. You might have to learn how to write a for loop in Scala, but it's just a subset of Scala for templates and views. SBT is used for the build system, but you don't have to learn or know SBT. All the old play commands still work, they're just powered by a different system.

After the demo, Sadek took over and started discussing the key features of Play 2.0.

To handle tremendous amounts of data, you need to do chunking of data and be able to process a stream of data, not just wait until it's finished. Java's InputStream is outdated and too low level. Its read() method reads the next byte of data from the input and this method can block until input data is available.

To solve this, Play includes a reactive programming feature, which they borrowed from Haskell. It uses Iteratee/Enumerator IO and leverages inversion of control (not like dependency injection, but more like not micro-managing). The feature allows you to have control when you need it so you don't have to wait for the input stream to complete. The Enumerator is the component that sends data and the Iteratee is the component that receives data. The Iteratee does incremental processing and can tell the Enumerator when it's done. The Iteratee can also send back a continuation, where it tells the Enumerator it wants more data and how to give it. With this paradigm, you can do a lot of cool stuff without consuming resources and blocking data flow.

Akka is an actor system that is a great model for doing concurrent code. An Actor could be both an Enumerator and an Iteratee. This vastly improves the expressiveness for concurrent code. For example, here's how you'd use Akka in Play:

def search(keyword: String) = Action {
  AsyncResult {
    // do something with result
  }
}

Play does not try to abstract data access because datastores are different now. You don't want to think of everything as objects if you're using something like MongoDB or navigating a Social Graph. Play 2.0 will provide some default modules for the different datastores, but they also expect a lot of contributed modules. Anorm will be the default SQL implementation for Play Scala and Ebean will be the default ORM implementation for Play Java. The reason they've moved away from Hibernate is because they needed something that was more stateless.

On the client side, there's so many technologies (LESS, CoffeeScript, DART, Backbone.js, jQuery, SASS), they didn't want to bundle any because they move too fast. Instead, there's plugins you can add that help you leverage these technologies. There's a lot of richness you can take advantage of on the client side and you need to have the tools for that.

Lastly, there's a new type of deployment: container-less deployment to the cloud. Akka allows you to distribute your jobs across many servers and Heroku is an implementation of elastic deployment that has built-in support for Play.

They've explained what they tried to design and the results of this new, clean architecture have been suprising. Side effects include: type-safety everywhere for rock-solid applications. There's an awesome performance boost from Scala. There's easier integration with existing projects via SBT. And it only takes 10 lines of code to develop an HTTP Server that responds to web requests.

The memory consumption is amazing: only 2MB of heap is used when a Play 2.0 app is started. Tests on Guillaume's laptop have shown that it can handle up to 40,000 requests per second, without any optimization of the JVM. Not only that, but after the requests subside, garbage collection cleans up everything and reduces the memory consumption back to 2MB.

At this point, Guillaume did another demo, showing how everything is type-safe in 2.0, including the routes file. If you mistype (or comment one out) any routes, the compiler will find it and notify you. Play 2.0 also contains a compiled assets feature. This allows you to use Google's Closure Compiler, CoffeeScript and LESS. If you put your LESS files in app/assets/stylesheets, compilation errors will show up in your browser. If you put JavaScript files in app/assets/javascripts, the Closure compiler will be used and compilation errors will show up in your browser.

Play 2.0 ships with 3 different sample applications, all implemented in both Java and Scala. HelloWorld is more than just text in a browser, it includes a form that shows how validation works. Another app is computer-database. When Guillaume started it, we saw how evolutions were used to create the database schema from the browser. The Play Team has done their best to make the development process a browser-based experience rather than having to look in your console. The computer-database is a nice example of how to do CRUD and leverages Twitter's Bootstrap for its look and feel.

The last sample application is zentasks. It uses Ajax and implements security so you can see how to create a login form. It uses LESS for CSS and CoffeeScript and contains features like in-place editing. If you'd like to see any of these applications in action, you can stop by the Typesafe booth this week at Devoxx.

Unfortunately, there will be no migrating path for Play 1.x applications. The API seems very similar, but there are subtle changes that make this difficult. The biggest thing is how templating has changed from Groovy to Scala. To migrate from 1.2.x would be mostly a copy/paste, modify process. There are folks working on getting Groovy templates working in 2.0. The good news is zenexity has hundreds of 1.x applications in production, so 1.x will likely be maintained for many years.

Summary
This was a great talk on what's new in Play 2.0. I especially like the native support for LESS and CoffeeScript and the emphasis on trying to keep developers using two tools: their editor and the browser. The sample apps look great, but the documentation look sparse. I doubt I'll get a chance to migrate my Play 1.2.3 app to 2.0 this month, but I hope to try migrating sometime before the end of the year.

More Scalate Goodness for Play

2011-11-07T14:07:40-07:00

This article is the 6th in a series on about my adventures developing a web application with HTML5, Play Scala, CoffeeScript and Jade. Previous articles can be found at:

Last week, I wrote about my adventures with Anorm and mentioned I'd made some improvements to Scalate Play interoperability. First of all, I've been using a Scalate trait and ScalateTemplate class to render Jade templates in my application. I described this setup in my first article on Scalate and Play.

Adding SiteMesh Features and Default Variables
When I started making my app look good with CSS, I started longing for a feature I've used in SiteMesh. That is, to have a body id or class that can identify the page and allow per-page CSS rules. To do this with SiteMesh, you'd have something like the following in your page:

  
<body id="signup"/>

And then read it in your decorator:

<body<decorator:getProperty property="body.class" writeEntireProperty="true"/>>

As I started looking into how to do this, I came across Play Scala's ScalaController and how it was populating Play's default variables (request, response, flash, params, etc.). Based on this newfound knowledge, I added a populateRenderArgs() method to set all the default variables and my desired bodyClass variable.

def populateRenderArgs(args: (Symbol, Any)*): Map[String, Any] = {
  val renderArgs = Scope.RenderArgs.current();

  args.foreach {
    o =>
      renderArgs.put(o._1.name, o._2)
  }

  renderArgs.put("session", Scope.Session.current())
  renderArgs.put("request", Http.Request.current())
  renderArgs.put("flash", Scope.Flash.current())
  renderArgs.put("params", Scope.Params.current())
  renderArgs.put("errors", validationErrors)
  renderArgs.put("config", Play.configuration)

  // CSS class to add to body
  renderArgs.put("bodyClass", Http.Request.current().action.replace(".", " ").toLowerCase)
  renderArgs.data.toMap
}

implicit def validationErrors:Map[String,play.data.validation.Error] = {
  import scala.collection.JavaConverters._
  Map.empty[String,play.data.validation.Error] ++ 
    Validation.errors.asScala.map( e => (e.getKey, e) )
}

After adding this method, I was able to access these values in my templates by defining them at the top:

-@ val bodyClass: String 
-@ val params: play.mvc.Scope.Params
-@ val flash: play.mvc.Scope.Flash

And then reading their values in my template:

body(class=bodyClass)
...
- if (flash.get("success") != null) {
  div(class="alert-message success" data-alert="alert")
    a(class="close" href="#") &×
    | #{flash.get("success")}
- }
...
  fieldset
    legend Leave a comment →
    div.clearfix
      label(for="author") Your name:
      input(type="text" name="author" class="xlarge" value={params.get("author")})
    div.clearfix
      label(for="content") Your message:
      textarea(name="content" class="xlarge") #{params.get("content")}
    div.actions
      button(type="submit" class="btn primary") Submit your comment
      button(type="reset" class="btn") Cancel

For a request like Home/index, the body tag is now rendered as:

<body class="home index">

This allows you to group CSS styles by Controller names as well as by method names.

Next, I started developing forms and validation logic. I quickly discovered I needed an action() method like the one defined in ScalaTemplate's TemplateMagic class.

def action(action: => Any) = {
  new play.mvc.results.ScalaAction(action).actionDefinition.url
}

Since TemplateMagic is an inner class, I determined that copying the method into my ScalateTemplate class was the easiest workaround. After doing this, I was able to import the method and use it in my templates.

-import controllers.ScalateTemplate._
...
form(method="post" class="form-stacked" id="commentForm"
     action={action(controllers.Profile.postComment(workout._1.id()))})

After getting the proper URL written into my form's action attribute, I encountered a new problem. The Play Scala Tutorial explains validation flow as follows:

if (Validation.hasErrors) {
  show(postId)
} else {
  Comment.create(Comment(postId, author, content))
  Action(show(postId))
}

However, when I had validation errors, I end up with the following error:

Could not load resource: [Timeline/postComment.jade]

To fix this, I added logic to my Scalate trait that looks for a "template" variable before using Http.Request.current().action.replace(".", "/") for the name. After making this change, I was able to use the following code to display validation errors.

if (Validation.hasErrors) {
  renderArgs.put("template", "Timeline/show")
  show(postId)
} else {
  Comment.create(Comment(postId, author, content))
  Action(show(postId))
}

Next, I wanted to give child pages the ability to set content in parent pages. With SiteMesh, I could use the <content> tag as follows:

<content tag="underground">
  HTML goes here
</content>

This HTML could then be retrieved in the decorator using the <decorator:getProperty> tag:

<decorator:getProperty property="page.underground"/>

With Scalate, I found it equally easy using the captureAttribute() method. For example, here's how I captured a list of an athlete's workouts for display in a sidebar.

- captureAttribute("sidebar")
  - Option(older).filterNot(_.isEmpty).map { workouts =>
    .older-workouts
      h3
        | Older workouts
        span.from from this app
      - workouts.map { workout =>
        - render("workout.jade", Map('workout -> workout, 'mode -> "teaser"))
      - }
  - }
- }

Then in my layout, I was able to retrieve this and display it. Below is a snippet from the layout I'm using (copied from Twitter's Bootstrap example). You can see how the sidebar is included in the .span4 at the end.

-@ val sidebar: String = ""
...
.container
  .content
    .page-header
      h1
        = pageHeader
        small
          = pageTagline
    .row
      .span10
        !~~ body
      .span4
        = unescape(sidebar)
  footer

View vs. Render in Scalate
In the sidebar code above, you might notice the render() call. This is the Scalate version of server-side includes. It works well, but there's also a view() shortcut you can use if you want to have templates for rendering your model objects. I quickly discovered it might be difficult to use this feature in my app because my object was Option[(models.Workout, models.Athlete, Seq[models.Comment])] instead of a simple object. You can read the view vs. render thread on the Scalate Google Group if you're interested in learning more.

Scalate as a Module
The last enhancement I attempted to make was to put Scalate support into a Play module. At first, I tried overriding Play's Template class but ran into compilation issues. Then Guillaume Bort (Play's lead developer) recommended I stick with the trait approach and I was able to get everything working. I looked at the outdated play-scalate module to figure out how to add Scala support to build.xml and copied its 500.scaml page for error reporting.

In order to get line-precise error reporting working, I had to wrap a try/catch around calling Scalate's TemplateEngine.layout() method. Again, most of this code was copied from the outdated play-scalate module.

case class Template(name: String) {
  
  def render(args: (Symbol, Any)*) = {
    val argsMap = populateRenderArgs(args: _*)
    
    val buffer = new StringWriter()
    var context = new DefaultRenderContext(name, scalateEngine, new PrintWriter(buffer))
    
    try {
      val templatePath = new File(Play.applicationPath+"/app/views","/"+name).toString
        .replace(new File(Play.applicationPath+"/app/views").toString,"")
      scalateEngine.layout(templatePath + scalateType, argsMap)
    } catch {
      case ex:TemplateNotFoundException => {
        if(ex.isSourceAvailable) {
          throw ex
        }
        val element = PlayException.getInterestingStrackTraceElement(ex)
        if (element != null) {
           throw new TemplateNotFoundException(name, 
             Play.classes.getApplicationClass(element.getClassName()), element.getLineNumber());
        } else {
           throw ex
        }
      }  
      case ex:InvalidSyntaxException => handleSpecialError(context,ex)
      case ex:CompilerException => handleSpecialError(context,ex)
      case ex:Exception => handleSpecialError(context,ex)
    } finally {
      if (buffer.toString.length > 0)
        throw new ScalateResult(buffer.toString,name)
    }
  }
}
...
private def handleSpecialError(context:DefaultRenderContext,ex:Exception) {
  context.attributes("javax.servlet.error.exception") = ex
  context.attributes("javax.servlet.error.message") = ex.getMessage
  try {
    scalateEngine.layout(scalateEngine.load(errorTemplate), context)
  } catch {
    case ex:Exception =>
      // TODO use logging API from Play here...
      println("Caught: " + ex)
      ex.printStackTrace
  }
}

private def errorTemplate:String = {
  val fullPath = new File(Play.applicationPath,"/app/views/errors/500.scaml").toString 
  fullPath.replace(new File(Play.applicationPath+"/app/views").toString,"")
}

Once I had this in place, error messages from Scalate are much better. Not only do I see the error in my browser, but I can click on the offending line to open it directly in TextMate.

I've published my play-scalate module on GitHub so others can try it out. To give it a whirl, add the following to your dependencies.yml:

    - upgrades -> play-scalate 0.1

repositories:
    - upgrades:
        type: http
        artifact: "http://static.raibledesigns.com/[module]-[revision].zip"
        contains:
            - upgrades -> *

Then add with play.modules.scalate.Scalate to your controllers and call the render() method.

Summary
After using Scalate and Play for almost 3 months, I'm really enjoying the combination. When I first integrated Scalate with a simple trait, the error messages were always in the console. Now that I've borrowed some smarts from Play's ScalaController and play-scalate's error reporting, I feel like it's practically a built-in solution. I was easily able to integrate my desired SiteMesh features and it even allows reusable template blocks. In the end, it's just Scala and Scalate does a good job of allowing you to leverage that.

Other thoughts:

If you're writing a lot of Jade and familiar with HTML, Don Park's html2jade is a great tool that comes with Scalate support.
I'm really enjoying writing CSS with LESS, particularly the ability to nest rules and have programming features. The only issue I've seen is IntelliJ's LESS plugin only does code-completion for variables rather than CSS values.
The Play Framework Cookbook is a great reference for learning how to write modules. Not only does it explain how to create modules, it has some great real-world examples for doing bytecode enhancement, implementing message queues, using Solr and how to do production monitoring.

If this series of articles has intrigued you and you'll be at Devoxx next week, you should stop by my talk on Thursday afternoon. In addition, there's several other Play talks at Devoxx and a possible meetup on Wednesday. Check out the Devoxx, anyone? thread for more information.

Update: There's one thing I forgot to mention about the Play Scalate Module. When I had Scalate integrated in my app with a trait, I only included the scalate-core and scalate-util JARs in dependencies.yml:

- org.fusesource.scalate -> scalate-core 1.5.2-scala_2.8.1:
    transitive: false
- org.fusesource.scalate -> scalate-util 1.5.2-scala_2.8.1:
    transitive: false

However, when I created the play-scalate module, I allowed more dependencies.

- org.fusesource.scalate -> scalate-core 1.5.2-scala_2.8.1:
    exclude:
        - javax.servlet -> *
        - com.sun.jersey -> *
        - org.osgi -> *
- org.fusesource.scalate -> scalate-util 1.5.2-scala_2.8.1

Because Scalate depends on Logback, debug messages started showing up in my console. To fix this, I created conf/logback.xml in my project and filled it with the following XML.

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
      <encoder>
          <pattern>%msg%n</pattern>
      </encoder>
  </appender>

  <root level="info">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

This reduces the logging and allows me to increase Scalate's logging if I ever have the need.

Integrating Scalate and Jade with Play 1.2.3

2011-09-07T13:21:41-06:00

At the beginning of this year, I decided I wanted to learn Scala. Since I'm a Web Frameworks Aficionado, I figured the best way to do that would be to learn Lift. I entered these two items on my todo list and let them lie for a couple months. After attending TSSJS 2011 and having a conversation with James Strachan, I added a couple more technologies to my learning list. James had great things to say about both CoffeeScript and Jade and I decided to learn those as well.

In May, Devoxx announced their Call For Papers and I started reminiscing about how awesome last year's trip was. I decided I'd try to get accepted again and started brainstorming about talks I'd like to give. I came up with "Comparing Scala Web Frameworks" and "HTML5 with Play Scala, CoffeeScript and Jade". The reason I chose Play over Lift for the latter talk is because I think it fits a lot more with the MVC mindset I have and the easy-to-learn nature of web frameworks I enjoy using. Both topics sounded very interesting to me, and I figured they'd also inspire me to learn the technologies in a brute-force fashion; where I was under a time constraint and would be embarrassed in front of a large audience if I didn't succeed.

In mid-July, I got an email from Stephan inviting me to speak again at the 10th edition of Devoxx. I smile splashed across my face and I quickly realized I had a lot to learn. Since I was still in vacation mode after summer vacation in Montana, I decided to wait until I returned from Cape Cod to start studying. While on my 2nd summer vacation, I received an email from Devoxx stating that they'd like me present "HTML5 with Play/Scala, CoffeeScript and Jade".

To learn all these technologies, I decided on an In Anger approach - where I would study minimally and learn mostly by doing. I ordered CoffeeScript on August 8th and Programming in Scala, 2nd Edition the following week (August 17th). I started reading both books while traveling the following week (I found CoffeeScript and Scala to be very similar, so I don't know if I'd recommend learning them at the same time). That same week, I started integrating Scalate (Jade) into a new Play Scala application.

Scalate advertises on their homepage that it works with Play via the play-scalate module. They neglect to mention that this module hasn't been updated in over a year or what version of Play it works with. I tried to use the scalate-0.7.2 version and quickly ran into issues. I posted a message to the Scalate Google Group explaining my compilation issues and stacktraces. The response? Crickets.

Next, I tried posting to the Play Google Group and got a much better response. Here's what they said:

Looking at the Scalate module code, I don't think it can work as is with Play Scala 0.9.1. The latest version is more than 1 year old, and we have made a lot of changes in the API.
...
The integration of Scalate is pretty difficult if you plan to get the same kind of experience than the native Play scala template regarding auto-reload and error reports.

You can try to port the module to 0.9.1, basically all it has to do is to provide a plugin that detect changes to scaml file, and recompile them. No special integration with the Play API is needed.

After learning that play-scalate was out-of-date, I contacted the project owner via GitHub and tried to get everything working with Play 1.2.3 and Scalate 1.5.1. I updated the dependencies in the project, fixed compilation issues and tried to build. No dice:

build: 
    [mkdir] Created dir: /Users/mraible/dev/play-scalate/tmp/classes 
   [scalac] Compiling 7 source files to /Users/mraible/dev/play-scalate/tmp/classes 
   [scalac] error: class file needed by Binding is missing. 
   [scalac] reference type Serializable of package scala refers to nonexisting symbol. 
   [scalac] one error found

I posted this error to the Play Group, discovered it was caused by Scalate 1.5.1 requiring Scala 2.9. I downgraded to Scalate 2.4.1 and got another nice cryptic error:

build: 
    [mkdir] Created dir: /Users/mraible/dev/play-scalate/tmp/classes 
   [scalac] Compiling 7 source files to /Users/mraible/dev/play-scalate/tmp/classes 
   [scalac] error: class file needed by ScalaController is missing. 
   [scalac] reference value dispatch of package <root> refers to nonexisting symbol. 
   [scalac] one error found

Apparently, this was caused by another Scala versioning issue and I was offered a much simpler solution for integrating Scalate. Below are the steps I performed to integrate Scalate 1.4.1 with Play 1.2.3.

Updated dependencies.yml to references Scala and Scalate dependencies.

require:
    - play
    - play -> scala 0.9.1
    - org.fusesource.scalate -> scalate-core 1.4.1:
        transitive: false
    - org.fusesource.scalate -> scalate-util 1.4.1:
        transitive: false

Added Scalate configuration elements to application.conf.
```
scalate=jade
jvm.memory=-Xmx256M
```

Created a ScalateTemplate class to contain the Scalate Engine and render the template.

package controllers

import play.Play

object ScalateTemplate {

  import org.fusesource.scalate._
  import org.fusesource.scalate.util._

  lazy val scalateEngine = {
    val engine = new TemplateEngine
    engine.resourceLoader = new FileResourceLoader(Some(Play.getFile("/app/views")))
    engine.classpath = Play.getFile("/tmp/classes").getAbsolutePath
    engine.workingDirectory = Play.getFile("tmp")
    engine.combinedClassPath = true
    engine.classLoader = Play.classloader
    engine
  }

  case class Template(name: String) {
    val scalateType = "." + Play.configuration.get("scalate");

    def render(args: (Symbol, Any)*) = {
      scalateEngine.layout(name + scalateType, args.map {
        case (k, v) => k.name -> v
      } toMap)
    }
  }

  def apply(template: String) = Template(template)
}

Created a Scalate trait to override the render() method in Play's Controller class.

package controllers

import play.mvc.Http

trait Scalate {

  def render(args: (Symbol, Any)*) = {
    def defaultTemplate = Http.Request.current().action.replace(".", "/")
    ScalateTemplate(defaultTemplate).render(args: _*);
  }
}

Created an Application.scala controller with a default index method.

package controllers

import play.mvc._
import models._

object Application extends Controller with Scalate {

  def index = {
    render('user -> User("Raible"))
  }
}

The models/User.scala class is very simple:

package models

case class User(name:String)

Lastly, I created an index.jade file in views/Application.

-@ var user: models.User
p Hi #{user.name},
- for(i <- 1 to 3)
  p= i
p See, I can count!

After getting all this working, I decided it was time to get it into production. As luck would have it, Heroku had just announced Play support a few days earlier. I heard through the grapevine that Play Scala would work, so I gave it a try. It was amazingly easy. All I had to do was create an account, create a "Procfile" in my application's root directory and run a heroku command followed by a git push. It all looked great until Play tried to compile my Jade templates as Groovy templates:

Cannot start in PROD mode with errors 
Template compilation error (In /app/views/Application/index.jade around line 2) 
The template /app/views/Application/index.jade does not compile : #{user.name} is not closed. 
play.exceptions.TemplateCompilationException: #{user.name} is not closed. 
       at play.templates.TemplateCompiler.generate(TemplateCompiler.java:102) 
       at play.templates.TemplateCompiler.compile(TemplateCompiler.java:15) 
       at play.templates.GroovyTemplateCompiler.compile(GroovyTemplateCompiler.java:4 1)

The solution from Guillaume was simple enough and I renamed my "views" directory to "templates" and updated ScalateTemplate.scala accordingly. You can see the deployed application at http://play-more.herokuapp.com.

The next steps in my Play Scala adventure will be trying to get the CoffeeScript module to work. I also hope to integrate HTML5 Boilerplate with Jade and Scalate Layouts. I'll be doing this with the mindset that HTML and JavaScript aren't that bad. I expect a lot from CoffeeScript and Jade and hope I enjoy them as much as Strachan.

In the meantime, here's some interesting links I've seen recently encountered that discuss Scala and/or Play. I dig the passion and activity that exists in these communities.

Integrating OAuth with AppFuse and its REST API

2011-07-05T10:56:48-06:00

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
Integrate Spring Security and OAuth
Authenticate and Retrieve Data with Client

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Java Web Application Security - Part V: Penetrating with Zed Attack Proxy

2011-06-21T07:45:41-06:00

Web Application Security is an important part of developing applications. As developers, I think we often forget this, or simply ignore it. In my career, I've learned a lot about web application security. However, I only recently learned and became familiar with the rapidly growing "appsec" industry.

I found a disconnect between what appsec consultants were selling and what I was developing. It seemed like appsec consultants were selling me fear, mostly because I thought my apps were secure. So I set out on a mission to learn more about web application security and penetration testing to see if my apps really were secure. This article is part of that mission, as are the previous articles I've written in this series.

When I first decided I wanted to do a talk on Webapp Security, I knew it would be more interesting if I showed the audience how to hack and fix an application. That's why I wrote it into my original proposal:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

At the time, I hadn't done much webapp pentesting. You can tell this from the fact that I mentioned WebGoat as the pentesting tool. From WebGoat's Project page:

WebGoat is a deliberately insecure J2EE web application maintained by OWASP designed to teach web application security lessons. In each lesson, users must demonstrate their understanding of a security issue by exploiting a real vulnerability in the WebGoat application. For example, in one of the lessons the user must use SQL injection to steal fake credit card numbers. The application is a realistic teaching environment, providing users with hints and code to further explain the lesson.

What I really meant to say and use was Zed Attack Proxy, also known as OWASP ZAP. ZAP is a Java Desktop application that you setup as a proxy for your browser, then use to find vulnerabilities in your application. This article explains how you can use ZAP to pentest a web applications and fix its vulnerabilities.

The application I'll be using in this article is the Ajax Login application I've been using throughout this series. I think it's great that projects like Damn Vulnerable Web App and WebGoat exist, but I wanted to test one that I think is secure, rather than one I know is not secure. In this particular example, I'll be testing the Spring Security implementation, since that's the framework I most often use in my open source projects.

Zed Attack Proxy Tutorial

Download and Run the Application
Install and Configure ZAP
Perform a Scan
Fix Vulnerabilities
Summary

Download and Run the Application
To begin, download the application and expand it on your hard drive. This app is the completed version of the Ajax Login application referenced in Java Web Application Security - Part II: Spring Security Login Demo. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and you need to login to do anything.

Install and Configure ZAP
The Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. Download the latest version (I used 1.3.0) and install it on your system. After installing, launch the app and change the proxy port to 9000 (Tools > Options > Local Proxy). Next, configure your browser to proxy requests through port 9000 and allow localhost requests to be proxied. I used Firefox 4 (Preferences > Advanced > Network > Connection Settings). When finished, your proxy settings should look like the following screenshot:

Another option (instead of removing localhost) is to add an entry to your hosts file with your production domain name. This is what I've done for this demo.

127.0.0.1       demo.raibledesigns.com

I've also configured Apache to proxy requests to Jetty with the following mod_proxy settings in my httpd.conf:

<IfModule mod_proxy.c>
    ProxyRequests Off 
    ProxyPreserveHost Off 

    <VirtualHost *:80>
       ProxyPass  /  http://localhost:8080/
    </VirtualHost>

    <VirtualHost *:443>
        SSLEngine on
        SSLProxyEngine on
        SSLCertificateFile "/etc/apache2/ssl.key/server.crt"
        SSLCertificateKeyFile "/etc/apache2/ssl.key/server.key"

        ProxyPass  /  https://localhost:8443/
    </VirtualHost>
</IfModule>

Perform a Scan
Now you need to give ZAP some data to work with. Using Firefox, I navigated to http://demo.raibledesigns.com and browsed around a bit, listing users, added a new one and deleted an existing one. After doing this, I noticed a number of flags in the ZAP UI under Sites. I then right-clicked on each site (one for http and one for https) and selected Attack > Active Scan site. You should be able to do this from the "Active Scan" tab at the bottom of ZAP, but there's a bug when the URLs are the same. After doing this, I received a number of alerts, ranging from high (cross-site scripting) to low (password autocomplete). The screenshot below shows the various issues.

Now let's take a look at how to fix them.

Fix Vulnerabilities
One of the things not mentioned by the scan, but #1 in Seven Security (Mis)Configurations in Java web.xml Files, is Custom Error Pages Not Configured. Custom error pages are configured in this app, but error.jsp contains the following code:

<% if (exception != null) { %>
    <% exception.printStackTrace(new java.io.PrintWriter(out)); %>
<% } else { %>
    Please check your log files for further information.
<% } %>

Stack traces can be really useful to an attacker, so it's important to start by removing the above code from src/main/webapp/error.jsp.

The rest of the issues have to do with XSS, autocomplete, and cookies. Let's start with the easy ones. Fixing autocomplete is easy enough; simply changed the HTML in login.jsp and userform.jsp to have autocomplete="off" as part of the <form> tag.

Then modify web.xml so http-only and secure cookies are used. While you're at it, add session-timeout and tracking-mode as recommended by the aforementioned web.xml misconfigurations article.

<session-config>
    <session-timeout>15</session-timeout>
    <cookie-config>
        <http-only>true</http-only>
        <secure>true</secure>
    </cookie-config>
    <tracking-mode>COOKIE</tracking-mode>
</session-config>

Next, modify Spring Security's Remember Me configuration so it uses secure cookies. To do this, add use-secure-cookies="true" to the <remember-me> element in security.xml.

<remember-me user-service-ref="userService" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"
             use-secure-cookie="true"/>

Unfortunately, Spring Security doesn't support HttpOnly cookies, but will in a future release.

The next issue to solve is disabling directory browsing. You can do this by copying Jetty's webdefault.xml (from the org.eclipse.jetty:jetty-webapp JAR) into src/test/resources and changing its "dirAllowed" <init-param> to false:

<servlet>
  <servlet-name>default</servlet-name>
  <servlet-class>org.mortbay.jetty.servlet.DefaultServlet</servlet-class>
  <init-param>
    <param-name>acceptRanges</param-name>
    <param-value>true</param-value>
  </init-param>
  <init-param>
    <param-name>dirAllowed</param-name>
    <param-value>false</param-value>
  </init-param>
  <init-param>

You'll also need to modify the plugin's configuration to point to this file by adding it to the <webAppConfig> section in pom.xml.

<configuration>
    <webAppConfig>
        <contextPath>/</contextPath>
        <defaultsDescriptor>src/test/resources/webdefault.xml</defaultsDescriptor>
    </webAppConfig>

Of course, if you're running in production you'll want to configure this in your server's settings rather than in your pom.xml file.

Next, I set out to fix secure page browser cache issues. I had the following settings in my SiteMesh decorator:

<meta http-equiv="Cache-Control" content="no-store"/>
<meta http-equiv="Pragma" content="no-cache"/>

However, according to ZAP, the first meta tag should have "no-cache" instead of "no-store", so I changed it to "no-cache".

After making all these changes, I created a new ZAP session and ran an active scan on both sites again. Below are the results:

I believe the first issue (parameter tampering) is because I show the error page when a duplicate user exists. To fix this, I changed UserFormController so it catches a UserExistsException and sends the user back to the form.

try {
    userManager.saveUser(user);
} catch (UserExistsException uex) {
    result.addError(new ObjectError("user", uex.getMessage()));
    return "userform";
}

However, this still doesn't seem to cause the alert to go away. This is likely because I'm not filtering/escaping HTML when it's first submitted. I believe the best solution for this would be to use something like OWASP's ESAPI to filter parameter values. However, I was unable to find integration with Spring MVC's data binding, so I decided not to try and fix this vulnerability.

Finally, I tried to disable jsessionid in URLs using suggestions from Stack Overflow. The previous setting in web.xml (<tracking-mode>COOKIE</tracking-mode>) should do this, but it doesn't seem to work with Jetty 8. The other issues (secure page browser cache, HttpOnly cookies and secure cookies), I was unable to solve. The last two are issues caused by Spring Security as far as I can tell.

Summary
In this article, I've shown you how to pentest a web application using Firefox and OWASP's Zed Attack Proxy (ZAP). I found ZAP to be a nice tool for figuring out vulnerabilities, but it'd be nice if it had a "retest" feature to see if you fixed an issue for a particular URL. It does have a "resend" feature, but running it didn't seem to clear alerts after I'd fixed them.

The issues I wasn't able to solve seemed to be mostly related to frameworks (e.g. Spring Security and HttpOnly cookies) or servers (Jetty not using cookies for tracking). My suspicion is the Jetty issues are because it doesn't support Servlet 3 as well as it advertises. I believe this is fair; I am using a milestone release after all. I tried scanning http://demo.raibledesigns.com/ajax-login (which runs on Tomcat 7 at Contegix) and confirmed that no jsessionid exists.

Hopefully this article has helped you understand how to figure out security vulnerabilities in your web applications. I believe ZAP will continue to get more popular as developers become aware of it. If you feel ambitious and want to try and solve all of the issues in my Ajax Login application, feel free to fork it on GitHub.

If you're interested in talking more about Webapp Security, please leave a comment, meet me at Jazoon later this week or let's talk in July at Über Conf.

Java Web Application Security - Part IV: Programmatic Login APIs

2011-06-06T21:44:09-06:00

Over the last month, I've posted a number of articles on implementing authentication with Java EE 6, Spring Security and Apache Shiro. One of the things I demonstrated in my live demos (at Utah's JUG Meetings) was programmatic authentication. I left this out of my screencasts and previous tutorials because I thought it'd fit better in a comparison article.

In this article, I'd like to show you how you can programmatically login to an application using the aforementioned security frameworks. To do this, I'll be using my ajax-login application that I wrote for Implementing Ajax Authentication using jQuery, Spring Security and HTTPS.

To begin, I implemented a LoginController as a Spring MVC Controller that returns JSON.

package org.appfuse.examples.webapp.security;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginController {

    @Autowired
    LoginService loginService;

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public LoginStatus getStatus() {
        return loginService.getStatus();
    }

    @RequestMapping(method = RequestMethod.POST)
    @ResponseBody
    public LoginStatus login(@RequestParam("j_username") String username,
                             @RequestParam("j_password") String password) {

        return loginService.login(username, password);
    }
}

This controller delegates its logic to a LoginService interface.

package org.appfuse.examples.webapp.security;

public interface LoginService {

  LoginStatus getStatus();

  LoginStatus login(String username, String password);
}

The Client
The client for this controller is the same as mentioned in my previous article, but I'll post it again for your convenience. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginController.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('Login failed, please try again.');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "${ctx}/api/login.json",
        type: "POST",
        beforeSend: function(xhr) {
            xhr.withCredentials = true;
        },
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href = getHost() + '${ctx}/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

The biggest secret to making this all work (the HTTP -> HTTPS communication, which is considered cross-domain), is the window.name Transport and the jQuery plugin that implements it. To make this plugin work with Firefox 3.6, I had to implement a Filter that adds Access-Control headers.

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "http://" + req.getServerName());
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");
        response.setHeader("Access-Control-Allow-Credentials", "true");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Java EE 6 LoginService
Java EE 6 has a few new methods in HttpServletRequest:

authenticate(response)
login(user, pass)
logout()

In this example, I'll use the new login(username, password) method. The hardest part about getting this working was finding the right Maven dependency. At first, I tried the one that seemed to make the most sense:

<dependency>
    <groupId>javax</groupId>
    <artifactId>javaee-web-api</artifactId>
    <version>6.0</version>
</dependency>

Unfortunately, this resulted in a strange error that means the dependency has the interfaces, but not the implementation classes. I ended up using GlassFish's dependency instead (thanks to Stack Overflow for the tip).

<dependency>
    <groupId>org.glassfish</groupId>
    <artifactId>javax.servlet</artifactId>
    <version>3.0</version>
    <scope>provided</scope>
</dependency>

Since Servlet 3.0 doesn't appear to be in Maven Central, I had to add the GlassFish Repository to my pom.xml's <repositories> element.

<repository>
    <id>glassfish-repo</id>
    <url>http://download.java.net/maven/glassfish</url>
</repository>

After that, it was easy to implement the LoginService interface with a JavaEELoginService class:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;

@Service("javaeeLoginService")
public class JavaEELoginService implements LoginService {
    private Log log = LogFactory.getLog(JavaEELoginService.class);

    @Autowired
    HttpServletRequest request;

    public LoginStatus getStatus() {
        if (request.getRemoteUser() != null) {
            return new LoginStatus(true, request.getRemoteUser());
        } else {
            return new LoginStatus(false, null);
        }
    }

    @Override
    public LoginStatus login(String username, String password) {
        try {
            if (request.getRemoteUser() == null) {
                request.login(username, password);
                log.debug("Login succeeded!");
            }
            return new LoginStatus(true, request.getRemoteUser());
        } catch (ServletException e) {
            e.printStackTrace();
            return new LoginStatus(false, null);
        }
    }
}

I tried to use this with "mvn jetty:run" (with version 8.0.0.M2 of the jetty-maven-plugin), but I got the following error:

javax.servlet.ServletException
        at org.eclipse.jetty.server.Request.login(Request.java:1927)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.springframework.beans.factory.support.AutowireUtils$ObjectFactoryDelegatingInvocationHandler.invoke(AutowireUtils.java:178)
        at $Proxy52.login(Unknown Source)
        at org.appfuse.examples.webapp.security.JavaEELoginService.login(JavaEELoginService.java:30)

This lead me to believe that Servlet 3 is not quite implemented, so I tried it with Tomcat 7.0.8. To support SSL and container-managed authentication, I had to create a certificate keystore and uncomment the SSL Connector in $CATALINA_HOME/conf/server.xml. I also had to add an "admin" user with roles="ROLE_ADMIN" to $CATALINA_HOME/conf/tomcat-users.xml.

<user username="admin" password="admin" roles="ROLE_ADMIN"/>

With Tomcat 7, I was able to login successfully, proven by the following logging.

DEBUG - JavaEELoginService.login(31) | Login succeeded!

However, in the UI, I still got a "Login failed, please try again." message. Recalling that I had some issues with ports previous, I configured Apache to proxy the default http/https ports to 8080/8443 and tried again. This time it worked!

Spring Security LoginService
Spring Security offers a programmatic API and I was able to implement its LoginService as follows:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Service;

@Service("springLoginService")
public class SpringSecurityLoginService implements LoginService {
    private Log log = LogFactory.getLog(SpringSecurityLoginService.class);

    @Autowired(required = false)
    @Qualifier("authenticationManager")
    AuthenticationManager authenticationManager;

    public LoginStatus getStatus() {
        Authentication auth = SecurityContextHolder.getContext().getAuthentication();
        if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
            return new LoginStatus(true, auth.getName());
        } else {
            return new LoginStatus(false, null);
        }
    }

    public LoginStatus login(String username, String password) {
        UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
        User details = new User(username);
        token.setDetails(details);

        try {
            Authentication auth = authenticationManager.authenticate(token);
            log.debug("Login succeeded!");
            SecurityContextHolder.getContext().setAuthentication(auth);
            return new LoginStatus(auth.isAuthenticated(), auth.getName());
        } catch (BadCredentialsException e) {
            return new LoginStatus(false, null);
        }
    }
}

I then modified the LoginService dependency in LoginController so this implementation would be used.

@Autowired
@Qualifier("springLoginService")
LoginService loginService;

Since Spring's API doesn't depend on Servlet 3, I tried it in Jetty using "mvn jetty:run". Of course, I modified my web.xml accordingly for Spring Security before doing so. Interestingly enough, I found that the my SpringSecurityLoginService seemed to work:

DEBUG - SpringSecurityLoginService.login(39) | Login succeeded!

But in the UI, the login failed with a "Login failed, please try again." message. Using the standard ports with Apache in front of Jetty solved this issue.

Apache Shiro LoginService
Apache Shiro is nice enough to offer a programmatic API as well. I was able to implement a ShiroLoginService as follows:

package org.appfuse.examples.webapp.security;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.shiro.SecurityUtils;
import org.apache.shiro.authc.AuthenticationException;
import org.apache.shiro.authc.UsernamePasswordToken;
import org.apache.shiro.subject.Subject;
import org.springframework.stereotype.Service;

@Service("shiroLoginService")
public class ShiroLoginService implements LoginService {
    private Log log = LogFactory.getLog(ShiroLoginService.class);

    public LoginStatus getStatus() {
        Subject currentUser = SecurityUtils.getSubject();
        if (currentUser.isAuthenticated()) {
            return new LoginStatus(true, currentUser.getPrincipal().toString());
        } else {
            return new LoginStatus(false, null);
        }
    }

    public LoginStatus login(String username, String password) {
        if (!getStatus().isLoggedIn()) {
            UsernamePasswordToken token = new UsernamePasswordToken(username, password);
            Subject currentUser = SecurityUtils.getSubject();
            try {
                currentUser.login(token);
                log.debug("Login succeeded!");
                return new LoginStatus(currentUser.isAuthenticated(),
                        currentUser.getPrincipal().toString());
            } catch (AuthenticationException e) {
                return new LoginStatus(false, null);
            }

        } else {
            return getStatus();
        }
    }
}

Then I modified the LoginService dependency in LoginController so this implementation would be used.

@Autowired
@Qualifier("shiroLoginService")
LoginService loginService;

Next, I modified my web.xml for Apache Shiro and tried "mvn jetty:run". Again, the login appeared to succeed (based on log messages) on the server, but failed in the UI. When using http://localhost instead of http://localhost:8080, everything worked fine.

Summary
This article has shown you how you can programmatically login using Java EE 6, Spring Security and Apache Shiro. Before Java EE 6 (and Servlet 3), there was no API to programmatically login, so this is a welcome addition. The fact that my Ajax login example didn't work when ports differed is because of browsers' same origin policy, which specifies the ports have to be the same. Specifying no ports (the defaults), seems to be the loophole.

On a related note, I've discovered some interesting articles recently from the AppSec Blog.

The 2nd article has an interesting paragraph:

... there's Apache Shiro (FKA JSecurity and then later as Apache Ki), another secure framework for Java apps. Although it looks simpler to use and understand than ESAPI and covers most of the main security bases (authentication, authorization, session management and encryption), it doesn't help take care of important functions like input validation and output encoding. And Spring users have Spring Security (Acegi) a comprehensive, but heavyweight authorization and authentication framework.

So according to this blog, the security frameworks discussed here aren't the best.

The most comprehensive, up-to-date choice for Java developers is OWASP's ESAPI Enterprise Security API especially now that the 2.0 release has just come out.

I haven't heard of many organizations adopting ESAPI over Java EE 6, Spring Security or Apache Shiro, but maybe I'm wrong. Is ESAPI something that's being used out there by companies?

Java Web Application Security - Part III: Apache Shiro Login Demo

2011-05-26T16:43:22-06:00

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Store Users in a Database
Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

AppFuse 2.1 Released!

2011-04-04T09:38:05-06:00

The AppFuse Team is pleased to announce the release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are JPA 2, JSF 2, Tapestry 5 and Spring 3. In addition, we've migrated from XFire to CXF and enabled REST for web services. There's even a new appfuse-ws archetype that leverages Enunciate to generate web service endpoints, documentation and downloadable clients. This release fixes many issues with archetypes, improving startup time and allowing jetty:run to be used for quick turnaround while developing. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications with Java quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with IntelliJ, you can achieve zero-turnaround in your project and develop features without restarting the server.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose with persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run "appfuse:full-source".

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. Light archetypes allow code generation and full-source features, but do not currently support Stripes or Wicket. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket. You can see demos of these archetypes at http://demo.appfuse.org.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the mailing list or create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Adding Search to AppFuse with Compass

2011-03-15T17:11:12-06:00

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

Added Searchable annotations to the User and Address.
Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
Defined CompassTemplate (similar in concept to HibernateTemplate).
Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

To index on startup, I added compassGps.index() to the StartupListener..
In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code.

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

JSR 303 and JVM Web Framework Support

2011-03-08T11:33:24-07:00

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

Fixing XSS in JSP 2

2011-02-28T14:08:46-07:00

Way back in 2007, I wrote about Java Web Frameworks and XSS. My main point was that JSP EL doesn't bother to handle XSS.

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

A couple months later, I proposed a Tomcat enhancement to escape JSP's EL by default. I also entered an enhancement request for this feature and attached a patch. That issue has remained open and unfixed for 3 and 1/2 years.

Yesterday, Chin Huang posted a handy-dandy ELResolver that XML-escapes EL values.

I tried Chin's resolver in AppFuse today and it works as well as advertised. To do this, I copied his EscapeXML*.java files into my project, changed the JSP API's Maven coordinates from javax.servlet:jsp-api:2.0 to javax.servlet.jsp:jsp-api:2.1 and added the listener to web.xml.

With Struts 2 and Spring MVC, I was previously able to have ${param.xss} and pass in ?xss=<script>alert('gotcha')</script> and it would show a JavaScript alert. After using Chin's ELResolver, it prints the string on the page instead of displaying an alert.

Thanks to Chin Huang for this patch! If you're using JSP, I highly recommend you add this to your projects as well.

Implementing Ajax Authentication using jQuery, Spring Security and HTTPS

2011-02-23T16:55:55-07:00

I've always had a keen interest in implementing security in webapps. I implemented container-managed authentication (CMA) in AppFuse in 2002, watched Tomcat improve it's implementation in 2003 and implemented Remember Me with CMA in 2004. In 2005, I switched from CMA to Acegi Security (now Spring Security) and never looked back. I've been very happy with Spring Security over the years, but also hope to learn more about Apache Shiro and implementing OAuth to protect JavaScript APIs in the near future.

I was recently re-inspired to learn more about security when working on a new feature at Overstock.com. The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience. User experience aside, I think it's interesting to see what's required to implement such a feature.

To demonstrate how we did it, I whipped up an example using AppFuse Light, jQuery and Spring Security. The source is available in my ajax-login project on GitHub. To begin, I wanted to accomplish a number of things to replicate the Overstock environment:

Force HTTPS for authentication.
Allow testing HTTPS without installing a certificate locally.
Implement a RESTful LoginService that allows users to login.
Implement login with Ajax, with the request coming from an insecure page.

Forcing HTTPS with Spring Security
The first feature was fairly easy to implement thanks to Spring Security. Its configuration supports a requires-channel attribute that can be used for this. I used this to force HTTPS on the "users" page and it subsequently causes the login to be secure.

<intercept-url pattern="/app/users" access="ROLE_ADMIN" requires-channel="https"/>

Testing HTTPS without adding a certificate locally
After making the above change in security.xml, I had to modify my jWebUnit test to work with SSL. In reality, I didn't have to modify the test, I just had to modify the configuration that ran the test. In my last post, I wrote about adding my 'untrusted' cert to my JVM keystore. For some reason, this works for HttpClient, but not for jWebUnit/HtmlUnit. The good news is I figured out an easier solution - adding the trustStore and trustStore password as system properties to the maven-failsafe-plugin configuration.

<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.2</version>
<configuration>
    <includes>
        <include>**/*WebTest.java</include>
    </includes>
    <systemPropertyVariables>
      <javax.net.ssl.trustStore>${project.build.directory}/ssl.keystore</javax.net.ssl.trustStore>
      <javax.net.ssl.trustStorePassword>appfuse</javax.net.ssl.trustStorePassword>
    </systemPropertyVariables>
</configuration>

The disadvantage to doing things this way is you'll have to pass these in as arguments when running unit tests in your IDE.

package org.appfuse.examples.web;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginService {

  @Autowired
  @Qualifier("authenticationManager")
  AuthenticationManager authenticationManager;

  @RequestMapping(method = RequestMethod.GET)
  @ResponseBody
  public LoginStatus getStatus() {
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
      return new LoginStatus(true, auth.getName());
    } else {
      return new LoginStatus(false, null);
    }
  }

  @RequestMapping(method = RequestMethod.POST)
  @ResponseBody
  public LoginStatus login(@RequestParam("j_username") String username,
                           @RequestParam("j_password") String password) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    User details = new User(username);
    token.setDetails(details);

    try {
      Authentication auth = authenticationManager.authenticate(token);
      SecurityContextHolder.getContext().setAuthentication(auth);
      return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
      return new LoginStatus(false, null);
    }
  }

  public class LoginStatus {

    private final boolean loggedIn;
    private final String username;

    public LoginStatus(boolean loggedIn, String username) {
      this.loggedIn = loggedIn;
      this.username = username;
    }

    public boolean isLoggedIn() {
      return loggedIn;
    }

    public String getUsername() {
      return username;
    }
  }
}

To verify this class worked as expected, I wrote a unit test using JUnit and Mockito. I used Mockito because Overstock is transitioning to it from EasyMock and I've found it very simple to use.

package org.appfuse.examples.web;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Matchers;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.TestingAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.context.SecurityContextImpl;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class LoginServiceTest {

  LoginService loginService;
  AuthenticationManager authenticationManager;

  @Before
  public void before() {
    loginService = new LoginService();
    authenticationManager = mock(AuthenticationManager.class);
    loginService.authenticationManager = authenticationManager;
  }

  @After
  public void after() {
    SecurityContextHolder.clearContext();
  }

  @Test
  public void testLoginStatusSuccess() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    SecurityContext context = new SecurityContextImpl();
    context.setAuthentication(auth);
    SecurityContextHolder.setContext(context);

    LoginService.LoginStatus status = loginService.getStatus();
    assertTrue(status.isLoggedIn());
  }

  @Test
  public void testLoginStatusFailure() {
    LoginService.LoginStatus status = loginService.getStatus();
    assertFalse(status.isLoggedIn());
  }

  @Test
  public void testGoodLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    when(authenticationManager.authenticate(Matchers.<Authentication>anyObject())).thenReturn(auth);
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertTrue(status.isLoggedIn());
    assertEquals("foo", status.getUsername());
  }

  @Test
  public void testBadLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(false);
    when(authenticationManager.authenticate(Matchers.anyObject()))
        .thenThrow(new BadCredentialsException("Bad Credentials"));
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertFalse(status.isLoggedIn());
    assertEquals(null, status.getUsername());
  }
}

Implement login with Ajax
The last feature was the hardest to implement and still isn't fully working as I'd hoped. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

Passing in the "ajax=true" parameter disables SiteMesh decoration on the login page, something that's described in my Ajaxified Body article.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

Instead of adding a click handler to a specific id, it's probably better to use a CSS class that indicates authentication is required for a link, or -- even better -- use Ajax to see if the link is secured.

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginService.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('<div class="error">Login failed, please try again.</div>');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "/api/login.json",
        type: "POST",
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href= getHost() + '/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Issues
I encountered a number of issues when implementing this in the ajax-login project.

If you try to run this with ports (e.g. 8080 and 8443) in your URLs, you'll get a 501 (Not Implemented) response. Removing the ports by fronting with Apache and mod_proxy solves this problem.
If you haven't accepted the certificate in your browser, the Ajax request will fail. In the example, I solved this by clicking on the "Users" tab to make a secure request, then going back to the homepage to try and login.
The jQuery window.name version 0.9.1 doesn't work with jQuery 1.5.0. The error is "$.httpSuccess function not found."

Finally, even though I was able to authenticate successfully, I was unable to make the authentication persist. I tried adding the following to persist the updated SecurityContext to the session, but it doesn't work. I expect the solution is to create a secure JSESSIONID cookie somehow.

@Autowired
SecurityContextRepository repository;

@RequestMapping(method = RequestMethod.POST)
@ResponseBody
public LoginStatus login(@RequestParam("j_username") String username,
                         @RequestParam("j_password") String password,
                         HttpServletRequest request, HttpServletResponse response) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    ...

    try {
        Authentication auth = authenticationManager.authenticate(token);
        SecurityContextHolder.getContext().setAuthentication(auth);
        // save the updated context to the session
        repository.saveContext(SecurityContextHolder.getContext(), request, response);
        return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
        return new LoginStatus(false, null);
    }
}

Conclusion
This article has shown you how to force HTTPS for login, how to do integration testing with a self-generated certificate, how to implement a LoginService with Spring MVC and Spring Security, as well as how to use jQuery to talk to a service cross-domain with the window.name Transport. While I don't have everything working as much as I'd like, I hope this helps you implement a similar feature in your applications.

One thing to be aware of is with lightbox/dialog logins and HTTP -> HTTPS is that users won't see a secure icon in their address bar. If your app has sensitive data, you might want to force https for your entire app. OWASP's Secure Login Pages has a lot of good tips in this area.

Update: I've posted a demo of the ajax-login webapp. Thanks to Contegix for hosting the demo and helping obtain/install an SSL certificate so quickly.

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

2011-02-10T16:53:27-07:00

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter Name	Dispatchers
securityFilter	request
rewriteFilter	request, forward
struts-prepare	request
sitemesh	request, forward, include
staticFilter	request, forward
struts	request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter Name	Dispatchers
rewriteFilter	request
securityFilter	request, forward, include
struts-prepare	request, forward
sitemesh	request, forward, include
struts	forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.

How I Calculated Ratings for My JVM Web Frameworks Comparison

2010-12-06T11:55:18-07:00

When I re-wrote my Comparing JVM Web Frameworks presentation from scratch, I decided to add a matrix that allows you to rate a framework based on 20 different criteria. The reason I did this was because I'd used this method when choosing an Ajax framework for Evite last year. The matrix seemed to work well for selecting the top 5 frameworks, but it also inspired a lot of discussion in the community that my ratings were wrong.

I expected this, as I certainly don't know every framework as well as I'd like. The mistake I made was asking for the community to provide feedback on my ratings without describing how I arrived at them. From Peter Thomas's blog:

What you are doing is adjusting ratings based on who in the community shouts the loudest. I can't help saying that this approach comes across as highly arrogant and condescending, you seem to expect framework developers and proponents to rush over and fawn over you to get better ratings, like waiters in a restaurant trying to impress a food-critic for Michelin stars.

I apologize for giving this impression. It certainly wasn't my intent. By having simple numbers (1.0 == framework does well, 0.5 == framework is OK and 0 == framework not good at criteria) with no rationalization, I can see how the matrix can be interpreted as useless (or to put it bluntly, as something you should wipe your ass with). I don't blame folks for getting angry.

For my Rich Web Experience presentation, I documented why I gave each framework the rating I did. Hopefully this will allow folks to critique my ratings more constructively and I can make the numbers more accurate. You can view this document below or on Google Docs.

In the end, what I was hoping to do with this matrix was to simply highlight a technique for choosing a web framework. Furthermore, I think adding a "weight" to each criteria is important because things like books often aren't as important as REST support. To show how this might be done, I added a second sheet to the matrix and made up some weighting numbers. I'd expect anyone that wants to use this to downloaded the matrix, verify the ratings are accurate for your beliefs and weight the criteria accordingly.

Of course, as I and many others have said, the best way to choose a web framework is to try them yourself. I emphasized this at the end of my presentation with the following two slides.

My Everything You Ever Wanted To Know About Online Video Presentation

2010-12-03T10:16:44-07:00

This week I've had the pleasure of speaking at The Rich Web Experience in Fort Lauderdale. I did two talks, one on Comparing JVM Web Frameworks and one titled Everything You Ever Wanted To Know About Online Video. Both talks had full rooms and very engaged audiences.

In the video talk, there were some audience members that knew way more than me about the topic. This made for a very interactive session and one of the most fun presentations I've ever done. It was also cool to talk about a lot of things I've learned over the last year (for more details on that, check out my team status or team hiring posts). If you don't have Flash installed, you can download a PDF of this presentation.

The first talk about Comparing JVM Web Frameworks was largely an extension of the one I presented at Devoxx two weeks ago. The main differences between this one and the last one is I extended it a bit and took into account some community feedback. However, this seemed to simply inspire anger, so I'll pass on embedding it here. You can view it on Slideshare or download the PDF.

My Comparing Web Frameworks slides often inspire harsh words, but folks really seem to like the presentation. I encourage you to watch my Devoxx presentation on Parleys.com to see for yourself.

This marks the end of 2010 conferences for me. I had a blast speaking at The Rich Web Experience, as well as TheServerSide Java Symposium, The Irish Software Show and Devoxx. Now it's time to sit back, relax, get some powder days in and find my next gig.

Hope y'all have a great holiday season!

My Comparing JVM Web Frameworks Presentation from Devoxx 2010

2010-11-18T05:23:10-07:00

This week, I've been having a great time in Antwerp, Belgium at the Devoxx Conference. This morning, I had the pleasure of delivering my Comparing JVM Web Frameworks talk. I thoroughly enjoyed giving this presentation, especially to such a large audience. You can view the presentation below (if you have Flash installed) or download it here.

Unlike previous years, I chose to come up with a spreadsheet matrix that shows why I chose the 5 I did. This spreadsheet and rankings given to each framework are likely to be debated, as I don't know all the frameworks as well as I'd like to. Also, the missing column on this spreadsheet is a "weighting" column where you can prioritize certain criteria like I've done in the past when Comparing Ajax Frameworks. If you believe there are incorrect numbers, please let me know and I'll try to get those fixed before I do this talk again at The Rich Web Experience.

One thing that doesn't come across in this presentation is that I believe anyone can use this matrix, and weightings, to make any of these frameworks come out on top. I also believe web frameworks are like spaghetti sauce in The Ketchup Conundrum. That is, the only way to make more happy spaghetti sauce lovers was to make more types of spaghetti sauce. You can read more about this in my There is no "best" web framework article.

Update: If you disagree with the various ratings I gave to web frameworks in this presentation, please provide your opinions by filling out this survey. Thanks to Sebastien Arbogast for setting this up.

Update: Sebastien has posted his survey results at JVM Web Framework Survey, First Results.

Update 12/6: A video of this presentation is now available on Parleys.com.

P.S. My current gig is ending in mid-December. If you're looking for a UI Architect with a passion for open source frameworks, please let me know.

AppFuse 2.1 Milestone 2 Released

2010-11-15T15:28:57-07:00

I'm pleased to announce the 2nd milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Spring 3 and Struts 2.1. This release fixes many issues with archetypes and contains many improvements to support Maven 3. For more details on specific changes see the 2.1.0 M2 release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with AppFuse, you can achieve zero-turnaround in your project and develop features without restarting the server.

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. In the final 2.1.0 release, the light archetypes will allow code generation like the basic and modular archetypes. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for Spring MVC and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for the 2.1 final release.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

RE: Moving from Spring to Java EE 6: The Age of Frameworks is Over

2010-10-16T15:19:07-06:00

Last Tuesday, Cameron McKenzie wrote an interesting article on TheServerSide titled Moving from Spring to Java EE 6: The Age of Frameworks is Over. In this article, Cameron says the following:

J2EE represents the past, and Java EE 6 represents the future. Java EE 6 promises us the ability to go beyond frameworks. Frameworks like Spring are really just a bridge between the mistakes of the J2EE past and the success of the Java EE 6 future. Frameworks are out, and extensions to the Java EE 6 platform are in. Now is the time to start looking past Spring, and looking forward to Seam and Weld and CDI technologies.

He then links to an article titled Spring to Java EE - A Migration Experience, an article written by JBoss's Lincoln Baxter. In this article, Lincoln talks about many of the technologies in Java EE 6, namely JPA, EJB, JSF, CDI and JAX-RS. He highlights all the various XML files you'll need to know about and the wide variety of Java EE 6 application servers: JBoss AS 6 and GlassFish v3.

I don't have a problem with Lincoln's article, in fact I think it's very informative and some of the best documentation I've seen for Java EE 6.

I do have some issues with Cameron's statements that frameworks are mistakes of the J2EE past and that Java EE 6 represents the future. Open source frameworks made J2EE successful. Struts and Hibernate came out in the early days of J2EE and still exist today. Spring came out shortly after and has turned into the do-everything J2EE implementation it was trying to fix. Java EE 6 might be a better foundation to build upon, but it's certainly not going to replace frameworks.

To prove my point, let's start by looking at the persistence layer. We used to have Hibernate based on JDBC, now we have JPA implementations built on top of the JPA API. Is JPA a replacement for all persistence frameworks? I've worked with it and think it's a good API, but the 2.0 version isn't available in a Maven repo and Alfresco recently moved away from Hibernate (which == JPA IMO) to iBATIS for greater data access layer control and scalability. Looks like the age of frameworks isn't over for persistence frameworks.

The other areas that Java EE 6 covers that I believe frameworks will continue to excel in: EJB, CDI, JSF and JAX-RS. Personally, I don't have a problem with EJB 3 and think it's a vast improvement on EJB 2.x. I don't have an issue with CDI either, and as long as it resembles Guice for dependency injection, it works for me. However, when you get into the space I've been living in for the last couple years (high-traffic public internet sites), EJB and things like the "conversation-scope" feature of CDI don't buy you much. The way to make web application scale is to eliminate state and cache as much as possible, both of which Java EE doesn't provide much help for. In fact, to disable sessions in a servlet-container, you have to write a Filter like the following:

public class DisabledSessionFilter extends OncePerRequestFilter {

    /**
     * Filters requests to disable URL-based session identifiers.
     */
    @Override
    protected void doFilterInternal(final HttpServletRequest request,
                                    final HttpServletResponse response,
                                    final FilterChain chain)
            throws IOException, ServletException {

        HttpServletRequestWrapper wrappedRequest = new HttpServletRequestWrapper(request) {

            @Override
            public HttpSession getSession(final boolean create) {
                if (create) {
                    throw new UnsupportedOperationException("Session support disabled");
                }
                return null;
            }

            @Override
            public HttpSession getSession() {
                throw new UnsupportedOperationException("Session support disabled");
            }
        };

        // process next request in chain
        chain.doFilter(wrappedRequest, response);
    }
}

What about JAX-RS? Does it replace the need for frameworks? I like the idea of having a REST API in Java. However, its reference implementation is Jersey, which seems more like a framework than just Java EE. If you choose to use JAX-RS in your application, you still have to choose between CXF, Jersey, RESTEasy and Restlet. I compared these frameworks last year and found the Java EE implementation lacking in the features I needed.

Finally, let's talk about my-least-framework-web-framework: JSF. The main reason I don't like JSF is because of its 1.x version. JSF 1.0 was released a year before the Ajax term was coined (see timeline below). Not only did it take forever to develop as a spec, but it tried to be a client-component framework that was very stateful by default.

Now that JSF 2.0 is out, it has Ajax integrated and allows you to use GET instead of POST-for-everything. However, the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn't be developing your UI). Also, the best component development platform for the web is JavaScript. I recommend using an Ajax framework for your components if you really want a rich UI.

Sure you can use the likes of Tapestry and Wicket if you like POJO-based web development, but if you're looking to develop a webapp that's easy to maintain and understand, chances are that you'll do much better with traditional MVC frameworks like Spring MVC and Struts 2. The simplicity and popularity of Rails and Grails further emphasize that developers prefer these types of web frameworks.

Another reason I don't like JSF: there's very few developers in the wild happy with it. The major promoters of JSF are book authors, trainers, Java EE Vendors and MyFaces developers. Whenever I speak at conferences, I ask folks to raise their hands for the various web frameworks they're using. I always ask the JSF users to keep their hands up if they like it. Rarely do they stay up.

So it looks like we still need web frameworks.

Eberhard Wolff has an interesting post where he defends Spring and talks about the productivity comparisons between Spring and Java EE. He recommends using Grails or Spring Roo if you want the level of productivity that Ruby on Rails provides. That's a valid recommendation if you're building CRUD-based webapps, but I haven't developed those in quite some time. Nowadays, the apps I develop are true SOFEA apps, where the backend serves up XML or JSON and the frontend client is HTML/JavaScript/CSS, Android, iPad or Sony Blu-Ray players. On my current project, our services don't even talk to a database, they talk to a CMS via RESTful APIs. We use Spring's RestTemplate for this and HttpClient when it doesn't have the features we need. Not much in Java EE 6 for this type of communication. Sure, Jersey has a client, but it's certainly not part of the Java EE spec.

As far as getting Ruby on Rails' zero-turnaround productivity, I don't need Grails or Spring Roo, I simply use IDEA and JRebel.

Conclusion
I don't see how new features in Java EE 6 can mean the age of frameworks is over. Java SE and J2EE have always been foundations for frameworks. The Java EE 6 features are often frameworks in themselves that can be used outside of a Java EE container. Furthermore, Java EE 6 doesn't provide all the features you need to build a high-scale web app today. There's no caching, no stateless web framework that can serve up JSON and HTML and no hot-reload productivity enhancements like JRebel. Furthermore, there's real excitement in Javaland for languages like Scala, Groovy and JRuby. All of these languages have web frameworks that've made many developers happy.

Here's to the Age of Frameworks - may it live as long as the JVM!

P.S. If you'd like to hear me talk about web frameworks on the JVM, I'll be speaking at The Colorado Springs Open Source Meetup and Devoxx 2010 in the near future.

My Incredible Trip to Ireland

2010-06-14T23:42:55-06:00

If you ever get a chance to travel to Ireland, take it! I don't know when I heard these words, or how they came into my head, but I remembered them clearly when I was first introduced to Barry Alistair by Jeff Genender. Soon after, I was able to negotiate my way into being a speaker at The 2010 Irish Software Show.

The show was last week and I had a blast traveling to Dublin to speak and explore. My sister came me on this trip, but missed a connection in Seattle and had to join me a day late. I left Denver at noon on Monday and arrived at Dublin Airport at 7 am. I was on the same flight as Josh Long and thoroughly enjoyed my iPad as a travel companion. When I got off the plane, my battery life was at 60% and I'd been watching movies and listening to music for 6 hours.

I took a cab through the misty, cool morning to my hotel. I grabbed a coffee, cleaned up, and walked a few blocks to Trinity College for the conference. I made it in time for the opening keynote by Chris Horn. It was an interesting talk, focusing on what needed to happen to make Ireland the Innovation Hub of Europe. After that, I attended Tim Berglund's session on Complexity Theory and Software Development. After lunch and a few more talks, I teamed up with Andres Almiray and Josh Long for a pint at the hotel bar.

That evening, we attended Jeff Genender's talk on Getting into Open Source. The free drinks loosened everyone up and Jeff did a great job with a humorous presentation on how to get Committer Status. After Jeff's talk, about 10 of us headed to a Moroccan restaurant for a late dinner. I was in bed around midnight.

Wednesday morning, my sister arrived in my hotel room at 8 and promptly fell into bed. I set my alarm to sleep an hour and closed the Vegas-style, no-light-allowed curtains. We awoke much later (12:30) than we'd planned (9:00). We quickly got up and headed for some sight-seeing in Dublin. First off, we hit Dublinia and Christ Church Cathedral. Both sites were spectacular and we both learned a lot about the history of Dublin. From there, we skipped across the bridge to The Old Jameson Distillery for a tour and a bit of whiskey.

The picture below was taken on the Ha'penny Bridge as we were heading back from Jameson. The expression of the girl on the left is priceless.

A couple hours later and I was delivering my talk on The Future of Web Frameworks. The crowd was lively; the Guinness I drank while talking was lovely. My session was followed by a Web Framework Experts Panel with Peter Ledbrook (Grails), Jamie van Dyke (Rails), Shay Friedman (ASP.NET MVC), Julian Fitzell (Seaside) and myself (Java Frameworks). The debate was good and there was much discussion about the right apps for each framework and how important statelessness is for scalable applications. After 3 hours of talking, my sister and I headed back to the hotel. I was particularly happy about the evening since it was the first time a family member of mine had seen me speak.

Correction from my Dad: This wasn't the first time a family member saw me speak. He attended my talk at ApacheCon EU 2007.

A block from the hotel, we spotted a nice looking pub (Doyles) and stopped in for a pint. As we bellied up to the end of the bar, we recognized Jamie (from the panel) and got introduced to his friend Rob. We quickly got lost in conversation, stories and laughter and were surprised when we discovered it was 2:30am. Since I had a talk first thing in the morning, we ducked out shortly after.

Thursday started with my talk Comparing Kick-Ass Web Frameworks. Then my sister and I did some more site-seeing, starting at the Guinness Storehouse. We met Josh and John Willis as they were leaving and they advised we go straight to The Gravity Bar at the top. We took there advise and were getting great views of Dublin and savoring sweet pints of Guinness moments later. The tour facility was freakin' awesome and I loved how it was shaped like a pint glass.

We grabbed some gear from the gift shopped and landed (by accident) at The Brazen Head (Ireland's Oldest Pub, Est. 1198) for a pint of cider and Guinness. Since my sister used to be in the cider business, she was particularly happy there was so much on tap in Ireland.

From the pub, we headed to John Willis's session on The Cambrian Cloud Explosion. Following John's session, we headed to the Speaker's Dinner for a very fun evening with the hosts and speakers of the conference.

On Friday, we woke up in the early afternoon and quickly decided the Book of Kells was our best chance of getting some site seeing in. After visiting the Book of Kells, my favorite quote of the conference happened in the courtyard.

Josh looked at Jamie (with his bad hangover) and exclaimed, "My God Man. Your skin is so white it's hurting my eyes!". You probably had to be there (or know Josh) to enjoy the humor, but I wanted to capture the memory in this post so I could laugh whenever I read this in the future. After that, Jamie, Josh, Kalin and I enjoyed a Starbuck's patio talking about living in the South of France for a couple hours. Then we walked 2 blocks to the Porterhouse Brewing Co. to watch the World Cup and enjoy more interesting conversations.

Jamie left the conference that evening and we joined a whole slew of other speakers for dinner at an excellent Lebanese restaurant near Temple Bar. Good times where had afterwards at a nearby Silent Disco.

Saturday, we woke up early to catch a tour bus out to Glendalough with Josh and John. The bus ride was not pleasant, but the destination was spectacular. We hung out there for several hours, exploring the buildings, walking to the lake and humoring each other.

Our last night in Dublin was an early, relaxing one. As you can tell, I really enjoyed this trip, particularly hanging out with my sister and all the cool people we met. I can easily say that this trip registers as one of my favorite conference experiences to date.

To see all the pictures I took on this trip, check out my Irish Software Show 2010 set on Flickr.

My Presentations from The Irish Software Show 2010

2010-06-10T07:11:35-06:00

This week I've been enjoying Dublin, Ireland thanks to the 2nd Annual Irish Software Show. On Wednesday night, I spoke about The Future of Web Frameworks and participated in a panel with Grails, Rails, ASP.NET MVC and Seaside developers. It was a fun night with lots of lively discussion. Below is my presentation from this event.

This morning, I delivered my Comparing Kick-Ass Web Frameworks talk. This presentation contains updated statistics for various metrics comparing Rails vs. Grails and Flex vs. GWT.

Thanks to all who attended my talks this week!

P.S. I believe audio was recorded on Wednesday night, but I'm unsure how it turned out. I'm pretty sure no recordings were done on this morning's session.

2009 - A Year in Review

2010-01-11T10:06:09-07:00

I wrote my first "year in review" post in 2005 and continued the tradition in 2006 and 2008. This year, my December was filled with unplanned circumstances, a new job and a houseful of family for the holidays, so I never had time to sit down and write this post. As things are returning to normal, I figured it's about time I kicked off 2010 with one of my favorite writing reflections.

2009 started off with a bang: I wrote about my Mom nailing a bear's nuts to a tree after she killed it. My new gig (at Evite) started out fun with choosing an Ajax framework and a Tech Meetup in LA. My bike got stolen, I started running to work and traveled to Tahoe without an ID.

February started off with an epic weekend at Silverton. I went to Web Directions North and attended many good talks:

I wrote my first GWT-related entry and a few posts related to independent consulting.

I ended February with a couple more GWT-related entries.

In March, I bought a new 15" MacBook Pro and shipped it back shortly after to get a 256 GB SSD. I still have nothing but good things to say about the machine. I discovered Nexus is awesome and the kids and I went on our first hike of the year.

My GWT Journey continued with GXT MVC, modularization and optimization. I got a new office and new bike and proudly witnessed my Dad's Retirement.

April came and I got Drunk on Software, had a blast at Holly and Jason's Wedding and published our Ajax Framework Analysis Results.

At the end of April, I started building the kids a treehouse and inspired smiles with two new kittens. On May 7th, I had PRK eye surgery and wrote about my experience in early June. I continue to be extremely happy with the results.

May ended with Ryan and Breanne's Wedding in Playa Del Carmen. Having so many great friends around and the Nuggets vs. Lakers playoffs the same week made this one of the best vacations of my life.

June brought the news that the Colorado Software Summit was over. There's been several times in the past few months that I've missed the annual experience. Can someone please start a conference at a Colorado mountain resort in the near future? Pretty please! I've always experienced this conference with Bruce and we continued another tradition (riding to Red Rocks) with 2nd Row seats at Big Head Todd.

My GWT posts continued with a Facebook-style Autocomplete, Implementing OAuth, JSON Parsing with JavaScript Overlays and a preview of GWT 2.0. I wrote about implementing SOFEA with GWT and Grails at Evite.com and had a blast at the Great Sand Dunes on Father's Day.

My job hunt began and I started a month-long vacation in Montana with Raible Road Trip #13. Vacationing for a summer month in Montana has been one of my goals for several years. Accomplishing it this year made me extremely happy and I hope to make it a summer tradition.

July was an awesome month in 2009. Granted, April and May were special with tropical weddings, but Montana in July is a particularly tasty treat. My Summer Vacation in Montana attempts to capture how much fun we had. It was particularly enjoyable because my parents, children and many life-long friends were involved.

As summer began to set, I decided to get back into speaking at conferences, starting with the Rich Web Experience. I wrote about initial GWT work at my new client, which included an interesting experience with Java REST Frameworks. The month ended with one of my favorite holidays: Jack's Birthday.

September was nice and uneventful. I learned about Concurrency on the JVM Using Scala, started using MVP with GWT and learned how to do more with less. I also ran in a 10K.

October started out with a family trip to Washington for my sister's fabulous wedding. People flew in from all over the US and we had a sweet condo on Lake Chelan for the week. Playing golf, wine tasting and celebrating with good people made for a great start to the fall season.

In November, I started writing more, mostly because I was gearing up for upcoming talks and thinking about / working on AppFuse. I celebrated Abbie's Birthday, did some website optimization and chuckled at the comments about my hunting season adventure.

Right before Thanksgiving, I got a call from my client letting me know that their budget had run out my contract would end soon. Luckily, I had an interview setup the next day and had great success in finding a new gig.

I ended November with a trip to Oregon for Thanksgiving and ran in the Oregon Mid-Valley Road Race. The followed week, I flew with my kids and parents to The Rich Web Experience and learned about Objective-J and Cappuccino. I had a near-perfect (high 70s, no lines) Disney World experience with my family, watched the Ducks with the Civil War and compared kick-ass web frameworks.

In December, I didn't do much blogging - mostly because I arrived home from Florida to discover a waterfall in my guest room. The water pipes were routed through the ceiling, had busted from the cold, and water was pouring everywhere. Dealing with that and starting a new job occupied most of my time and I never got a chance to write much down. I ran in the Jingle Bell 5K and watched the Broncos lose a lot. Shortly after, my family came for Christmas and a good time was had by all.

As I reflect back on last year, my biggest surprise is that I got into running. I ended up running in 5 races last year and even enjoyed doing it a few times. It's still not my favorite activity (skiing and mountain biking win that title), but I enjoy it enough to do it a couple times each week. The goals I wrote down for last year were: visit 3 foreign countries, take 3 months of vacation and spend 1 month in Montana. I only made it to 1 foreign country (Mexico), but I did take 2 months of vacation and got my month in Montana. I'll take that.

In 2010, I hope to speak at (or attend) 3 conferences, finish up The Bus and do a whole bunch of skiing and mountain biking. More than anything, I plan to continue having a lot of fun with my family and implementing a lot of cool technologies along the way.

It's gonna be a great year.

Introduction to Objective-J and Cappuccino with Tom Robinson

2009-12-03T11:21:59-07:00

This morning, I attended Tom Robinson's talk on Objective-J and Cappuccino. Tom is one of the founders of 280 North and creators of the Cappuccino framework and Objective-J language, so I was very interested in hearing about Cappuccino from the source. The text below are my notes, but they're also mostly Tom's words, not mine. I've added a "Thoughts" section at the end that are my words.

Tom's Team was Cocoa programmers before they started building Cappuccino. They wanted to focus on building Desktop Class Web Applications (for example, Google Maps, Meebo and 280 Slides). Tom showed a demo of 280 Slides and how it can rotate and scale images very easily, something you don't often see in web applications.

To build desktop class web applications, you can use Flash or Silverlight, but they're controlled by Adobe and Microsoft. Also, they have no iPhone support and poor Mac and Linux performance. The other option is JavaScript + DOM. They're open standards, available almost everywhere (including mobile devices) and its a very rich ecosystem with lots of competition. The downside to JavaScript is standards bodies, many incompatibilities, technical limitations (e.g. can't access web cam) and the DOM is very document-centric.

The bottom line is we can't fix Flash, but we can fix JavaScript.

This is what 280 North is trying to do with Objective-J. It's a proper superset of JavaScript, has a few syntax additions, has a powerful runtime and is implemented in JavaScript. Objective-J is analogous to Objective-C. It adds to JavaScript like Objective-C adds to C.

One of the first things Objective-J adds is Dependency Management. You can import from search paths:

@import <Foundation/CPObject.j>

Or from relative paths:

@import "ApplicationController.j"

@import prevents duplicate loads has asynchronous downloading and synchronous execution. That means all files are downloaded before evaluation begins, but to the programmer, it seems to happen synchronously.

The thing that sets Cappuccino apart from other libraries is its inheritance model. It uses classical OO inheritance (using Objective-C syntax).

@implementation Person : CPObject {
    String firstName @accessors;
    String lastName @accessors;
}

- (String) fullName {
    return firstName + lastName;
}

@end

The type definitions (String) are ignored for now and primarily used for documentation. In the future, they plan to add optional static typing, hence the reason for having them. Tom is unsure if you can leave off the String type or if the compiler requires it.

@implementation has proper support for super and language syntax support. One of the reasons they chose Objective-C is because classical inheritance works great for UI Frameworks.

Objective-J uses "send a message" syntax instead of "call a method" syntax. In the code snippets below, the first line is JavaScript, the second is Objective-J:

object.method()
[object method]

object.methodWithFoot(arg1)
[object methodWithFoo:arg1]

object.methodWithFooBar(arg1, arg2)
[object methodWithFoo:arg1 bar:arg2]

Dynamic Dispatch is one of the most interesting parts of Objective-J. forwardInvocation in Objective-C is like method_missing in Ruby. Methods can be used as references, for example:

var action = @selector(someMethod:);

Runtime mutability is important for KeyValueCoding (KVC) and KeyValueObserving (KVO). KVC allows you to swap classes at runtime and KVO allows you to listen for when property values change. At runtime, a $KVO_ClassName is generated. This class notifies any registered observers when values are changed and then calls the original class to change the property.

Cappuccino
Cappuccino is an application framework, not a library. It uses the Hollywood Principle: "Don't call us, we'll call you".

The Framework handles document management (open, save, revert), content editing (undo, redo, copy, paste) and graphics manipulation. The DOM is designed for documents (same is true for HTML and CSS). Tom doesn't like the DOM as its not a good API for building applications. Proof is all the JavaScript libraries built to make the DOM better.

Cappuccino has an MVC framework and CPView is its View. It's analogous to a <div> and represents a rectangle on the screen. Everything visible is a CPView or one of its subclasses. It defines resizing and layout behavior. CoreGraphics is Cappuccino's canvas-like drawing API. It uses VML on IE, canvas on everything else.

Very little of the code in Cappuccino talks to the DOM (less than 2%). It's not just about providing widgets that work in all browsers, it's a way to write platform independent display code.

Events are done very differently than most JavaScript libraries. Browser's dispatching is not used. A single event listener is registered for each type of event on the window. These events are captured and sent to the objects that need to know about them. This allows for consistent events across all browsers, even keyboard events. It also allows for creating custom event flows and easily creating custom events. Cappuccino events allow you to get around a common problem with DOM Events where you can't click on overlapping rectangles.

Notifications can be registered and sent very easily. Both "scoped" and private notifications can be created.

Undo Management is included in Cappuccino. It manages a stack of undos for you. Redos are "free" and undo functionality is part of the document architecture. This makes it easy to integrate with auto-save functionality.

Run loops (also called event loops) are an advanced feature of Cappuccino. They allow you to perform actions on every run loop. This enables complex optimizations for DOM/Graphic operations and undo grouping.

The final part of Cappuccino is Keyed Archiving. Keyed Archiving stores a graph of Objective-J objects. It handles reference cycles, conditional inclusions, has an efficient data format and works on the client and server (Objective-J can be run on the server). The data format is similar like binary, but it's UTF-8. Keyed Archiving is used for archiving views and used heavily in 280 Slides for storing, retrieving, and exporting presentations.

Other applications implemented with Cappuccino include almost.at and Mockingbird. EnStore uses it too, but only for its admin interface.

An interesting extension for Rails developers is CPActiveRecord, a reimplementation of Rails' ActiveRecord in Cappuccino.

There are several tools included with Cappuccino:

objj: command line Objective-J
objcc: "compile" ahead of time
press: optimize code and resources
nib2cib: convert Mac OS X nibs
capp: project creation

All these tools are built on Narwhal (which conforms to the CommonJS standard).

CommonJS
CommonJS is an effort among server-side JavaScript projects to standardize non-browser JavaScript APIs. There's numerous API specifications (so far):

Binary, File, IO
stdin, stdout, stderr, args, env
Web server gateway (JSGI) - similar to WSGI and Rack for Python and Ruby

To learn more about CommonJS, see CommonJS effort sets JavaScript on path for world domination.

Narwhal is 280 North's implementation of CommonJS APIs. It works with multiple JavaScript engines, including Rhino, JavaScriptCore (SquirrelFish) and XUL Runner. According to Tom, Rhino is an order of magnitude slower than JavaScriptCore and V8. Of course, Narwhal supports Objective-J too.

Aristo
Aristo is the new default theme in Cappuccino and was created by the Sofa design firm. It includes windows, tabs and menus and is open source so you can modify.

Atlas
Atlas is an IDE for Cappuccino, focused on building user interfaces graphically. Atlas is a downloadable application for OS X. It's written almost entirely in Cappuccino. The desktop version bridges Cappuccino windows to native windows. Tom did a demo of Atlas and showed how its layout feature allows you pin, center and align very easily. It's all done with JavaScript because doing layouts with CSS is often very painful. After that, he showed us how can you Atlas to very easily build a Web Application and then export it as a native OS X application without changing a line of code. Atlas includes Mozilla's Bespin for code editing.

To learn more about Aristo and Atlas, you might want to checkout Ajaxian's Big News from Cappuccino: Aristo and Atlas from earlier this year.

Atlas has a $20 Beta Program if you're interested in trying it out.

My Thoughts
Cappuccino looks like a very cool web framework. It reminds me of GWT in that you have to learn a new language to use it. However, Atlas takes a lot of that pain away. I particularly like how it has document and undo/redo support built-in. On my current GWT project, this would be very useful as we've had to build this functionality by hand.

AppFuse 2.1 Milestone 1 Released

2009-11-19T07:16:36-07:00

The AppFuse Team is pleased to announce the first milestone release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate, Spring and Tapestry 5.

What is AppFuse?
AppFuse is an open source project and application that uses open source tools built on the Java platform to help you develop Web applications quickly and efficiently. It was originally developed to eliminate the ramp-up time found when building new web applications for customers. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose which persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run appfuse:full-source.

In addition, AppFuse Light has been converted to Maven and has archetypes available. AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket.

Other notable improvements:

Added Compass support thanks to a patch from Shay Banon.
Upgraded from XFire to CXF for Web Services.
Moved Maven repository to Sonatype's OSS Repository Hosting for snapshots and releasing to Maven Central. There are no longer any AppFuse-specific artifacts, all are available in central. Thanks to Sonatype for this great service and its excellent repository manager.
Upgraded to Canoo WebTest 3.0. Now if we could just get its Maven Plugin moved to Codehaus.
Added Ajaxified Body to AppFuse Light archetypes.
Infrastructure upgrades, including JIRA 4, Confluence 3, FishEye 2, Bamboo 2 and Crowd 1.6. Many thanks to Atlassian and Contegix for their excellent products and services.
For more details on specific changes see the release notes.

Please note that this release does not contain updates to the documentation. Code generation will work, but it's likely that some content in the tutorials won't match. For example, you can use annotations (vs. XML) for dependency injection and Tapestry is a whole new framework. I'll be working on documentation over the next several weeks in preparation for Milestone 2.

AppFuse is available as several Maven archetypes. For information on creating a new project, please see the QuickStart Guide.

To learn more about AppFuse, please read Ryan Withers' Igniting your applications with AppFuse.

The 2.x series of AppFuse has a minimum requirement of the following specification versions:

Java Servlet 2.4 and JSP 2.0 (2.1 for JSF)
Java 5+

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find bugs, please create an issue in JIRA.

Thanks to everyone for their help contributing code, writing documentation, posting to the mailing lists, and logging issues.