Search for [appfuse] in weblog rd

Life Update: New Job, Same House, and Same Awesome Family!

2024-11-16T09:31:40-07:00

It's been over eight years since I wrote a life update blog post. At that time (2016), we'd just moved into the "Raible Ranch", the bus was finished, and the kids were in high school. I'd just started one of my few-and-far-between full-time jobs at Stormpath.

Fast-forward to today. Our kids are in college at the University of Colorado during the Coach Prime Era, we're empty nesters with an affection for live music, and our pet zoo is cuter than ever.

You probably know me. But in case you don't, my name is Matt Raible, and I'm married to a fabulous woman named Trish McGinity. She's been a hero in my life ever since I met her in 2010. She dazzled me so much, I asked her to marry me at Versailles in 2011. We married in my hometown (Condon, Montana) in 2013, on the most beautiful lake you've ever seen.

When I met Trish, I was an independent consultant, and she was in cybersecurity sales. On a fateful night in 2010, on a friend Jarvis's birthday, I asked her what she had done for a living. She said, "Have you ever heard of OWASP?"

My heart lifted and I felt a moment of joy. This woman was speaking my language! 14 years later and Oh My!, Trish has had an incredible influence on my life and career.

You might think of me as the AppFuse guy, the JHipster promoter, or some dude who talked about Okta a lot. I'm cool with that, but it's irrefutable to say that Trish hasn't influenced my career. She encouraged me to lean into cybersecurity and constantly said I was "doing it wrong" with my carefree attitude toward security.

These days, I'm a Senior Cloud Engineer at CrowdStrike, the foremost cybersecurity company in the world. My previous gig was with Okta, where I spent seven years educating developers about OAuth and OpenID Connect.

I'll admit, I had a lot of anxiety about joining CrowdStrike. In late June, after enjoying funemployment, I had three excellent offers from FusionAuth, Yugabyte, and CrowdStrike. It's the first time in my life that I've had more than one offer at the same time.

The first two opportunities were excellent. But, both had maybe equity. As an old fogey in the twilight of my career, I thought guaranteed equity was a better opportunity. I chose the company trajectory over my passion for open source and Java. I accepted the CrowdStrike offer on July 3rd as we were departing for a 4th of July trip to Montana.

At that time, I felt totally confident about my decision. I was taking the path less traveled and abandoning all my Java celebrityism for a skyrocketing company that was kicking ass and taking names. You're supposed to do things outside your comfort zone, right?

Take the road less traveled.

I was a little worried about the equity though. After all, CrowdStrike's stock price was at an all-time high.

I turned 50 on July 16th. We hosted a backyard party at the ranch the weekend before (July 13th), complete with a local Grateful Dead band. Shakedown Street was excellent and everyone had a good time.

The Incident

On my birthday, while blowing out candles, I wished for CrowdStrike's stock price to drop.

On Friday (July 19th), I woke up to more unread texts than I received on my milestone birthday. A lot of friends were aware that I'd accepted a new job at CrowdStrike and were texting me about an incident.

Apparently, CrowdStrike had just caused a worldwide computer outage. If you were flying that day, you were probably affected. I received more text messages that morning than on my birthday.

My response was positive: this situation will be very good for my equity grant's strike price!

I quickly realized that CrowdStrike was way bigger than I originally thought. There were computer issues at the golf club I ventured into that morning. Their systems were down and they apologized. I smirked, apologized back, and mentioned I recently accepted a job offer from CrowdStrike.

I started working for CrowdStrike on July 29th. 10 days after the incident. I quickly realized that making developers aware of CrowdStrike wouldn't be a problem. Everyone knew about us because of July 19th. In a sense, it was a blessing in my DevRel journey at CrowdStrike.

I worked at Okta for seven years and there were a couple of incidents. CrowdStrike's response was waayyyy different than Okta's.

First of all, CrowdStrike's incident wasn't a breach. It was a bug that they shipped. It was totally their fault. Okta had a full-on breach that they didn't disclose for a month. Regardless of the incident details, as a tech worker for said company, it's a severe morale hit when it happens to you.

That's been one of my biggest struggles. I was extremely proud to join CrowdStrike as a Senior Engineer. Heck, I passed the coding test using Java, and they were a Go shop! When this all happened in June, I was so proud of my non-traditional path to excellence in the tech industry.

I wasn't hired as a senior engineer who cranks out code, maintains functionality, and implements new features. Instead, I was hired as a "Senior Engineer - Cloud" that's focused on DevRel. I'm still a developer advocate, but my title indicates I can still sling some code.

I've been coming up to speed on Go and Python as fast as I can. I've also been doing a lot of CrowdStrike Foundry tutorials, finding/fixing issues, and improving our learning materials as fast as I can. I even published my first CrowdStrike blog post!

Get Started with Falcon Foundry

Now that I've filled you in on the life of Matt Raible, I feel like I've ignored the best parts of my life: Trish, Abbie, and Jack.

Trish continues to excel as the Head of Cybersecurity Communications at Empower. Yes, the Empower that has the naming rights to the Empower Field at Mile High where the Denver Broncos play. We haven't experienced box seats yet, but we're still hopeful. Trish is the star of her team, and it's her favorite job, ever. She's delighted at the fact that I'm asking her if SIEM and SOAR are acronyms or just fancy industry terms.

You're probably wondering if we're evil parents at this point. You've learned about Trish and me, but what about the kids, the pets, and our parents? I mean, this is a life update after all!

The kids are doing great. As a casual blogger, I still managed to post about both of their birthdays this year. Jack and Abbie are thriving!

They're still in school at CU Boulder and enjoying the Coach Prime Era. Abbie's a senior and already has a job. That's quite an accomplishment, and we're very impressed!

Jack's thinking about dating. He didn't try out for the CU basketball team, but he's still interested in making it happen.

The pets are currently spoiled. Trish's best friend, Caroline, has moved in. She takes the two dogs (Daisy and Kai) on daily walks. I tend to like the same activity, and they're very good at pretending it's their first walk of the day with both of us!

Our parents are thriving too! I recently attended Trish's Dad's 85th birthday celebration. He's doing splendid and likes to swim and play tennis daily.

My parents are still loving life in Montana. A friend of theirs has a photo trap set up for wildlife on the homestead. He captures some amazing photos! Check out his website for more fabulous shots.

Trish has more to her story. She's experienced some of the most extreme ups and downs of her life. In July, shortly after we celebrated my birthday, we spent a week in Estes Park. The reason was simple: she wanted to compete in a horse show and I wanted to support her, explore the nearby trails, and visit the local breweries.

We decided to make her lifelong dreams come true and leased the most kick-ass horse she could find. His name is Athos, and he was the horse she competed with in Estes Park. He's 18.2 hands tall, which means he's 6' at the saddle. Trish is only 5'1". They won Reserve Champion on their first collaboration!

Trish and Athos were a match made in heaven. Until they weren't.

A month ago, Trish fell off Athos after a jump in a training lesson. It was a tack failure while demoing a new saddle, and the stirrup strap slid right off. She almost saved it, but proceeded to fall and injure herself badly in the process. When I met her at the ER (Emergency Room), her pain level was 9.5/10 for her wrist and 3/10 for her ankle. She broke both her wrist and her ankle in the same accident. It wasn't Athos' fault, but rather a faulty stirrup bar.

Since this happened, we've learned I'm a terribly grumpy caregiver, she's had her wrist surgically repaired, and her ankle had surgery a week later.

Breaking two bones at the same time really limits your mobility. I wouldn't wish it upon anyone. Trish is optimistic that she'll compete with horses in the future, but she's also realizing it gets harder as you get older.

What's Next?

Trish, Jack, and I have all had midlife crises this year. Trish cranked things up a notch to be a world-class eventer with horses, then got hurt. Jack was going to be an aeronautics engineer, then took a helicopter flight over Glacier National Park and changed his mind. I abandoned all my Java experience and chose CrowdStrike, a Go and Python shop.

Abbie's a senior in college and already has a job. I think she's winning the family badass contest!

We have no idea what's next. But we're proud of Abbie for securing her first job.

As for the rest of us? We'll figure it out. Raibles and McGinitys are known for thriving in the face of adversity. 💪😃

For a photo and video experience of our summer adventures, see my Summer Adventures 2024 album on Flickr.

JHipster Conf 2018: Summer Solstice in Paris

2018-06-28T10:13:27-06:00

Last week, I journeyed to Paris with my son, Jack. It was his first time in Europe and I brought him along for good reason. I’d been invited to the first ever JHipster Conf, and I was eager to attend. We were both pretty excited when we left Denver last Monday.

Our adventure to Paris begins! #jhipsterconf

A post shared by Matt Raible (@vwsforlife) on Jun 18, 2018 at 4:01pm PDT

My Background with JHipster

I’ve been a part of the JHipster community for a few years now. I joined by accident, really. I was trying to market myself as an independent consultant by spouting my knowledge of Spring Boot and Angular with an InfoQ mini-book. Since JHipster leveraged both to jumpstart app development, it seemed like a perfect fit. I’ve been a long-time fan of app jumpstarts, having developed my own called AppFuse in days long gone.

Through the process of writing the mini-book, finding issues, and submitting pull requests, I eventually found myself to be a member of the JHipster development team. Through my relationship with JHipster, and it’s 3.0 release, I found myself intrigued my microservices and how to develop them with Spring Boot, Spring Cloud, all through the generation expertise of JHipster.

I’ve learned a ton by being part of the project and trying to figure out how all of its options work.

When I found myself with a full-time job at Stormpath, I did my best to create a Stormpath module for JHipster. When Okta acquired Stormpath, I added a similar module to my list of things I wanted to write.

When it came time to implement an Okta module, I discovered JHipster’s OAuth support only worked internally, not with an external OAuth provider, also known as an Identity Provider, or IdP. I mentioned to the JHipster team I thought we could do better and add support for external providers instead. They agreed, and I went to work.

In hindsight, it was a great decision and not terribly difficult to implement thanks to Spring Security, Keycloak, and Docker. We had a ton of help from the community along the way, and as of last October, JHipster added support for single sign-on with OIDC (tested with Keycloak and Okta).

JHipster 5.0: Spring Boot 2.0, Angular 6, and React

It’s been a fabulous adventure on the JHipster train and it’s still going strong. We just released version 5.0 with React and Spring Boot 2.0 support, there’s client generators for Ionic and React Native, and we just hosted a kick-ass conference about JHipster in Paris.

JHipster Conf 2018

The conference featured members of the core team, the well-dressed and fit Joe Kutner from Heroku, as well as Java celebrities like Ray Tsang and Josh Long. You can read about the festivities and presentations from JHipster's founder, Julien Dubois.

The conference was a little over 24 hours long, starting with a speaker’s dinner on Wednesday evening. Before attending, Jack and I spent the day strolling around Versailles. Versailles is a special place in my life since I proposed to my double rainbow there after Devoxx Belgium in 2011.

?? Versailles #working #jhipsterconf

A post shared by Matt Raible (@vwsforlife) on Jun 20, 2018 at 9:15am PDT

The conference kicked off with a keynote by the JHipster's co-leads: Julien and Deepu.

Full house at #JHipsterConf @java_hipster #paris pic.twitter.com/nnZ6JmlXrW
— Deepu K Sasidharan (@deepu105) June 21, 2018

JHipster Conf 2018 keynote from Julien Dubois

During my talk, I had Jack join me on stage for an intro, and tried to give him a taste of public speaking in front of hundreds.

After a fabulous day at #JHipsterConf and a fun rooftop party at @ippon_technologies , Jack and I enjoyed a romantic dinner and a bit of Fête de la Musique. ??

A post shared by Matt Raible (@vwsforlife) on Jun 21, 2018 at 2:44pm PDT

My talk went well, with some successful and some failed demos. Hopefully people got the point that it’s cool to store your users outside of JHipster so you can share them between apps. I also tried to show that OAuth and OIDC are excellent for securing APIs. You can download my presentation from Add JHipster to Your JHipster Apps with OIDC or view it below.

I created a tutorial of the app I showed in my talk and published it to the Okta developer blog: Build a Photo Gallery PWA with React, Spring Boot, and JHipster. If you like React and OAuth, you're gonna love this guide!

Jack and I had a day in Paris after the conference, so we made the most of it. We hit the Eiffel Tower, hiked the stairs, and marveled at the view. After, we waited in a long line for The Catacombs and walked among the dead.

I had a wonderful time this week in Paris and at @jhipsterconf. Many thanks to the @java_hipster developers, community, @juliendubois, @deepu105, and @ippontech for making it all possible! #JHipsterConf pic.twitter.com/61ZfRZxg8G
— Matt Raible (@mraible) June 23, 2018

Kudos to the JHipster Community!

What a trip! It’s so much fun to be a part of JHipster’s thriving open source community. It’s not just the project itself; it’s all the projects we build upon, from Java to TypeScript to Spring Boot to Spring Data to Spring Security to Angular to React to webpack to Bootstrap. It’s a conglomeration of all of my favorite tools and open source developers encompassed in several awesome projects!

Life as an open source developer is pretty fun. I encourage you to get involved in open source too! I started way back in the early 2000s with Struts and Ant, and it’s done wonders for my career.

Viva La Open Source!

2017 - A Year in Review

2018-01-31T16:16:39-07:00

2017 was a year with many changes. We changed presidents, Stormpath joined forces with Okta, our daughter started high school, and we lost two of our precious pets. On the upside, I traveled to many beautiful places, talked to developers around the world, and became a Devoxx Champion. Not only that, but our two classic VWs ran like champs the whole year. We put upwards of 10K adventure miles on our Syncro, and another couple thousand on Hefe the Bus.

I'm going to look back on 2017 using the following categories.

Professional

2017 was the first year I started with a full-time job in January since the turn of the century. I'd been an independent consultant for most of my career. Stormpath enticed me enough with their people and vision that I became a full-time employee with them in September 2016.

January started slow, but I started to hit my stride in February when I traveled to Sweden for Jfokus and skiing. Coincidentally, it was shortly after the conference that I found myself negotiating with Eric Berg about joining forces with Okta. I remember it well: I was sitting in the hotel lobby, with James Ward and Ray Tsang, when it all happened. Shortly after, we were sitting in the back of the bus on our way to a skiing adventure.

A week later, I was back in the states and the deal with Okta was happening. Throughout January, I thought the possibility was 20%. When I returned home from Jfokus, it was full-on happening. In fact, I was one of the last ones to sign the deal, causing panic and a few phone calls in the process. Most of the folks at Stormpath became Okta employees on February 27.

The awkward part was we knew about the deal when we traveled to Devnexus. We had a Stormpath booth there, and we were unable to tell everyone there would be no more signups after Friday, February 24.

The following week, we had a massive onboarding with everyone. There was excitement in the room and many high spirits, so it was a fun day. Shortly after, we entered into a 90-day quiet period for Okta's IPO. It was strange to start a new job with a new company and not be able to talk about it.

Okta didn't have much for SDKs when we started writing blog posts in March. However, it supports standards like SAML and OAuth, so I was able to put together some posts using Spring Security.

I wrote about my anticipation for Devoxx US and France in mid-March.

Towards the end of March and early April, I had some luck integrating with Angular, both with Okta's Sign-In Widget and OpenID Connect.

In early April, I hosted an Angular Workshop and spoke about Cloud Native Progressive Web Apps with Josh Long at Devoxx France.

We're about to kick off the part deux of our Cloud Native Apps series with @mraible @DevoxxFR pic.twitter.com/0GCSAiUgs4
— Josh Long (???, ???) (@starbuxman) April 6, 2017

On my flight home from that trip — on April 7 — Okta had its IPO. Things started trucking along after the IPO. I wrote a plethora of blogs posts in April and published them over the spring and summer months.

Speaking

This year, speaking became a part of my job, so I spoke frequently. According to TripIt, I took 29 trips, to 41 cities, and 14 countries. I traveled 115,456 miles and spent most of my time on United. By my count, I spoke at 22 events.

I wrote about my experiences at J-Spring, Devoxx UK, GeeCON, and Spring I/O in May, and Devoxx Poland in June. Spring I/O was especially fun because Trish met me in Barcelona.

I was featured on the main Okta blog in late September in Developer DNA: A Day in the Life of Matt Raible.

As the year was waning, I wrote about life as an open source developer and my adventures to Devoxx Belgium and Devoxx Morocco.

In my blog post about SpringOne and The Rich Web Experience, I included a picture of my 2017 conference lanyard collection.

To end the year, I published some popular posts on the Okta Developer blog:

Devoxx4Kids

We held two Devoxx4Kids Denver workshops in 2017: MIT App Inventor - Anyone Can Build Apps and Fruit Ninja with Scratch. These classes were taught by Regina Peyfuss and Melissa McKay, respectively. I'm happy to announce they've both since joined the board for Devoxx4Kids Denver and have been doing a great deal to help organize things.

I ?? the smell of learning in the morning. #Devoxx4Kids Denver in full swing w/ @rpeyfuss at @ThriveWorkplace. Awesome space and instructor! pic.twitter.com/a6Ac1qAJh0
— Matt Raible (@mraible) June 3, 2017

Our next meetup will be on March 10 and will be taught by Melissa's daughter, Kaitlyn! I think it's pretty cool we have an opportunity to have kids teaching kids. I've also been working with Jay Zimmerman and UberConf to have an UberKids this summer.

Projects

AppFuse is no more, so I primarily work on JHipster and the JHipster-Mini Book these days. I refactored JHipster's OAuth authentication in September to use an external identity provider (e.g., Keycloak or Okta) instead of an embedded one. I wrote about how to use OpenID Connect with JHipster on the Okta Developer blog in October.

With the help of InfoQ, we released v4.0 of the JHipster Mini-Book on September 22nd. It's hard to believe, but it's still up-to-date!

Personal

I wrote down my goal at the beginning of 2017: to drive Hefe up the California coast. In July, I wrote about why that goal got moved to my bucket list:

I wanted to spend our summer vacation driving our VWs up the California coast, on a mammoth 3500-mile road trip over two weeks. However, when a landslide happened near Big Sur, I knew it was probably best to move this road trip from my yearly goals to my bucket list. Instead, we opted to drive to Montana and spend a couple of weeks vacationing in my childhood playground.

We had a blast road-tripping, hiking, and rafting in Montana. A highlight for me was hiking to Rumble Lake, somewhere I hadn't been to since I was a kid.

The drive home through Jackson Hole and the Flaming Gorge was pretty nice as well!

My only other personal post in the first half of 2017 was Farewell to the 2016-17 Ski Season.

I had a blast skiing in Winter Park, Copper, Crested Butte, Sweden, Steamboat, Tahoe, and Montana.

The highlight of the season was skiing at Big Sky Resort in Montana. We're super pumped that next year's ski pass includes Big Sky!

In early August, we rafted the Salmon River in Idaho with our rafting family and had an epic trip.

Jack turned 13 in late August and we had a splendid weekend with my folks in town.

Abbie and Tucker won their division at the SummerFest Horse Show in September!

Abbie didn't want a birthday party this year, so we surprised her with one instead. It's hard to believe she's a freshman in high school. She's even driving us around with her learner's permit these days! Her birthday was a bittersweet celebration since we had to say goodbye to Jake earlier that day.

We welcomed a new Australian Shepperd-Border Collie mix puppy into our home in December. Her name is Daisy.

A post shared by Matt Raible (@vwsforlife) on Dec 2, 2017 at 3:24pm PST

Stout and Hefe

Stout 5.0 and Hefe 3.0 were released in early May. Stout had his hood painted and Hefe got a kick-ass stereo system installed.

For Hefe's stereo, I tried going phone-only for a controller. This turned out to be a bad idea, mostly due to bit Play HD and its terrible mobile app. Also, Hefe is lowered and a bit bumpy in the front, so trying to use a touch screen while driving doesn't work very well. He's in the shop now getting a new deck installed.

We drove Stout over 5000 miles this summer and he's been super dependable. Now that he's reliable, we often marvel that he really is the greatest car ever built.

2018

Professionally, I'll be speaking at a lot of Java User Groups in 2018. The Developer Relations team at Okta is focusing on large US cities this year, so you won't see me overseas as much. This is nice because it means I get to spend more time with my family. I have received offers to speak in other countries where they'll pay for all my expenses. If expenses are covered, Okta will allow me to travel overseas. However, I've been telling people I'm only willing to speak if they'll pay for a family member's flight too. It seems to be a win-win so far: most conferences have declined, and I don't have to travel as much. If I find a conference that is willing pay, I'll get to travel with someone from my awesome family!

I'll be coding loads of example apps and writing heaps of tutorials for the Okta Developer blog and I'm super excited about our Iterate Conference next month. I'll also continue to help organize DJUG and Devoxx4Kids Denver.

Romantically, Trish and I hope to never stop dating with the 2-2-2 rule. I look forward to a year with running VWs, long dog walks, and disconnecting to connect. As for goals, I'll go for the ol' standby: to be happy and content with what I have. 😊

2016 - A Year in Review

2017-02-01T17:46:59-07:00

When I wrote my 2015 year in review blog post, I was certain my '66 VW Bus would finally be finished. AND IT IS! Do I need to even write this year's post? Yes, because I want to tell you how awesome it is to own this incredible-looking, awesomely-fast, mean machine.

But first, let's review the year using the following categories.

Professional

I had two different clients in 2016: CA Technologies and Stormpath. I worked full-time for CA in January and February, helping them adopt AngularJS. To help them learn about Angular 2, I rewrote my AngularJS getting started and testing tutorials for Angular 2. The first versions were published at Getting Started with Angular 2 and Testing Angular 2 Applications. I refactored both tutorials to use Angular CLI in August and published Getting Started + Testing with Angular CLI and Angular 2 (RC5). Since then, I've been maintaining an up-to-date version on GitHub.

In April, I started working half-time for CA and half-time for Stormpath. For Stormpath, I worked on their Java SDK and helped them launch their Java SDK 1.0. I really enjoyed working with the team at Stormpath. This led to me think about my priorities in life. I realized that I wanted to work remotely, get paid to speak at conferences, and get paid to work on open source. Stormpath provided me with all of these opportunities and I started working full-time for them on September 26, 2016.

In May, I joined the board of the Denver Java User Group. I've been helping organize meetups, find speakers, and secure location sponsors. If you're interested in speaking at DJUG in 2017, please let me know!

I wrote about life as an open source developer in early November. At that time, I mentioned being more stressed than I was used to. My stress levels continued to be high through the end of the year because I get to create my own job. I get to choose the conferences I submit too, the talks I create, the example apps, the blog posts, everything. There's so much I want to do that I've had to learn to scale back and not try to do so much at once.

I didn't blog much on raibledesigns.com in Q4 of 2016, but I wrote a bunch of technical posts on the Stormpath blog.

According to TripIt, I flew 59K miles in 2016.

Speaking

I spoke at nine different events in 2016. You can find links to my presentations on my presentations page.

February: Denver Open Source Users Group
April: Colorado Springs Open Source Software Meetup Group and Devoxx France
June: Devoxx UK and GeekOut
August: HTML5 Denver Users Group
September: vJUG24
November: Rocky Mountain Software Symposium
December: The Rich Web Experience

My most memorable conference in 2016 was Devoxx France because I took Abbie and we experienced springtime in Paris together.

Becoming a Java Champion at Devoxx France was certainly a surprise too!

I really enjoyed taking my mom with me to Devoxx UK and GeekOut. It was my first time speaking at both conferences and I was impressed by both personnel and venues.

Devoxx4Kids

We only had two Devoxx4Kids Denver workshops in 2016: Introduction to Object-Oriented JavaScript using Bitsbox and Exploring JavaScript. The second one was taught by the world-famous Venkat Subramaniam.

The great @venkat_s teaching #Denver kids about JavaScript. So awesome! #Devoxx4Kids pic.twitter.com/CqOW5ma6Cq
— Matt Raible (@mraible) September 10, 2016

I'd like to plan a few Devoxx4Kids Denver events in 2017, so please let me know if you have a topic you'd like to present.

Projects

I stopped working on AppFuse in February and announced I was mothballing the project in April. Reminder: the demos, documentation and bug tracker will cease to exist on April 30, 2017.

I spent many late nights in July through October updating the JHipster Mini-Book for JHipster 3.x. On the book's blog, I wrote about what's changed and how to buy a print version.

Personal

I wrote down my goals at the beginning of 2016:

Finish the bus
Win a trophy
Ski a lot
Raft a lot
Stay in shape
July in Montana
Happy Trish
Ionic/Angular2 app
Learn how to make a good dirty martini

I'm proud to the bus is finished and it won Best in Class at VWs on the Green.

Today was a great day! Won best in class Bus, Camper, Pickup Custom 1949-67 Split Window. #vwbus

A photo posted by Matt Raible (@vwsforlife) on May 15, 2016 at 2:41pm PDT

We skied a lot, but never took our own raft on river. I stayed in shape and have records to prove it thanks to 21-Points Health and dailymile. This summer, I spent seven weekends in a row away from home; with three weeks in Montana. Not quite a month, but close. Trish was happy because we journeyed to Hawaii to celebrate her parents' 50th anniversary.

Her happiness continued when she found her dream home in July. We moved into The Raible Ranch in September.

Weeks later, she and Abbie bought their dream horse named Tucker. I wrote about him in Abbie's 14th birthday post.

I learned how to make an excellent dirty martini, but did not find the time to create an Ionic/Angular2 app.

The Ski Bus (aka Stout)

Stout the Syncro spent a couple months getting upgrades in late 2015: transmission rebuilt, fridge installed, Propex heater installed, and new headlights. We leveraged these improvements on an epic family road trip to Jekyll Island (Georgia) for my Dad's 70th birthday.

We made it back to Denver just in time for the AFC Championship game and thoroughly enjoyed ourselves tailgating beforehand.

Watching the Broncos win Super Bowl 50 was definitely a highlight of 2016.

We took Stout on a bunch of ski trips and winter camped several times. The Propex heater and Reflectix insulation worked great and kept it around 50°F even when it was 0°F out.

I bought a new mountain bike (Yeti SB 95) in May and Trish and I drove the van to Syncro Solstice in Moab. We were hoping for a weekend of mountain biking and 4-wheeling, but Trish had a bad crash and stabbed herself in the chest with a handlebar. That ended our 4x4 aspirations for the weekend, but the van ran great!

The other major road trip for Stout was when I drove from Colorado to Montana over four days. I drove to Grand Junction on Monday evenning, worked during the day on Tuesday, went mountain biking in the afternoon, then drove to Jackson Hole that night. Wednesday, I worked in Jackson Hole, went mountain biking, then drove to Montana that night. I woke up early on Thursday and made it to Missoula to start working promptly at 8am. That evening, I drove to Seeley Lake and setup camp at the first annual Bob Marshall Music Festival. Trish flew in on Friday and we danced and celebrated my birthday all weekend.

The Porsche Bus (aka Hefe)

Hefe's debut at VWs on the Green was a wild success and we had a lot of fun that day. In June, we showed him at Colorado Concours d'Elegance and won a blue ribbon. We trailered him to Montana and drove him in the Swan Valley parade and the Seeley Lake parade.

We hauled him back in Denver to be in the Colorado Bug-In at Bandimere Speedway, where he got second in the custom/radical class.

On the way home from that event, one of the rear shocks blew up from rubbing against the axle. That was the beginning of the end for my air suspension. I drove Hefe to Kustom Coach Werks in early August to have fix the doors (they had gaps) and tune the suspension. When they finished, I called it "release 1.5".

I drove Hefe for a month without rear shocks and didn't have too many issues; it was just bouncy in the back. Buses at the Brewery in Colorado Springs was a fun event and the first time that Hefe went viral on Facebook. This has happened a few times since then. He gets thousands of likes from being featured on a VW fan page.

Before the drive to Colorado Springs, I noticed the rear suspension was missing bolts, causing the rear end to shimmy. I was able to find a new bolt and fix it, but the problem with bolts falling out continued in September. I tried smaller wheels (15" Fuchs), but the suspension was still really stiff in front. Because of the suspension issues, I shipped it back to KCW in October. They ripped out the air suspension and replaced it with coil over shocks, making for a much better ride. It now can cruise at 90 mph easily and looks great all the time! After these repairs, it's now Hefe 2.0.

If you're interested in reading about the suspension decision process, or to see pictures, see this thread on KCW's forums.

2017

Professionally, I'll be speaking at a lot of conferences and meetups this year. As a Developer Evangelist for Stormpath, I'm expected to speak at least once a month. I did a webinar on Getting Started with Angular in mid-January and I'm traveling next week to Jfokus in Stockholm. See my future engagements for more scheduled events.

I suspect most of the coding I'll be doing will be related to Stormpath's SDKs, both Angular and Java. Of course, I'll help out on JHipster as much as I can too. With JHipster 4 on the horizon, there will be a lot of work to update the JHipster Mini-Book.

Hefe 3.0 will be released this spring after I get a sweet sound system installed. As far as personal goals for 2017, I have only one: to drive Hefe up the California coast. With a new house, two running VWs, and a job that's a helluva lot of fun, life is pretty awesome right now.

A Delightful Trip to Devoxx UK and GeekOut 2016

2016-06-12T11:13:17-06:00

I had the pleasure of traveling to London, England and Tallin, Estonia this past week. In London, I spoke at Devoxx UK. In Tallin, I spoke at GeekOut. I took my mom (or mum, if you prefer) and we explored the sights, enjoyed local cuisines and savored a few beverages. Our trip started with a direct flight from Denver to London. We arrived on Tuesday, June 7, around noon.

We were only in London for two nights, but it was enough time for us to savor excellent Indian food, fancy a walk through London, and order a bow tie. I forgot the bow tie for my JHipster outfit. Luckily, I found a good replacement and was able to order it for next-day delivery. I had to order it by 5pm and the site declined both my credit cards with time running out. I ended up using PayPal and got my order placed in the nick of time: 16:59:51.

The big news announced at Devoxx UK is that Devoxx is coming to the United States in 2017! I'm on the program committee for this conference, so I look forward to helping make it spectacular.

The first ever #DevoxxUS is happening March 21-23, 2017 in San Jose, CA! https://t.co/jtQe95LNOL
— Devoxx US (@devoxxus) June 8, 2016

I thought both my talks went well, but I did have some issues during my Angular 2 talk. I used Angular CLI to do the demo and IDEA kept indexing, which caused my live templates to fail. I later learned from an attendee (thanks!) that excluding the dist and tmp directories helps prevents this. You can find the source code for my demo on GitHub.

You can view my "Art of Angular in 2016" presentation on SlideShare, watch it on YouTube, or just click through it below.

I also did a presentation on JHipster, leveraging JDL Studio heavily. You can click here to see the JDL I used for my blog demo. A video of this session is on YouTube.

We closed out our visit to London with a fun speaker's dinner and an early morning flight to Helsinki, then Tallinn.

In Tallinn, we wandered through its Old Town, had some great meals and marveled at how late the sun set. We stopped in at the GeekOut conference's party on Thursday night. The ZeroTurnaround guys did a heckuva job planning this conference. I loved how both the party and the conference were in large, industrial-but-modern buildings with great sound systems.

My JHipster talk went well on Friday (slides here) and my mom and I celebrated at Rataskaevu 16 that evening. Then we shared laughs and learned how to play jackbox.tv at ZeroTurnaround's post-conference party.

Our last night was spent in Helsinki, a 20-minute flight from Tallinn. Since my grandparents are Finnish, my mom delighted in hearing words from her youth along the streets. We both took advantage and relaxed in our hotel's roof top saunas too. We indulged in a food/wine-paired dinner over three hours that evening.

Thanks to the organizers and good friends from Devoxx UK and GeekOut. We had a great time and are still smiling about all our new memories.

Update on June 20, 2016: Devoxx UK has posted a channel on YouTube with all of its sessions. You can view my Art of Angular talk or Get Hip with JHipster.

Moving AppFuse into the Attic

2016-04-28T15:40:16-06:00

In mid-February, I decided to stop working on AppFuse. My reason was simple: I was no longer getting any value from my contributions to the project. I sent a message to the developers mailing list the next day:

Hello everyone,

Last night, I started working on AppFuse 4.0, with the following features from the roadmap:

Remove XML wherever possible

Java 8

Spring Boot

Spring Data

JSR 303 (might require removing or developing client-side support)

As I started removing XML and integrating Spring Boot and Spring Data, it quickly became apparent that it’d be a lot of work to make all of these changes. My guess is it’d take over 100 hours of my time to do everything. This is time I’d be taking away from my family and personal time.

At the end of last year, I wanted to make AppFuse 4.0 happen because I thought it’d help me stay up-to-date with Java technologies and learn some things along the way. As I dug into the codebase last night, I realized it’d be more of a headache than a learning experience. It seems there would be little reward for all the work.

Because there’s little-to-no activity on the mailing list these days, it seems like it’s the right time to shutdown the project and dedicate my free time to other open source endeavors. As you might know, I’m a big fan of JHipster (http://jhipster.github.io/). It combines AngularJS and Spring Boot and has all the features that AppFuse has - but with a more modern technology stack.

If we had everything hosted on GitHub, I think it’d make sense to add a line to the README that says “This project is no longer maintained”. However, since there’s a lot hosted on appfuse.org (with Confluence), it might not be that easy. Maybe it’s possible to export everything from Confluence to static HTML pages and host them somewhere with the same URLs so there’s not a bunch of 404s from shutting down the project.

Thank you for your contributions over the years. AppFuse was pretty cool back in the day, but now there’s better solutions.

Cheers,

Matt

The good news is I've worked out a deal with Contegix to keep appfuse.org up and running for the next year. The demos, documentation and bug tracker will be available until April 30, 2017. Bamboo and FishEye will be discontinued in the next week since they're too memory intensive for a smaller server. I'd love to figure out a way to export all the documentation from Confluence to Asciidoctor so everything can be on GitHub for years to come. However, there's something to be said for just letting a project fade away rather than holding onto nostalgic artifacts.

On a related note, Java.net will be closing in a year from today. AppFuse started on SourceForge, but moved to appfuse.java.net shortly after. Today, the only thing left on java.net are AppFuse's mailing lists. I suppose it makes sense that both projects will cease to exist around the same time.

AppFuse's source code will remain on GitHub. I have no plans to delete it.

Thanks to everyone that used and contributed to AppFuse over the years. It was a pretty wild and crazy ride from 2003-2007!

2015 - A Year in Review

2016-01-12T16:21:15-07:00

2015 was the year The Bus was supposed to be finished. If you read my year in review from last year, you'll see I was certain of it. To be fair, I did have estimates from people that had me expecting it to be done in July. The good news is the interior was finished in July. Since then, it's been back at Reincarnation getting the finishing touches applied. I believe if it was worked on for a week straight, it could be finished. It's that close. So close I can taste it. THIS will be the year!

For this Year in Review post, I'll same the format I've used the last few years.

Professional

I had four different clients in 2015. The first was in the healthcare industry, the second in the API hosting space, one in the fashion industry and one in computer software. For the first client, I wrote about integrating Node.js, Ruby and Spring with Okta's SAML support. I also helped them adopt and learn AngularJS. Learning about Foundation and Angular was a nice treat too.

In March, I revisited how to setup your own software company. In that post, I wrote about how I felt when valuing time over money.

Earlier this year, I had the opportunity to work 20 hours per week instead of 40. It was one of the greatest work-life experiences I've had to date. I was still able to pay all my bills, and I had time during each-and-every-day to do something fun. When working 40 hours per week, exercising and cooking dinner were somewhat of a chore. When I flipped to working less, work became the chore and exercise and cooking became the fun parts of my day. I read somewhere recently that if Americans valued health over wealth, we'd be a lot better off. I felt like I did this when working less and that I was rich in time.

The second client asked me to do a research project where I tried a bunch of API platforms (e.g. Apigee and 3Scale) and wrote up an analysis about five different stories they provided. It was a fun project where I learned a lot.

In April, I went to work at a fashion industry startup in downtown Denver. I wrote JavaScript, CSS and HTML for them for most of the year. The location afforded me the ability to discontinue my office at Thrive, a co-working space in Denver. While I missed having kombucha and local craft beer on tap, I loved the standup desk and the office environment with conversations, banter and laughter between desks.

In November, I picked up a new client that I'm currently working with. They've recently adopted Angular and I've enjoyed the experience of working with a team of UI developers. We're putting a fresh UI on a legacy product and the team has been able to move really quickly. They're also in the Bay Area so it's been fun traveling to Silicon Valley once a month. My contract with them ends this spring.

Speaking

I spoke about Angular at Denver's Open Source User Group in February and about JHipster at Denver's Java User Group in April. In the second half of the year, I re-joined the conference circuit, starting with UberConf in July. September brought me to SpringOne 2GX in Washington, DC and the Angular Summit in Boston.

Trish and I giggled with glee as we returned to the Devoxx Belgium in November.

My JHipster talk was a hit and we loved our brief tour of Brugge.

As always, the people and the memories were the best part.

To end the year, I journeyed to Fort Lauderdale for the Rich Web Experience. After speaking, I worked remotely and networked for a few days. The networking lasted until the wee hours of the morning and good fun was had by all.

Devoxx4Kids

I didn't plan any Devoxx4Kids events until later in the year. The Denver Chapter watched Arun and Aditya Gupta speak about modding Minecraft in August. I also presented on setting up a Minecraft server in the cloud. In October, the talented Kellyn Pot'Vin-Gorman taught a class about coding simple games with the Raspberry Pi. It's always fun to see the kids' faces light up when they solve a programming problem.

This year, with the help of Tuliva, we hope to host one Devoxx4Kids class per quarter. If there's a particular topic or speaker you'd like to see, please let me know.

Projects

I released a new version of AppFuse in February and started working on the JHipster Mini-Book in March. To help provide updates about the book, I created a blogging app with JHipster and started www.jhipster-book.com. I wrote the book and created 21-Points Health as its example application. I open-sourced 21-Points on GitHub during my Devoxx presentation. Because of my writing and speaking about JHipster, I was invited to join the JHipster Team. I happily accepted; it's been a great group to work with.

The JHipster Mini-Book 1.0 was released on October 30, 2015. I wrote about how I wrote it a few days later.

Personal

I wrote down my goals at the beginning of 2015:

21-Points Fitness App
JHipster Mini Book (InfoQ)
Finish Bus
New House
Good Blood Pressure

I'm proud to say I accomplished three out of five. The Bus didn't get finished and we failed to sell our house. I did create 21-Points Health, wrote The JHipster Mini-Book and maintained good health all year.

We weren't planning on selling our house this year; we wanted to find a new house and rent out our current one. However, we found a house we loved (small house, lots of land) in the heart of DTC and putting our house on the market was the best way to try and get it. Our offer failed and we endured the house-is-on-the-market dance through July and August. It was pretty sweet living in such a clean house, but we got tired of living in a de-personalized abode. We took it off the market in September and plan on renting it once we find our next home.

The worst part of the year started with a phone call from a friend in the early afternoon on October 7. My good friend, Cletus, called to notify me that he'd found our friend, Jason Miller, dead that morning. Jason was a pledge class brother of Cletus and mine in the Chi Phi Fraternity at DU. You might remember Jason from my trip to Oktoberfest in 2008. He was one of my best friends.

Jason was a heckuva guy and an awesome friend. He always inspired laughter and smiles with his witty humor. He will be missed, but not forgotten. Jason had a zest for life and a good time that we've resolved to remember annually with a brother's trip to an exotic gambling destination. This year, we'll be traveling to Las Vegas for St. Patty's Day and March Madness.

The Ski Bus

The Syncro Ski Bus had a pretty good year. We took it on many skiing adventures, with trips to Steamboat, Crested Butte, and Telluride.

We enjoyed a couple overnights and tailgated a few times at our favorite ski area: Mary Jane.

I took Jack and we met my parents in Moab in May for Syncro Solstice 2015. We took it for some off-roading, busted a cooling hose a couple times, and almost made it back to Denver.

That next weekend, we took it to VWs on the Green and showed it alongside The (under construction) Bus.

In July, we journeyed to Montana for the 4th of July. We drove our Westy in the Swan Valley parade while the kids rode on top and threw water balloons and candy to the crowd. We drove back from Montana in a single day (900 miles). I remember Wyoming well. Sirius XM was broadcasting the last Grateful Dead concert from Soldier Field and Trish was driving. She sang for hours, driving through the rain and fog while I dozed periodically.

Sewfine finished The Bus's interior in July. I wrote about this and many other things in Life Update: The Bus Project, New Gigs, New House and More.

A week later, I won a runner-up trophy for the Syncro at The Colorado Bug-In at Bandimere Speedway. On my birthday, July 16, we drove it to Dinosaur National Monument for a week-long raft trip, starting at Gates of Lodore. I hit my first deer that night, driving at 3am and doing minimal damage to our German-engineered RV. The deer broke the headlight, but not the bulb.

We had some more camping/rafting adventures with it in the fall, but eventually had to take it to the shop for a transmission rebuild in November. While it was getting repairs, we upgraded a few things. It now has a South African grill, modern headlights, a new fridge and a Propex heater. It's finally become the ultimate camping vehicle, for summer or winter.

2016

Professionally, I plan to continue speaking at conferences this year. I've submitted a few proposals to Devoxx France, Devoxx UK and I've agreed to speak at Geek Out. And did you hear that SpringOne 2016 is in Las Vegas?!

I suspect this will be the year of Angular 2 and microservices. Watch this project to see how JHipster implements microservices. I plan to pump out a release of AppFuse in the near future, implementing #NoXML and moving to Java 8.

Trish and I hope to move into a new house this year, but won't be disappointed if we don't find the right one. We have another year before Abbie starts high school at 7am.

I have no real big projects planned, except for driving The Bus. We do have a few fun vacations planned. In fact, I'm writing this while riding in the back of our Westy. My parents and I are road-tripping from Denver to Jekyll Island, Georgia. We're taking the scenic route to see some places we've never been and staying at RV Parks with the top popped. We arrive in New Orleans this afternoon and will be staying two nights to enjoy the culture and cuisine.

My Dad turns 70 this weekend and we planned a family vacation to celebrate. His brother and sister are joining us and Trish and the kids are flying in this weekend. We figured a road trip across the country was appropriate since we've done so many together and I happen to own the ultimate vehicle for car camping. Below is a map of our route.

In June, we'll be celebrating Trish's parents 50th wedding anniversary in Kauai. With Trish's brother, sister and children joining in, everyone is sure to have a fantastic vacation. I hope to continue the fun by working remotely from Montana in July.

When I drive the finished bus in the Swan Valley parade on the 4th of July, I'll be smiling from ear-to-ear. 2016 is the year! The long wait will be over soon...

Google's Mirror of Maven Central 25% Faster

2015-11-10T00:13:51-07:00

Last week, Takari announced that Google is Maven Central's New Best Friend. While writing a news article about this for InfoQ, I decided to run a small test to see the speed of the default Maven Central versus the new Google Cloud Storage instance. This micro benchmark didn't seem worthy of including in the article, but I think it's interesting to see the speed improvements I found.

I ran rm -rf ~/.m2/repository, then mvn install with the default repository configured. I ran the commands again with Google Cloud Storage. I found that the downloading of dependencies, compilation and running unit tests on AppFuse's web projects averaged 4 minutes, 30 seconds. With Google Cloud Storage, the same process averaged 3 minutes and 37 seconds. By my calculations, this means you speed up artifact resolution for your Maven projects by 25% by switching to Google. To do that, create a ~/.m2/settings.xml file with the following contents.

<settings>
  <mirrors>
    <mirror>
      <id>google-maven-central</id>
      <name>Google Maven Central</name>
      <url>https://maven-central.storage.googleapis.com</url>
      <mirrorOf>central</mirrorOf>
    </mirror>
  </mirrors>
</settings>

Benchmark Details
My tests were run on a Mac Pro (late 2013) with a 3.5 GHz 6-Core Intel Xeon E5 processor and 32 GB of RAM. Bandwidth speeds during this test averaged 57 Mbps down, 6 Mbps up. Below are the timing numbers (in minutes) from my test:

Default: 4:33, 4:36, 4:32, 4:24, 4:09
Google: 5:13, 3:35, 2:15, 3:38, 3:39

Google had some wide variances in its results, with five minutes and two minutes. Because of this, I dropped the low and high numbers for each service before calculating the average. My math with raw numbers is below.

Default:
273, 276, 272, 264, 213 = 260, 4:20
276, 272, 264 = 270, 4:30

Google:
313, 215, 135, 218, 219 = 220, 3.66 = 3:40
215, 218, 219 = 3:37

Chen Eric commented on the InfoQ article to note that Chinese programmers are blocked from using Google.

Update: Jason Swank of Sonatype has done some more extensive benchmarking and found different results.

We found that average unprimed Google API (first mvn run) caching performed 30% slower than Maven Central. Primed Google API cache performance (second run) was 3% faster then Maven Central (second run). We also ran a number of cloud-based tests with similar results.

SpringOne 2GX 2015: My Presentations on Comparing Hot JavaScript Frameworks and NoXML

2015-09-20T12:29:00-06:00

Last week, I had the pleasure of traveling to Washington, DC to speak at the annual SpringOne 2GX conference. I was pretty stressed for the last few weeks because I had to create two new presentations from scratch, and both had to be 90 minutes long. I was also hoping to finish the JHipster Book before the conference started. I was able to finish both presentations in the nick of time, but did not find the time to write the last chapter in the JHipster Book.

The first presentation was titled Comparing Hot JavaScript Frameworks: AngularJS, Ember.js and React.js. I started by revisiting the Comparing JVM Web Frameworks talk I did at vJUG last February. I explained how I think traditional web frameworks are no longer relevant in 2015, but I do believe server-side rendering is still very relevant. From there, I used Yevgeniy Brikman’s framework scorecard (from his Node.js vs. Play Framework presentation) to rank each framework by a number of different criteria. You can see the final results on slide 160. Since the scores were so close, I believe you could tweak some scores a bit (or add weights to the different criteria) and make any of the frameworks come out on top.

You can click through the presentation below, download it from my presentations page, or see it on SlideShare.

I started writing the second presentation a week before I had to deliver it. On Thursday, September 10th, I stayed up late, trying to figure out how to create a good presentation on NoXML and finish the last part of the JHipster Book. Then it came to me, I needed to parallelize and do them both at the same time. I decided to compare AppFuse (which is similar to a legacy Spring application with lots of XML) to JHipster (which hardly contains any XML).

I wrote a 10-page Google Doc on how I planned to do this, then went rafting and camping with my family for the weekend. I finished most of the presentation on Monday night, but then realized the presentation wouldn't be long enough to fill 90 minutes. So I hunkered down at midnight, created a new AppFuse application and removed a bunch of its XML. This took me until 3:30am, and I was able to accomplish the following tasks:

Spring XML to Java
Spring Security Configuration to Java
web.xml to WebApplicationInitializer
Spring MVC to Java
Migrated to Spring Boot
Maven to Groovy

I was pretty pumped when I completed my final goal: converting to Spring Boot and getting a test to pass. I made commits to an appfuse-noxml project on GitHub as I accomplished each step. You can see all the changes in the project's commit log. While I'd figured everything out, I still needed to complete the presentation. Luckily, I found time to do this the night before, the morning of, and in the final hour before I had to deliver the talk. You can imagine my relief when I was done delivering both talks.

You can click through the presentation below, download it from my presentations page, or view it on SlideShare.

While I didn't get to spend much time at the conference, I did have a lot of fun while I was there. I got to meet some new folks, reconnect with old friends, and enjoy beers and dinner with a smiling crew on Thursday night. The Broncos victory late that night was the icing on the cake.

Getting Hip with JHipster at Denver's Java User Group

2015-04-09T08:31:54-06:00

Last night, I had the pleasure of speaking at Denver's Java User Group Meetup about JHipster. I've been a big fan of JHipster ever since I started using it last fall. I developed a quick prototype for a client and wrote about solving some issues I had with it on OS X. I like the project because it encapsulates the primary open source tools I've been using for the last couple of years: Spring Boot, AngularJS and Bootstrap. I also wrote about its 2.0 release on InfoQ in January.

To add some humor to my talk, I showed up as a well-dressed Java Developer. Like a mature gentleman might do, I started the evening with a glass of scotch (Glenlivet 12). Throughout the talk I became more hip and adjusted my attire, and beverage, accordingly. As you might expect, my demos had failures. The initial project creation stalled during Bower's download all JavaScript dependencies. Luckily, I had a backup and was able to proceed. Towards the end, when I tried to deploy to Heroku, I was presented with a lovely message that "Heroku toolbelt updating, please try again later". I guess auto-updating has its downsides.

After finishing the demo, I cracked open a cold PBR to ease my frustration.

I did two live coding sessions during this presentation; standing on the shoulders of giants to do so. I modeled Josh Long's Getting Started with Spring Boot to create a quick introduction to Spring Boot. IntelliJ IDEA 14.1 has a nice way to create Spring Boot projects, so that came in handy. For the JHipster portion, I created a blogging app and used relationships and business logic similar to what Julien Dubois did in his JHipster for Spring Boot Webinar. Watching Josh and Julien's demos will give you a similar experience to what DJUG attendees experienced last night, without the download/deployment failures.

You can click through my presentation below, download it from my presentations page, or view it on SlideShare.

You might notice my announcement on slide #32 that I've signed up to write a book on JHipster.

I haven't started writing the book yet, but I have been talking with InfoQ and other folks about it for several months. I plan to use Asciidoctor and Gradle as my authoring tools. If you have experience writing a book with these tools, I'd love to hear about it. If you've developed an application with JHipster and have some experience in the trenches, I'd love to hear your stories too.

As I told DJUG last night, I plan to be done with the book in a few months. However, if you've been a reader of this blog, you'll know I've been planning to be done with my '66 VW Bus in just a few more months for quite some time, so that phrase has an interesting meaning for me.

How To Setup Your Own Software Development Company, 6 Years Later

2015-03-02T09:26:01-07:00

Just over six years ago, I wrote a popular post titled How To Setup Your Own Software Development Company. I'd just left LinkedIn a few months earlier and was enjoying consulting life again, working with a group of friends at Evite. In the article, I wrote about how I liked consulting because it forces you to keep your skills up-to-date and it pays a lot better. I also talked about the type of legal entity you should form (I have an S Corp), what business insurance you should buy, what I had for health insurance and how I automated payroll and tax payments.

I recently received an email from a reader, asking me if I had any updated thoughts.

It's been nearly six years since you wrote the article about starting your own business ... and thanks, by the way.

I am starting my venture into independent contract work as a software engineer (Java technology) in California and most likely will setup an S corp entity.

Seeing that you wrote this six years ago and things have considerably changed in the U.S. (economy, health care, etc.), I was wondering if you had some updated thoughts to share, perhaps some learned lessons even.

And also, I have questions about business insurance: what type of insurance should I opt for? Is there really an umbrella insurance out there? Or does each (or many) clients out there dictate the insurance you need?

Yes, a lot has changed in the last six years. The economy has improved and health care costs have risen. Through this time, I've been able to continue to operate as an independent software developer and keep the contracts flowing. Personally, the biggest changes in my life have been outside of work. I met an exceptional woman, traveled to conferences all around the world with her, got married, traveled some more, then bought a VW Westfalia so we could have lots of fun traveling in our own backyard. All the while, I've worked for some great clients. I built a team of hot shots at Time Warner Cable (many of them still work there), I skied the awesome powder of Utah while working at Overstock and I enjoyed a long-term contract at Oracle. After Oracle, I got into the healthcare industry and I've been working in it ever since.

In fact, I just finished working for a healthcare company last week and I'm on the hunt for my next gig in April. Check out my LinkedIn profile if you'd like to see my résumé.

I've learned quite a few lessons over the last several years. As an independent developer, the biggest thing I've learned is marketing is key. I've always known this, but I've been reminded of its importance a few times. When I worked at Taleo (after Overstock), I was on a 3-month contract that turned into a 9-month contract that got a 1-year extension when Oracle bought them. The work was challenging, but the application was outdated. Getting them to adopt new technologies like Bootstrap and AngularJS was difficult. When Oracle took over, they offered me a 1-year contract at a great rate. I accepted, never thinking it would be difficult to get paid from someone like Oracle. It took them over three months to pay my first invoice and it took me another three months to get payments flowing regularly. I felt like I was trapped. I felt like I could quit, but that wouldn't speed up the process of getting my invoices paid. From this experience, I'm hesitant to start with any contract that's longer than three months.

During my time at Oracle, I didn't blog as much as I had previously (because the day-to-day work wasn't that exciting), but I did still speak at conferences. Last year, I took the year off from speaking at conferences altogether. Speaking is an excellent marketing tool. Because of my lack of speaking, I saw a downturn in contract opportunities in Q4 last year.

As far as health insurance is concerned, I continued to have a disaster prevention plan, with a $5K per year deductible. I paid around $300/month for this, and rarely used it. By riding my bike to my office in downtown Denver, and skiing a bunch in the winter, I felt like I was pretty healthy. After I stopped eating sugar last fall, I became much healthier. So much healthier that I've stopped taking high blood pressure medication. Today, I don't pay for health insurance. Trish went back to IT Security Sales in November and she was able to get me on her company's plan for $100 cheaper than what I was paying. I didn't have dental insurance for the last five years and I did have to shell out $5K for a tooth implant at one point.

For business insurance, I have the Business Owner's Policy from The Hartford. I pay around $600/year and I've gotten that back when I've had laptops stolen or accidentally killed my iPhone. I've got automated backups going all the time, so I haven't lost any data in several years. This insurance policy and its liability coverage has been "good enough" for all my clients, including the big ones.

I think the biggest lesson I've learned in the last several years is that the best way to be rich is to be rich in time. I've always dreamed of making $500/hour and working 20 hours per week. While $500/hour sounds crazy, you know there's consultants out there that are making that kinda cash. They're probably not in software, maybe they're political consultants, or former professional athletes, but those consulting rates do exist. In software, there's certainly companies that bill those kinda rates. My rates for the last several years haven't been that good, but they've been pretty awesome.

Earlier this year, I had the opportunity to work 20 hours per week instead of 40. It was one of the greatest work-life experiences I've had to date. I was still able to pay all my bills, and I had time during each-and-every-day to do something fun. When working 40 hours per week, exercising and cooking dinner were somewhat of a chore. When I flipped to working less, work became the chore and exercise and cooking became the fun parts of my day. I read somewhere recently that if Americans valued health over wealth, we'd be a lot better off. I felt like I did this when working less and that I was rich in time.

Related to feeling better over making more, I've started to target employment opportunities that offer a good team to work with. For the last year, most of my contracts have been with remote clients, where they haven't required me to travel onsite. While this sounds great in theory, I do miss the comradery that exists when working with a team. Working with someone over a Skype/HipChat call is nothing like sitting next to each other and cracking jokes while writing code. Don't get me wrong, I love remote work, but I do think it's important to be onsite and collaborating face-to-face at least once per month.

To those individuals looking to start their own Solopreneurship, I hope this advice helps. It's been a great experience for me.

AppFuse 3.5 Released!

2015-02-20T09:08:53-07:00

The AppFuse Team is pleased to announce the release of AppFuse 3.5. This release contains a number of improvements.

XML reduced by 8x in projects generated with AppFuse
CRUD generation support for Wicket, as well as AppFuse Light archetypes (Spring Security, Spring FreeMarker and Stripes)
Upgraded Tapestry to 5.4
Integrated Spring IO Platform for dependency management
Refactored unit tests to use JUnit 4
Renamed maven-warpath-plugin to warpath-maven-plugin
Upgraded to jWebUnit 3 for AppFuse Light integration tests
Updated all AppFuse Light modules to be up-to-date

For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is a full-stack framework for building web applications on the JVM. It was originally developed to eliminate the ramp-up time when building new web applications. Over the years, it has matured into a very testable and secure system for creating Java-based webapps.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the users mailing list. You can also post them to Stack Overflow with the "appfuse" tag.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix, and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

2014 - A Year in Review

2015-01-31T09:37:43-07:00

2014 was destined to be a spectacular year. When I wrote my thoughts down last January, I thought the Broncos would win the Super Bowl and my VW Bus restoration project would be finished by summer. To focus on finishing the bus project, I didn't submit any talks to conferences. Instead of traveling to exotic locations, we opted to visit a bunch in our own backyard instead.

I should've known it'd be an interesting year when the Broncos flopped in the Super Bowl.

For this Year in Review post, I'll use the format I've used the last couple of years.

Professional

I had three different clients throughout the year, all in the health care industry. I continued working on a project with John Muir Health where we developed a hybrid mobile app. I wrote about what I learned in Documenting your Spring API with Swagger and Developing an iOS Native App with Ionic. Since I wrote the article about Swagger's Spring MVC support, it has become much easier to integrate; now it only requires an @EnableSwagger annotation.

In April, I started consulting with a small company in Alabama. I helped them modernize their tech stack and implemented a number of web services with Apache Camel. I wrote about this in a four-part series in September and October.

I continued leveraging Spring Boot in my client projects and wrote about Building a REST API with JAXB, Spring Boot and Spring Data at the end of that engagement.

In November, I had time off between clients. I visited my parents and their awesome retirement home in Montana. I also took some time to refactor AppFuse to be a bit leaner and meaner. I signed up with a new health care client in December. It's been an enjoyable experience so far, especially since I'm only working 20 hours per week this quarter.

I'm currently booked with clients through the end of March, but hoping to find one with a challenging project in April.

Speaking

To optimize cash flow towards the bus project, I didn't submit any talks to conferences in 2014. However, I did talk about Comparing JVM Web Frameworks at the Denver Open Source Users Group and vJUG in February. I also spoke at local user groups (DeRailed and HTML5 Users) about The Art of AngularJS in the first part of the year. In the fall, I participated in panels on unit testing at Denver Startup Week and building a personal brand at AirConf.

I hope to attend and speak at more conferences in 2015. To help facilitate this, I've published a list of talks I'd like to present.

Devoxx4Kids Denver

2014 was the inaugural year for Devoxx4Kids Denver. In April, I announced our first meetup. It was an Introduction to Server-Side Minecraft Programming, taught by Denver's own Scott Davis. It was a wild success and the kids had a great time hacking.

I didn't plan any events during the summer, but we held a couple workshops in the fall: one on littleBits and a second on Greenfoot. The kids had a lot of fun with littleBits thanks to Juan Sanchez of Tack Mobile.

Unfortunately, I wasn't able to attend the Greenfoot workshop with Steve Pousty, but I heard it was awesome. Thanks again to Dan Allen for assisting with this class.

I haven't scheduled any Devoxx4Kids Denver meetups for 2015 yet. If you're interested in speaking, please let me know!

Projects

AppFuse: I didn't have many posts about AppFuse in 2014, but I did work on it from time to time. Upgrading its dependencies was a good way to stay up-to-date with open source frameworks, in addition to lots of reading. InfoQ covered the AppFuse 3.0 release and I continue to write for them.

In December, I removed 10,000 lines of XML from the project without affecting any functionality. In the past month, I've added support for generating CRUD for all its web frameworks (including AppFuse Light's). This includes Spring + FreeMarker, Stripes and Wicket. GWT support is still in progress. I hope to do a release soon; there's only 10 issues left as of this writing.

Roller: I didn't contribute any code to Apache Roller in 2014, but I upgraded this site to the latest release, upgraded to Java 8 + Tomcat 7, and integrated Wufoo Forms. I hope to make some scalability improvements to how I've deployed Roller in the near future.

Personal

I started off the year with a trip of a lifetime: Heli Skiing in British Columbia with CMH Gothics. An excerpt from my blog post:

Everyday was awesome, but Wednesday was incredible. It snowed 10 cm on Tuesday night, which turned into knee-deep powder and face shots. The hoots and hollers from everyone floating down forced my face into a sorta perma-grin. For many of us, it was the best skiing day of our entire lives. The video below tries to capture how much fun we had throughout the week, but ultimately fails. It's one of those things you have to experience to really appreciate. I ended up skiing around 80,000 vertical feet in four days.

After returning from British Columbia, Trish and I skied Breckenridge and Copper before taking the kids to Aspen for three feet of powder on Super Bowl weekend.

We skied all over Colorado in February and March. We even did some winter camping in the Syncro for Valentine's Day. It snowed a lot that night.

I wrote about all our skiing adventures in Farewell to the 2013-2014 Ski Season.

On April 10th, I reminisced that 10 years ago today, I bought a VW Bus. In that post, I explained some body-work quality issues the current restoration shop found. Nevertheless, I still thought it would be done in just a few more months.

We journeyed to Florida with Abbie and Jack for Spring Break. We relished in a super-fun family vacation where we golfed, swam and even spent a day frolicking with their Mom and fiancé (now husband), Dave. We're pretty proud that we're all still a family even though Julie and I are divorced. I wrote about our vacation in Spring Break in Florida: Golf, Beaches and Boats!

The hospitality from Trish's parents was like no other. I especially enjoyed a fishing trip with her Dad Joe in the Everglades. I love these people!

Our winter activities in the Syncro were incredible. Trish sold her Nissan Xterra (the one with the sick bass) in December. Yes, that's right - we only had one vehicle during ski season last year. The fact that it worked so well is amazing. In May, we kicked it up a knotch and drove it to Moab for Syncro Solstice. My parents went with us and we experienced fabulous weather and met a bunch of super-cool Vanagon owners.

We made record time from Moab back to Denver (5 hours). The next day is when things started to go downhill. I washed the engine for the VWs on the Green show. On that Monday, the engine started "lurching" where it seemed like it wasn't getting gas (it turned out to be caused by a lack of air). We drove the lurcher for several weeks, journeying to Campout for the Cause and a exhilarating raft trip down the Yampa River.

We had the lurching examined in mid-June by Rocky Mountain Westy; we also had them install some bumpers (for a better trailer hitch) and refreshed the tires.

The ol' van looked pretty good when our Dads arrived in Denver for some whitewater on the Arkansas.

In July, we skipped the annual road trip to The Cabin in favor of a romantic road trip to see old friends in Santa Fe. The Syncro's engine seized on that trip and I experienced my favorite breakdown ever. The engine froze at a gas station near our Ski Shack and I was able to walk home. Luckily, Trish's Dad had recently (over Father's Day weekend) gifted his Nissan Maxima to us. After a train ride to fetch our other car, our road trip continued and we celebrated our first anniversary at The Broadmoor.

After we returned from Santa Fe, I turned 40 years old. We rocked out a Journey concert to celebrate and Trish surprised me with a birthday visit from my kick-ass sister. The following evening, a bunch of friends took me on a Craft Brewery Tour in a school/prison bus. Both nights were epic.

At the end of July, Trish had one of her favorite photos featured on the cover of Whisky Magazine.

In August, we flew to Boston to celebrate Trish's Dad's 75th birthday. My parents joined us, we visited my Dad's family in New York and took the kids to the Big Apple. They loved the Statue of Liberty.

Abbie started middle school (6th grade) in mid-August and Jack started 4th grade. They looked great on their first day of school.

We were pumped about our new engine in the Syncro when we left Denver for Rafting the Green River through Desolation Canyon. We celebrated Jack's 10th birthday on that trip and had a very memorable expedition with some really cool people.

Trish's photo from our last night on the river makes me want do it all again.

In October, I started considering health over wealth when we embarked on a 21-day Sugar Detox. I'm proud to say that, as a result, I'm off blood pressure medication and continue to lose weight and feel better. We're cooking at home more than ever, exercising daily and I've lost almost 20 pounds in the process.

Medallion, the horse that Trish and Abbie were leasing (and almost ready to buy) got sick with colic on Friday, October 10. I remembered the day without looking at a calendar because Trish called me to tell me about his illness. She was out of town on a business trip and was absolutely distraught. She told me about the situation and asked me to visit Medallion while he endured extreme pain. I worked out of my van at the stables that day, checking on him every couple hours. They had to put him down that evening. Abbie took it pretty well when I told her a couple hours later, but there were heaps of tears later that night.

Our little girl turned 12 a few weeks later. The kids put on some dark costumes for Halloween.

A week later, I visited my parents in Montana. We played cribbage, ate scrumptious meals and trounced around the woods looking for bucks. I marveled at the beauty of my parents' retirement home.

I neglected to blog or tweet about my self-caused Syncro accidents in Q4 of 2014. Long story short: it's been getting body work done for quite some time (two accidents!) and I look forward to getting it back in tip-top-shape next week. Thank goodness for USAA insurance.

The biggest event of the year happened just before Christmas: THE BUS IS PAINTED!! HOLY CHRISTMAS PRESENT BATMAN!

2015

It's hard to believe that 2015 could possibly top 2014. I started the year heli-skiing and ended it with a painted bus. However, the bus isn't done yet. Last year, I wrote:

My goal for 2014 is singular: finish The Bus.

I continue that mission this year. I've negotiated a deal with ReinCARnation to have it done by April 1st. Yes, it seems strange that April Fools is the finished bus delivery date. However, I'm confident they'll make it happen. This could be the year I've dreamed about since I was in high school. It was in 1991 that I first fantasized about putting a 911 engine in a VW Bus.

This year, I hope to be cruise around Colorado in two running VW Busses. From there, I plan to surround myself with the laughter of our children, the enthusiasm of our border collies and the warmness of Trish's love. It's gonna be a really great year!

Integrating Node.js, Ruby and Spring with Okta's SAML Support

2015-01-08T11:43:47-07:00

Security has always piqued my interest, ever since I first developed AppFuse and figured out how to make J2EE security work back in 2004. I hacked AppFuse to have Remember Me functionality, then moved onto Acegi/Spring Security. Spring Security had the features I needed, even if it did require almost 100 lines of XML to configure it. These days, it's much better and its JavaConfig - combined with Spring Boot - is pretty slick.

That was the first part of my security life. The second phase began the night I met Trish, and learned she sold security products. She knew of OWASP and their top 10 rules. It was Trish that inspired me to write my Java Web Application Security presentation. I really enjoyed writing that presentation, comparing Apache Shiro, Spring Security and Java EE's security frameworks. I followed up the first time I presented it with a number of blog posts and screencasts. Hmmmm, maybe I should update the presentation/screencasts to use Java configuration only (#NoXML) and submit it to a couple conferences this year? I digress.

I had to do a security-related spike over the last couple weeks. I was trying to get SAML authentication working with Okta and my client's Active Directory server. Luckily, someone setup the AD integration so all I had to do was try a few different languages/frameworks. I searched and found ThoughtWorks' okta-samples, which includes examples using Node.js and Sinatra (Ruby + JRuby). I also found a Spring SAML example that includes one of my favorite things in JavaLand: Java-based configuration.

I'm happy to report I was able to get all of these applications working with my client's Okta setup. This article will tell you how I did it. For each application, I created a new application on Okta using its "Template SAML 2.0 Application" and added myself in the application's "People" tab. Each section below contains the configuration I used for Okta. The instructions below assume you're similar to me, a developer that has Java 8, Node and Ruby installed, but none of the specific frameworks. As I write this, I have everything working on my Mac with Yosemite, but I wrote the instructions below using one of my old laptops, fresh after a Yosemite upgrade.

The first thing I did was checkout ThoughtWorks samples.

git clone https://github.com/ThoughtWorksInc/okta-samples.git

Node.js

I started by getting the Node.js sample working. For Okta's configuration, I used:

Setting	Value
Application label	Okta Node.js Example
Force Authentication	false
Post Back URL	http://localhost:3000/login/callback
Name ID Format	EmailAddress
Recipient	http://localhost:3000/
Audience Restriction	http://localhost:3000/
authnContextClassRef	PasswordProtectedTransport
Response	Signed
Assertion	Signed
Request	Compressed
Destination	http://localhost:3000/login/callback
Attribute Statements	email\|${user.email},firstName\|${user.firstName}

The Node.js sample uses express, as well as passport and passport-saml. The passport packages are used to handle the SAML authentication and connect is used to compress the requests from your local server.

The only thing I needed to do to make the Node.js app work was to paste the X509 cert string and target URL into its config.json from the Okta app. In Okta's Admin interface, I clicked on the "Sign On" tab and clicked its "View Setup Instructions" button. I copied the "Redirect Login URL" value and copied it into config.json's entryPoint value. I then downloaded the certificate and opened it in vi. I ran the following two commands to remove ^M and line endings (more details here).

:%s/<Ctrl-V><Ctrl-M>//g
:%s/\n//g

Next, I copied everything between -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- and pasted it into the cert value of config.json. I had to remove the comments from config.json for everything to work.

After making these changes, I was able to run "npm install" and "npm start" and successfully login at http://localhost:3000.

Ruby

The Ruby sample uses Sinatra, omniauth and omniauth-saml. To run the okta-ruby-sinatra application, I had to start by installing Bundler.

sudo gem install bundler

Then I installed all the required gems for this project using the following command.

bundle install

This resulted in the following error:

An error occurred while installing nokogiri (1.6.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.1'` succeeds before bundling

I tried Bundler's suggestion, but it failed:

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb
mkmf.rb can't find header files for ruby at /System/Library/Frameworks/Ruby.
framework/Versions/2.0/usr/lib/ruby/include/ruby.h

I then tried upgrading to Xcode 6.1.1. I received the same error and running "bundle update sinatra" and "sudo gem update --system" didn't help anything. I found an old Stack Overflow answer that suggested running "xcode-select --install" to install Xcode's Command Line Developer Tools. After doing so, I ran "sudo gcc" to accept to all Apple's licensing agreements. I ran "bundle install" again and this time it failed with the following error:

-----
libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
-----
...
An error occurred while installing nokogiri (1.6.1), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.6.1'` succeeds before bundling.

I tried Bundler's suggested again: "sudo gem install nokogiri -v '1.6.1'". This didn't work, so I tried "bundle update" and it finally worked. I ran "bundle install" for the final time, followed by "ruby app.rb". WEBrick started and I created a "Okta Ruby Example" application on Okta with the following settings.

Setting	Value
Application label	Okta Ruby Example
Force Authentication	false
Post Back URL	http://localhost:4567/auth/saml/callback
Name ID Format	EmailAddress
Recipient	http://localhost:4567
Audience Restriction	http://localhost:4567
authnContextClassRef	PasswordProtectedTransport
Response	Signed
Assertion	Signed
Request	Compressed
Destination	http://localhost:4567/auth/saml/callback
Attribute Statements	email\|${user.email},firstName\|${user.firstName}

To configure Sinatra with Otka's settings, I started by renaming the config.yml.sample file:

mv config.yml.sample config.yml

In Otka's Admin UI for the application, I clicked on the "Sign On" tab and clicked its "View Setup Instructions" button. I copied the "Redirect Login URL" value and copied it into config.yml's target_url value. I then downloaded the certificate and ran the the following command in the directory I downloaded it to.

openssl x509 -noout -fingerprint -in "okta.cert"

I copied the fingerprint into config.yml's fingerprint value and restarted the app. I opened http://localhost:4567 in my browser and was able to successfully login.

JRuby

To start with JRuby, I first read the project's README. It mentioned issues with "nokogiri" and explains the project contains a patched release of nokogiri 1.6.0. Since I knew there was a later release, I modified Gemfile and removed the version and path information from the last line. I copied the config.yml from the Ruby project and ran the following commands to install Bundler, the project's dependencies and start the app.

jruby -S gem install bundler
jruby -S bundle install

Running the second command resulted in the following error:

Your jruby version is 1.7.18, but your Gemfile specified jruby 1.7.4

I modified Gemfile to specify "1.7.18" and tried again. This time it worked. I started the application using the following command:

jruby app.rb

NOTE: If you see the the following in your browser window, it means you forgot to copy config.yml from the Ruby project.

undefined method `auth' for Sinatra::Application:Class

When I tried to login at http://localhost:4567, I saw an infinite redirect and the following error in my console.

W, [2015-01-08T08:53:22.514000 #56144]  WARN -- : attack prevented by Rack::Protection::SessionHijacking
0:0:0:0:0:0:0:1 - - [08/Jan/2015 08:53:22] "GET / HTTP/1.1" 302 - 0.0190
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:53:22 MST] "GET / HTTP/1.1" 302 0

Stack Overflow indicated this is a problem caused by an old version of rack-protection. Running "jruby -S bundle update rack-protection" updated the project to use rack-protection 1.5.3 (was 1.5.1). After restarting and trying again, I received the following error:

I, [2015-01-08T08:59:32.679000 #56176]  INFO -- omniauth: (saml) Callback phase initiated.
E, [2015-01-08T08:59:34.747000 #56176] ERROR -- omniauth: (saml) Authentication failure! invalid_ticket: Onelogin::Saml::ValidationError, Digest mismatch
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:34 -0700] "POST /auth/saml/callback HTTP/1.1" 302 9 2.0760
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:34 -0700] "GET /auth/failure?message=invalid_ticket&strategy=saml HTTP/1.1" 404 449 0.0080
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:34 MST] "GET /auth/failure?message=invalid_ticket&strategy=saml HTTP/1.1" 404 449
- -> /auth/failure?message=invalid_ticket&strategy=saml
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:34 -0700] "GET /__sinatra__/404.png HTTP/1.1" 200 18893 0.0200
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:32 MST] "POST /auth/saml/callback HTTP/1.1" 302 9
- -> /auth/saml/callback
0:0:0:0:0:0:0:1 - - [08/Jan/2015:08:59:34 MST] "GET /__sinatra__/404.png HTTP/1.1" 200 18893
http://localhost:4567/auth/failure?message=invalid_ticket&strategy=saml -> /__sinatra__/404.png

At this point, the only thing different from my working version and my old laptop was the version of Java. My old laptop had "build 1.8.0_05-b13", so I upgraded to the latest version of Java 8 (update 25). This didn't help, so I tried updating all bundles with "jruby -S bundle update". This failed too, so I figured I'd try to use the version of JRuby that was on my working laptop (version 1.7.16.1). I installed Homebrew, ran "brew install jruby", removed the newer version from my path and downgraded the version in Gemfile. I had to re-install Bundler and the projects dependencies with the following commands.

jruby -S gem install bundler
jruby -S bundle install

Same error again. I reverted Gemfile.lock and ran the only bundle update command I'd run on my working laptop:

$ jruby -S bundle update sinatra

Unfortunately, this still didn't fix the issue. I copied the project from my working laptop and tried running that project. It failed, proving that it was an environment issue, not a project/code issue. I tried rebooting and when that didn't work, I gave up. It's pretty strange this didn't work on a fresh Yosemite install - it took me less than 10 minutes to get it working originally.

Spring

The Spring sample I got working with Okta was Vincenzo De Notaris' spring-boot-security-saml-sample. This project uses Spring Boot and Spring Security SAML. I created a "Okta Spring Example" application on Okta with the following settings.

Setting	Value
Application label	Okta Spring Example
Force Authentication	false
Post Back URL	http://localhost:8080/saml/SSO
Name ID Format	EmailAddress
Recipient	http://localhost:8080/saml/SSO
Audience Restriction	com:vdenotaris:spring:sp
authnContextClassRef	PasswordProtectedTransport
Response	Signed
Assertion	Signed
Request	Uncompressed
Destination	http://localhost:8080/saml/SSO
Attribute Statements	email\|${user.email},firstName\|${user.firstName}

The biggest thing I learned while trying to get these values correct was that Request needs to be set to Uncompressed.

After cloning the GitHub project to my hard drive, I added a new SSO provider by adding a new bean to WebSecurityConfig.java. The URL I got from Okta's Admin UI: Sign On > View Setup Instructions > Public Link (near the bottom of the page).

@Bean(name = "idp-okta")
public ExtendedMetadataDelegate ssoOktaExtendedMetadataProvider()
      throws MetadataProviderException {
    @SuppressWarnings({ "deprecation"})
    HTTPMetadataProvider httpMetadataProvider
          = new HTTPMetadataProvider("https://client.okta.com/app/random-key-here/sso/saml/metadata", 5000);
    httpMetadataProvider.setParserPool(parserPool());
    ExtendedMetadataDelegate extendedMetadataDelegate =
          new ExtendedMetadataDelegate(httpMetadataProvider, extendedMetadata());
    extendedMetadataDelegate.setMetadataTrustCheck(false);
    extendedMetadataDelegate.setMetadataRequireSignature(false);
    return extendedMetadataDelegate;
}

For the SSL connection to work, I had to download the certificate and import it into the application's keystore. To do this in Chrome, I went to https://client.okta.com, clicked on the lock icon in the address bar, then dragged/dropped the certificate image to my desktop. This resulted in a *.okta.com.cer file on my desktop. I added it to the keystore using the following commands (thanks Stack Overflow).

keytool -importcert -file ~/Desktop/\*.okta.com.cer -keystore src/main/resources/saml/samlKeystore.jks

When prompted for the password, I entered "nalle123". This value is specified in WebSecurityConfig.java's keyManager bean. I then added this provider to the list of providers in the metadata bean.

@Bean
@Qualifier("metadata")
public CachingMetadataManager metadata() throws MetadataProviderException {
        List<MetadataProvider> providers = new ArrayList<MetadataProvider>();
    providers.add(ssoOktaExtendedMetadataProvider());
    providers.add(ssoCircleExtendedMetadataProvider());
    return new CachingMetadataManager(providers);
}

After making these changes, I started the application using "mvn spring-boot:run". I navigated to http://localhost:8080, chose Okta as my Idp and logged in successfully!

Summary

This article shows you how I got Node.js, Ruby and Spring applications working with Okta's SAML support. My experience with this when I first tried it: Node was super-easy, Ruby was a bit more difficult, JRuby was a cinch and Spring took several days. As you can tell from this article, Ruby/JRuby were the most difficult to make work on a clean machine.

All in all, working with Okta has been a pleasant experience so far. Hopefully this article helps make it a good experience for you as well.

AppFuse, Reduced

2014-12-16T06:03:31-07:00

In November, I had some time off between clients. To occupy my time, I exercised my body and brain a bit. I spent a couple hours a day exercising and a few hours a day working on AppFuse. AppFuse isn't used to start projects nearly as much as it once was. This makes sense since there's been a ton of innovation on the JVM and there's lots of get-started-quickly frameworks now. Among my favorites are Spring Boot, JHipster, Grails and Play.

You can see that AppFuse's community activity has decreased quite a bit over the years by looking at its mailing list traffic.

Even though there's not a lot of users talking on the mailing list, it still seems to get quite a few downloads from Maven Central.

I think the biggest value that AppFuse provides now is a learning tool for those who work on it. Also, it's a good place to show other developers how they can evolve with open source frameworks (e.g. Spring, Hibernate, JSF, Tapestry, Struts) over several years. Showing how we migrated to Spring MVC Test, for example, might be useful. The upcoming move to Spring Data instead of our Generic DAO solution might be interesting as well.

Regardless of whether AppFuse is used a lot or not, it should be easy to maintain. Over the several weeks, I made some opinionated changes and achieved some pretty good progress on simplifying things and making the project easier to maintain. The previous structure has a lot of duplicate versions, properties and plugin configurations between different projects. I was able to leverage Maven's inheritance model to make a number of improvements:

Changed AppFuse's parent to be based on the Spring IO Platform. This project is a dependency manager that defines version numbers for open source projects that work well with Spring.
Defined plugins, their versions and configurations in <pluginManagement>.
Defined dependencies, their versions and exclusions in <dependencyManagement>.
Simplified archetypes so new projects have minimal dependencies. For example, here's a basic project's pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.company</groupId>
    <artifactId>springmvc-project</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>AppFuse Spring MVC Application</name>

    <parent>
        <groupId>org.appfuse</groupId>
        <artifactId>appfuse-web</artifactId>
        <version>3.5.0-SNAPSHOT</version>
    </parent>

    <build>
        <plugins>
            <plugin>
                <groupId>de.juplo</groupId>
                <artifactId>hibernate4-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>dbunit-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.appfuse</groupId>
            <artifactId>appfuse-${web.framework}</artifactId>
            <version>${appfuse.version}</version>
            <type>pom</type>
        </dependency>
    </dependencies>

    <properties>
        <amp.genericCore>true</amp.genericCore>
        <amp.fullSource>false</amp.fullSource>
        <dao.framework>hibernate</dao.framework>
        <db.name>mydatabase</db.name>
        <web.framework>spring</web.framework>

        <!-- Framework/Plugin versions -->
        <appfuse.version>3.5.0-SNAPSHOT</appfuse.version>
        <java.version>1.7</java.version>
    </properties>

    <profiles>
        <profile>
            <id>itest</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.cargo</groupId>
                        <artifactId>cargo-maven2-plugin</artifactId>
                    </plugin>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>webtest-maven-plugin</artifactId>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <reporting>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>webtest-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </reporting>
</project>

The pull request for these changes says it all:

AppFuse +4,822 -14,369
AppFuse Light +776 -4,687

That's right, I was able to eliminate a good chunk of code without affecting any of AppFuse's functionality¹. I think we can all agree that less code == easier maintenance. This theme will continue as we work on future releases.

Other improvements include migrating all tests to use JUnit4, integrating Spring MVC Test, and configuring the surefire plugin to run tests in parallel. I also The build-helper-maven-plugin is now used to find open ports for Cargo to run and a lot of testing was done to ensure you can build/test multiple AppFuse-derived projects at the same time. Finally, I migrated to the hibernate4-maven-plugin and upgraded to Tapestry 5.4.

In the next version of AppFuse, I plan to remove as much XML as possible - moving all of the configuration to Spring's JavaConfig. We'll also be moving to Java 8 as a minimum. I'm even considering getting rid of all the pom.xml files in favor of another build language that requires less code.

In other words, the upcoming 3.5 release will be the last release that supports Java 7 and uses Spring's XML for configuration. AppFuse 4.0 will strive for #NoXML. The project's roadmap has more details on additional hopes and dreams.

We'd love to hear your feedback on these change. Do you like the simplification theme? Are you OK with having AppFuse as a parent in your projects?

1. For project and code stats, see AppFuse on Open Hub.

Developing Services with Apache Camel - Part IV: Load Testing and Monitoring

2014-10-15T10:04:01-06:00

Welcome to the final article in a series on my experience developing services with Apache Camel. I learned how to implement CXF endpoints using its Java DSL, made sure everything worked with its testing framework and integrated Spring Boot for external configuration. For previous articles, please see the following:

This article focuses on load testing and tools for monitoring application performance. In late July, I was asked to look into load testing the new Camel-based services I'd developed. My client's reason was simple: to make sure the new services were as fast as the old ones (powered by IBM Message Broker). I sent an email to the Camel users mailing list asking for advice on load testing.

I'm getting ready to put a Camel / CXF / Spring Boot application into production. Before I do, I want to load test and verify it has the same throughput as a the IBM Message Broker system it's replacing. Apparently, the old system can only do 6 concurrent connections because of remote database connectivity issues.

I'd like to write some tests that make simultaneous requests, with different data. Ideally, I could write them to point at the old system and find out when it falls over. Then I could point them at the new system and tune it accordingly. If I need to throttle because of remote connectivity issues, I'd like to know before we go to production. Does JMeter or any Camel-related testing tools allow for this?

In reply, I received suggestions for Apache's ab tool and Gatling. I'd heard of Gatling before, and decided to try it.

Gatling

I don't remember where I first heard of Gatling, but I knew it had a Scala DSL and used Akka under the covers. I generated a new project using a Maven archetype and went to work developing my first test. My approach involved three steps:

Write tests to run against current system. Find the number of concurrent requests that make it fall over.
Run tests against new system and tune accordingly.
Throttle requests if there are remote connectivity issues with 3rd parties. If I needed to throttle requests, I was planning to use Camel's Throttler.

To develop the first test, I started with Gatling's Recorder. I set it to listen on port 8000, changed my DrugServiceITest to use the same port and ran the integration test. This was a great way to get started because it recorded my requests as XML files, and used clean and concise code.

I ended up creating a parent class for all simulations and named it AbstractSimulation. This was handy because it allowed me to pass in parameters for all the values I wanted to change.

import io.gatling.core.scenario.Simulation
import io.gatling.http.Predef._

/**
 * Base Simulation class that allows passing in parameters.
 */
class AbstractSimulation extends Simulation {

  val host = System.getProperty("host", "localhost:8080")
  val serviceType = System.getProperty("service", "modern")
  val nbUsers = Integer.getInteger("users", 10).toInt
  val rampRate = java.lang.Long.getLong("ramp", 30L).toLong

  val httpProtocol = http
    .baseURL("http://" + host)
    .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
    .doNotTrackHeader("1")
    .acceptLanguageHeader("en-US,en;q=0.5")
    .acceptEncodingHeader("gzip, deflate")
    .userAgentHeader("Gatling 2.0")

  val headers = Map(
    """Cache-Control""" -> """no-cache""",
    """Content-Type""" -> """application/soap+xml; charset=UTF-8""",
    """Pragma""" -> """no-cache""")
}

The DrugServiceSimulation.scala class posts a SOAP request over HTTP.

import io.gatling.core.Predef._
import io.gatling.http.Predef._

import scala.concurrent.duration._

class DrugServiceSimulation extends AbstractSimulation {

  val service = if ("modern".equals(serviceType)) "/api/drugs" else "/axis2/services/DrugService"

  val scn = scenario("Drug Service :: findGpiByNdc")
    .exec(http(host)
    .post(service)
    .headers(headers)
    .body(RawFileBody("DrugServiceSimulation_request.xml")))

  setUp(scn.inject(ramp(nbUsers users) over (rampRate seconds))).protocols(httpProtocol)
}

To run tests against the legacy drug service with 100 users over 60 seconds, I used the following command:

mvn test -Dhost=legacy.server:7802 -Dservice=legacy -Dusers=100 -Dramp=60

The service property's default is "modern" and determines the service's URL. To run against the local drug service with 100 users over 30 seconds, I could rely on more defaults.

mvn test -Dusers=100

The name of the simulation to run is configured in pom.xml:

<plugin>
    <groupId>io.gatling</groupId>
    <artifactId>gatling-maven-plugin</artifactId>
    <version>${gatling.version}</version>
    <configuration>
        <simulationsFolder>src/test/scala</simulationsFolder>
        <simulationClass>com.company.app.${service.name}Simulation</simulationClass>
    </configuration>
    <executions>
        <execution>
            <phase>test</phase>
            <goals>
                <goal>execute</goal>
            </goals>
        </execution>
    </executions>
</plugin>

When the simulations were done running, the console displayed a link to some pretty snazzy HTML reports. I ran simulations until things started falling over on the legacy server. That happened at around 400 requests per second (rps). When I ran them against a local instance on my fully-loaded 2013 MacBook Pro, errors started flying at 4000/rps while 3000/rps performed just fine.

Jenkins

I configured simulations to run in Jenkins with the Gatling Plugin. It's a neat plugin that allows you to record and compare results over time. After initial setup, I found I didn't use it much. Instead, I created a Google Doc with my findings and created screenshots of results so my client had it in an easy-to-read format.

Data Feeders

I knew the results of the simulations were likely skewed, since the same request was used for all users. I researched how to make dynamic requests with Gatling and found Feeders. Using a JDBC Feeder I was able make all the requests contain unique data for each user.

I added a feeder to DrugServiceSimulation, added it to the scenario and changed to use an ELFileBody so the feeder would substitute a ${NDC} variable in the XML file.

val feeder = jdbcFeeder("jdbc:db2://server:50002/database", "username", "password",
    "SELECT NDC FROM GENERICS")

val scn = scenario("Drug Service")
        .feed(feeder)
        .exec(http(host)
        .post(service)
        .headers(headers)
        .body(ELFileBody("DrugServiceSimulation_request.xml")))

I deployed the new services to a test server and ran simulations with 100 and 1000 users.

100 users over 30 seconds

Neither service had any failures with 100 users. The max response time for the legacy service was 389 ms, while the new service was 172 ms. The mean response time was lower for the legacy services: 89 ms vs. 96 ms.

1000 users over 60 seconds

When simulating 1000 users against the legacy services, 50% of the requests failed and the average response time was over 40 seconds. Against the new services, all requests succeeded and the mean response time was 100ms.

I was pumped to see the new services didn't need any additional performance enhancements. These results were enough to convince my client that Apache Camel was going to be a performant replacement for IBM Message Broker.

I wrote more simulations for another service I developed. In doing so, I discovered I missed implementing a couple custom routes for some clients. The dynamic feeders made me stumble onto this because they executed simulations for all clients. After developing the routes, the dynamic data helped me uncover a few more bugs. Using real data to load test with was very helpful in figuring out the edge-cases our routes needed to handle.

Next, I started configuring logging for our new Camel services.

Logging with Log4j2

Log4j 2.0 had just been released and my experience integrating it in AppFuse motivated me to use it for this project. I configured Spring to use Log4j 2.0 by specifying the following dependencies. Note: Spring Boot 1.2+ has support for Log4j2.

<log4j.version>2.0</log4j.version>
...

<!-- logging -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.7.7</version>
</dependency>
<!-- Necessary to configure Spring logging with log4j2.xml -->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-jcl</artifactId>
    <version>${log4j.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
    <version>${log4j.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-web</artifactId>
    <version>${log4j.version}</version>
</dependency>

I created a src/main/resources/log4j2.xml file and configured a general log, as well as one for each route. I configured each route to use "log:com.company.app.route.input" and "log:com.company.app.route.output" instead of "log:input" and "log:output". This allowed the log-file-per-route configuration you see below.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Properties>
        <Property name="fileLogDir">/var/log/app-name</Property>
        <Property name="fileLogPattern">%d %p %c: %m%n</Property>
        <Property name="fileLogTriggerSize">1 MB</Property>
        <Property name="fileLogRolloverMax">10</Property>
    </Properties>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d [%-15.15t] %-5p %-30.30c{1} %m%n"/>
        </Console>
        <RollingFile name="File" fileName="${fileLogDir}/all.log"
                         filePattern="${fileLogDir}/all-%d{yyyy-MM-dd}-%i.log">
            <PatternLayout pattern="${fileLogPattern}"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="${fileLogTriggerSize}"/>
            </Policies>
            <DefaultRolloverStrategy max="${fileLogRolloverMax}"/>
        </RollingFile>
        <RollingFile name="DrugServiceFile" fileName="${fileLogDir}/drug-service.log"
                         filePattern="${fileLogDir}/drug-service-%d{yyyy-MM-dd}-%i.log">
            <PatternLayout pattern="${fileLogPattern}"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="${fileLogTriggerSize}"/>
            </Policies>
            <DefaultRolloverStrategy max="${fileLogRolloverMax}"/>
        </RollingFile>
        <!-- Add a RollingFile for each route -->
    </Appenders>

    <Loggers>
        <Logger name="org.apache.camel" level="info"/>
        <Logger name="org.springframework" level="error"/>
        <Logger name="com.company.app" level="info"/>

        <Root level="error">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="File"/>
        </Root>

        <Logger name="com.company.app.drugs" level="debug">
            <AppenderRef ref="DrugServiceFile"/>
        </Logger>
        <!-- Add a Logger for each route -->
    </Loggers>
</Configuration>

I did run into some issues with this configuration:

The /var/log/app-name directory has to exist or there's a stacktrace on startup and no logs are written.
When deploy from Jenkins, I ran into permissions issues between deploys. To fix this, I chowned the directory before restarting Tomcat.
```
chown -R tomcat /var/log/app-name
/etc/init.d/tomcat start
```

Monitoring

While I was configuring the new services on our test server, I also installed hawtio at /console. I had previously configured it to run in Tomcat when running "mvn tomcat7:run":

<plugin>
    <groupId>org.apache.tomcat.maven</groupId>
    <artifactId>tomcat7-maven-plugin</artifactId>
    <version>2.2</version>
    <configuration>
        <path>/</path>
        <webapps>
            <webapp>
                <contextPath>/console</contextPath>
                <groupId>io.hawt</groupId>
                <artifactId>hawtio-web</artifactId>
                <version>1.4.19</version>
                <type>war</type>
                <asWebapp>true</asWebapp>
            </webapp>
        </webapps>
    </configuration>
    ...
</plugin>

hawtio has a Camel plugin that's pretty slick. It shows all your routes and their runtime metrics; you can even edit the source code for routes. Even though I used a Java DSL, my routes are only editable as XML in hawtio. Claus Ibsen has a good post on Camel's new Metrics Component. I'd like to learn how to build a custom dashboard for hawtio - Claus's example looks pretty nice.

The Spring Boot plugin for hawtio is not nearly as graphic intensive. Instead, it just displays metrics and their values in a table format.

There's some good-looking Spring Boot Admin UIs out there, notably JHipster's and the one in spring-boot-admin. I hope the hawtio Spring Boot plugin gets prettier as it matures.

I wanted more than just monitoring, I wanted alerts when something went wrong. For that, I installed New Relic on our Tomcat server. I'm fond of getting the Monday reports, but they only showed activity when I was load testing.

I believe all these monitoring tools will be very useful once the app is in production. My last day with this client is next Friday, October 24. I'm trying to finish up the last couple of services this week and next. With any luck, their IBM Message Broker will be replaced this year.

Summary

This article shows how to use Gatling to load test a SOAP service and how to configure Log4j2 with Spring Boot. It also shows how hawtio can help monitor and configure a Camel application. I hope you enjoyed reading this series on what I learned about developing with Camel over the past several months. If you have stories about your experience with Camel (or similar integration frameworks), Gatling, hawtio or New Relic, I'd love to hear them.

It's been a great experience and I look forward to developing solid apps, built on open source, for my next client. I'd like to get back into HTML5, AngularJS and mobile development. I've had a good time with Spring Boot and JHipster this year and hope to use them again. I find myself using Java 8 more and more; my ideal next project would embrace it as a baseline. As for Scala and Groovy, I'm still a big fan and believe I can develop great apps with them.

If you're looking for a UI/API Architect that can help accelerate your projects, please let me know! You can learn more about my extensive experience from my LinkedIn profile.

Getting Started with JHipster on OS X

2014-09-08T11:30:33-06:00

Last week I was tasked with developing a quick prototype that used AngularJS for its client and Spring MVC for its server. A colleague developed the same application using Backbone.js and Spring MVC. At first, I considered using my boot-ionic project as a starting point. Then I realized I didn't need to develop a native mobile app, but rather a responsive web app.

My colleague mentioned he was going to use RESThub as his starting point, so I figured I'd use JHipster as mine. We allocated a day to get our environments setup with the tools we needed, then timeboxed our first feature spike to four hours.

My first experience with JHipster failed the 10-minute test. I spent a lot of time flailing about with various "npm" and "yo" commands, getting permissions issues along the way. After getting thinks to work with some sudo action, I figured I'd try its Docker development environment. This experience was no better.

JHipster seems like a nice project, so I figured I'd try to find the causes of my issues. This article is designed to save you the pain I had. If you'd rather just see the steps to get up and running quickly, skip to the summary.

The "npm" and "yo" issues I had seemed to be caused by a bad node/npm installation. To fix this, I removed node and installed nvm. Here's the commands I needed to remove node and npm:

sudo rm -rf /usr/local/lib/node_modules
sudo rm -rf /usr/local/include/node
sudo rm /usr/local/bin/node

sudo rm -rf /usr/local/bin/npm
sudo rm /usr/local/share/man/man1/node.1
sudo rm -rf /usr/local/lib/dtrace/node.d
sudo rm -rf ~/.npm

Next, I ran "brew doctor" to make sure Homebrew was still happy. It told me some things were broken:

$ brew doctor
Warning: Broken symlinks were found. Remove them with `brew prune`:
  /usr/local/bin/yo
  /usr/local/bin/ionic
  /usr/local/bin/grunt
  /usr/local/bin/bower

I ran brew update && brew prune, followed by brew install nvm. Next, I added the following to my ~/.profile:

source $(brew --prefix nvm)/nvm.sh

To install the latest version of node, I ran the commands below and set the latest version as the default:

nvm ls-remote
nvm install v0.11.13
nvm alias default v0.11.13

Once I had a fresh version of Node.js, I was able to run JHipster's local installation instructions.

npm install -g yo
npm install -g generator-jhipster

Then I created my project:

yo jhipster

I was disappointed to find this created all the project files in my current directory, rather than in a subdirectory. I'd recommend you do the following instead:

mkdir ~/projectname && cd ~/projectname && yo jhipster

Before creating your project, JHipster asks you a number of questions. To see what they are, see its documentation on creating an application. Two things to be aware of:

Hot reloading Java code doesn't work well (yet) with Java 8
Its OAuth2 implementation doesn't work with WebSockets

In other words, I'd recommend using Java 7 + (cookie-based authentication with websockets) or (oauth2 authentication w/o websockets).

After creating my project, I was able to run it using "mvn spring-boot:run" and view it at http://localhost:8080. To get hot-reloading for the client, I ran "grunt server" and opened my browser to http://localhost:9000.

JHipster + Docker on OS X

I had no luck getting the Docker instructions to work initially. I spent a couple hours on it, then gave up. A couple of days ago, I decided to give it another good ol' college-try. To make sure I figured out everything from scratch, I started by removing Docker.

I re-installed Docker and pulled the JHipster image using the following:

sudo docker pull jdubois/jhipster-docker

The error I got from this was the following:

2014/09/05 19:43:38 Post http:///var/run/docker.sock/images/create?fromImage=jdubois%2Fjhipster-docker&tag=:
dial unix /var/run/docker.sock: no such file or directory

After doing some research, I learned I needed to run boot2docker init first. Next I ran boot2docker up to start the Docker daemon. Then I copied/pasted "export DOCKER_HOST=tcp://192.168.59.103:2375" into my console and tried to run docker pull again.

It failed with the same error. The solution was simpler than you might think: don't use sudo.

$ docker pull jdubois/jhipster-docker
Pulling repository jdubois/jhipster-docker
01bdc74025db: Pulling dependent layers
511136ea3c5a: Download complete
...

The next command that JHipster's documentation recommends is to run the Docker image, forward ports and share folders. When you run it, the terminal seems to hang and trying to ssh into it doesn't work. Others have recently reported a similar issue. I discovered the hanging is caused by a missing "-d" parameter and ssh doesn't work because you need to add a portmap to the VM to expose the port to your host. You can fix this by running the following:

boot2docker down
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containerssh,tcp,,4022,,4022"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containertomcat,tcp,,8080,,8080"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containergruntserver,tcp,,9000,,9000"
VBoxManage modifyvm "boot2docker-vm" --natpf1 "containergruntreload,tcp,,35729,,35729"
boot2docker start

After making these changes, I was able to start the image and ssh into it.

docker run -d -v ~/jhipster:/jhipster -p 8080:8080 -p 9000:9000 -p 35729:35729 -p 4022:22 -t jdubois/jhipster-docker
ssh -p 4022 jhipster@localhost

I tried creating a new project within the VM (cd /jhipster && yo jhipster), but it failed with the following error:

/usr/lib/node_modules/generator-jhipster/node_modules/yeoman-generator/node_modules/mkdirp/index.js:89
                    throw err0;
                          ^
Error: EACCES, permission denied '/jhipster/src'

The fix was giving the "jhipster" user ownership of the directory.

sudo chown jhipster /jhipster

After doing this, I was able to generate an app and run it using "mvn spring-boot:run" and access it from my Mac at http://localhost:8080. I was also able to run "grunt server" and see it at http://localhost:9000

However, I was puzzled to see that there was nothing in my ~/jhipster directory. After doing some searching, I found that the docker run -v /host/path:/container/path doesn't work on OS X.

David Gageot's A Better Boot2Docker on OSX led me to svendowideit/samba, which solved this problem. The specifics are documented in boot2docker's folder sharing section.

I shutdown my docker container by running "docker ps", grabbing the first two characters of the id and then running:

docker stop [2chars]

I started the JHipster container without the -v parameter, used "docker ps" to find its name (backstabbing_galileo in this case), then used that to add samba support.

docker run -d -p 8080:8080 -p 9000:9000 -p 35729:35729 -p 4022:22 -t jdubois/jhipster-docker
docker run --rm -v /usr/local/bin/docker:/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba backstabbing_galileo

Then I was able to connect using Finder > Go > Connect to Server, using the following for the server address:

cifs://192.168.59.103/jhipster

To make this volume appear in my regular development area, I created a symlink:

ln -s /Volumes/jhipster ~/dev/jhipster

After doing this, all the files were marked as read-only. To fix, I ran "chmod -R 777 ." in the directory on the server. I noticed that this also worked if I ran it from my Mac's terminal, but it took quite a while to traverse all the files. I noticed a similar delay when loading the project into IntelliJ.

Summary

Phew! That's a lot of information that can be condensed down into four JHipster + Docker on OS X tips.

Make sure your npm installation doesn't require sudo rights. If it does, reinstall using nvm.
Add portmaps to your VM to expose ports 4022, 8080, 9000 and 35729 to your host.
Change ownership on the /jhipster in the Docker image: sudo chown jhipster /jhipster.
Use svendowideit/samba to share your VM's directories with OS X.

Why I prefer IntelliJ IDEA over Eclipse

2014-07-21T13:33:55-06:00

Over the last couple months, I've received a few emails asking why I prefer IntelliJ IDEA over Eclipse. They usually go something like this:

I keep seeing you recommending IntelliJ. I keep trying it intermittently with using Eclipse, but I feel like I'm missing something obvious that makes so many people think it's better. Granted having the usual plugins incorporated is nice, but other things like the build process and debugger sometimes seems a step back from Eclipse. Could you please blog a '10 reasons why I love IntelliJ' or point me to something that would clue me in?

I grew to love IntelliJ for a few reasons. It all started in 2006 when I decided to migrate AppFuse from Ant to Maven. Before that, I was a huge Eclipse fan (2002 - 2006). Before Eclipse, I used HomeSite, an HTML Editor to write all my Java code (1999-2002). Eclipse was the first IDE that didn't hog all my system's memory and was pleasant to work with.

The reason I started using IntelliJ in 2006 was because of it's multi-module Maven support. Eclipse's Maven support was terrible, and m2e hasn't gotten a whole lot better in recent years AFAIK.

Back then, I used to think everything should be built and run from the command line. A couple years later, I realized it was better to run tests and debug from an IDE. Now I'm more concerned with the ability to run tests and debug in an IDE than I am from the build system.

In 2009, I started doing a lot more front-end work: writing HTML, CSS and JavaScript. I also started digging into alternate languages for these: Jade, GWT, CoffeeScript, LESS, SASS - even Scala. I found IntelliJ's support, and plugins, to be outstanding for these languages and really enjoyed how it would tell me I had invalid JavaScript, HTML and CSS.

My original passion in software was HTML and JavaScript and I found that hasn't changed in the last 15 years. AFAIK, Eclipse still has terrible web tools support; it excels at Java (and possibly C++ support). Even today, I write most of my HTML code (for InfoQ and this blog) in IntelliJ.

In reality, it probably doesn't matter which IDE you use, as long as you're productive with it. Once you learn one IDE well, the way others do things will likely seem backwards. I'm so familiar with debugging in IntelliJ, that when I tried to use Eclipse's debugger a few weeks ago, it seemed backwards to me.

In a nutshell: the technologies I've worked with have been better embraced by IntelliJ. Has this happened to you? Have certain technologies caused you to use one IDE over another?

This site now powered by Java 8, Tomcat 7 and Wufoo Forms

2014-04-09T07:37:59-06:00

I recently upgraded this site to use the latest version of Apache Roller. It was a minor release (5.0.3), but I figured I'd document the steps in case "early onset" comes soon. First of all, to download raibledesigns.com and get it running locally, I perform the following steps:

Backup everything using ~/bin/backup.sh on raibledesigns.com
scp backup file to local hard drive and expand
Copy ROOT, skins and repository directories to local webapps
Make sure activation, mail, mysql and jta JARs are in $CATALINA_HOME/lib
Copy roller-custom.properties from raibledesigns.com's $CATALINA_HOME/lib
Copy context files from hosted $CATALINA_HOME/conf/Catalina to local directory
Import database and change roller-custom.properties to match local credentials

Next, to upgrade to the latest Roller release, I do the following:

Download latest Roller release and expand
Copy JARs (from WEB-INF/lib) to existing install (to upgrade dependencies)
Delete any lower-versioned JARS from WEB-INF/lib directory
Copy JSPs (from WEB-INF/jsps) to existing install
Run database migration scripts from WEB-INF/classes/dbscripts
Use a diff tool (like SmartSynchronize) to compare new vs. existing
Test and troubleshoot (if there's startup errors)

This process has worked well for the last 10 years, and it's been in my head the whole time. It's bound to escape someday.

Contact Form Enhancements
In addition to upgrading Roller, I also upgraded Tomcat 6 to Tomcat 7.0.52. In doing so, I found Jakarta's Mailer Taglib doesn't work with Tomcat 7. As you can tell from the aforementioned thread, I've known this for several years. That's the only thing that's stopped me from upgrading Tomcat the past couple years.

Rather than using my own code to send email, I figured I'd explore my options. I looked into Kontactr, EmailMeForm and Wufoo. The first two allowed me to customize their colors, but Wufoo was the only one that had a transparent background feature. Because Wufoo form's are transparent, and I'd copied their CSS for AppFuse's forms in the past, I decided to use it. To make it work with both of this site's themes (light and dark), I did need to create two forms on Wufoo and do some local JavaScript modifications. You can see the results on my contact page (view-source to see the JavaScript). If you switch the theme (by clicking the little rectangles in the top-right corner), you can see how the form is redrawn with the proper color scheme.

If I'd found it sooner, I likely would've used Brace Forms for my contact form.

Theme Improvements
Ever since I switched to a new look and feel, there's been improvements I wanted to make.

The stylesheet switching doesn't happen as fast as I'd hoped (there's a flash even if using cookies), so I'll likely be converting some theme-setting logic to the server-side. The HTML5 version of the FaceBook Like Button requires you to specify the "data-colorscheme" in markup so this further supports moving to the server.

To fix this issue, I tried creating a cookie-reader plugin for Roller. This worked, but not as well as I'd hoped. The theme cookie was read the first time the page loaded, but then cached for subsequent requests. If a user changed their theme, the new value would not be read by the cached page. My solution now is putting the JavaScript set-default-theme logic right after the stylesheet to set it before the body loads.

Powered by Java 8
To make this site run on Java 8, I had to move to a new server. Keith did a bunch of work on my old server to get Java 8 working, but found Linux JDK 7 code has dependancies on the 2.6 kernel. If you're reading this post, the DNS changes have propagated. This means I have my first Java 8 app in production!

Secured by SSL
The last change I made was to start using my SSL certificate on this site. You can read about the challenges I encountered on the Roller mailing list. The final solution involved changing my Absolute URL to use https and then changing my theme to use the following for its base.

<base href="$absBaseURL.replace('https:', '')" />

I also had to change a number of iframes to use src="//..." instead of src="//...". To force HTTPS for the admin pages, Keith configured Apache to do the redirect. Using Roller to redirect resulted in infinite loop errors.

This means you can now access this site using https://raibledesigns.com or http://raibledesigns.com, whichever you prefer. Speaking of SSL, you might wonder if this new server is affected by The Heartbleed Bug. According to Keith, it is not.

This server is still running openSSL 1.0.0 which was shipped with Centos 6.1, and the Heartbleed vulnerability appeared in OpenSSL 1.0.1 which shipped Centos 6.5.

Summary
This site has undergone quite a few updates this month: Roller 5.0.3, Tomcat 7.0.52, a Wufoo contact form (+ recent tweets on the same page), Java 8 and optional SSL. If you see any issues, please let me know.

May 8, 2014 Update: I decided to revert to "http" for the absolute URL instead of "https". However, I've updated all "src" attributes on <iframe> and <img> tags to use schema-less URLs (// instead of http://). This means that both secure and non-secure URLs should work.

2013 - A Year in Review

2014-01-31T08:53:10-07:00

2013 was an amazing year: Trish and I got married, celebrated on a 'round-the-world honeymoon and invested in a new 4x4 VW Bus. I finally achieved my goal of vacationing 25% and I got to spend more than two months in the presence of my wonderful parents.

For this Year in Review post, I'll use the same format as I did last year:

Professional

For the last few years, I've generally had one client per year. That changed this year when my contract with Oracle ended in May. Fortunately, I had the opportunity to develop a cool dashboard application before I finished. I wrote about it in a four-part series.

I engaged in a month-long contract with Travelport to keep me busy in June. They hired me to develop a portal consolidation prototype, which I did with AngularJS, Grails and LDAP. I showcased that application at Devoxx in November.

In September, I started a new gig with John Muir Health. I was hired to help with their mobile architecture, and spent my first couple months doing front-end optimizing and helping get their MyJMH product released. Once the release was complete, I proposed a mobile architecture and started assisting with the development of their mobile application.

We're developing the Mobile API with Spring Boot. I wrote about my initial experience in A Webapp Makeover with Spring 4 and Spring Boot. My contract with John Muir Health is through the end of March and I hope to start something new shortly after.

In October, I started Writing for InfoQ and I've really enjoyed it so far. You can see the articles I've written on my author page or by clicking the links below:

Speaking

I spoke at five events in 2013:

Denver JUG on The Modern Java Web Developer and Java Web Security
Devoxx France on Comparing JVM Web Frameworks and Play vs. Grails
HTML5 Denver on Bootstrap
JavaOne: I spoke for the first time and received a Rock Star Award
Devoxx and a Nordic Countries Speaking Tour on The Modern Java Web Developer

It seems that folks liked my presentations since they were in the top 1% of most viewed on SlideShare in 2013.

Trish took many beautiful photos as part of our trip to Devoxx France.

We stopped in Iceland on the way home to see the Northern Lights.

Projects

AppFuse: I released AppFuse Light 2.2.1 in January and switched AppFuse from JSF's MyFaces to PrimeFaces in February. I blogged about AppFuse's GWT integration in March and celebrated the project's 10 year anniversary in April. AppFuse 3.0 was released just before Christmas.

Roller: I didn't contribute any code to the Roller project in 2013, but I did make this site responsive with CSS media queries.

Personal

Trish and I started our world travels in March with a trip to Magnificent Mexico. We had a wonderful time playing with old friends and renting a house on the beach.

Our trip to Paris and Iceland was a dream come true for Trish. Watching the Aurora Borealis dance in the sky is something I'll never forget.

To end the ski season in April, we completed the trifecta (3 ski resorts in 3 days) with a fun family weekend.

The last week of the ski season was epic, as I described in a life update.

The last week in Winter Park was the best skiing of the year. The first (Tuesday) afternoon, I started skiing around 1pm and it snowed all afternoon - resulting in several inches by the end of the day. The next day was smooth and empty, followed by a day of deep powder and knee-deep runs down Eagle Wind. I had one of the best runs of my life that day.

I wrote about the '66 Bus Project and how I removed it from Motorworks Restorations after six years.

We spent the first weekends in June on the Colorado River. One of the biggest highlights for me was catching a 16" Trout, right after Trish saw it jump and told me exactly where to cast my line. The fact that I was able to cast it to the precise spot was cool enough, but getting the fish on my line moments later was exhilarating. Shortly after, a majestic Bald Eagle flew over us and we all dropped our jaws in amazement.

Trish and I started a two month sabbatical in July to get married in proper fashion in the town I grew up in. She looked absolutely stunning.

During the ceremony, I surprised her with a 162-page book I built with my words and her pictures.

When asked if I'd take her as my lawfully-wedded wife, I shouted as "YES!" at the top of my lungs. Trish agreed enthusiastically and we were pronounced husband and wife.

Our wedding week was an unbelievably fun experience with many, many friends and family. We look forward to celebrating it over and over again as long as we live.

We had a day of rest following the wedding, then drove back to Denver to begin our 'round-the-world honeymoon without phones. We spent a month traveling to Ireland, Italy, Thailand and Fiji. It was a wonderful journey and we greatly enjoyed all the people, places and experiences.

We arrived back in the US at the end of August, just in time to celebrate Jack's 9th birthday.

In September, we got loud at the Broncos Home Opener, and experienced beautiful Colorado fall weather with trips to Estes Park and Aspen. In October, we traveled to Minnesota for a best friend's wedding. We also visited Pennsylvania to spend some time with Trish's parents.

We Bought a Ski Bus in October and celebrated Abbie's birthday in November. Shortly after, we departed on a whirlwind six-country speaking tour in Europe. We visited 14 countries in 2013.

In early December, we flew to Idaho to pickup our new Syncro. We anticipated a few VW Adventures when we bought the bus and haven't been disappointed.

The Syncro drove back to Denver just fine, but started experiencing cooling issues driving around town. We got it fixed and drove to Montana for Christmas. We broke down on Christmas Eve in Bozeman, tried to fix it, but eventually gave up and rented a car. Straightaway Motors replaced the thermostat over the next week and we were able to drive it back to Denver with no issues. It started overheating again a couple days later. We've only recently got it running smoothly, with help from Rocky Mountain Westy. You can read the full story on thesamba.com.

Christmas Vacation was spent at The Raible Homestead, enjoying my parents new retirement cabin and a sweet sledding hill.

We finished the year skiing at Big Mountain, where I learned how to downhill ski in grade school.

2014

My goal for 2014 is singular: finish The Bus. I finally found the restoration shop I wish I would've found years ago. They're very quick and efficient, and send me daily (picture) updates of their progress. With any luck, it'll be done in a couple months (sound familiar?).

Last year, I wanted to slow down and I feel like I did that on our honeymoon. My wife scoffs at that notion. This year, I hope to simply take more time to do things, instead of trying to cram many things into tight timelines.

I plan to spend less time traveling and speaking at conferences and more time at Volkswagen shows. With our 4x4 VW Syncro, we hope to camp, raft and spend quite a bit of time in the Rocky Mountains. We also hope to take the busses to a few art shows to show off Trish's majestic photos.

Professionally, I hope to continue developing HTML5 apps and APIs with JavaScript, CSS, Java and Groovy. I might dive back into Scala and I'm intrigued by Node.js and Spring Boot. Since I won't be doing as many conferences, the technologies I use will likely be driven by client engagements. I expect 2014 to be a big year for HTTP/2.

Last year, and the year before, we watched the Broncos fail miserably in the NFL playoffs. This year, they're in the Super Bowl. With a kick-ass Ski Bus, a Porsche Bus on the way, awesome kids and a great wife - 2014 is destined to be spectacular. I hope the Broncos are too!

Using Grunt with AngularJS for Front End Optimization

2014-01-15T12:15:52-07:00

I'm passionate about front end optimization and have been for years. My original inspiration was Steve Souders and his Even Faster Web Sites talk at OSCON 2008. Since then, I've optimized this blog, made it even faster with a new design, doubled the speed of several apps for clients and showed how to make AppFuse faster. As part of my Devoxx 2013 presentation, I showed how to do page speed optimization in a Java webapp.

I developed a couple AngularJS apps last year. To concat and minify their stylesheets and scripts, I used mechanisms that already existed in the projects. On one project, it was Ant and its concat task. On the other, it was part of a Grails application, so I used the resources and yui-minify-resources plugins.

The Angular project I'm working on now will be published on a web server, as well as bundled in an iOS native app. Therefore, I turned to Grunt to do the optimization this time. I found it to be quite simple, once I figured out how to make it work with Angular. Based on my findings, I submitted a pull request to add Grunt to angular-seed.

Below are the steps I used to add Grunt to my Angular project.

Install Grunt's command line interface with "sudo npm install -g grunt-cli".
Edit package.json to include a version number (e.g. "version": "1.0.0").

Add Grunt plugins in package.json to do concat/minify/asset versioning:

    "grunt": "~0.4.1",
    "grunt-contrib-concat": "~0.3.0",
    "grunt-contrib-uglify": "~0.2.7",
    "grunt-contrib-cssmin": "~0.7.0",
    "grunt-usemin": "~2.0.2",
    "grunt-contrib-copy": "~0.5.0",
    "grunt-rev": "~0.1.0",
    "grunt-contrib-clean": "~0.5.0"

Run "sudo npm install" to install the project's dependencies.

Create a Gruntfile.js that runs all the plugins.

module.exports = function (grunt) {

    grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),

        clean: ["dist", '.tmp'],

        copy: {
            main: {
                expand: true,
                cwd: 'app/',
                src: ['**', '!js/**', '!lib/**', '!**/*.css'],
                dest: 'dist/'
            },
            shims: {
                expand: true,
                cwd: 'app/lib/webshim/shims',
                src: ['**'],
                dest: 'dist/js/shims'
            }
        },

        rev: {
            files: {
                src: ['dist/**/*.{js,css}', '!dist/js/shims/**']
            }
        },

        useminPrepare: {
            html: 'app/index.html'
        },

        usemin: {
            html: ['dist/index.html']
        },

        uglify: {
            options: {
                report: 'min',
                mangle: false
            }
        }
    });

    grunt.loadNpmTasks('grunt-contrib-clean');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-contrib-concat');
    grunt.loadNpmTasks('grunt-contrib-cssmin');
    grunt.loadNpmTasks('grunt-contrib-uglify');
    grunt.loadNpmTasks('grunt-rev');
    grunt.loadNpmTasks('grunt-usemin');

    // Tell Grunt what to do when we type "grunt" into the terminal
    grunt.registerTask('default', [
        'copy', 'useminPrepare', 'concat', 'uglify', 'cssmin', 'rev', 'usemin'
    ]);
};

Add comments to app/index.html so usemin knows what files to process. The comments are the important part, your files will likely be different.

<!-- build:css css/app-name.min.css -->
<link rel="stylesheet" href="lib/bootstrap/bootstrap.min.css"/>
<link rel="stylesheet" href="lib/font-awesome/font-awesome.min.css"/>
<link rel="stylesheet" href="lib/toaster/toaster.css"/>
<link rel="stylesheet" href="css/app.css"/>
<link rel="stylesheet" href="css/custom.css"/>
<link rel="stylesheet" href="css/responsive.css"/>
<!-- endbuild -->
...

<!-- build:js js/app-name.min.js -->
<script src="lib/jquery/jquery-1.10.2.min.js"></script>
<script src="lib/bootstrap/bootstrap.min.js"></script>
<script src="lib/angular/angular.min.js"></script>
<script src="lib/angular/angular-animate.min.js"></script>
<script src="lib/angular/angular-cookies.min.js"></script>
<script src="lib/angular/angular-resource.min.js"></script>
<script src="lib/angular/angular-route.min.js"></script>
<script src="lib/fastclick.min.js"></script>
<script src="lib/toaster/toaster.js"></script>
<script src="lib/webshim/modernizr.min.js"></script>
<script src="lib/webshim/polyfiller.min.js"></script>
<script src="js/app.js"></script>
<script src="js/services.js"></script>
<script src="js/controllers.js"></script>
<script src="js/filters.js"></script>
<script src="js/directives.js"></script>
<!-- endbuild -->

A couple of things to note: 1) the copy task copies the "shims" directory from Webshims lib because it loads files dynamically and 2) setting "mangle: false" on the uglify task is necessary for Angular's dependency injection to work. I tried to use grunt-ngmin with uglify and had no luck.

After making these changes, I'm able to run "grunt" and get an optimized version of my app in the "dist" folder of my project. For development, I continue to run the app from my "app" folder, so I don't currently have a need for watching and processing assets on-the-fly. That could change if I start using LESS or CoffeeScript.

The results speak for themselves: from 27 requests to 5 on initial load, and only 3 requests for less than 2K after that.

	YSlow	Page Speed
No optimization	75 27 HTTP requests / 464K	55/100
Apache optimization (gzip and expires headers)	89 initial load: 26 requests / 166K primed cache: 4 requests / 40K	88/100
Apache + concat/minified/versioned files	98 initial load: 5 requests / 136K primed cache: 3 requests / 1.4K	93/100

Update: Andreas Andreou has a nice tip on how to reduce the LOC in this example.

Add "matchdep" as a dependency in package.json (or run "sudo npm install matchdep --save-dev").
```
"matchdep": "~0.3.0"
```

Replace all the grunt.loadNpmTasks(...) calls with the following:

require('matchdep').filterDev('grunt-*').forEach(grunt.loadNpmTasks);

Thanks Andreas!

AppFuse 3.0 Released!

2013-12-23T14:31:15-07:00

The AppFuse Team is pleased to announce the release of AppFuse 3.0. This release is AppFuse's first release as a 10-year old and includes a whole slew of improvements.

Java 7 and Maven 3 are now minimal requirements
Replaced MyFaces and Tomahawk with PrimeFaces for JSF
- Removed SiteMesh in favor of JSF's built-in layout support
Added Wicket support
Migrated from jMock to Mockito for tests
Integrated wro4j and WebJars
Migrated to Bootstrap 3 and defaulted to Bootswatch's Spacelab theme

In addition, this release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Spring 4, Spring Security 3.2 and Bootstrap 3. For more details on specific changes see the release notes.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

Happy 10 Year AppFuse!

2013-04-05T08:56:45-06:00

10 years ago yesterday, I released the first version of AppFuse. It started with XDoclet generating ActionForms from POJOs and became very popular for Struts developers that wanted to use Hibernate. The project's popularity peaked in 2006, as you can see from the mailing list traffic below.

It's possible the decrease in traffic is because we re-wrote everything to be based on Maven. It's also possible it was because of more attractive full-stack frameworks like Grails and Rails. However, the real reason is likely that I stopped working on it all the time due to getting a divorce becoming an awesome dad.

Below is a timeline of how the project evolved over its first 4 years.

AppFuse has been a great project for me to work on and it's been a large source of my knowledge about Java, Web Frameworks, Spring, Hibernate - as well as build systems like Ant and Maven. We started with CVS, moved to SVN and now we're on GitHub. We've experienced migrating from Tapestry 4 to Tapestry 5 (thanks Serge Eby!), upgrading to JSF 2 and enjoyed the backwards compatibility of Spring and Struts 2 throughout the years. We've also added REST support, a Web Services archetype and kept up with the latest Spring and Hibernate releases.

Last year, we added Bootstrap and jQuery as foundational front-end frameworks. For our next release, we're switching to PrimeFaces, adding Wicket and changing from jMock to Mockito. Most of these changes are already in source control, we just need to polish them up a bit and add AMP support. I hope to release 3.0 before the bus is done.

Thanks to all the enthusiastic users of and contributors to AppFuse over the years. It's been a great ride!

Integrating GWT into AppFuse

2013-03-07T18:49:28-07:00

I've been interested in integrating GWT into AppFuse ever since I blogged about it 4 years ago. A few months after that post, I wrote about Enhancing Evite.com with GWT and Grails. After Evite, I had a gig near Boston where I developed with GXT for the remainder of the year. When all was said and done, I ended up spending a year with GWT and really enjoyed my experience. I haven't used it much since.

GWT is scheduled to be integrated into AppFuse in version 4.0. That's quite a ways off. The good news is you might not have to wait that long, thanks to Iván García Sainz-Aja. Iván let us know about his work a couple weeks ago in an email to the appfuse-dev mailing list.

It's still work in progress but it has already most of AppFuse functionality..

If you want to give it a try

https://github.com/ivangsa/appfuse.git

the quickest way to have a go would be

web/gwt> mvn -P gwtDebug -Dgwt.inplace=true gwt:compile jetty:run

at the moment it still requires this fork of gwt-bootstrap to be compiled first

https://github.com/ivangsa/gwt-bootstrap.git

It needs a lot of testing yet but it's getting quite there

As you can imagine, I was very excited to hear about Iván's work. So I cloned his repo, built gwt-bootstrap locally and checked it out. Functionality wise, it was great! However, when I dug into the source code, I found a whole lotta code.

To see how the GWT flavor compared to the other implementations in AppFuse, I created a cloc report on the various web frameworks in AppFuse. I'm sure these reports could be adjusted to be more accurate, but I believe they give a good general overview. I posted some graphs that displays my findings in visual form.

When I sent this to the mailing list, Ivan responded that it was a lot of code and estimated 12 new files would be needed to CRUD an entity. This sure seems like a lot to me, but he defended this yesterday and noted that his implementation follows many of GWT's latest best practices: MVP pattern, Activities and Places, EventBus, Gin and Guice. He also shared a wiki page with explanations and diagrams of how things work.

The reason I'm writing this post is to get more feedback on this implementation. First of all, does GWT really require this much code? Secondly, are there other GWT implementations that reduce a lot of the boilerplate? SmartGWT, Vaadin* and Errai come to mind.

If you were starting a new GWT project and using AppFuse, how would you want it implemented?

* Vaadin 7 claims it can be used as a drop-in replacement for GWT. I tried replacing the gwt-servlet and gwt-user dependencies with Vaadin's, but it didn't work.

Switching AppFuse from MyFaces to PrimeFaces

2013-02-06T12:19:34-07:00

When describing my bias against JSF back in November, I wrote:

... there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

Shortly after the AppFuse 2.2.1 release in December, Gilberto Andrade contributed a sample project that used Mojarra (the JSF RI) and PrimeFaces instead of MyFaces and its Tomahawk components. Last week, I spent a few hours integrating Gilberto's changes into AppFuse's master branch. You can see all the changes I made (which include a Jetty plugin upgrade and some cleanup) in this Crucible review. Feel free to leave comments on ask questions in the review itself.

The first thing I noticed when integrating PrimeFaces is you have to add a custom repository in order to get its artifacts via Maven.

<repositories>
    <repository>
        <id>prime-repo</id>
        <name>Prime Repo</name>
        <url>http://repository.primefaces.org</url>
    </repository>
</repositories>

This is unfortunate since all of AppFuse's other dependencies can be found in Maven Central. It means that if you're using a JSF archetype, the PrimeFaces repo will be checked for artifacts first, causing an unnecessary slowdown in artifact resolution. I hope the PrimeFaces developers fix this soon.

While integrating these two frameworks, I ran into a number of issues.

An IllegalStateException on startup when using "mvn jetty:run"
The first issue I encountered was that I was unable to run the app in Jetty. It worked fine in Tomcat but I got the following error in Jetty:

2013-01-31 22:28:07.683:WARN:/:unavailable
java.lang.IllegalStateException: Application was not properly initialized at startup, could not find Factory: javax.faces.context.FacesContextFactory
at javax.faces.FactoryFinder$FactoryManager.getFactory(FactoryFinder.java:951)
at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:316)
at javax.faces.webapp.FacesServlet.init(FacesServlet.java:302)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:492)
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:312)

I found the fix for this on Stack Overflow and added the following listener to my web.xml to solve it.

<listener>
    <listener-class>com.sun.faces.config.ConfigureListener</listener-class>
</listener>

Conditionally rendering a button disables its click-ability
The next thing I noticed was the Delete button didn't work when editing a user. It was hidden correctly when adding a user, but clicking on it to delete a user simply refreshes the page. Below is the code I used successfully with MyFaces. For some reason, this doesn't work with PrimeFaces.

<c:if test="${not empty userForm.user.id}">
<h:commandButton value="#{text['button.delete']}" action="#{userForm.delete}"
    styleClass="btn" onclick="return confirmMessage(msgDelConfirm)"/>
</c:if>

I also tried the following, but no dice. This is currently an open issue.

<h:commandButton rendered="${not empty userForm.user.id}" value="#{text['button.delete']}" 
    action="#{userForm.delete}" styleClass="btn" onclick="return confirmMessage(msgDelConfirm)"/>

The PrimeFaces Bootstrap theme 404s on some images
After integrating PrimeFaces' Bootstrap theme, the following error shows up in server logs.

[INFO] [talledLocalContainer] Feb 02, 2013 10:40:25 PM com.sun.faces.application.resource.ResourceHandlerImpl logMissingResource
[WARNING] [talledLocalContainer] WARNING: JSF1064: Unable to find or serve resource, images/ui-bg_highlight-hard_70_000000_1x100.png, from library, primefaces-bootstrap.

This seems to have happened before in previous releases and is currently an open issue.

Canoo WebTest doesn't work with fileUpload nor to set checkbox values
We use Canoo WebTest to run integration tests on the UI in AppFuse. For some reason, performing file uploads and setting checkbox values works fine with MyFaces/Tomahawk, but not with Mojarra/PrimeFaces. I'm not sure if this is caused by the JSF core or the component library, but it remains an open issue. For now, I've just commented out the parts of tests that used to do this.

On a related note, getting the real path of a resource from the ServletContext worked fine before the switch, but results in a null value now.

String uploadDir = getServletContext().getRealPath("/resources") + "/" + request.getRemoteUser() + "/";

PrimeFaces resources served up at /javax.faces.resource/* not found
While I didn't have problems with this in AppFuse, I did encounter it in AppFuse Light. I don't know why there was a difference between the two, but it turned out to be caused by the UrlRewriteFilter and my desire for extensionless URLs. The outbound-rule to strip .xhtml from URLs was the culprit. Adding a condition to it solved the problem. Yeah, the condition seems backwards, but it works.

<outbound-rule match-type="regex">
    <condition type="query-string" operator="equal">ln=primefaces</condition>
    <from>^(.*)\.xhtml(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

Summary
The initial switch to Mojarra/PrimeFaces was pretty easy thanks to Gilberto's sample project. However, the small issues encountered after that turned out to be quite frustrating and you can see that several are still not fixed. I guess it just goes to show that not all web frameworks are perfect. Hopefully we'll get these minor issues fixed before the next release. In the meantime, you can checkout the updated demos for AppFuse JSF and AppFuse Light JSF.

AppFuse Light 2.2.1 Released!

2013-01-24T19:43:20-07:00

In December, the AppFuse Team released 2.2.1. Right before that release, I decided to wait on enhancing its "light" modules, a.k.a. AppFuse Light. I'm glad I did, because it took some effort to get jQuery and Bootstrap integrated, as well as to make it more secure.

The good news is AppFuse Light 2.2.1 is released and it's sitting out on the Central Repository. This release is a refactoring of all archetypes to be up-to-date with the AppFuse 2.2.1 release. This means Java 7 compatibility, Servlet 3, Bootstrap/jQuery integration, Tapestry 5.3.6 upgrade and security improvements. I integrated Bootstrap and jQuery using WebJars Servlet 3 support since it was simple and straightforward.

You can create projects using AppFuse's light archetypes using a command such as the following:

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes 
  -DarchetypeArtifactId=appfuse-light-spring-freemarker-archetype -DarchetypeVersion=2.2.1 
  -DgroupId=com.mycompany -DartifactId=myproject

The list of archetypes is as follows:

appfuse-light-jsf-archetype
appfuse-light-spring-archetype
appfuse-light-spring-freemarker-archetype
appfuse-light-spring-security-archetype
appfuse-light-stripes-archetype
appfuse-light-struts-archetype
appfuse-light-tapestry-archetype
appfuse-light-wicket-archetype

The QuickStart Guide will help you get setup and demos are available at the following links:

If you have questions about AppFuse, we invite you to ask them on the users mailing list or tweet using #appfuse.

For those enjoying Bootstrap in your apps, I encourage you to check out {wrap}bootstrap and Bootswatch.

2012 - A Year in Review

2013-01-08T13:15:24-07:00

I wrote my first year in review blog entry way back in 2005. That means this year's is number 8. Since they keep getting longer every year, I figured I'd try something different this year and use sections similar to Remy Sharp.

Professional

I spent the entirety of the year with one client: Taleo. Oracle bought them in February. In June, the transition to Oracle happened. My tasks and projects haven't changed much since the transition, but it has been a real pain to get paid on time. My contract with them is through the end of May. I hope to take July off (to get married) and August off (to honeymoon) and start a new gig in September.

I did minimal Java work throughout the year and spent most of my time doing CSS and JavaScript. I love doing front-end work much more than back-end, so day-to-day, it was very satisfying.

Business

Raible Designs had a great year, our best ever financially. I hired Trish as an assistant in January, doubling the size of the company. We had our Annual Shareholders Meeting on a Disney Cruise in April. I have no plans to do increase the size of the company in 2013, mostly because I enjoy not having the responsibility of employees. I've employed sub-contractors in the past and it always seems to create more of a headache than it's worth.

We moved our offices from downtown Littleton to Thrive (in LoDo) last April and I plan on staying there. It's an easy 6-mile bicycle commute and the guys there have been great to work with. I highly recommend it if you're looking for co-working space.

Speaking

I spoke at 5 events in 2012:

Silicon Valley Spring User Group on What's New in Spring 3.1.
JFokus on Comparing Web Frameworks and HTML5 with Play Scala.
Spring I/O on Comparing JVM Web Frameworks.
JavaPosse Roundup on How to build a Shot-Ski.
ÜberConf on Play vs. Grails.

Traveling to Stockholm and Madrid for JFokus and Spring I/O was spectacular.

I took the 2nd half of the year off from speaking, celebrating my temporary retirement at a Def Leppard concert in June. When I decided to do this, I was planning on my VW Bus being finished. I was planning on switching from Java Conferences to VW Shows as a hobby. Unfortunately, now it's January and The Bus still isn't done. I have high hopes for it being finished in 2013.

I paid for 2 conferences in 2012, the JavaPosse Roundup in March and Monktoberfest in October. While I enjoyed both, Monktoberfest will likely become a tradition.

Projects

AppFuse: I started the year by refreshing AppFuse's UI with Twitter Bootstrap and blogged about the Twitter's Summit on Bootstrap shortly after. At the same time, I was overhauling Taleo's UI with HTML5, Bootstrap and CSS3, but waited until August to blog about it.

In the fall, the AppFuse Team migrated to GitHub and integrated Hibernate Search. AppFuse 2.2.1 was released in December.

Happy Trails: In May and June, I worked on Happy Trails with James Ward as part of our Play vs. Grails Smackdown. This was a very enjoyable experience and I learned a lot about GitHub, Cloudbees and Heroku.

Play More: In June, I wrote about Upgrading to Play 2: Anorm and Testing. A couple weeks later, I wrote about integrating Validation and Secure Social into Play More. In the aforementioned post, I also posted my ÜberConf presentation, the source code and my thoughts on Play 2.

I found Anorm and Scalate to be huge time sinks and don't know if I'd recommend using either one in a Play 2 project. I'm sure Scalate will be easier to use as its Play 2 integration gets more refined, but I don't know if there's any hope for a JDBC abstraction that doesn't produce error messages when things go south.

Roller: Although I didn't contribute much to Apache Roller in 2012, I do continue to use it frequently. This site celebrated its 10-year anniversary in August. It got a new look and feel shortly after.

Personal

Last year was a fantastic year for family time. Our cruise around the Western Caribbean was money well spent.

My goal was to spend a month with my parents this year and we satisfied that by July. In total, we enjoyed their company for 8 weeks in 2012.

The worst ski season on record ended in April and Trish and I headed for wine country in May.

Abbie and Jack's school year ended on June 5th and we celebrated Father's Day on the Oregon coast.

Raible Road Trip #17 began in late June and we spent two weeks parading, rafting and golfing in Montana. From there, we flew to Maui for two weeks. I wrote about it all in Summer Vacation 2012 in Montana and Maui.

After returning from Hawaii, we dropped the kids off at their Mom's and headed to my cousin's wedding in Beacon, NY for the weekend. From there, my parents, Trish and I rode the train to Boston, spent a few days in Cape Cod and returned home in time for the beginning of school.

The kids started the school year in mid-August and we celebrated Jack's 8th birthday a couple weeks later.

My lack of speaking engagements and work-related travel gave us more time to frolic in the Colorado mountains, so we bought a boat for Labor Day weekend. That Saturday, we took it on its Maiden Voyage on the Colorado River, floating from Radium to Rancho del Rio. I guided and rowed the boat most of the time while our 7 passengers (and 2 dogs) enjoyed cold beverages, great scenery and relaxing in the sun.

The next day, Abbie played her first game of golf.

In October, I finished our deck project and work continued on our kitchen remodel (started in July by Ted Pepper). Our kitchen was far enough along for us to host our 1st Annual Halloween Party.

Abbie turned 10 the first week in November. That Friday, we headed to Kauai to celebrate my parents' 40th Anniversary. We spent Thanksgiving in Wenatchee, Christmas in Boston/Vermont and New Years with good friends in Fraser.

2013

My goals for 2013 are to be happy and to slow down. I don't plan on traveling any less, or spending less time with my family. I'd just like to walk slower, smell the good ol' Rocky Mountain air more and enjoy life. I believe our raft will help us facilitate this and we're already planning trips to the Green River in Utah and Schafer Meadows in the Bob.

The big event for the year will be Trish and my wedding this summer. We'll be doing it in my hometown in Montana and we expect it to be quite the shindig. Afterward, we plan on traveling the world for our honeymoon.

Of course, if the bus gets finished, that's a huge bonus too.

Professionally, I'm starting an exciting project at Taleo that will involve heavy JavaScript, CSS3 and maybe even Canvas. I have speaking engagements lined up at Denver JUG in February and HTML5 Denver in April. Also, I just submitted a couple talks to Devoxx France.

For AppFuse, I'd love to get 3-4 releases out this year, but 2 is probably more realistic. The AppFuse Roadmap shows our future releases + features. I'd also like to work a bit more on my Play 2 app; finishing the mobile client and possibly switching from Anorm to Slick or Spring Data.

While Oracle has been good to me, I look forward to finding and working for a new client in September.

Last year at this time, we watched the Broncos execute the Mile High Miracle over the Steelers. With tickets to both this week's game (vs. The Ravens) and next week's game (vs. The Patriots), I wouldn't be surprised if January turns out to be a great month. However, with the Powder Days, The Super Bowl, the raft trips, the Bus Shows and The Wedding, there's a good chance that all the months in 2013 are epic.

AppFuse 2.2.1 Released!

2012-12-11T15:21:44-07:00

The AppFuse Team is pleased to announce the release of AppFuse 2.2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are Hibernate 4, Struts 2.3.7, Apache CXF 2.7.0 and Spring Security 3.1.3. In addition, we've integrated HTML5, Twitter Bootstrap, jQuery and replaced Compass with Hibernate Search. Last but not least, we've added full support for Java 7 and integrated many security improvements. For more details on specific changes see the release notes.

Demos for this release can be viewed at http://demo.appfuse.org. Please see the QuickStart Guide to get started with this release.

A number of blog posts were written about features that went into this release while it was being developed:

If you have questions about AppFuse, please read the FAQ or join the mailing list. If you find any issues, please report them on the users mailing list.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. We highly recommend using the new IntelliJ IDEA 12 for developing web applications.

Improving AppFuse's PageSpeed with Apache

2012-12-04T09:25:05-07:00

One of the most important things when developing webapps is to make them fast. With AppFuse, we've tried to incorporate many of the 14 rules for faster-loading websites. We had a gzip filter before it was cool (2003) and replaced it with the one from EhCache. However, users experienced issues with both of these, both with XFire/CXF and WebWork/Struts 2 and JSPs. Because of these issues, we disabled gzipping a few releases ago.

This article is designed to show you how you can make your AppFuse webapp faster, without modifying any code. The good news is this applies to any webapp that you can deploy behind Apache.

Last Friday, I sent an email to the good folks at Contegix to see if they could install mod_pagespeed on the Apache server that sits in front of *.appfuse.org. My goal was to improve the YSlow and PageSpeed scores of the apps hosted on demo.appfuse.org. I discovered they were getting a dismal score of 24 and figured we could do a lot better. mod_pagespeed speeds up your site and reduces page load time by automatically applying web performance best practices. It seemed like an easy solution.

Unfortunately, we were unable to use mod_pagespeed. From the guys at Contegix:

Attempting to install mod_pagespeed as you requested, we find that it requires Apache httpd 2.2 and libstdc++ 4.1.2, both of which are unsupported in RHEL4. To get mod_pagespeed to work on your present operating system basically means re-rolling the core components, which would make them unsupported. I'm afraid mod_pagespeed is simply not an option on your present configuration.

Since I still wanted to improve performance, I opted for another route instead: using mod_deflate (for gzipping) and mod_expires (for expires headers). I also turned on KeepAlive as recommended by PageSpeed Insights.

mod_deflate
mod_deflate was already installed in Apache (version 2.0.52), so all I had to do was configure it. On RHEL4, Apache is installed at /etc/httpd and there's a conf.d directory that contains all the configuration files. I created a file at /etc/httpd/conf.d/deflate.conf and populated it with the following:

#
# mod_deflate configuration
#
<IfModule mod_deflate.c>
    SetOutputFilter DEFLATE
    
    AddOutputFilterByType DEFLATE text/plain text/html text/xml text/css application/xml application/xhtml+xml application/rss+xml application/javascript application/x-javascript
    
    DeflateCompressionLevel 9
    
    BrowserMatch ^Mozilla/4 gzip-only-text/html
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    
    DeflateFilterNote Input instream
    DeflateFilterNote Output outstream
    DeflateFilterNote Ratio ratio
    
    LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
    
</IfModule>

At first, I had separate lines for all the different content types (as recommended by this article). The Contegix support crew figured out the solution (everything needed to be on one line) in 14 minutes, updated the config and verified it worked using an http compression testing page.

mod_expires
mod_expires was already installed, so I added a config file at /etc/httpd/conf.d/expires.conf. I used this howto and asked Contegix for help when it didn't work. Their response took quite a bit longer this time (49 minutes), but they once again figured it out:

It appears that FilesMatch does not like to play will JkMount. It does work using content type.

My final config for expires.conf:

<IfModule mod_expires.c>
    ExpiresActive On
    
    <FilesMatch "\.(jpe?g|png|gif|js|css)$">
        ExpiresDefault "access plus 1 week"
    </FilesMatch>
    
    ExpiresByType image/jpeg "access plus 1 week"
    ExpiresByType image/png "access plus 1 week"
    ExpiresByType image/gif "access plus 1 week"
    ExpiresByType text/css "access plus 1 week"
    ExpiresByType application/javascript "access plus 1 week"
    ExpiresByType application/x-javascript "access plus 1 week"
</IfModule>

I used "1 week" because we're changing things quite a bit right now and we haven't integrated resource fingerprinting yet.

KeepAlive
The last thing I did to improve performance was to turn on KeepAlive by editing /etc/httpd/conf/httpd.conf and changing Off to On.

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

Summary
As a result of these changes, our PageSpeed score went from 24 to 96 and YSlow went from a 90 to a 98. When I started this experiment, I was only trying to fix demo.appfuse.org. However, it also improved the speed of all the other *.appfuse.org sites, including Confluence, Bamboo, JIRA and FishEye. Thanks for all the help Contegix! There's a good chance you've given me back a few minutes in each day.

Originally posted on the AppFuse Blog.

AppFuse News: Liquibase and wro4j Tutorials from J. García

2012-11-30T14:32:47-07:00

New committer J. García has been doing a lot of work to improve i18n in AppFuse 2.2, as well as our Struts 2 support. In addition, he's written a couple articles that show you how to integrate Liquibase and wro4j in your AppFuse applications. Thanks for the great documentation J!

The 2.2 release is coming along, and we're down to 16 open issues. I've updated the Hibernate, JPA, Services and Web Services tutorials and hope to finish the web tutorials in the next week. You can try the latest code using the QuickStart Guide or check it out on the demo site:

Please see this thread on the mailing list if you have any questions or suggestions.

In related news, Roger Hughes has a good article titled Ten Things You Can Do With Spring Security. Since AppFuse uses Spring Security extensively, hopefully you can use some of Roger's tips to improve the security of your app.

Why the bias against JSF?

2012-11-08T09:24:27-07:00

In my last post about InfoQ's Top 20 Web Frameworks for the JVM, I received a thought-provoking comment from henk53:

There is one little thing that does bother me in those presentations, and that's your fairly obvious bias against JSF.
...
If you are presenting yourself as, more or less, an authority on comparing web frameworks, then having a fairly obvious biased against one of them is just peculiar. I, all of my team, and various clients distrust your ranking of JSF. We do look at your data if the choice is between other frameworks, but as soon as JSF comes into the picture we just look elsewhere.

I'm not really sure where this bias comes from. Yes, JSF 1.0 sucked and 1.2 was only marginally better, but 2.0 is really cool and productive and there are SUPERB component and utility libraries now like PrimeFaces and OmniFaces. As a researcher of this topic I think you should keep up the date and not stick to some old grudge.

This is true, I am biased against JSF. It all started with my first JSF experience back in August 2004. If you remember correctly, 2004 was a big year: JSF 1.0, Spring 1.0 and Flex 1.0 were all released. The "AJAX" term was coined in early 2005.

By 2007 and 2008, JSF still hadn't gotten any better. In late 2009, JSF 2.0 was released and I upgraded in March 2011. As you can see from the aforementioned post, I ran into quite a few issues upgrading. JSF was also the hardest one to get working with extension-less URLs.

Most of my issues with JSF come from having maintained an application built with it since 2004. If I were to start a new application without any legacy migration issues, I imagine it wouldn't be as difficult. However, if you compare it to Struts 2 and Spring MVC, I've had little-to-no issues upgrading those applications over the years.

Also, I'm not just biased against JSF, but most component-based web frameworks. Just ask the Tapestry and Wicket folks. They've felt my criticisms over the years. My reason for preferring request-based frameworks like Struts 2/Spring MVC and Grails/Play has been because I've never seen the appeal in component-based frameworks. Often I've found that their components are just widgets that you can get from any decent JavaScript framework. And chances are that JavaScript framework can work with any web framework. Also, I've worked on a lot of high-traffic web applications that require statelessness for scalability.

I see the value in component-based frameworks, I just don't think components should be authored on the server-side. Most of the Java-based component frameworks require 2+ files for components (one for the component, one for the view, possibly one for the config). I love GWT's component concept in that you can just extract a class and re-use it. With JS frameworks, you can often just include a script. These days, when I think of good component-based frameworks, I think of jQuery UI and Twitter Bootstrap.

All that being said, there's a lot of folks praising JSF 2 (and PrimeFaces moreso). That's why I'll be integrating it (or merging your pull request) into the 2.3 release of AppFuse. Since PrimeFaces contains a Bootstrap theme, I hope this is a pleasant experience and my overall opinion of JSF improves.

In other component-based frameworks in AppFuse news, Tapestry 5 has gotten really fast in the last year. I imagine this is because we have a Tapestry expert, Serge Eby, working on it. And we're planning on adding Wicket in the 2.3 release.

So even though I prefer request-based frameworks with REST support and Bootstrap, that doesn't mean everyone does. I'll do my best to be less-biased in the future. However, please remember that my view on web frameworks is as a developer, not an analyst. And aren't developers supposed to be opinionated?

InfoQ's Top 20 Web Frameworks for the JVM

2012-11-06T12:04:28-07:00

Back in early October, InfoQ.com published a community research article titled Top 20 Web Frameworks for the JVM. Their goal seemed to be fairly simple:

Using the new community research tool, we at InfoQ want to get YOUR opinions on the relative importance and maturity of a variety of web frameworks that are targeted for the JVM. Please vote by dragging each practice across two dimensions – how important is the framework relative to the other frameworks, and how much is it actually used in real teams and projects.

When I first saw this article, I noticed some strange web frameworks listed. Namely, Netty, SiteMesh and Spark. I haven't heard of many folks using Netty for a web framework, but I'm sure it's possible. SiteMesh is certainly not a web framework and I've never even heard of Spark. And where is GWT and Vaadin? Regardless of the choices, I went ahead and voted.

Last week, InfoQ posted their top content for October on Facebook.

Top 20 Web Frameworks for the JVM 25,992 PV
Ruby on Rails vs. Node.js at LinkedIn 11,904 PV
Mobile Webdev: The Horror 11,150 PV
A Brief Introduction to REST 8,872 PV
Kanban and Scrum - making the most of both 7,665 PV

First of all, it's interesting to see that JVM Web Frameworks is still a hot topic for developers. Whenever I do my Comparing JVM Web Frameworks talk at conferences, I always see a few jabs about "he's still doing that talk!?" Yes, it seems strange that a talk I first did in 2004 is still in high demand.

Secondly, I think InfoQ does good in showing how the frameworks ranked and showing their heatmaps. Below are their rankings from 1109 participants.

According to this research, the top 5 web frameworks for the JVM are Spring MVC, Play, Grails, JSF and Struts (I hope those surveyed meant Struts 2, not Struts 1).

In my research from last February (slide 21), I ranked them (with no particular weightings) as Grails, GWT, JRuby on Rails, Spring MVC and Vaadin. So I guess you could say I got 2 out of 5 right (Grails and Spring MVC). Not bad considering InfoQ didn't even consider GWT and Vaadin.

Another intriguing data point in this study is each frameworks' heatmap. For example, below are heatmaps for the top 4 frameworks.

Notice how Grails and Spring MVC are both hotter in the bottom right corner? It seems the community's overall opinions of these two frameworks are more aligned than JSF and Play, which a fair amount of folks rank as hyped and unimportant.

What I really like about this research is it's the community's opinions, visualized. It also confirms that some of my favorite frameworks are still on top. I don't know if JSF belongs as a top framework, however it seems a lot of folks do. I recently thought about removing it from AppFuse, but decided to keep it (at least for the next release). I hope InfoQ does more research projects like this, especially if they get their list of web frameworks right.

AppFuse News: GitHub, Hibernate Search and The Future

2012-09-25T10:42:14-06:00

It's been a while since I've written anything about AppFuse, but since the project has had quite a bit of activity lately, now seems like a good time.

GitHub
First of all, we moved the source code from java.net to GitHub way back in June. Thanks to Serge for helping with this process and making it quick and painless. For some reason, shortly after moving, we started having quite a few build issues with Bamboo. I was able to diagnose the problem as not enough memory on our server. Thankfully, Contegix was able to add another 2GB of RAM to our box and get everything back up-to-snuff.

New Committer: J. Garcia
J. Garcia has been a regular voice on the users and developers mailing list for several months. He's recently started contributing a lot of patches in JIRA and seems genuinely interested in the success of AppFuse. That's why we voted and added him as a committer. To prove this was a smart move, he recently replaced Compass with Hibernate Search and upgraded to Hibernate 4. As part of this work, he removed iBATIS support, which brings me to my next point.

The Future
In mid-August, I sent an email to the community, asking them "Anyone using iBATIS?"

I'm thinking of replacing AppFuse's Data Tier with Spring Data, especially because it has NoSQL and REST support. There's a good intro on InfoQ today:

http://www.infoq.com/articles/spring-data-intro

Does anyone see an issue with this? The lack of iBATIS support could be an issue, but I doubt it since if we wanted to continue supporting it, we should move to MyBATIS.

Everyone agreed this was a good idea and it seemed like a logical time to remove iBATIS support. In addition, I posted a roadmap I jotted down in early May. Since we've missed all the dates so far, I've removed them from the listing below. We hope to get 1-2 releases done by the end of this year, with 2.2 in the next 2-3 weeks.

2.2
Hibernate 4
Hibernate Search
Bootstrap
H5BP
2.3
AMP for all light modules
Wicket
PrimeFaces
2.4
JSR 303 (might require removing or developing client-side support)
Mockito instead of jMock/EasyMock
2.5
AMP one-to-many
Spring Data
MyBatis (if there's interest in adding it back in)
2.6
wro4j for concatenation and minimizing JS and CSS
pjax - https://github.com/defunkt/jquery-pjax
2.7
Scala example
Gradle example
Article about examples
3.0
GWT

Maven Central Statistics
To prove there's still a fair amount of folks using AppFuse, here's some statistic from Sonatype's OSS Repository Hosting Service.

From this screenshot, you can see that AppFuse artifacts are downloaded around 7,000 times per month. The following graph is even more interesting. Apparently, around 3,000 new projects are created with AppFuse archetypes each month.

The AppFuse Name
Finally, I recently discovered that ShoreTel decided to name a new product AppFuse. I guess this signifies two things: 1) it's a good name for a product and 2) someone didn't do their research before naming it. At this point, I'm not too concerned, but it is an interesting development.

Twitter's Open Source Summit: Bootstrap 2.0 Edition

2012-02-01T11:28:40-07:00

Every few months, Twitter hosts an Open Source Summit to talk about tools they're using. Since I happened to be near San Fransisco, I happily attended their latest #ossummit to learn about Bootstrap 2.0. Below are my notes from last night's event.

95% of Twitter's infrastructure is powered by open source. They hope to contributing back to open source by doing 2-3 summits per year. Without open source, there would be no Twitter. You can find a bunch of Twitter's open source contributions at twitter.github.com. They're also big fans of Apache and commit to a wide variety of projects there.

Bootstrap
Bootstrap is developed by two main guys: @mdo and @fat. Mark (@mdo) has been a designer at Twitter for 2 years. He started on the Revenue Team with ads, but has been working on redesign for last 4 months. Has been doing HTML and CSS for about 11 years. He used Notepad on Windows to build his GeoCities site.

boot·strap: simple and flexible HTML, CSS, and JavaScript for popular user interface components and interactions.

Work on Bootstrap started about a 1.5 years ago. Internal tools didn't get the proper attention they needed. They figured out there was a lot of people that wanted good looking UIs and interactions. It became Twitter Blueprint and was mostly used internally. Jacob (@fat) started shortly after first release and decided to add some JavaScript on top of it. The JavaScript for Bootstrap was originally the "Twitter Internal Toolkit" or "TIT" and was built on Moo Tools. Jacob was like "we gotta open source this, it's gonna be huge!" (he was right).

The 1.0 release supported Chrome, Safari and Firefox (everyone at Twitter was on Macs). 1.3 added cross-browser support and JavaScript plugins.

Now there's Bootstrap 2 (just released!). They rewrote all the documentation and components and removed legacy code.

So, what's new? The biggest thing is the docs. Previously had live examples, now shows live examples and why you would do something, as well as additional options. The "topbar" has been renamed to "navbar", but it's still got all the hotness. It's responsive with CSS media queries for small devices, tablets, small desktops and large desktops. This means the layout breaks at certain points and stacks elements to fit on smaller screens.

CSS: smarter defaults, better classes. In 1.4, all forms were stacked. Now they can flow horizontally. Tables are now namespaced so Bootstrap's styles don't apply to all tables. The available table, form and navigation classes are as follows:

// Tables
.table { ... }
.table-striped { ... }
.table-bordered { ... }
.table-condensed { ... }

// Forms
.form-inline { ... }
.form-search { ... }
.form-horizontal { ... }

// Nav
.nav { ... }
.nav-tabs { ... }
.nav-pills { ... }

The goals with 2.0 are consistency, simplification and future-proofing styles. With 1.4, buttons used "btn primary" and it caused problems if you wanted to have a "primary" class in your project. With 2.0, buttons and all elements are namespaced to avoid collisions (now it's .btn-primary).

After Mark finished talking about the design of Bootstrap, Jacob (@fat) started talking about Bootstrap's JavaScript. Jacob works on The Platform Team at Twitter and claims he made a lot of mistakes with 1.x. However, thanks to semantic versioning, 2.0 is a new version and he got to start over!

The biggest change in 2.0 is the use of data attributes (a.k.a. data-*). They were using them in 1.x, but not to the full potential of what they can be and should be. The first class API for Bootstrap JavaScript is data attributes (or HTML), not JavaScript.

With 1.x, you could add an anchor to close modals and alerts.

// 1.x closing modal/alerts
<a class="close" href="#">×</a>

However, if you put your alerts in your modals, you close them all when you likely only wanted to close one. 2.0 uses a "data-dismiss" attribute.

<a class="close" data-dismiss="model">×</a>

This allows you to target what you want closed (modals or alerts, etc.). You know exactly what's going to happen just by reading the code. Another example is the "href" attribute of an anchor. Rather than using "href", you can now use "data-target".

// 1.x href = target
<a href="#myModal" data-toggle="modal">Launch</a>

// 2.x data-target = target
<a data-target=".fat" data-toggle="modal">Launch</a>

If you'd rather turn off the data attribute API, or just part of it, you can do so by using the following:

// Turn off all data-api
$('body').off('.data-api')

// Turn off alert data-api
$('body).off('.alert.data-api')

2.0's JavaScript API has the same stuff, but better. You can turn off the data-api and do everything with JavaScript. They copied jQuery UI in a lot of ways (defaults, constructors, etc.). Bootstrap's JavaScript has 12 plugins. New ones include collapse, carousel and typeahead.

Customize - a new tab that lets you customize and download Bootstrap. It's basically an alternative to customizing .less files and allows you to choose components, select jQuery plugins, customize variables (colors, font-sizes, backgrounds) and download.

What does the future hold? Internationalization, improving responsiveness, more new features and bug fixes.

After both Mark and Jacob gave their talks, they talked together about Community and how great it's been. Even if you're not into writing CSS and JavaScript, they mentioned they still wanted to hear from you. To give an example of great community contributions, one guy opened 50 issues in the last 2 days.

Someone in the audience asked why they used LESS over SASS. Jacob said the main reason they use LESS is because they're good friends with the guy who invented it (Alexis). SASS turns CSS into a programming language, but they wanted to maintain the approachability of CSS, which LESS does. There's no plans to do an official SASS port, but there is talk of doing one. One advantage of the current LESS compiler is they rewrote it to have better output so it's far more readable.

NASA
After Mark and Jacob finished, there was a 5 minute break to grab beers and snacks. Then Sean Herron (@seanherron) (a.k.a. "NASA Bro") talked about Bootstrap at NASA. He actually didn't talk about Bootstrap much, except that they used it for code.NASA. He talked about NASA and how it's playing a key role in the movement towards open data, open source and open standards in our federal government. He mentioned how data.NASA was launched last August and that they helped develop OpenStack. Finally, he mentioned open.NASA, which is a collaborative approach to open, direct and transparent communication about our space program.

Hogan.js
Next up, Rob Sayre (@sayrer) talked about Hogan.js. Rob has been at Twitter for a few months, before that he wrote JavaScript at other places. Hogan.js is a compiler for Mustache templates.

Why Mustache? Because it's similar to HTML and easy to edit. You can mock data as JSON files and programmers are not required.

At Twitter, designers can do the CSS and Mustache without connecting to the backend. It has cross-language support in Ruby, Java and JavaScript. However, client-side template compilation has performance problems, especially in IE7 on a Windows XP box with 4 viruses.

So they had a few choices: work on mustache.js, or use Dust.js or Handlebars.js. The compilers are very nice for Dust.js and Handlebars.js, but they're huge. Handlebar's parser is 4000 lines. The entire Hogan.js file is 500 lines. They decided they were too large to send to the browser's of their users, so they chose to write a better compiler for Mustache.

Hogan.js's main features:

Compile on the server
Parser API
Performance

Performance is much better with Hogan.js than Mustache.js. On IE7 - Hogan is 5x faster than Mustache. On an iPhone, it's about the same (and an iPhone's browser is faster than IE7 on a decent computer). With modern browsers (Chrome 17, Safari 5 and Firefox 10), it's more than 10x faster.

Hogan.js is currently used at Twitter for Tweet embedding, the Bootstrap build process and soon, Twitter.com.

It's been awhile since I got excited about an open source project. Bootstrap has helped me a lot recently, in my Play More! mobile app, on some client projects and I'm in the process of refreshing AppFuse's UI to use it. I love how you can add a class or two to an element and all of a sudden they pop with good looks. The main problem with Bootstrap at this point is that a lot of Bootstrapped apps look the same. There's talk of adding themes in a future release to help alleviate this problem. In the meantime, there's a lot to get excited about with 2.0.

Thanks to Twitter for hosting this event and kudos to Mark and Jacob (and the community!) for such a fantastic project.

Refreshing AppFuse's UI with Twitter Bootstrap

2012-01-31T17:12:17-07:00

The last time AppFuse had an update done to its look and feel was in way back in 2006. I've done a lot of consulting since then, which has included a fair bit of page speed optimization, HTML5 development and integrating smarter CSS. It was way back in '05 when we first started looking at adding a CSS Framework to AppFuse. It was Mike Stenhouse's CSS Framework that provided the inspiration and my CSS Framework Design Contest that provided its current themes (puzzlewithstyle, andreas01 and simplicity).

Since then, a lot of CSS Frameworks have been invented, including Blueprint in 2007 and Compass in 2008. However, neither has taken the world by storm like Twitter Bootstrap. From Building Twitter Bootstrap:

A year-and-a-half ago, a small group of Twitter employees set out to improve our team’s internal analytical and administrative tools. After some early meetings around this one product, we set out with a higher ambition to create a toolkit for anyone to use within Twitter, and beyond. Thus, we set out to build a system that would help folks like us build new projects on top of it, and Bootstrap was conceived.
...
Today, it has grown to include dozens of components and has become the most popular project on GitHub with more than 13,000 watchers and 2,000 forks.

The fact that Bootstrap has become the most popular project on GitHub says a lot. For AppFuse.next, I'd like to integrate a lot of my learnings over the past few years, as well as support HTML5 and modern browsers as best we can. This means page speed optimizations, getting rid of Prototype and Scriptaculous in favor of jQuery, adding wro4j for resource optimization and integrating HTML5 Boilerplate. I've used Twitter Bootstrap for my Play More! app, as well as some recent client projects. Its excellent documentation has made it easy to use and I love the way you can simply add classes to elements to make them transform into something beautiful.

Last week, I spent a couple late nights integrating Twitter Bootstrap 2.0 into the Struts 2 and Spring MVC versions of AppFuse. The layout was pretty straightforward thanks to Scaffolding. Creating the Struts Menu Velocity template to produce dropdowns wasn't too difficult. I added class="table table-condensed" to the list screen tables, class="well form-horizontal" to forms and class="btn primary" to buttons.

I also added validation errors with the "help-inline" class. This is also where things got tricky with Struts and Spring MVC. For the form elements in Bootstrap, they recommend you use a "control-group" element that contains your label and a "controls" element. The control contains the input/select/textarea and also the error message if there is one. Here's a sample element waiting for data:

<div class="control-group">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name">
    </div>
</div>

Below is what that element should look like to display a validation error:

<div class="control-group error">
    <label for="name" class="control-label">Name</label>
    <div class="controls">
        <input type="text" id="name" name="name" value="">
        <span class="help-inline">Please enter your name.</span>
    </div>
</div>

You can see this markup is pretty easy, you just need to add an "error" class to control-group and span to show the error message. With Struts 2, this was pretty easy thanks to its customizable templates for its tags. All I had to do was create a "template/css_xhtml" directory in src/main/webapp and modify checkbox.ftl, controlfooter.ftl, controlheader-core.ftl and controlheader.ftl to match Bootstrap's conventions.

Spring MVC was a bit trickier. Since its tags don't have the concept of writing an entire control (label and field), I had to do a bit of finagling to get things to work. In the current implementation, Struts 2 forms have a single line for a control-group and its control-label and controls.

<s:textfield key="user.firstName" required="true"/>

With Spring MVC, it's a bit more complex:

<spring:bind path="user.firstName">
<fieldset class="control-group${(not empty status.errorMessage) ? ' error' : ''}">
</spring:bind>
    <appfuse:label styleClass="control-label" key="user.firstName"/>
    <div class="controls">
        <form:input path="firstName" id="firstName" maxlength="50"/>
        <form:errors path="firstName" cssClass="help-inline"/>
    </div>
</fieldset>

You could probably overcome this verbosity with Tag Files.

Figuring out if a control-group needed an error class before the input tag was rendered was probably the hardest part of this exercise. This was mostly due to Bootstrap's great documentation and useful examples (viewed by inspecting the markup). Below are some screenshots of the old screens and new ones.

Check out the full set on Flickr if you'd like a closer look.

Even though I like the look of the old UI, I can't help but think a lot of the themes are designed for blogs and content sites, not webapps. The old Wufoo forms were a lot better looking though. And if you're going to develop kick-ass webapps, you need to make them look good. Bootstrap goes a long way in doing this, but it certainly doesn't replace a good UX Designer. Bootstap simply helps you get into HTML5-land, start using CSS3 and it takes the pain out of making things work cross-browser. Its fluid layouts and responsive web design seems to work great for business applications, which I'm guessing AppFuse is used for the most.

I can't thank the Bootstrap developers enough for helping me make this all look good. With Bootstrap 2 dropping this week, I can see myself using this more and more on projects. In the near future, I'll be helping integrate Bootstrap into AppFuse's Tapestry 5 and JSF versions.

What do you think of this CSS change? Do you change your CSS and layout a fair bit when starting with AppFuse archetypes? What can we do to make AppFuse apps look better out-of-the-box?

Update: I updated AppFuse to the final Bootstrap 2.0 release. Also, Johannes Geppert wrote a Struts 2 Bootstrap Plugin. I hope to integrate this into AppFuse in the near future.

2011 - A Year in Review

2012-01-11T09:45:20-07:00

2010 was an awesome year, but 2011 rocked the house! At the end of last year, I kept my goals simple:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus. Everything else is gravy.

As predicted, it was a spectacular year, but I only accomplished 50% of my goals. That is, Trish and I had a great time skiing (especially in Utah), moving in together, traveling the world and getting engaged in Versailles. I even satisfied some 5-year goals: building a sauna in my basement and getting a ski shack in the mountains.

However, I didn't get much done with The Bus. Or rather, the guys at MotorWorks Restorations didn't drain me for all I'm worth in 2011. We did make good progress with estimating the final cost and obtaining many hard-to-find parts though. I now have a Porsche 911 Engine (1983 3.0L 6 cylinder), a Porsche 901 5 speed transmission, Porsche "Turbo Twist" wheels and a Custom Air Ride Front Beam from Franklin's VW Works. The thing that slowed our progress the most was the custom beam, as it took almost 6 months from order to delivery. When it arrived in September, I decided to put things on hold. I didn't want to get my bus back in the midst of winter and not be able to drive it.

In January, I started working at Overstock.com and flew out to Salt Lake City the first two weeks to ramp up. In setting up my development environment, I discovered some inefficiencies and wrote about how I fixed them in Making Code Generation Smarter with Maven. I also discovered that JRebel slowed my build process down by almost 50% and stopped using it as part of my daily process. Now I simply run in debug mode or use fancy frameworks that allow instant reload.

At the end of the month, I skied Alta for the first time on a blue bird day. That night, I flew back to Denver, had a gorgeous lunch ride in 70°F weather on Friday afternoon, then picked up my lovely girlfriend, Trish, and drove to Crested Butte for a weekend of skiing with James Ward and friends. It was my first time visiting Crested Butte and I can see why people call it Paradise.

As of February 1st, Trish and I both had 17 ski days. As I'm writing this, we're both up to 14 for this season and very proud to say Abbie and Jack each have 10. Both kids are shredding like I've always dreamed they would.

My parents (both retired) came to visit for the month of February and helped me start building a sauna in my basement. This was something I've wanted to do for years, so it was a lot of fun getting things started. While we didn't finish construction, we did get the plumbing/electrical done, stove installed and walls framed. We also had a fun night with Chef Navarro and Friends.

The rest of my posts in February were purely technical. I wrote about how I implemented extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF. Then I started delving into security-related topics, largely motivated by a new feature I was working on at Overstock. When I wrote about Implementing Ajax Authentication using jQuery, Spring Security and HTTPS, I explained what I was trying to accomplish.

The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience.

I was able to get everything working, posted a demo and wrote about upcoming conferences in Las Vegas and Kraków. In that article, I described a new talk I was going to create: Webapp Security: Develop. Penetrate. Protect. Relax. I ended the month with a post on how to fix XSS in JSP 2.

On February 25th, I experienced The Greatest Snow on Earth with thigh-deep powder and face shots on every run.

That night, we saw Hot Buttered Rum and woke up early for 27" of fresh powder at Alta. The picture of Trish free-heeling at Alta is one of my favorites of the year.

In early March, we experienced a suite night at the Avs game, then I kept it technical and upgraded this blog to Apache Roller 5.0, RC4. I upgraded AppFuse to JSF 2, wrote about JSR 303 and web framework support and added search to AppFuse using Compass. When writing about JSF 2, I mentioned replacing Ajax4JSF with RichFaces. 9 months later, it seems like PrimeFaces has more momentum, so I may try that instead.

For St. Patrick's Day, we flew to Vegas for TSSJS and had a blast. Not only did my talks go well, but we had a bunch of friends in town and had much success gambling the nights away.

After delivering my Comparing JVM Web Frameworks talk, I had a very interesting conversation with James Strachan. He recommended I take a look at CoffeeScript and Jade. A couple months later, when I was thinking of talks for Devoxx, I added these to my list of technologies to learn and submitted a talk about HTML5 with Play Scala, CoffeeScript and Jade.

That Friday, we returned to Denver, got a good night's sleep then headed on a hut trip in Colorado's High Country. The name of the hut was Peter Estin Hut and it was a bit of a hike to get to. My friend, Joe, set my expectations correctly when he warned me it'd be a 5 hour death march. It took us 4 hours, 30 minutes and we skied up 2200 vertical feet of switchbacks.

Trish and I started off April with a bang by moving in together, then embarking on a whirlwind trip of two opening days with a stopover in Kraków. We flew Business Class from Chicago to Munich (releasing AppFuse 2.1 just before departing) and were very impressed with Kraków's Old Town upon arrival.

On that trip, I especially enjoyed meeting Trish's brother for the first time and watching the Red Sox win their first game of the year.

After returning, I realized I lost a lot of data in my Roller 5 upgrade and wrote about what happened and how I recovered it. Then we bid farewell to the 2010-2011 ski season and got ready for mountain bike season.

For next year, I think I'll keep my goal at 30 days. If everything works out as planned, we'll have a place in the mountains this fall and it'll be a bit easier to hit the slopes without sitting in traffic. For now, I'm pumped about the beginning of mountain bike season.

May arrived and Trish and I started looking for a place in the mountains. We found an very cool cabin, but later decided it was too nice and we'd rather have a condo. I finished The Basement Sauna Project and wrote about Java Web Application Security: Java EE 6, Spring Security and Apache Shiro.

On May 21st, we again visited Winter Park and Fraser to look at condos, then drove back to Denver for U2 at Mile High Stadium. I'm not a big fan of U2, but this was quite possibly the best concert I've ever been to. The experience of 70,000+ fans, all screaming, doing the wave, and loving life was epic. We decided that night that we liked one of the condos more than the rest and put in an offer the following week. The kids headed off to Florida for a month with their Mom and we drove to Utah for Desert Rocks and mountain biking in Moab.

I Upgraded to Roller 5.0 in June and showed how to add a Facebook Like button. Next, I transcribed a couple more Java Web Application Security articles: programmatic login APIs and penetrating with Zed Attack Proxy.

The kids returned to us just before the annual Father's Day Camping Trip. We drove to Lake Granby and stayed at Stillwater Campground for the weekend.

From my Father's Day post:

I really enjoyed spending so much time with my family this weekend. The good news is we get to do it again when I meet them at The Cabin in Montana next week. In the meantime, Trish and I are off to Jazoon for a few days of fun in Switzerland.

A few days later, we were in Zurich for the first time, enjoying our accommodations and marveling at Switzerland's excellent transportation system. I experienced an interesting situation while speaking, but quickly forgot when we visited Rhine Falls, celebrated our anniversary at Gruelich and played in the Swiss Alps.

We arrived back in Denver at 6pm on Sunday, June 26th and I flew to Montana at 8am the next morning. I worked from The Cabin that week and wrote a book review for Agile Hiring.

I think Sean [Landis] does a great job in showing how you can apply agile principles (people over process, tracking tools, failing fast, continuous improvement, constant feedback) to improve your hiring process. As a person that interviews often, I think it also gives great insights into how companies interview and what they're looking for. I've often thought that being honest about my skills and what I'm looking for is a good tactic and this book seems to confirm that.

That weekend, it was the "90s Reunion" for the Seeley Swan High School, which I attended my freshman and sophomore year of high school. There were several highlights of the trip, but one of my favorites was our float in the Swan Valley Parade. Abbie came up with the idea (Happy Birthday America) and we decorated an old trailer with streamers, balloons and a birthday cake to celebrate. Abbie made up a song and sang it throughout the entire parade (with a portable microphone and amp).

I went swimming with my iPhone 4 that Sunday and Trish and her Dad showed up a few hours later. On the 4th, we had a blast celebrating the holiday with mine and my parents' old friends. The next day, we adventured to Glacier National Park, where Trish shot an awesome sunrise at Lake McDonald. She also captured a beautiful picture of The Cabin at sunset that week.

In between vacationing in Montana and writing about it, I wrote some technical posts:

We returned to Denver in mid-July and shipped the kids off to Florida just before my annual Birthday Golf Debacle. We teed off at Wellshire Golf Course and had one of the most well-behaved rounds in a few years. The next weekend, we closed on our "Ski Shack" in Fraser and I penned Another Dream Realized: Mountain Views. Trish's October Sunset photo is one of my favorite glimpses of our view.

Next, I blogged about How do you get started in programming? and we headed to the airport a few hours later for a whirlwind trip from Oregon to Cape Cod. That was a wonderful trip and it took me almost a month to write about it. We flew into Portland and spent the weekend hiking in Silver Falls and drinking good beer at the Oregon Brewers Festival and Edgefield. We stayed with my good friends Clint and Autumn (married in Costa Rica in 2008), who had their first child only a few weeks after our visit (August 26th).

We had an awful early flight out of Portland to Boston that Sunday. On Monday afternoon, we picked up Abbie and Jack from the Boston airport. They flew up from West Palm Beach on their first unaccompanied minors flight. We drove them to Cape Cod and settled into Trish's parents house near the water. The rest of the week, we had a blast with our friends Chris and Julie and the entire McGinity clan.

We spent the rest of August going to concerts at Red Rocks, beer festivals in Winter Park, mountain biking, hiking, taking the kids to the Ski Shack and playing with Crazy Sagan. In my only August post, I talked a lot about the Broncos, Hawaii, Devoxx and leaving Overstock.

That pretty much catches you up-to-date with the goings on in my life. As you can tell, it's been a great summer so far. I'm really looking forward to the fall too. Broncos season, a trip to Hawaii, learning new technologies, talking about them at Devoxx and a new gig.
...
Tomorrow is Jack's birthday. I gotta run ... it's time to go buy a bike, play a little golf, hit the pool and go to the BBQ before tonight's CD Release Party.

We watched proudly as Jack turned 7 the next day and loved his excitement when he got his first mountain bike.

September started with mountain breezes and fishing at Lake Granby. Lots of friends, family and delicious food make for a heckuva 3-day weekend. Jack was the only one to catch a fish and you couldn't wipe the smile off his face for the rest of the day.

I described my Digital Diet and wrote my first article related to my Devoxx talk: Integrating Scalate and Jade with Play 1.2.3. The 2nd weekend in September, we enjoyed a September Sports Weekend, going to two Rockies games and the Monday night Broncos Opener before heading to Kauai. While the Broncos didn't win that game, Trish's shot of Brett, Joe and I sums up many-a-game this season.

As expected, Kauai blew our minds. We saw several spectacular sunsets, did some Stand Up Paddling (it's harder than it looks), played golf, took a doorless helicopter ride and kayaked along the Na Pali coast.

We never got bored of the view from our room.

Upon our return, I tried to make CoffeeScript work with Scalate and integrated HTML5 Boilerplate with Play.

October was a quiet month on this blog. I only wrote two articles, one about increasing my developer happiness and another about developing with HTML5, CoffeeScript and Twitter's Bootstrap. Watching the Broncos in October was painful at first, buy many rejoiced when Tebow started against Miami (10/23) and we had the amazing comeback to win. We went to the next game against the Lions and were quickly shown they had a long ways to go. We dressed up as the Scary Family for Halloween and had a ball.

I made up for October's lack of writing by writing 9 articles in November. I started with complaints about Play Scala's Anorm and PostgreSQL, then celebrated Abbie's 9th Birthday with a Magic Show.

The next week, I wrote about my new MacBook Pro and iMac and more Scalate goodness for Play. At this point, I was scrambling to finish up my Play More! application for Devoxx and worked a lot of late nights. That weekend, I used PhoneGap to turn my webapp into a native app and finished everything in the nick of time. We flew to Paris that Monday, took the train to Antwerp on Tuesday and settled in for a memorable week of interesting talks.

That Saturday, we took the train back to Paris and checked into our hotel for a few days of romance. I proposed to Trish at Versailles on Sunday at sunset and we rolled around in our joy for a few days before we told any of our friends.

We flew back to the US the Wednesday before Christmas and spent the weekend in Boston with Trish's family. For full details of that trip, see Our Engaging Trip to Paris and Antwerp.

To end the year, I kept it light and only spun up one blog post in December: What have I been working on at Taleo? A week after writing that post, we moved to the Ski Shack for 3 weeks and enjoyed a wonderful holiday break with many visitors.

Yes, 2011 was a spectacular year. We traveled to four different countries, did a lot of camping and mountain biking and accomplished 2/3 of my 5 year goals (basement sauna and mountain views). I wrote a lot of technical blog posts in the 2nd half of the year; most inspired my extracurricular activities. I plan to continue that this year as I learn more about HTML5, Play Framework, Scala and CoffeeScript. At the end of last year's Year in Review, I wrote:

In 2011, I plan on doing two main things: keep rockin' it with Trish and finishing The Bus.

I think I can safely say I kept it rockin' with Trish - we're living together, engaged and still amazed we found one another. As for The Bus, I think MotorWorks Restorations and I are in a good position to make a final push and finish the sucker. My hope is to have it done by May so I can start driving it and taking it to VW Shows. In the meantime, Trish has started working on McGinity Photo full-time and loving it. We have trips planned to Stockholm (for Jfokus), Madrid (for Spring I/O) and Paris (for Devoxx Paris).

Last weekend, we witnessed one of the greatest playoff games ever when the Broncos beat the Steelers. I think the picture of Trish celebrating the final touchdown pretty much sums up 2011.

I look forward to all the skiing, celebrating, camping, traveling and bus rides in 2012. It's gonna be a fantastic year.

Upgrading AppFuse to Spring Security 3.1 and Spring 3.1

2012-01-05T08:58:21-07:00

Before the holiday break, I spent some time upgrading AppFuse to use the latest releases of Spring and Spring Security. I started with Spring Security in early December and quickly discovered its 3.1 XSD required some changes. After changing to the 3.1 XSD in my security.xml, I had to change its <http> element to use security="none" instead of filters="none". With Spring Security 3.0.5, I had:

<http auto-config="true" lowercase-comparisons="false">
    <intercept-url pattern="/images/**" filters="none"/>
    <intercept-url pattern="/styles/**" filters="none"/>
    <intercept-url pattern="/scripts/**" filters="none"/>

After upgrading to 3.1, I had to change this to:

<http pattern="/images/**" security="none"/>
<http pattern="/styles/**" security="none"/>
<http pattern="/scripts/**" security="none"/>

<http auto-config="true">

The next thing I had to change was UserSecurityAdvice.java. Instead of using Collection<GrantedAuthority> for Authentication's getAuthority() method, I had to change it to use Collection<? extends GrantedAuthority>.

Authentication auth = ctx.getAuthentication();
Collection<? extends GrantedAuthority> roles = auth.getAuthorities();

Lastly, I discovered that SPRING_SECURITY_CONTEXT_KEY moved to HttpSessionSecurityContextRepository. Click here to see the changelog for this upgrade in AppFuse's FishEye.

You can read more about what's new in Spring Security 3.1 on InfoQ. I'm especially pumped to see http-only cookie support for Servlet 3.0. I discovered Spring Security didn't support this when Pen-Testing with Zed Attack Proxy.

Upgrading to Spring Framework 3.1
Compared to the Spring Security upgrade, upgrading to Spring 3.1 was a breeze. The first thing I discovered after changing my pom.xml's version was that Spring Security required some additional exclusions in order to get the latest Spring versions. Of course, this was communicated to me through the following cryptic error.

-------------------------------------------------------------------------------
Test set: org.appfuse.dao.LookupDaoTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.004 sec <<< FAILURE!
testGetRoles(org.appfuse.dao.LookupDaoTest)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.NoSuchMethodError: org.springframework.context.support.GenericApplicationContext.getEnvironment()Lorg/springframework/core/env/ConfigurableEnvironment;
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:97)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:1)
	at org.springframework.test.context.support.DelegatingSmartContextLoader.loadContext(DelegatingSmartContextLoader.java:228)
	at org.springframework.test.context.TestContext.loadApplicationContext(TestContext.java:124)
	at org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:148)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:109)

Without these additional exclusions, Spring Security pulled in Spring 3.0.6. I had to exclude spring-expression, spring-context and spring-web from spring-security-taglibs to get the 3.1.0.RELEASE version of Spring.

<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.security.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-expression</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
    </exclusions>
</dependency>

I also had to exclude spring-context from spring-security-config and spring-context and spring-expression from spring-security-core. Isn't Maven wonderful?

<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-core</artifactId>
    <version>${spring.security.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-expression</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-config</artifactId>
    <version>${spring.security.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
    </exclusions>
</dependency>

After making these changes, I got a bit further, but ended up being blocked by a bug in Tapestry 5's Spring support. Basically, after upgrading to Spring 3.1, I started seeing the following error:

java.lang.RuntimeException: Service id 'environment' has already been defined by 
org.apache.tapestry5.services.TapestryModule.buildEnvironment(PerthreadManager)

Luckily, I was able to easily fix this with advice I found on Tapestry's mailing list. Unfortunately, even though I submitted a fix on December 15th, it didn't make it into Tapestry's 5.3.1 release on December 21st. As soon as Tapestry 5.3.2 is released, I hope to get the AppFuse's build passing again (it's currently failing).

I hope this article helps you upgrade your AppFuse-started applications to the latest versions of Spring and Spring Security. Over the next few weeks, I'll be exploring many of Spring 3.1's new features and implementing them as I see fit. Right now, I'm thinking environments/profiles, Servlet 3 / Java 7 support and Hibernate 4 support. These seem to be the best new features to learn about for my talk in a few weeks.

Installing OpenJDK 7 on OS X

2011-07-12T14:11:44-06:00

Last week, I scanned an article and saw there was a Java 7 Webinar. At first, I thought Java 7 was released, but soon after realized it was a Developer Preview. Unfortunately, the download page doesn't have support for OS X. Since it took me a bit of work to figure out how to install OpenJDK 7 on OS X (I'm running Snow Leopard 10.6.7), I figured I'd write down how I did it.

I started off by downloading "OpenJDK 1.7 universal (32/64 bits) from Mac OS/X branch" from the openjdk-osx-build project's downloads (direct link). After downloading, I installed the dmg as normal.

Update Jan 27, 2012:
After installing the dmg, add the following to your ~/.profile and you should be good to go. Thanks to Mark Beaty for the tip.

function setjdk() { if [ $# -ne 0 ];then export JAVA_HOME=`/usr/libexec/java_home -v $@`; fi; java -version; }

Continue with the instructions below if you don't like this technique for some reason.

I don't use Java Preferences to set my JDK, instead I use David Blevin's handy setjdk script. To make this script work with JDK 7 on OS X, I had to make one minor change. On line 40, I added "Contents" to the path for JAVA_HOME:

export JAVA_HOME=$vmdir/$ver/Contents/Home

Update Jan 27, 2012: You no longer need to make this change.

From there, I had to setup some symlinks so everything would work as expected:

cd /System/Library/Java/JavaVirtualMachines/
sudo ln -s /Library/Java/JavaVirtualMachines/1.7.0.jdk

Update Jan 27, 2012: The latest version installs at a different location so the symlink command above should be changed to:

sudo ln -s /Library/Java/JavaVirtualMachines/1.7.0u.jdk 1.7.0.jdk

Lastly, I had my JAVA_HOME set to "/System/Library/Frameworks/JavaVM.framework/Home". I like the shorter (and seemingly more common) "/Library/Java/Home", so I set it back to that in my ~/.profile:

export JAVA_HOME=/Library/Java/Home

On my system, /Library/Java/Home had a symlink to /System/Library/Frameworks/JavaVM.framework/Home, so I changed it to the CurrentJDK that Java Preferences and setjdk use.

cd /Library/Java
rm Home
ln -s /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Contents/Home

Then I had to add a symlink for 1.7 in the Versions directory.

cd /System/Library/Frameworks/JavaVM.framework/Versions
sudo ln -s /System/Library/Java/JavaVirtualMachines/1.7.0.jdk/Contents 1.7

After making these changes, I was able to switch to JDK 7 easily.

$ setjdk 1.7
Setting this terminal's JDK to 1.7 ... openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-b00)
OpenJDK 64-Bit Server VM (build 21.0-b17, mixed mode)

I was also able to switch back to JDK 6.

$ setjdk 1.6
Setting this terminal's JDK to 1.6 ... java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-384, mixed mode)

Maven Issues
Next, I tried using JDK 7 to build AppFuse. I ran into two issues when I tried to do this. The first was caused by the native2ascii plugin, which has been known to cause issues on non-Mac platforms. Adding the following profile seemed to solve the problem.

<profile>
    <activation>
        <jdk>1.7</jdk>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>native2ascii-maven-plugin</artifactId>
                <dependencies>
                    <dependency>
                      <groupId>com.sun</groupId>
                      <artifactId>tools</artifactId>
                      <version>1.7.0</version>
                      <scope>system</scope>
                      <systemPath>${java.home}/../lib/tools.jar</systemPath>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</profile>

The next issue was with Enunciate and its maven-enunciate-cxf-plugin.

[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] com/sun/mirror/apt/AnnotationProcessorFactory
com.sun.mirror.apt.AnnotationProcessorFactory
[INFO] ------------------------------------------------------------------------
[INFO] Trace
java.lang.NoClassDefFoundError: com/sun/mirror/apt/AnnotationProcessorFactory

It seemed like adding a profile that included tools.jar would solve this, but it doesn't. When I add the dependency directly to the plugin itself, I get the following error:

warning: The apt tool and its associated API are planned to be
removed in the next major JDK release.  These features have been
superseded by javac and the standardized annotation processing API,
javax.annotation.processing and javax.lang.model.  Users are
recommended to migrate to the annotation processing features of
javac; see the javac man page for more information.
[WARNING] Validation result has errors.
error: [core] java.lang.StackTraceElement: A TypeDefinition must have a public no-arg constructor or be annotated with a factory method.
1 error
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------

Hopefully this article helps you get started with Java 7 on OS X. If you have any additional tips, please leave a comment.

Integrating OAuth with AppFuse and its REST API

2011-07-05T10:56:48-06:00

One of the new features in AppFuse 2.1 is an appfuse-ws archetype. This archetype leverages Enunciate and CXF to create a project with a REST API and generated HTML documentation. Enunciate is a very useful tool, allowing you to develop web services with JAX-RS and JAX-WS annotations and have all types of client libraries generated. For me, it seems very useful for developing the backend of SOFEA (a.k.a. modern) applications.

Back in March, Ryan Heaton published a nice article on Securing Web Services in an Enunciate application. I decided to take his tutorial a step further and not only secure my web services, but also to integrate with OAuth 2. In this tutorial, I'll show you how to create a new application with AppFuse WS, secure it, add OAuth support, and then use a client app to authenticate and retrieve data.

Create a New AppFuse WS Project
Integrate Spring Security and OAuth
Authenticate and Retrieve Data with Client

Create a New AppFuse WS Project
To begin, I visited the Create AppFuse Archetypes page and created a new application using the "Web Services Only" option in the Web Framework dropdown. Below is the command I used to create the "appfuse-oauth" project.

mvn archetype:generate -B -DarchetypeGroupId=org.appfuse.archetypes \
-DarchetypeArtifactId=appfuse-ws-archetype -DarchetypeVersion=2.1.0 \
-DgroupId=org.appfuse.example -DartifactId=appfuse-oauth

After doing this, I started the app using mvn jetty:run and confirmed it started OK. At this point, I was able to view the generated documentation for the application at http://localhost:8080. The screenshot below shows what the app looks like at this point.

NOTE: You might notice the REST endpoint of /{username}. This is a bug in AppFuse 2.1.0 and has been fixed in SVN. It does not affect this tutorial.

Integrate Spring Security and OAuth
I originally tried to integrate Spring Security with Enunciate's Securing Web Services Tutorial. However, it only secures endpoints and doesn't do enough filtering for OAuth support, so I ended up using a custom web.xml. I put this file in src/main/resources and loaded it in my enunciate.xml file. I also upgraded Spring Security and imported my security.xml file.

  <?xml version="1.0"?>
  <enunciate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:noNamespaceSchemaLocation="http://enunciate.codehaus.org/schemas/enunciate-1.22.xsd">
      ...
      <webapp mergeWebXML="src/main/resources/web.xml"/>
      <modules>
      ...
          <spring-app disabled="false" springVersion="3.0.5.RELEASE">
              <springImport uri="classpath:/applicationContext-resources.xml"/>
              <springImport uri="classpath:/applicationContext-dao.xml"/>
              <springImport uri="classpath:/applicationContext-service.xml"/>
              <springImport uri="classpath:/applicationContext.xml"/>
              <springImport uri="classpath:/security.xml"/>
          </spring-app>
      </modules>
  </enunciate>

Then I created src/main/resources/web.xml with a filter for Spring Security and a DispatcherServlet for OAuth support.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

    <filter>
        <filter-name>securityFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>springSecurityFilterChain</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>securityFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>appfuse-oauth</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>appfuse-oauth</servlet-name>
        <url-pattern>/oauth/*</url-pattern>
    </servlet-mapping>
</web-app>

Next, I created a src/main/resources/security.xml and used it to secure my API, specify a login page, supply the users and integrate OAuth (see the last 4 beans below).

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security.xsd
                           http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

    <http auto-config="true">
        <intercept-url pattern="/api/**" access="ROLE_USER"/>
        <intercept-url pattern="/oauth/**" access="ROLE_USER"/>
        <intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
        <form-login login-page="/login.jsp" authentication-failure-url="/login.jsp?error=true"
                    login-processing-url="/j_security_check"/>
    </http>

    <authentication-manager>
        <authentication-provider>
            <user-service>
                <user name="admin" password="admin" authorities="ROLE_USER,ROLE_ADMIN"/>
                <user name="user" password="user" authorities="ROLE_USER"/>
            </user-service>
        </authentication-provider>
    </authentication-manager>

    <!--hook up the spring security filter chain-->
    <beans:alias name="springSecurityFilterChain" alias="securityFilter"/>

    <beans:bean id="tokenServices"
                class="org.springframework.security.oauth2.provider.token.InMemoryOAuth2ProviderTokenServices">
        <beans:property name="supportRefreshToken" value="true"/>
    </beans:bean>

    <oauth:provider client-details-service-ref="clientDetails" token-services-ref="tokenServices">
        <oauth:verification-code user-approval-page="/oauth/confirm_access"/>
    </oauth:provider>

    <oauth:client-details-service id="clientDetails">
        <!--<oauth:client clientId="my-trusted-client" authorizedGrantTypes="password,authorization_code,refresh_token"/>
        <oauth:client clientId="my-trusted-client-with-secret"
                      authorizedGrantTypes="password,authorization_code,refresh_token" secret="somesecret"/>
        <oauth:client clientId="my-less-trusted-client" authorizedGrantTypes="authorization_code"/>-->
        <oauth:client clientId="ajax-login" authorizedGrantTypes="authorization_code"/>
    </oauth:client-details-service>
</beans:beans>

I used the OAuth for Spring Security sample apps to figure this out. In this example, I used authorizedGrantTypes="authorization_code", but you can see from the commented <oauth:client> elements above that there's a few different options. You should also note that the clientId is hard-coded to "ajax-login", signifying I only want to allow a single application to authenticate.

At this point, I'd like to give a shoutout to Ryan Heaton for creating both Enunciate and Spring Security's OAuth support. Nice work Ryan!

At this point, I needed to do a number of additional tasks to finish integrating oauth. The first was to modify the Jetty Plugin's configuration to 1) run on port 9000, 2) load my custom files and 3) allow jetty:run to recognize Enunciate's generated files. Below is the final configuration in my pom.xml.

<plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>maven-jetty-plugin</artifactId>
    <version>6.1.26</version>
    <configuration>
        <connectors>
            <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <port>9000</port>
                <maxIdleTime>60000</maxIdleTime>
            </connector>
        </connectors>
        <webAppConfig>
            <baseResource implementation="org.mortbay.resource.ResourceCollection">
                <resourcesAsCSV>
                    ${basedir}/src/main/webapp,
                    ${project.build.directory}/${project.build.finalName}
                </resourcesAsCSV>
            </baseResource>
            <contextPath>/appfuse-oauth</contextPath>
        </webAppConfig>
        <webXml>${project.build.directory}/${project.build.finalName}/WEB-INF/web.xml</webXml>
    </configuration>
</plugin>

Next, I added the necessary OAuth dependencies for Spring Security to my pom.xml. Since the latest release is a milestone release, I had to add Spring's milestone repo too.

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>${spring.version}</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.1.2</version>
</dependency>
<dependency>
    <groupId>taglibs</groupId>
    <artifactId>standard</artifactId>
    <version>1.1.2</version>
</dependency>

Since I named my DispatcherServlet "appfuse-oauth" in web.xml, I created a src/main/webapp/WEB-INF/appfuse-oauth-servlet.xml to configure Spring MVC. I had to create the src/main/webapp/WEB-INF directory.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

    <!-- Scans the classpath of this application for @Components to deploy as beans -->
    <context:component-scan base-package="org.appfuse.examples.webapp"/>

    <!-- Configures the @Controller programming model -->
    <mvc:annotation-driven/>

    <!-- Resolves view names to protected .jsp resources within the /WEB-INF/views directory -->
    <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
        <property name="prefix" value="/"/>
        <property name="suffix" value=".jsp"/>
    </bean>
</beans>

In order to show the OAuth confirmation page, I needed to create src/main/java/org/appfuse/examples/webapp/AccessConfirmationController.java and map it to /oauth/confirm_access. I copied this from one of the sample projects and modified to use Spring's annotations.

package org.appfuse.examples.webapp;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.oauth2.provider.ClientAuthenticationToken;
import org.springframework.security.oauth2.provider.ClientDetails;
import org.springframework.security.oauth2.provider.ClientDetailsService;
import org.springframework.security.oauth2.provider.verification.ClientAuthenticationCache;
import org.springframework.security.oauth2.provider.verification.DefaultClientAuthenticationCache;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.TreeMap;

/**
 * Controller for retrieving the model for and displaying the confirmation page
 * for access to a protected resource.
 *
 * @author Ryan Heaton
 */
@Controller
@RequestMapping("/confirm_access")
public class AccessConfirmationController {

    private ClientAuthenticationCache authenticationCache = new DefaultClientAuthenticationCache();
    @Autowired
    private ClientDetailsService clientDetailsService;

    @RequestMapping(method = RequestMethod.GET)
    protected ModelAndView confirm(HttpServletRequest request, HttpServletResponse response) throws Exception {
        ClientAuthenticationToken clientAuth = authenticationCache.getAuthentication(request, response);
        if (clientAuth == null) {
            throw new IllegalStateException("No client authentication request to authorize.");
        }

        TreeMap<String, Object> model = new TreeMap<String, Object>();
        ClientDetails client = clientDetailsService.loadClientByClientId(clientAuth.getClientId());
        model.put("auth_request", clientAuth);
        model.put("client", client);

        return new ModelAndView("access_confirmation", model);
    }
}

This controller delegates to src/main/webapp/access_confirmation.jsp. I created this file and filled it with code to display Accept and Deny buttons.

<%@ page import="org.springframework.security.core.AuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.common.exceptions.UnapprovedClientAuthenticationException" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.BasicUserApprovalFilter" %>
<%@ page import="org.springframework.security.oauth2.provider.verification.VerificationCodeFilter" %>
<%@ page import="org.springframework.security.web.WebAttributes" %>
<%@ taglib prefix="authz" uri="http://www.springframework.org/security/tags" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<html>
<head>
    <title>Confirm Access</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>

<body>

<h1>Confirm Access</h1>

<div id="content">

    <% if (session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) != null && 
                 !(session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION) instanceof UnapprovedClientAuthenticationException)) { %>
    <div class="error">
        <h2>Woops!</h2>

        <p>Access could not be granted.
            (<%= ((AuthenticationException) session.getAttribute(WebAttributes.AUTHENTICATION_EXCEPTION)).getMessage() %>)</p>
    </div>
    <% } %>
    <c:remove scope="session" var="SPRING_SECURITY_LAST_EXCEPTION"/>

    <authz:authorize ifAnyGranted="ROLE_USER,ROLE_ADMIN">
        <h2>Please Confirm</h2>

        <p>You hereby authorize "<c:out value="${client.clientId}" escapeXml="true"/>" to access your protected resources.</p>

        <form id="confirmationForm" name="confirmationForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="authorize" value="Authorize" type="submit"></label>
        </form>
        <form id="denialForm" name="denialForm"
              action="<%=request.getContextPath() + VerificationCodeFilter.DEFAULT_PROCESSING_URL%>" method="POST">
            <input name="<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_REQUEST_PARAMETER%>"
                   value="not_<%=BasicUserApprovalFilter.DEFAULT_APPROVAL_PARAMETER_VALUE%>" type="hidden"/>
            <label><input name="deny" value="Deny" type="submit"></label>
        </form>
    </authz:authorize>
</div>
</body>
</html>

Finally, I needed to create src/main/webapp/login.jsp to allow users to login.

<%@ page language="java" pageEncoding="UTF-8" contentType="text/html;charset=utf-8" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt" %>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>

<html>
<head>
    <title>Login</title>
    <link rel="stylesheet" type="text/css" media="all"
          href="http://demo.appfuse.org/appfuse-struts/styles/simplicity/theme.css"/>
    <style type="text/css">
        h1 {
            margin-left: -300px;
            margin-top: 50px
        }
    </style>
</head>
<body>
<h1>Login</h1>

<form method="post" id="loginForm" action="<c:url value='/j_security_check'/>">
    <fieldset style="padding-bottom: 0">
        <ul>
            <c:if test="${param.error != null}">
                <li class="error">
                    ${sessionScope.SPRING_SECURITY_LAST_EXCEPTION.message}
                </li>
            </c:if>
            <li>
                <label for="j_username" class="required desc">
                    Username <span class="req">*</span>
                </label>
                <input type="text" class="text medium" name="j_username"
                       id="j_username" tabindex="1"/>
            </li>

            <li>
                <label for="j_password" class="required desc">
                    Password <span class="req">*</span>
                </label>
                <input type="password" class="text medium" name="j_password"
                       id="j_password" tabindex="2"/>
            </li>
            <li>
                <input type="submit" class="button" name="login" value="Login"
                       tabindex="3"/>
            </li>
        </ul>
    </fieldset>
</form>
</body>
</html>

All the changes described in the above section are necessary to implement OAuth if you create a project with AppFuse WS 2.1. It may seem like a lot of code, but I was able to copy/paste and get it all working in an app in under 5 minutes. Hopefully you can do the same. I'm also considering adding it by default to the next version of AppFuse. Now let's look at integrating OAuth into a client to authenticate and retrieve data from this application.

Authenticate and Retrieve Data with Client
I originally thought my GWT OAuth application would provide a nice client. However, after 30 minutes of trying to get GWT 1.7.1 and the GWT Maven plugin (1.1) working with my 64-bit Java 6 JDK on OS X, I gave up. So I opted to use the Ajax Login application I've been using in my recent security tutorials.

In this example, I used OAuth2RestTemplate from Spring Security OAuth. While this works, and works well, I'd still like to get things working with GWT (or jQuery) to demonstrate how to do it from a pure client-side perspective.

To begin, I got the latest source of Ajax Login from GitHub (as of this morning) and made some changes. First of all, I added the Spring Security OAuth dependencies to pom.xml:

<repository>
    <id>spring-milestone</id>
    <url>http://s3.amazonaws.com/maven.springframework.org/milestone</url>
</repository>
...
<dependency>
    <groupId>org.springframework.security.oauth</groupId>
    <artifactId>spring-security-oauth</artifactId>
    <version>1.0.0.M3</version>
    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Then I modified src/main/webapp/WEB-INF/security.xml and added an OAuth Token Service and defined the location of the OAuth server.

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/security"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xmlns:beans="http://www.springframework.org/schema/beans"
             xmlns:oauth="http://www.springframework.org/schema/security/oauth2"
             xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
              http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
              http://www.springframework.org/schema/security/oauth2 http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

...
    <oauth:client token-services-ref="oauth2TokenServices"/>

    <beans:bean id="oauth2TokenServices"
                class="org.springframework.security.oauth2.consumer.token.InMemoryOAuth2ClientTokenServices"/>

    <oauth:resource id="appfuse" type="authorization_code" clientId="ajax-login"
                    accessTokenUri="http://localhost:9000/appfuse-oauth/oauth/authorize"
                    userAuthorizationUri="http://localhost:9000/appfuse-oauth/oauth/user/authorize"/>

Next, I created a Controller that uses OAuth2RestTemplate to make the request and get the data from the AppFuse OAuth application's API. I created src/main/java/org/appfuse/examples/webapp/oauth/UsersApiController.java and filled it with the following code:

package org.appfuse.examples.webapp.oauth;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.oauth2.common.exceptions.InvalidTokenException;
import org.springframework.security.oauth2.consumer.*;
import org.springframework.security.oauth2.consumer.token.OAuth2ClientTokenServices;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;

import java.util.ArrayList;
import java.util.List;

@RequestMapping("/appfuse/users")
@Controller
public class UsersApiController {

    private OAuth2RestTemplate apiRestTemplate;
    @Autowired
    private OAuth2ClientTokenServices tokenServices;

    private static final String REMOTE_DATA_URL = "http://localhost:9000/appfuse-oauth/api/users";

    @Autowired
    public UsersApiController(OAuth2ProtectedResourceDetails resourceDetails) {
        this.apiRestTemplate = new OAuth2RestTemplate(resourceDetails);
    }

    @RequestMapping(method = RequestMethod.GET)
    @ResponseBody
    public List<User> getUsers() {
        try {
            List users = apiRestTemplate.getForObject(REMOTE_DATA_URL, List.class);
            return new ArrayList<User>(users);
        } catch (InvalidTokenException badToken) {
            //we've got a bad token, probably because it's expired.
            OAuth2ProtectedResourceDetails resource = apiRestTemplate.getResource();
            OAuth2SecurityContext context = OAuth2SecurityContextHolder.getContext();
            if (context != null) {
                // this one is kind of a hack for this application
                // the problem is that the sparklr photos page doesn't remove the 'code=' request parameter.
                ((OAuth2SecurityContextImpl) context).setVerificationCode(null);
            }
            //clear any stored access tokens...
            tokenServices.removeToken(SecurityContextHolder.getContext().getAuthentication(), resource);
            //go get a new access token...
            throw new OAuth2AccessTokenRequiredException(resource);
        }
    }
}

At this point, I thought everything would work and I spent quite some time banging my head against the wall when it didn't. As I was composing an email to the Enunciate users mailing list, I realized the issue. It appeared to be working, but from the server side, and the redirect back to the client was not happening. The Ajax Login app uses UrlRewriteFilter (for pretty URLs) to redirect from /app/* to /$1 and this redirect was losing the code parameter in the URL.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

To fix this, I added use-query-string="true" to the root element in src/main/webapp/WEB-INF/urlrewrite.xml:

<urlrewrite default-match-type="wildcard" use-query-string="true">

After making all these changes, I ran mvn jetty:run on both apps and opened http://localhost:8080/appfuse/users in my browser. It all worked and a smile crept across my face. I've checked in the client changes into ajax-login on GitHub and the appfuse-oauth example into AppFuse Demos on Google Code. If you'd like to see this example in action, I'd encourage you to checkout both projects and let me know if you find any issues.

Java Web Application Security - Part III: Apache Shiro Login Demo

2011-05-26T16:43:22-06:00

A couple weeks ago, I wrote a tutorial on how to implement security with Spring Security. The week prior, I wrote a similar tutorial for Java EE 6. This week, I'd like to show you how to implement the same features using Apache Shiro. As I mentioned in previous articles, I'm writing this because I told the audience at April's UJUG that I would publish screencasts of the demos.

Today, I've finished the third screencast showing how to implement security with Apache Shiro. Below is the presentation (with the screencast embedded on slide 22) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Store Users in a Database
Summary

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a shiro.ini file Shiro's configuration. Create src/main/resources/shiro.ini and populate it with the contents below:

[main]

[users]
admin = admin, ROLE_ADMIN

[roles]
ROLE_ADMIN = *

[urls]
/app/users = authcBasic

You can see this file has four sections and is pretty simple to read and understand. For more information about what each section is for, check out Shiro's configuration documentation.

Next, open src/main/webapp/WEB-INF/web.xml and add Shiro's IniShiroFilter:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.apache.shiro.web.servlet.IniShiroFilter</filter-class>
    <!-- no init-param means load the INI config from classpath:shiro.ini -->
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Then add Shiro's core and web dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.shiro</groupId>
    <artifactId>shiro-web</artifactId>
    <version>1.1.0</version>
</dependency>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Apache Shiro is easier to configure than Spring Security out-of-the-box, mostly because it doesn't require XML.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Shiro doesn't currently have a way to logout with its API. However, it will be added in the 1.2 release.

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Before you implement form-based authentication, I'd like to show you how easy it is to force SSL with Apache Shiro.

Force SSL
Apache Shiro allows you to force SSL on a URL by simply adding "ssl[port]" to a URL in the [urls] section. If you don't specify the port, it will use the default port (443). I'm not sure if it allows you to switch back to http like Spring Security's requires-channel, but I don't think it does. Modify the URLs section of your shiro.ini to have the following:

[urls]
/app/users = ssl[8443],authc

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll be prompted to accept the Untrusted Certificate and then redirected to https://localhost:8443/users after logging in.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a few lines to shiro.ini. First of all, since I'd rather not change the name of the input elements in login.jsp, override the default names in the [main] section:

# name of request parameter with username; if not present filter assumes 'username'
authc.usernameParam = j_username
# name of request parameter with password; if not present filter assumes 'password'
authc.passwordParam = j_password
authc.failureKeyAttribute = shiroLoginFailure

Then change the [urls] section to filter on login.jsp and use "authc" instead of "authcBasic":

[urls]
# The /login.jsp is not restricted to authenticated users (otherwise no one could log in!), but
# the 'authc' filter must still be specified for it so it can process that url's
# login submissions. It is 'smart' enough to allow those requests through as specified by the
# shiro.loginUrl above.
/login.jsp = authc
/app/users = ssl[8443],authc

Then change login.jsp so the form's action is blank (causing it to submit to itself) instead of j_security_check:

<form action="" id="loginForm" method="post">

Now, restart Jetty and you should be prompted to login with this JSP instead of the basic authentication dialog.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a few settings to shiro.ini to define your database and tables to use. Open src/main/resources/shiro.ini and add the following lines under the [main] section.

jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
#jdbcRealm.permissionsLookupEnabled=false
# If not filled, subclasses of JdbcRealm assume "select password from users where username = ?"
jdbcRealm.authenticationQuery = select user_pass from users where user_name = ?
# If not filled, subclasses of JdbcRealm assume "select role_name from user_roles where username = ?"
jdbcRealm.userRolesQuery = select role_name from users_roles where user_name = ?

ds = com.mysql.jdbc.jdbc2.optional.MysqlDataSource
ds.serverName = localhost
ds.user = root
ds.databaseName = appfuse
jdbcRealm.dataSource = $ds

This configuration is similar to what I did with the Java EE 6 tutorial where I'm pointing to a database other than the H2 instance that's used by the application. I believe Shiro can talk to a DAO like Spring Security, but I have yet to explore that option.

While you're at it, add the following lines to enable password encryption.

sha256Matcher = org.apache.shiro.authc.credential.Sha256CredentialsMatcher
jdbcRealm.credentialsMatcher = $sha256Matcher

You'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database using the following command:

mysql -u root -p -e 'create database appfuse'

Then create the tables necessary and populate it with an 'admin' user. Login using "mysql -u root -p appfuse" and execute the following SQL statements:

create table users (
  user_name         varchar(30) not null primary key,
  user_pass         varchar(100) not null
);

create table user_roles (
  user_name         varchar(30) not null,
  role_name         varchar(30) not null,
  primary key (user_name, role_name)
);

insert into users values ('admin', '22f256eca1f336a97eef2b260773cb0d81d900c208ff26e94410d292d605fed8');
insert into user_roles values ('admin', 'ROLE_ADMIN');

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Apache Shiro 1.1.0. I don't have a lot of experience with Apache Shiro, but I was able to get the basics working without too much effort. This tutorial doesn't show how to do Remember Me because I couldn't figure it out in 5 minutes, which means I have 5 more minutes before it fails the 10-minute test.

Shiro was formerly named JSecurity and has been an Apache project for less than a year. It seems to be more targeted towards non-web use, so its certainly something to look at if you're more interested in cryptography or non-web apps. I think there's a good chance this project will continue to grow and be used more as more developers learn about it. The Apache brand certainly doesn't hurt.

I didn't include a slide about the limitations I found with Shiro, mostly because I haven't used it much. I've used Java EE and Spring Security for several years. The main limitation I found was the lack of documentation, but I've heard it's improving rapidly.

In the next couple weeks, I'll post a Part IV on implementing programmatic login using the APIs of Java EE 6, Spring Security and Apache Shiro. I'll be presenting this topic at Jazoon as well as the long-form version (with hacking) at ÜberConf. Hopefully I'll see you at one of those conferences.

Update: Thanks to help from Les Hazlewood, I've figured out how to implement Remember Me with Apache Shiro. In the [urls] section of shiro.ini, the second url (shown below) says to Shiro "In order to visit the /app/users URL, you must be connecting via SSL on port 8443 and you must also be authenticated."

/app/users = ssl[8443],authc

Remembered users are not authenticated because their identity hasn't been proven during the current session. What I want Shiro to say is "In order to visit the /app/users URL, you must be connecting via SSL on 8443 and you must also be a known user. If you're not, you should login first." Where a known user is someone who has a recognized identity and has either authenticated during the current session or is known via RememberMe from a previous session. The documentation gives a good example with Amazon.com for why Shiro makes this distinction. It allows more control (usually necessary), but you can relax the control as you see fit.

So, to relax my configuration a bit to match what I want (known users), I updated shiro.ini's [urls] section to be as follows:

/app/users = ssl[8443],user

The key is that the /app/users url is now protected with the more relaxed user filter instead of the authc filter. However, you would typically want an account profile page (or credit card information page, or similar) protected with the authc filter instead to guarantee proof of identity for those sensitive operations.

Java Web Application Security - Part II: Spring Security Login Demo

2011-05-13T09:20:51-06:00

Last week, I wrote a tutorial on how to implement Security in Java EE 6. This week, I'd like to show you how to implement the same features using Spring Security. Before I begin, I'd like to explain my reason for writing this article.

Last month, I presented a talk on Java Web Application Security at the Utah JUG (UJUG). As part of that presentation, I did a number of demos about how to implement security with Java EE 6, Spring Security and Apache Shiro. I told the audience that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense.

Today, I've finished the second screencast showing how to implement security with Spring Security. Below is the presentation (with the screencast embedded on slide 16) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Download and Run the Application
Implement Basic Authentication
Force SSL
Implement Form-based Authentication
Add Remember Me
Store Users in a Database
Summary

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, you'll need to create a Spring context file that contains Spring Security's configuration. Create src/main/webapp/WEB-INF/security.xml and populate it with the contents below:

  <?xml version="1.0" encoding="UTF-8"?>
  <beans:beans xmlns="http://www.springframework.org/schema/security"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:beans="http://www.springframework.org/schema/beans"
               xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">

      <!-- New in Spring Security 3.1 -->
      <!-- <http pattern="/css/**" security="none"/> -->

      <http auto-config="true">
          <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN"/>
          <http-basic/>
      </http>

      <authentication-manager alias="authenticationManager">
          <authentication-provider>
              <password-encoder hash="sha"/>
              <user-service>
                  <user name="user" password="12dea96fec20593566ab75692c9949596833adc9" authorities="ROLE_USER"/>
                  <user name="admin" password="d033e22ae348aeb5660fc2140aec35850c4da997" authorities="ROLE_ADMIN"/>
              </user-service>
          </authentication-provider>
      </authentication-manager>

      <!-- Override userSecurityAdvice bean in appfuse-service to allow any role to update a user. -->
      <beans:bean id="userSecurityAdvice" class="org.appfuse.examples.webapp.security.UserSecurityAdvice"/>
  </beans:beans>

The last bean, userSecurityAdvice, is an aspect that's needed to override some behavior in AppFuse. You won't need this normally when implementing Spring Security.

Next, open src/main/webapp/WEB-INF/web.xml and add Spring's DelegatingFilterProxy:

<filter>
    <filter-name>securityFilter</filter-name>
    <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    <init-param>
        <param-name>targetBeanName</param-name>
        <param-value>springSecurityFilterChain</param-value>
    </init-param>
</filter>

And add its filter-mapping just after the rewriteFilter in the filter-mappings section (order is important!):

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

You don't need to add any dependencies in your pom.xml is because this project depends on AppFuse, which already contains these dependencies.

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. Spring Security is a bit easier to configure than Java EE 6 out-of-the-box, mostly because it doesn't require you to configure your container.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath()); 
}

NOTE: Spring Security has a way to configure "logout" to match a URL and get rid of a class like LogoutController. Since it was already in the project, I don't cover that in this tutorial.

Force SSL
Spring Security allows you to switch between secure (https) and non-secure (http) protocols using a simple requires-channel attribute on the <intercept-url> element. Possible values are "http", "https" and "any". Add requires-channel="https" to your security.xml file:

<intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>

In order for this to work, you have to configure Jetty to listen on an SSL port. Add the following just after the jetty-maven-plugin's </webAppConfig> element in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

You can switch between http and https protocols. With Java EE, you can only force https. You have to write a custom filter to switch back to http.
Redirecting to https actually works. With Java EE (on Jetty at least), a 403 is returned instead of redirecting the request.

Now let's look at how to have more control over the look-and-feel of the login screen, as well as how to make logout work with form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to add a <form-login> element in security.xml's <http> element:

<http auto-config="true">
    <intercept-url pattern="/app/users" access="ROLE_USER,ROLE_ADMIN" requires-channel="https"/>
    <form-login login-page="/login" authentication-failure-url="/login?error=true"
                login-processing-url="/j_security_check"/>
    <http-basic/>
</http>

You can leave the <http-basic> element since Spring Security is smart enough to serve up the form for browsers and use Basic Authentication for clients such as web services. The login.jsp page (that /login forwards to) already exists in the project, in the src/main/webapp directory. The forwarding is done by the UrlRewriteFilter with the following configuration in src/main/webapp/WEB-INF/urlrewrite.xml.

<rule>
    <from>/login</from>
    <to>/login.jsp</to>
</rule>

This JSP has 3 important elements: 1) a form that submits to "/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Add Remember Me
Remember Me is a feature you see in many web applications today. It's usually a checkbox on the login form that allows you to auto-login the next time you visit a site. This feature doesn't exist in Java EE security, but it does exist in Spring Security. To enable it, add the following just below <form-login> in security.xml:

<remember-me user-service-ref="userDao" key="e37f4b31-0c45-11dd-bd0b-0800200c9a66"/>

Next, open src/main/webapp/login.jsp and change the name of the "remember me" checkbox to be _spring_security_remember_me:

<input type="checkbox" name="_spring_security_remember_me" id="rememberMe"/>

After making these changes, you should be able to restart Jetty, go to http://localhost:8080/users, enter admin/adminjdbc, check the Remember Me checkbox and login. Then close your browser, and repeat the process. This time, you won't be prompted to login. For more information on this feature, see Spring Security's Remember Me documentation.

While storing usernames and passwords in a file is convenient for demos, it's not very real-world-ish. The next section shows you how to configure Spring Security to use a database for its user store.

Store Users in a Database
To store your users in a database instead of file, you'll need to add a user-service-ref attribute to the <authentication-provider> element. You can also delete the <user-service> element.

<authentication-manager alias="authenticationManager">
    <authentication-provider user-service-ref="userDao">
        <password-encoder hash="sha"/>
    </authentication-provider>
</authentication-manager>

The "userDao" bean is provided by AppFuse and its UserDaoHibernate.java class. This class implements Spring Security's UserDetailsService interface. With Java EE, I had to configure a database connection and make sure the JDBC Driver was in my container's classpath. With Spring Security, you can talk to the database you already have configured in your application.

Of course, you could do this with Java EE too. One thing I neglected to show in my last tutorial was that 1) the app uses H2 and 2) I had to configure Java EE's database to be MySQL. This was because when I tried to access my H2 instance, I got an error about two threads trying to access it at once.

2011-05-13 08:47:29.081:WARN::UserRealm Java EE Login could not connect to database; will try later
org.h2.jdbc.JdbcSQLException: Database may be already in use: "Locked by another process". 
        Possible solutions: close all other connection(s); use the server mode [90020-154]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:327)
	at org.h2.message.DbException.get(DbException.java:167)
	at org.h2.message.DbException.get(DbException.java:144)
	at org.h2.store.FileLock.getExceptionAlreadyInUse(FileLock.java:443)
	at org.h2.store.FileLock.lockFile(FileLock.java:338)
	at org.h2.store.FileLock.lock(FileLock.java:134)
	at org.h2.engine.Database.open(Database.java:535)
	at org.h2.engine.Database.openDatabase(Database.java:218)

The password for the "admin" user is configured in src/test/resources/sample-data.xml and it's loaded by DbUnit before the application starts. You can view your pom.xml and the dbunit-maven-plugin's configuration if you're interested in learning how this is done. The password is currently configured to "adminjdbc", but you can reset it by generating a new password and modifying sample-data.xml.

Now if you restart Jetty, you should be able to login with admin/adminjdbc and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using Spring Security 3.0.5. In addition to the basic XML configuration, Spring Security also provides a AOP support and annotations you can use to secure methods. It also has many more features than standard Java EE Security. In my opinion, it's the most mature security framework we have in Java today. Currently, I think its reference documentation is the best place to learn more.

There are a few limitations I found with Spring Security:

The authentication mechanism (file, database, ldap, etc.) is contained in the WAR
Securing methods only works on Spring beans
Remember Me doesn't work in my screencast (because I forgot to rename the checkbox in login.jsp)

Of course, you can configure Spring to load its configuration from outside the WAR (e.g. a file or JNDI), but it's not as easy as including the configuration in your app.

In the next couple weeks, I'll post Part III of this series, where I'll show you how to implement this same set of features using Apache Shiro. In the meantime, please let me know if you have any questions.

I created the screencasts with Camtasia. For small screens, and embedding in the presentation, I created it at 50% and used the SmartFocus feature to zoom in and out during the demo. For larger screens, I published another screencast at 100%, in HD. If you have a preference for which screencast is better, I'd love to hear about it.

Java Web Application Security - Part I: Java EE 6 Login Demo

2011-05-05T16:58:00-06:00

Back in February, I wrote about my upcoming conferences:

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks:

Implement Basic Authentication
Implement Form-based Authentication
Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs)
Force SSL for certain URLs
Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone)
Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone)
Encrypt Passwords
Secure methods with annotations

For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities.

I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial.

* You can also watch the screencast on YouTube or download the presentation PDF.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml:

<security-constraint>
    <web-resource-collection>
        <web-resource-name>users</web-resource-name>
        <url-pattern>/users</url-pattern>
        <http-method>GET</http-method>
        <http-method>POST</http-method>
    </web-resource-collection>
    <auth-constraint>
        <role-name>ROLE_ADMIN</role-name>
    </auth-constraint>
</security-constraint>

<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Java EE Login</realm-name>
</login-config>

<security-role>
    <role-name>ROLE_ADMIN</role-name>
</security-role>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after </webAppConfig> in the Jetty plugin's configuration.

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.HashLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/realm.properties</config>
    </loginService>
</loginServices>

The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath());
}

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to replace the <login-config> in your web.xml with the following:

<login-config>
    <auth-method>FORM</auth-method>
    <form-login-config>
        <form-login-page>/login.jsp</form-login-page>
        <form-error-page>/login.jsp?error=true</form-error-page>
    </form-login-config>
</login-config>

The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Force SSL
Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same <security-constraint> you already have in web.xml, add the following after </auth-constraint>:

<user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>

To configure Jetty to listen on an SSL port, add the following just after </loginServices> in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you.

Store Users in a Database
Finally, to store your users in a database instead of file, you'll need to change the <loginService> in the Jetty plugin's configuration. Replace the existing <loginService> element with the following:

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.JDBCLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/jdbc-realm.properties</config>
    </loginService>
</loginServices>

The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information.

jdbcdriver = com.mysql.jdbc.Driver
url = jdbc:mysql://localhost/appfuse
username = root
password =
usertable = app_user
usertablekey = id
usertableuserfield = username
usertablepasswordfield = password
roletable = role
roletablekey = id
roletablerolefield = name
userroletable = user_role
userroletableuserkey = user_id
userroletablerolekey = role_id
cachetime = 300

Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands:

mysql -u root -p -e 'create database appfuse'
curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql
mysql -u root -p appfuse < create-appfuse.sql

Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the <configuration> element (before <executions>) in pom.xml:

<dependencies>
    <!-- MySQL for JDBC Realm -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.14</version>
    </dependency>
</dependencies>

Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example:

$ sh jetty-password.sh javaeelogin
javaeelogin
OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur
MD5:53b176e6ce1b5183bc970ef1ebaffd44

The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement.

UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin';

Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0:

authenticate(response)
login(user, pass)
logout()

This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely:

No error messages for failed logins
No Remember Me
No auto-redirect from HTTP to HTTPS
Container has to be configured
Doesn’t support regular expressions for URLs

Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain.

In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions.

AppFuse 2.1 Released!

2011-04-04T09:38:05-06:00

The AppFuse Team is pleased to announce the release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are JPA 2, JSF 2, Tapestry 5 and Spring 3. In addition, we've migrated from XFire to CXF and enabled REST for web services. There's even a new appfuse-ws archetype that leverages Enunciate to generate web service endpoints, documentation and downloadable clients. This release fixes many issues with archetypes, improving startup time and allowing jetty:run to be used for quick turnaround while developing. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications with Java quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with IntelliJ, you can achieve zero-turnaround in your project and develop features without restarting the server.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose with persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run "appfuse:full-source".

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. Light archetypes allow code generation and full-source features, but do not currently support Stripes or Wicket. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket. You can see demos of these archetypes at http://demo.appfuse.org.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the mailing list or create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Adding Search to AppFuse with Compass

2011-03-15T17:11:12-06:00

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

Added Searchable annotations to the User and Address.
Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
Defined CompassTemplate (similar in concept to HibernateTemplate).
Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

To index on startup, I added compassGps.index() to the StartupListener..
In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code.

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

Upgrading to JSF 2

2011-03-07T13:24:53-07:00

Last week, I spent a few hours upgrading AppFuse from JSF 1.2 to JSF 2.0. In reality, I upgraded from MyFaces 1.2.7 to 2.0.4, but all JSF implementations should be the same, right? All in all, it was a pretty easy upgrade with a few minor AppFuse-specific things. My goal in upgrading was to do the bare minimum to get things working and to leave integration of JSF 2 features for a later date.

In addition to upgrading MyFaces, I had to upgrade Tomahawk by changing the dependency's artifactId to tomahawk20. I was also able to remove the following listener from my web.xml:

<listener>
    <listener-class>org.apache.myfaces.webapp.StartupServletContextListener</listener-class>
<listener>

After that, I discovered that MyFaces uses a new URI (/javax.faces.resource/) for serving up some of its resource files. I kindly asked Spring Security to ignore these requests by adding the following to my security.xml file.

<intercept-url pattern="/javax.faces.resource/**" filters="none"/>

Since JSF 2 includes Facelets by default, I tried removing Facelets as a dependency. After doing this, I received the following error:

ERROR [308855416@qtp-120902214-7] ViewHandlerWrapper.fillChain(158) | Error instantiation parent Faces ViewHandler
java.lang.ClassNotFoundException: com.sun.facelets.FaceletViewHandler
        at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
        at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at org.ajax4jsf.framework.ViewHandlerWrapper.fillChain(ViewHandlerWrapper.java:144)
        at org.ajax4jsf.framework.ViewHandlerWrapper.calculateRenderKitId(ViewHandlerWrapper.java:68)
        at org.apache.myfaces.lifecycle.DefaultRestoreViewSupport.isPostback(DefaultRestoreViewSupport.java:179)
        at org.apache.myfaces.lifecycle.RestoreViewExecutor.execute(RestoreViewExecutor.java:113)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:171)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Figuring this was caused by the following element in my web.xml ...

<context-param>
    <param-name>org.ajax4jsf.VIEW_HANDLERS</param-name>
    <param-value>com.sun.facelets.FaceletViewHandler</param-value>
</context-param>

... I removed it and tried again. This time I received a NoClassDefFoundError:

java.lang.NoClassDefFoundError: com/sun/facelets/tag/TagHandler
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:392)
        at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.myfaces.shared_impl.util.ClassUtils.classForName(ClassUtils.java:184)
        at org.apache.myfaces.view.facelets.util.ReflectionUtil.forName(ReflectionUtil.java:67)

Since everything seemed to work with Facelets in the classpath, I decided to save this headache for a later date. I entered two issues in AppFuse's JIRA, one for removing Facelets and one for replacing Ajax4JSF with RichFaces.

The next issue I encountered was redirecting from AppFuse's password hint page. The navigation-rule for this page is as follows:

<navigation-rule>
    <from-view-id>/passwordHint.xhtml</from-view-id>
    <navigation-case>
        <from-outcome>success</from-outcome>
        <to-view-id>/login</to-view-id>
        <redirect/>
    </navigation-case>
</navigation-rule>

With JSF 2.0, the rule changes the URL to /login.xhtml when redirecting (where it was left as /login with 1.2) and it was caught by the security setting in my web.xml that prevents users from viewing raw templates.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Protect XHTML Templates</web-resource-name>
        <url-pattern>*.xhtml</url-pattern>
    </web-resource-collection>
    <auth-constraint/>
</security-constraint>

To solve this issue, I had to make a couple of changes:

Comment out the security-constraint in web.xml and move it to Spring Security's security.xml file.
```
<intercept-url pattern="/**/*.xhtml" access="ROLE_NOBODY"/>
```
Add a rule to urlrewrite.xml that redirects back to login (since login.xhtml doesn't exist and I'm using extensionless URLs).
```
<rule match-type="regex">
    <from>^/login.xhtml$</from>
    <to type="redirect">%{context-path}/login</to>
</rule>
```

After getting the Password Hint feature passing in the browser, I tried running the integration tests (powered by Canoo WebTest). The Password Hint test kept failing with the following error:

[ERROR] /Users/mraible/dev/appfuse/web/jsf/src/test/resources/web-tests.xml:51: JavaScript error loading
page http://localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/passwordHint?username=admin: syntax error (http://
localhost:9876/appfuse-jsf-2.1.0-SNAPSHOT/javax.faces.resource/oamSubmit.js.jsf?ln=org.apache.myfaces#122)

Figuring this was caused by my hack to submit the form when the page was loaded, I turned to Pretty Faces, which allows you to call a method directly from a URL. After adding the Pretty Faces dependencies to my pom.xml, I created a src/main/webapp/WEB-INF/pretty-config.xml file with the following XML:

<url-mapping>
    <pattern value="/editProfile"/>
    <view-id value="/userForm.jsf"/>
    <action>#{userForm.edit}</action>
</url-mapping>

<url-mapping>
    <pattern value="/passwordHint/#{username}"/>
    <view-id value="/passwordHint.jsf"/>
    <action>#{passwordHint.execute}</action>
</url-mapping>

This allowed me to remove both editProfile.xhtml and passwordHint.xhtml, both of which simply auto-submitted forms.

At this point, I figured I'd be good to go and ran my integration tests again. The first thing I discovered was that ".jsf" was being tacked onto my pretty URL, most likely by the UrlRewriteFilter. Adding the following to my PasswordHint.java class solved this.

if (username.endsWith(".jsf")) {
    username = username.substring(0, username.indexOf(".jsf"));
}

The next thing was a cryptic error that took me a while to figure out.

DEBUG [1152467051@qtp-144702232-0] PasswordHint.execute(38) | Processing Password Hint...
2011-03-05 05:48:52.471:WARN::/passwordHint/admin
com.ocpsoft.pretty.PrettyException: Exception occurred while processing <:#{passwordHint.execute}> null
        at com.ocpsoft.pretty.faces.beans.ActionExecutor.executeActions(ActionExecutor.java:71)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.processEvent(PrettyPhaseListener.java:214)
        at com.ocpsoft.pretty.faces.event.PrettyPhaseListener.afterPhase(PrettyPhaseListener.java:108)
        at org.apache.myfaces.lifecycle.PhaseListenerManager.informPhaseListenersAfter(PhaseListenerManager.java:111)
        at org.apache.myfaces.lifecycle.LifecycleImpl.executePhase(LifecycleImpl.java:185)
        at org.apache.myfaces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:189)

Digging into the bowels of MyFaces, I discovered a class was looking for a viewId with an extension and no view-id was being set. Adding the following to the top of my execute() method solved this.

getFacesContext().getViewRoot().setViewId("/passwordHint.xhtml");

After making this change, all AppFuse's integration tests are passing and the upgrade seems complete. The only other issues I encountered were logging-related. The first is an error about Tomahawk that doesn't seem to affect anything.

Mar 5, 2011 6:44:01 AM com.sun.facelets.compiler.TagLibraryConfig loadImplicit
SEVERE: Error Loading Library: jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml
java.io.IOException: Error parsing [jar:file:/Users/mraible/.m2/repository/org/apache/myfaces/tomahawk/tomahawk20/1.1.10/tomahawk20-1.1.10.jar!/META-INF/tomahawk.taglib.xml]: 
        at com.sun.facelets.compiler.TagLibraryConfig.create(TagLibraryConfig.java:410)
        at com.sun.facelets.compiler.TagLibraryConfig.loadImplicit(TagLibraryConfig.java:431)
        at com.sun.facelets.compiler.Compiler.initialize(Compiler.java:87)
        at com.sun.facelets.compiler.Compiler.compile(Compiler.java:104)

The second is excessive logging from MyFaces. As far as I can tell, this is because MyFaces switched to java.util.logging instead of commons logging. With all the frameworks that AppFuse leverages, I think it has all the logging frameworks in its classpath now. I was hoping to fix this by posting a message to the mailing list, but haven't received a reply yet.

[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider newInstance
[WARNING] [talledLocalContainer] INFO: Creating instance of org.appfuse.webapp.action.BasePage
[WARNING] [talledLocalContainer] Mar 5, 2011 6:50:25 AM org.apache.myfaces.config.annotation.TomcatAnnotationLifecycleProvider destroyInstance
[WARNING] [talledLocalContainer] INFO: Destroy instance of org.appfuse.webapp.action.BasePage

After successfully upgrading AppFuse, I turned to AppFuse Light, where things were much easier.

Now that AppFuse uses JSF 2, I hope to start leveraging some of its new features. If you're yearning to get started with them today, I invite you to grab the source and start integrating them yourself.

Fixing XSS in JSP 2

2011-02-28T14:08:46-07:00

Way back in 2007, I wrote about Java Web Frameworks and XSS. My main point was that JSP EL doesn't bother to handle XSS.

Of course, the whole problem with JSP EL could be solved if Tomcat (and other containers) would allow a flag to turn on XML escaping by default. IMO, it's badly needed to make JSP-based webapps safe from XSS.

A couple months later, I proposed a Tomcat enhancement to escape JSP's EL by default. I also entered an enhancement request for this feature and attached a patch. That issue has remained open and unfixed for 3 and 1/2 years.

Yesterday, Chin Huang posted a handy-dandy ELResolver that XML-escapes EL values.

I tried Chin's resolver in AppFuse today and it works as well as advertised. To do this, I copied his EscapeXML*.java files into my project, changed the JSP API's Maven coordinates from javax.servlet:jsp-api:2.0 to javax.servlet.jsp:jsp-api:2.1 and added the listener to web.xml.

With Struts 2 and Spring MVC, I was previously able to have ${param.xss} and pass in ?xss=<script>alert('gotcha')</script> and it would show a JavaScript alert. After using Chin's ELResolver, it prints the string on the page instead of displaying an alert.

Thanks to Chin Huang for this patch! If you're using JSP, I highly recommend you add this to your projects as well.

Upcoming Conferences: TSSJS in Las Vegas and 33rd Degree in Kraków, Poland

2011-02-25T15:14:59-07:00

It's that time of year again - the beginning of Conference Season. I generally like to speak at a few conferences a year and 2011 is no different. For March Madness, I'll be heading to Las Vegas to speak at TheServerSide Java Symposium. I'll be giving updated talks similar to the ones I gave at last year's Rich Web Experience in Fort Lauderdale:

You might remember my Comparing JVM Web Frameworks talk from Devoxx 2010 and some of the interesting debate it caused. I've done some minor updates to my video presentation and some updates to my JVM Web Frameworks presentation as well. Most notably, I'll be including some findings from Peter Thomas's perfbench project. I also hope to update AppFuse to JSF 2 and integrate extensionless URLs in AppFuse Light. Marcin Zajaczkowski was nice enough to provide an upgrade to Wicket 1.4.15, so it'll be interesting to see how well Wicket supports extensionless URLs.

In April, I'll be presenting Comparing JVM Web Frameworks at the 33rd Degree Conference in Kraków, Poland. While I studied in Russia a couple summers in college, I've never been to Poland, so I'm really looking forward to this trip. With any luck, I'll have AppFuse 2.1 released by then and my knowledge of all its web frameworks' latest versions will be update-to-date. As you know, it's unlikely I'll recommend a best web framework (because there isn't one), but I hope to provide some techniques you can use to decide the best framework for your particular needs.

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and Über Conf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

If you're planning on attending TSSJS or 33rd Degree, hopefully I'll see you there.

Implementing Ajax Authentication using jQuery, Spring Security and HTTPS

2011-02-23T16:55:55-07:00

I've always had a keen interest in implementing security in webapps. I implemented container-managed authentication (CMA) in AppFuse in 2002, watched Tomcat improve it's implementation in 2003 and implemented Remember Me with CMA in 2004. In 2005, I switched from CMA to Acegi Security (now Spring Security) and never looked back. I've been very happy with Spring Security over the years, but also hope to learn more about Apache Shiro and implementing OAuth to protect JavaScript APIs in the near future.

I was recently re-inspired to learn more about security when working on a new feature at Overstock.com. The feature hasn't been released yet, but basically boils down to allowing users to login without leaving a page. For example, if they want to leave a review on a product, they would click a link, be prompted to login, enter their credentials, then continue to leave their review. The login prompt and subsequent review would likely be implemented using a lightbox. While lightboxes are often seen in webapps these days because they look good, it's also possible Lightbox UIs provide a poor user experience. User experience aside, I think it's interesting to see what's required to implement such a feature.

To demonstrate how we did it, I whipped up an example using AppFuse Light, jQuery and Spring Security. The source is available in my ajax-login project on GitHub. To begin, I wanted to accomplish a number of things to replicate the Overstock environment:

Force HTTPS for authentication.
Allow testing HTTPS without installing a certificate locally.
Implement a RESTful LoginService that allows users to login.
Implement login with Ajax, with the request coming from an insecure page.

Forcing HTTPS with Spring Security
The first feature was fairly easy to implement thanks to Spring Security. Its configuration supports a requires-channel attribute that can be used for this. I used this to force HTTPS on the "users" page and it subsequently causes the login to be secure.

<intercept-url pattern="/app/users" access="ROLE_ADMIN" requires-channel="https"/>

Testing HTTPS without adding a certificate locally
After making the above change in security.xml, I had to modify my jWebUnit test to work with SSL. In reality, I didn't have to modify the test, I just had to modify the configuration that ran the test. In my last post, I wrote about adding my 'untrusted' cert to my JVM keystore. For some reason, this works for HttpClient, but not for jWebUnit/HtmlUnit. The good news is I figured out an easier solution - adding the trustStore and trustStore password as system properties to the maven-failsafe-plugin configuration.

<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.2</version>
<configuration>
    <includes>
        <include>**/*WebTest.java</include>
    </includes>
    <systemPropertyVariables>
      <javax.net.ssl.trustStore>${project.build.directory}/ssl.keystore</javax.net.ssl.trustStore>
      <javax.net.ssl.trustStorePassword>appfuse</javax.net.ssl.trustStorePassword>
    </systemPropertyVariables>
</configuration>

The disadvantage to doing things this way is you'll have to pass these in as arguments when running unit tests in your IDE.

package org.appfuse.examples.web;

import org.appfuse.model.User;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
@RequestMapping("/api/login.json")
public class LoginService {

  @Autowired
  @Qualifier("authenticationManager")
  AuthenticationManager authenticationManager;

  @RequestMapping(method = RequestMethod.GET)
  @ResponseBody
  public LoginStatus getStatus() {
    Authentication auth = SecurityContextHolder.getContext().getAuthentication();
    if (auth != null && !auth.getName().equals("anonymousUser") && auth.isAuthenticated()) {
      return new LoginStatus(true, auth.getName());
    } else {
      return new LoginStatus(false, null);
    }
  }

  @RequestMapping(method = RequestMethod.POST)
  @ResponseBody
  public LoginStatus login(@RequestParam("j_username") String username,
                           @RequestParam("j_password") String password) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    User details = new User(username);
    token.setDetails(details);

    try {
      Authentication auth = authenticationManager.authenticate(token);
      SecurityContextHolder.getContext().setAuthentication(auth);
      return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
      return new LoginStatus(false, null);
    }
  }

  public class LoginStatus {

    private final boolean loggedIn;
    private final String username;

    public LoginStatus(boolean loggedIn, String username) {
      this.loggedIn = loggedIn;
      this.username = username;
    }

    public boolean isLoggedIn() {
      return loggedIn;
    }

    public String getUsername() {
      return username;
    }
  }
}

To verify this class worked as expected, I wrote a unit test using JUnit and Mockito. I used Mockito because Overstock is transitioning to it from EasyMock and I've found it very simple to use.

package org.appfuse.examples.web;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.mockito.Matchers;
import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.TestingAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContext;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.context.SecurityContextImpl;

import static org.junit.Assert.*;
import static org.mockito.Mockito.*;

public class LoginServiceTest {

  LoginService loginService;
  AuthenticationManager authenticationManager;

  @Before
  public void before() {
    loginService = new LoginService();
    authenticationManager = mock(AuthenticationManager.class);
    loginService.authenticationManager = authenticationManager;
  }

  @After
  public void after() {
    SecurityContextHolder.clearContext();
  }

  @Test
  public void testLoginStatusSuccess() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    SecurityContext context = new SecurityContextImpl();
    context.setAuthentication(auth);
    SecurityContextHolder.setContext(context);

    LoginService.LoginStatus status = loginService.getStatus();
    assertTrue(status.isLoggedIn());
  }

  @Test
  public void testLoginStatusFailure() {
    LoginService.LoginStatus status = loginService.getStatus();
    assertFalse(status.isLoggedIn());
  }

  @Test
  public void testGoodLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(true);
    when(authenticationManager.authenticate(Matchers.<Authentication>anyObject())).thenReturn(auth);
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertTrue(status.isLoggedIn());
    assertEquals("foo", status.getUsername());
  }

  @Test
  public void testBadLogin() {
    Authentication auth = new TestingAuthenticationToken("foo", "bar");
    auth.setAuthenticated(false);
    when(authenticationManager.authenticate(Matchers.anyObject()))
        .thenThrow(new BadCredentialsException("Bad Credentials"));
    LoginService.LoginStatus status = loginService.login("foo", "bar");
    assertFalse(status.isLoggedIn());
    assertEquals(null, status.getUsername());
  }
}

Implement login with Ajax
The last feature was the hardest to implement and still isn't fully working as I'd hoped. I used jQuery and jQuery UI to implement a dialog that opens the login page on the same page rather than redirecting to the login page. The "#demo" locator refers to a button in the page.

Passing in the "ajax=true" parameter disables SiteMesh decoration on the login page, something that's described in my Ajaxified Body article.

var dialog = $('<div></div>');

$(document).ready(function() {
    $.get('/login?ajax=true', function(data) {
        dialog.html(data);
        dialog.dialog({
            autoOpen: false,
	       title: 'Authentication Required'
        });
    });

    $('#demo').click(function() {
      dialog.dialog('open');
      // prevent the default action, e.g., following a link
      return false;
    });
});

Instead of adding a click handler to a specific id, it's probably better to use a CSS class that indicates authentication is required for a link, or -- even better -- use Ajax to see if the link is secured.

The login page then has the following JavaScript to add a click handler to the "login" button that submits the request securely to the LoginService.

var getHost = function() {
    var port = (window.location.port == "8080") ? ":8443" : "";
    return ((secure) ? 'https://' : 'http://') + window.location.hostname + port;
};

var loginFailed = function(data, status) {
    $(".error").remove();
    $('#username-label').before('<div class="error">Login failed, please try again.</div>');
};

$("#login").live('click', function(e) {
    e.preventDefault();
    $.ajax({url: getHost() + "/api/login.json",
        type: "POST",
        data: $("#loginForm").serialize(),
        success: function(data, status) {
            if (data.loggedIn) {
                // success
                dialog.dialog('close');
                location.href= getHost() + '/users';
            } else {
                loginFailed(data);
            }
        },
        error: loginFailed
    });
});

The biggest secret to making this all work (the HTTP -> HTTPS communication, which is considered cross-domain), is the window.name Transport and the jQuery plugin that implements it. To make this plugin work with Firefox 3.6, I had to implement a Filter that adds Access-Control headers. A question on Stackoverflow helped me figure this out.

public class OptionsHeadersFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        HttpServletResponse response = (HttpServletResponse) res;

        response.setHeader("Access-Control-Allow-Origin", "*");
        response.setHeader("Access-Control-Allow-Methods", "GET,POST");
        response.setHeader("Access-Control-Max-Age", "360");
        response.setHeader("Access-Control-Allow-Headers", "x-requested-with");

        chain.doFilter(req, res);
    }

    public void init(FilterConfig filterConfig) {
    }

    public void destroy() {
    }
}

Issues
I encountered a number of issues when implementing this in the ajax-login project.

If you try to run this with ports (e.g. 8080 and 8443) in your URLs, you'll get a 501 (Not Implemented) response. Removing the ports by fronting with Apache and mod_proxy solves this problem.
If you haven't accepted the certificate in your browser, the Ajax request will fail. In the example, I solved this by clicking on the "Users" tab to make a secure request, then going back to the homepage to try and login.
The jQuery window.name version 0.9.1 doesn't work with jQuery 1.5.0. The error is "$.httpSuccess function not found."

Finally, even though I was able to authenticate successfully, I was unable to make the authentication persist. I tried adding the following to persist the updated SecurityContext to the session, but it doesn't work. I expect the solution is to create a secure JSESSIONID cookie somehow.

@Autowired
SecurityContextRepository repository;

@RequestMapping(method = RequestMethod.POST)
@ResponseBody
public LoginStatus login(@RequestParam("j_username") String username,
                         @RequestParam("j_password") String password,
                         HttpServletRequest request, HttpServletResponse response) {

    UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(username, password);
    ...

    try {
        Authentication auth = authenticationManager.authenticate(token);
        SecurityContextHolder.getContext().setAuthentication(auth);
        // save the updated context to the session
        repository.saveContext(SecurityContextHolder.getContext(), request, response);
        return new LoginStatus(auth.isAuthenticated(), auth.getName());
    } catch (BadCredentialsException e) {
        return new LoginStatus(false, null);
    }
}

Conclusion
This article has shown you how to force HTTPS for login, how to do integration testing with a self-generated certificate, how to implement a LoginService with Spring MVC and Spring Security, as well as how to use jQuery to talk to a service cross-domain with the window.name Transport. While I don't have everything working as much as I'd like, I hope this helps you implement a similar feature in your applications.

One thing to be aware of is with lightbox/dialog logins and HTTP -> HTTPS is that users won't see a secure icon in their address bar. If your app has sensitive data, you might want to force https for your entire app. OWASP's Secure Login Pages has a lot of good tips in this area.

Update: I've posted a demo of the ajax-login webapp. Thanks to Contegix for hosting the demo and helping obtain/install an SSL certificate so quickly.

Integration Testing with HTTP, HTTPS and Maven

2011-02-11T15:54:16-07:00

Earlier this week, I was tasked with getting automated integration tests working in my project at Overstock.com. By automated, I mean that ability to run "mvn install" and have the following process cycled through:

Start a container
Deploy the application
Run all integration tests
Stop the container

Since it makes sense for integration tests to run in Maven's integration-test phase, I first configured the maven-surefire-plugin to skip tests in the test phase and execute them in the integration-test phase. I used the <id>default-phase</id> syntax to override the plugins' usual behavior.

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <executions>
    <execution>
      <id>default-test</id>
      <configuration>
        <excludes>
          <exclude>**/*Test*.java</exclude>
        </excludes>
      </configuration>
    </execution>
    <execution>
      <id>default-integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/*Test.java</include>
        </includes>
        <excludes>
          <exclude>none</exclude>
          <exclude>**/TestCase.java</exclude>
        </excludes>
      </configuration>
    </execution>
  </executions>
</plugin>

After I had this working, I moved onto getting the container started and stopped properly. In the past, I've done this using Cargo and it's always worked well for me. Apart from the usual setup I use in AppFuse archetypes (example pom.xml), I added a couple additional items:

Added <timeout>180000</timeout> so the container would wait up to 3 minutes for the WAR to deploy.
In configuration/properties, specified <context.path>ROOT</context.path> so the app would deploy at the / context path.
In configuration/properties, specified <cargo.protocol>https</cargo.protocol> since many existing unit tests made requests to secure resources.

I started by using Cargo with Tomcat and had to create certificate keystore in order to get Tomcat to start with SSL enabled. After getting it to start, I found the tests failed with the following errors in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: 
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1649)

Co-workers told me this was easily solved by adding my 'untrusted' cert to my JVM keystore. Once all this was working, I thought I was good to go, but found that some tests were still failing. The failures turned out to be because they were talking to http and https was the only protocol enabled. After doing some research, I discovered that Cargo doesn't support starting on both http and https ports.

So back to the drawing board I went. I ended up turning to the maven-jetty-plugin and the tomcat-maven-plugin to get the functionality I was looking for. I also automated the certificate keystore generation using the keytool-maven-plugin. Below is the extremely-verbose 95-line profiles section of my pom.xml that allows either container to be used.

Sidenote: I wonder how this same setup would look using Gradle?

<profiles>
  <profile>
    <id>jetty</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <build>
      <plugins>
        <plugin>
          <groupId>org.mortbay.jetty</groupId>
          <artifactId>maven-jetty-plugin</artifactId>
          <version>6.1.26</version>
          <configuration>
            <contextPath>/</contextPath>
            <connectors>
              <connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
                <!-- forwarded == true interprets x-forwarded-* headers -->
                <!-- http://docs.codehaus.org/display/JETTY/Configuring+mod_proxy -->
                <forwarded>true</forwarded>
                <port>8080</port>
                <maxIdleTime>60000</maxIdleTime>
              </connector>
              <connector implementation="org.mortbay.jetty.security.SslSocketConnector">
                <forwarded>true</forwarded>
                <port>8443</port>
                <maxIdleTime>60000</maxIdleTime>
                <keystore>${project.build.directory}/ssl.keystore</keystore>
                <password>overstock</password>
                <keyPassword>overstock</keyPassword>
              </connector>
            </connectors>
            <stopKey>overstock</stopKey>
            <stopPort>9999</stopPort>
          </configuration>
          <executions>
            <execution>
              <id>start-jetty</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
              <configuration>
                <daemon>true</daemon>
              </configuration>
            </execution>
            <execution>
              <id>stop-jetty</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>stop</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
  <profile>
    <id>tomcat</id>
    <build>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>tomcat-maven-plugin</artifactId>
          <version>1.1</version>
          <configuration>
            <addContextWarDependencies>true</addContextWarDependencies>
            <fork>true</fork>
            <path>/</path>
            <port>8080</port>
            <httpsPort>8443</httpsPort>
            <keystoreFile>${project.build.directory}/ssl.keystore</keystoreFile>
            <keystorePass>overstock</keystorePass>
          </configuration>
          <executions>
            <execution>
              <id>start-tomcat</id>
              <phase>pre-integration-test</phase>
              <goals>
                <goal>run-war</goal>
              </goals>
            </execution>
            <execution>
              <id>stop-tomcat</id>
              <phase>post-integration-test</phase>
              <goals>
                <goal>shutdown</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </build>
  </profile>
</profiles>

With this setup in place, I was able to automate running our integration tests by simply typing "mvn install" (for Jetty) or "mvn install -Ptomcat" (for Tomcat). For running in Hudson, it's possible I'll have to further enhance things to randomize the port and pass that into tests as a system property. The build-helper-maven-plugin and its reserve-network-port goal is a nice way to do this. Note: if you want to run more than one instance of Tomcat at a time, you might have to randomize the ajp and rmi ports to avoid collisions.

The final thing I encountered was our app didn't shutdown gracefully. Luckily, this was fixed in a newer version of our core framework and upgrading fixed the problem. Here's the explanation from an architect on the core framework team.

The hanging problem was caused by the way the framework internally aggregated statistics related to database connection usage and page response times. The aggregation runs on a separate thread but not as a daemon thread. Previously, the aggregation threads weren't being terminated on shutdown so the JVM would hang waiting for them to finish. In the new frameworks, the aggregation threads are terminated on shutdown.

Hopefully this post helps you test your secure and unsecure applications at the same time. At the same time, I'm hoping it motivates the Cargo developers to add simultaneous http and https support.

Update: In the comments, Ron Piterman recommended I use the Maven Failsafe Plugin because its designed to run integration tests while Surefire Plugin is for unit tests. I changed my configuration to the following and everything still passes. Thanks Ron!

<plugin>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<plugin>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <includes>
      <include>**/*Test.java</include>
    </includes>
    <excludes>
      <exclude>**/TestCase.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
    </execution>
    <execution>
      <id>verify</id>
      <phase>verify</phase>
      <goals>
        <goal>verify</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Update 2: In addition to application changes to solve hanging issues, I also had to change my Jetty Plugin configuration to use a different SSL connector implementation. This also required adding the jetty-sslengine dependency, which has been renamed to jetty-ssl for Jetty 7.

<connector implementation="org.mortbay.jetty.security.SslSelectChannelConnector">
...
<dependencies>
  <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-sslengine</artifactId>
    <version>6.1.26</version>
  </dependency>
</dependencies>

Implementing Extensionless URLs with Tapestry, Spring MVC, Struts 2 and JSF

2011-02-10T16:53:27-07:00

For the past couple of weeks, I've spent several evening hours implementing extensionless URLs in AppFuse. I've been wanting to do this ever since I wrote about how to do it a few years ago. This article details my experience and will hopefully help others implement this feature in their webapps.

First of all, I used the UrlRewriteFilter, one of my favorite Java open source projects. Then I followed a pattern I found in Spring's "mvc-basic" sample app from MVC Simplifications in Spring 3.0. The app has since changed (because SpringSource integrated UrlRewriteFilter-type functionality in Spring MVC), but the pattern was basically path-matching instead of extension-mapping. That is, the "dispatcher" for the web framework was mapped to /app/* instead of *.html.

Prior to the move to extensionless URLs, AppFuse used *.html for its mapping and this seemed to cause users problems when they wanted to serve up static HTML files. To begin with, I removed all extensions from URLs in tests (Canoo WebTest is used for testing the UI). I also did this for any links in the view pages and redirects in the Java code. This provided a decent foundation to verify my changes worked. Below are details about each framework I did this for, starting with the one that was easiest and moving to hardest.

Tapestry 5
Tapestry was by far the easiest to integrate extensionless URLs into. This is because it's a native feature of the framework and was already integrated as part of Serge Eby's Tapestry 5 implementation. In the end, the only things I had to do where 1) add a couple entries for CXF (mapped to /services/*) and DWR (/dwr/*) to my urlrewrite.xml and 2) change the UrlRewriteFilter so it was only mapped to REQUEST instead of both REQUEST and FORWARD. Below are the mappings I added for CXF and DWR.

<urlrewrite default-match-type="wildcard">
    ...
    <rule>
        <from>/dwr/**</from>
        <to>/dwr/$1</to>
    </rule>
    <rule>
        <from>/services/**</from>
        <to>/services/$1</to>
    </rule>
</urlrewrite>

Spring MVC
I had a fair amount of experience with Spring MVC and extensionless URLs. Both the Spring MVC applications we developed last year at Time Warner Cable used them. To change from a *.html mapping to /app/* was pretty easy and involved removing more code than I added. Previously, I had a StaticFilter that looked for HTML files and if it didn't find them, it dispatched to Spring's DispatcherServlet. I was able to remove this class and make the web.xml file quite a bit cleaner.

To make UrlRewriteFilter and Spring Security play well together, I had to move the securityFilter so it came after the rewriteFilter and add an INCLUDE dispatcher so included JSPs would have a security context available to them.

<filter-mapping>
    <filter-name>rewriteFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>
    <filter-name>securityFilter</filter-name>
    <url-pattern>/*</url-pattern>
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
    <dispatcher>INCLUDE</dispatcher>
</filter-mapping>

The only other things I had to change were security.xml and dispatcher-servlet.xml to remove the .html extensions. The urlrewrite.xml file was fairly straightforward. I used the following at the bottom as a catch-all for dispatching to Spring MVC.

<rule>
    <from>/**</from>
    <to>/app/$1</to>
</rule>
<outbound-rule>
    <from>/app/**</from>
    <to>/$1</to>
</outbound-rule>

Then I added a number of other rules for j_security_check, DWR, CXF and static assets (/images, /scripts, /styles, /favicon.ico). You can view the current urlrewrite.xml in FishEye. The only major issue I ran into was that Spring Security recorded protected URLs as /app/URL so I had to add a rule to redirect when this happened after logging in.

<rule>
    <from>/app/**</from>
    <to last="true" type="redirect">%{context-path}/$1</to>
</rule>

Struts 2
Using extensionless URLs with Struts 2 is likely pretty easy thanks to the Convention Plugin. Even though this plugin is included in AppFuse, it's not configured with the proper constants and I have struts.convention.action.disableScanning=true in struts.xml. It looks like I had to do this when I upgraded from Struts 2.0.x to Struts 2.1.6. It's true AppFuse's Struts 2 support could use a bit of love to be aligned with Struts 2's recommended practices, but I didn't want to spend the time doing it as part of this exercise.

With Struts 2, I tried the path-mapping like I did with Spring MVC, but ran into issues. Instead, I opted to use an ".action" extension by changing struts.action.extension from "html" to "action," in struts.xml. Then I had to do a bunch of filter re-ordering and dispatcher changes. Before, with a .html extension, I had all filters mapped to /* and in the following order.

Filter Name	Dispatchers
securityFilter	request
rewriteFilter	request, forward
struts-prepare	request
sitemesh	request, forward, include
staticFilter	request, forward
struts	request

Similar to Spring MVC, I had to remove the rewriteFilter in front of the securityFilter and I was able to remove the staticFilter. I also had to map the struts filter to *.action instead of /* to stop Struts from trying to catch static asset and DWR/CXF requests. Below is the order of filters and their dispatchers that seems to work best.

Filter Name	Dispatchers
rewriteFilter	request
securityFilter	request, forward, include
struts-prepare	request, forward
sitemesh	request, forward, include
struts	forward

From there, it was a matter of modifying urlrewrite.xml to have the following catch-all and rules for static assets, j_security_check and DWR/CXF.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.action$3</to>
</rule>
<outbound-rule match-type="regex">
    <from>^(.*)\.action(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

JSF
JSF was by far the most difficult to get extensionless URLs working with. I'm not convinced it's impossible, but I spent a several hours over a few days and was unsuccessful in completely removing them. I was able to make things work so I could request pages without an extension, but found when clicking buttons and links, the extension would often show up in the URL. I'm also still using JSF 1.2, so it's possible that upgrading to 2.0 would solve many of the issues I encountered.

For the time being, I've changed my FacesServlet mapping from *.html to *.jsf. As with Struts, I had issues when I tried to map it to /app/*. Other changes include changing the order of dispatchers and filters, the good ol' catch-all in urlrewrite.xml and modifying security.xml. For some reason, I wasn't able to get file upload working without adding an exception to the outbound-rule.

<rule match-type="regex">
    <from>^([^?]*)/([^?/\.]+)(\?.*)?$</from>
    <to last="true">$1/$2.jsf</to>
</rule>
<outbound-rule match-type="regex">
  <!-- TODO: Figure out how to make file upload work w/o using *.jsf -->
    <condition type="path-info">selectFile</condition>
    <from>^(.*)\.jsf(\?.*)?$</from>
    <to last="false">$1$2</to>
</outbound-rule>

I also spent a couple hours trying to get Pretty Faces to work. I wrote about my issues on the forums. I tried writing a custom Processor to strip the extension, but found that I'd get into an infinite loop where the processor kept getting called. To workaround this, I tried using Spring's RequestContextHolder to ensure the processor only got invoked once, but that proved fruitless. Finally, I tried inbound and outbound custom processors, but failed to get those working. The final thing I tried was url-mappings for each page in pretty-config.xml.

<url-mapping>
  <pattern value="/admin/users"/>
  <view-id value="/admin/users.jsf"/>
</url-mapping>
<url-mapping>
  <pattern value="/mainMenu"/>
  <view-id value="/mainMenu.jsf"/>
</url-mapping>

The issue with doing this was that some of the navigation rules in my faces-config.xml stopped working. I didn't spend much time trying to diagnose the problem because I didn't like having to add an entry for each page in the application. The one nice thing about Pretty Faces is it did allow me to do things like the following, which I formerly did with a form that auto-submitted when the page loaded.

<url-mapping>
  <pattern value="/passwordHint/#{username}"/>
  <view-id value="/passwordHint.jsf"/>
  <action>#{passwordHint.execute}</action>
</url-mapping>

Conclusion
My journey implementing extensionless URLs was an interesting one, and I solidified my knowledge about ordering of filters, dispatchers and the UrlRewriteFilter. I still think I have more to learn about properly implementing extensionless URLs in Struts 2 and JSF and I hope to do that in the near future. I believe Struts' Convention Plugin will help me and JSF 2 + Pretty Faces will hopefully work nicely too. Of course, it'd be great if all Java Web Frameworks had an easy mechanism for producing and consuming extensionless URLs. In the meantime, thank goodness for the UrlRewriteFilter.

If you'd like to try AppFuse and its shiny new URLs, see the QuickStart Guide and choose the 2.1.0-SNAPSHOT version.