Matt RaibleMatt Raible is a Web Developer and Java Champion. Connect with him on LinkedIn.

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Java Web Application Security - Part I: Java EE 6 Login Demo

Back in February, I wrote about my upcoming conferences:

In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk:

Webapp Security: Develop. Penetrate. Protect. Relax.
In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL.

After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators.

Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks:

  • Implement Basic Authentication
  • Implement Form-based Authentication
  • Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs)
  • Force SSL for certain URLs
  • Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone)
  • Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone)
  • Encrypt Passwords
  • Secure methods with annotations

For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities.

I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial.


Java EE 6 Login Tutorial

Download and Run the Application
To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users.

Implement Basic Authentication
The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml:

<security-constraint>
    <web-resource-collection>
        <web-resource-name>users</web-resource-name>
        <url-pattern>/users</url-pattern>
        <http-method>GET</http-method>
        <http-method>POST</http-method>
    </web-resource-collection>
    <auth-constraint>
        <role-name>ROLE_ADMIN</role-name>
    </auth-constraint>
</security-constraint>

<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Java EE Login</realm-name>
</login-config>

<security-role>
    <role-name>ROLE_ADMIN</role-name>
</security-role>

At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after </webAppConfig> in the Jetty plugin's configuration.

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.HashLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/realm.properties</config>
    </loginService>
</loginServices>

The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login.

After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out.

public void logout(HttpServletResponse response) throws ServletException, IOException {
    request.getSession().invalidate();
    response.sendRedirect(request.getContextPath());
}

You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication.

Implement Form-based Authentication
To change from basic to form-based authentication, you simply have to replace the <login-config> in your web.xml with the following:

<login-config>
    <auth-method>FORM</auth-method>
    <form-login-config>
        <form-login-page>/login.jsp</form-login-page>
        <form-error-page>/login.jsp?error=true</form-error-page>
    </form-login-config>
</login-config>

The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog.

Force SSL
Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same <security-constraint> you already have in web.xml, add the following after </auth-constraint>:

<user-data-constraint>
    <transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>

To configure Jetty to listen on an SSL port, add the following just after </loginServices> in your pom.xml:

<connectors>
    <connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8080</port>
    </connector>
    <connector implementation="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <forwarded>true</forwarded>
        <port>8443</port>
        <maxIdleTime>60000</maxIdleTime>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <password>appfuse</password>
        <keyPassword>appfuse</keyPassword>
    </connector>
</connectors>

The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>keytool-maven-plugin</artifactId>
    <version>1.0</version>
    <executions>
        <execution>
            <phase>generate-resources</phase>
            <id>clean</id>
            <goals>
                <goal>clean</goal>
            </goals>
        </execution>
        <execution>
            <phase>generate-resources</phase>
            <id>genkey</id>
            <goals>
                <goal>genkey</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <keystore>${project.build.directory}/ssl.keystore</keystore>
        <dname>cn=localhost</dname>
        <keypass>appfuse</keypass>
        <storepass>appfuse</storepass>
        <alias>appfuse</alias>
        <keyalg>RSA</keyalg>
    </configuration>
</plugin>

Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you.

Store Users in a Database
Finally, to store your users in a database instead of file, you'll need to change the <loginService> in the Jetty plugin's configuration. Replace the existing <loginService> element with the following:

<loginServices>
    <loginService implementation="org.eclipse.jetty.security.JDBCLoginService">
        <name>Java EE Login</name>
        <config>${basedir}/src/test/resources/jdbc-realm.properties</config>
    </loginService>
</loginServices>

The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information.

jdbcdriver = com.mysql.jdbc.Driver
url = jdbc:mysql://localhost/appfuse
username = root
password =
usertable = app_user
usertablekey = id
usertableuserfield = username
usertablepasswordfield = password
roletable = role
roletablekey = id
roletablerolefield = name
userroletable = user_role
userroletableuserkey = user_id
userroletablerolekey = role_id
cachetime = 300

Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands:

mysql -u root -p -e 'create database appfuse'
curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql
mysql -u root -p appfuse < create-appfuse.sql

Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the <configuration> element (before <executions>) in pom.xml:

<dependencies>
    <!-- MySQL for JDBC Realm -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>5.1.14</version>
    </dependency>
</dependencies>

Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example:

$ sh jetty-password.sh javaeelogin
javaeelogin
OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur
MD5:53b176e6ce1b5183bc970ef1ebaffd44

The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement.

UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin';

Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users.

Summary
In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0:

  • authenticate(response)
  • login(user, pass)
  • logout()

This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely:

  • No error messages for failed logins
  • No Remember Me
  • No auto-redirect from HTTP to HTTPS
  • Container has to be configured
  • Doesn’t support regular expressions for URLs

Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain.

In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions.

Posted in Java at May 05 2011, 04:58:00 PM MDT 9 Comments

Be careful when switching MySQL to UTF-8

Earlier this week, I noticed a couple strange issues on this blog and sent an email to the roller-user mailing list. I figured both issues were caused by my upgrade to Roller 5. Basically, my tag cloud wasn't working and I noticed a bunch of blog entries that had truncated data. I'd provide links to the truncated posts, but I believe I've fixed them, so links would be useless. This post is to make others aware of something I wasn't: be careful when switching MySQL to UTF-8.

The first issue, 404s from my tag cloud links, was something my theme was missing. It seems the tagIndex action is new in Roller 5 and required for tag clouds to work. To fix this issue, I had to add the following XML to my theme's theme.xml file.

<template action="tagsIndex">
    <name>TagsIndex</name>
    <description>Tag index page</description>
        <link></link>
    <navbar>false</navbar>
    <hidden>true</hidden>
    <templateLanguage>velocity</templateLanguage>
    <contentType>text/html</contentType>
    <contentsFile>Weblog.vm</contentsFile>
</template>

Since I wanted to replicate Roller 4's behavior, pointing the contentsFile to Weblog.vm worked just fine. The nice thing is I can always change it to another page and customize it to show more information about the selected tag.

The 2nd issue, data truncation, was a bit trickier. I thought it might've been something Roller did when upgrading my database from Roller 4 to 5. I didn't suspect upgrading from MySQL 3 to 5 would cause it. From my previous upgrade post:

At this point, I figured my database might be slightly hosed, but since it was simply creating tables, I was probably OK. I restarted Tomcat and left the old version in place while I waited for a MySQL 5 database instance from my hosting provider, KGB Internet. Once I got the new instance, I imported my backed-up database, ran the upgrade script and everything worked just peachy.

Keith at KGB looked into my issue and thought the problem was the charset. My old MySQL 3.x database used latin1 while my MySQL 5.x database uses UTF-8. The symptom looked familiar:

Be careful when switching to UTF-8. Once you have converted your data, any program/webapp that uses the database will have to check that the data they are sending to the database is valid UTF-8. If it isn't then MySQL will silently truncate the data after the invalid part, which can cause all sorts of problems.

Luckily, I had a backup of my pre-upgrade database and was able to convert and recover everything successfully with a little iconv, perl and numerous mysqldump and mysql import commands. Of course, it's possible there's still some jacked entries and comments. If you noticed any truncation, please let me know and I'll get them fixed.

Posted in Roller at Apr 28 2011, 08:11:40 PM MDT 2 Comments

Farewell to the 2010-2011 Ski Season

I'd call the 2010-2011 the best ski season ever, but it's really just the best ski season so far. In 2008, I wrote about a great 21-day season. This time last year, I wrote about an amazing 25-day season. This year, I took it up a notch and aimed for 30 days. I'm proud to say I accomplished my goal and had an awesome time doing it. I skied with more people I'd never skied with before (largely in part to my cool co-workers from Overstock) and shared many days with the lovely Trish McGinity.

The season started with a trip to Copper, shortly after Abbie's 8th Birthday. I remember that day clearly as the kids were a bit rusty and had a heckuva time on their first run. Sobbing, whining and fear surrounded them the entire time. After the first run, I had some hot chocolate with them, calmed them down and then proceeded to the bunny slope for some turns. The lift was broken when we got there so we had to hike for a few runs. Amazingly, Abbie said it was the most fun she'd ever had skiing, which surprised me after her meltdown on the first run.

It's fun to compare that day to the last day I took them this season. We did the same run (a blue at Copper) and both kids were doing parallel turns and having a blast. Actually, Jack was the only one doing parallel turns, Abbie was flying down the mountain, not turning at all. She was going so fast her legs looked like rubber bands, weaving and bobbing over the bumps in the snow. I'm awful proud of my little skiers.

As for me, I happened to land a new gig in Utah, home of the greatest snow on Earth. My interview with Overstock.com was two days, with the 2nd day on the slopes at Snowbird. It was easily the best interview I've ever had.

Snowbird! Mike, Sean and Chris Sun over Snowbird Back of Snowbird

That week, I returned to Denver for 3 days of skiing Breckenridge and A-Basin for Trish's Birthday Weekend. After returning from Christmas in Florida, I got a couple days in at Mary Jane and then accomplished 10 days before 2011 while skiing in sub-zero temperatures at Steamboat for New Years.

Good Morning from Steamboat! Sunrise over Steamboat

The next 4 months of skiing were fantastic with many firsts. I experienced Alta, Crested Butte and thigh-deep powder for the first time.

Speed Racer! Top of Crested Butte

Free Heeling at Alta Free Heeling at Alta

We finished up the season with a hut trip after TSSJS in Vegas, a weekend with the kids at Copper (as mentioned above) and Spring Splash at Winter Park.

For next year, I think I'll keep my goal at 30 days. If everything works out as planned, we'll have a place in the mountains this fall and it'll be a bit easier to hit the slopes without sitting in traffic. For now, I'm pumped about the beginning of mountain bike season. I took Trish and I's Gary Fisher Hi-Fi Plus's to the shop for tune-ups yesterday and we have a trip planned to Moab for Memorial Day. It's gonna be a great summer. :)

Posted in General at Apr 28 2011, 09:40:08 AM MDT 3 Comments

Two Opening Days with a Stopover in Kraków

Opening Day is a special event in Denver. The night before, it feels like the whole city is alive in anticipation of the big event. On Opening Day, it's typically a gorgeous spring day and serves as a great kickoff to baseball season. This year, we decided to take things up a notch and hit two opening days instead of one. The dates just happened to line up so we could go to the Rockies Home Opener on April 1st, fly to Kraków for the 33rd Degree Conference and make it back to Boston for the Red Sox Home Opener. Since Trish's brother lives near Boston, and I have good friends there, it sounded like the perfect vacation. To make a crazy vacation schedule even crazier, Trish and I moved in together the day before it all started. With moving and trying to finish my basement sauna before we left, we've definitely had a hectic few weeks.

Nightmare with water? Yeah, Trish'll do that to ya! Sarah and Joe Rockies Opener! Cargo!

After attending the Rockies Home Opener and having a great time with friends, we got to bed early and woke up on Saturday for our flight to Kraków. It was a 2 o'clock flight, so we got lots of sleep and then proceeded to thoroughly enjoy our flight when we upgraded to Business Class from Chicago to Munich. Business Class is the way to travel internationally. We arrived just after noon on Sunday and spent the afternoon exploring Kraków's Old Town and trying to stay awake. The weather was beautiful and it seemed like it might've been the warmest day of the year.

St. Mary's Basilica, Kraków Main Market Square St. Mary's Flowers

On Monday, we spent more time in the center of Kraków, wandering through the Main Market Square, Wawel Castle and the very cool Dragon's Den. We had lunch outside, again enjoying the great weather and some local beers. We were surprised to find that kamikaze shots are served in groups of four, rather than just one like it's done in the US. That evening, we enjoyed an excellent Italian dinner at Aqua e Wino.

The Grunwald Monument Church of St. Adalbert Renaissance courtyard of Wawel Castle Wawel Hill

On Tuesday, we headed to Aushwitz. This was a very sobering experience, but I'm glad we did it. It made me wonder if this type of thing could happen again, only to realize that it has. That evening, we sipped on martinis at the Metropolitan.

Auschwitz concentration camp Rudolf Höss was hanged here on 16 April 1947. Auschwitz Martinis at Metropolitan

On Wednesday, I delivered my talk on Comparing JVM Web Frameworks. You can download the PDF or view the presentation on Slideshare if you're interested. The conference itself had a spectacular schedule and speaker lineup, so I was a little disappointed I didn't attend any sessions. We did make it to the ZeroTurnaround Party that night and had a lot of fun talking to Grzegorz, Martijn and Anton.

We woke up Thursday and headed to the airport for our flight back to the US. We landed in Boston at 6:30 pm and headed to my friend Chris's house in Concord. You might remember Chris from my first game at Fenway Park. Friday, we joined other friends, hopped on the train and headed to Yawkey Way for a beer before the game. Our seats were in the bleachers, but we had a fantastic time watching the Red Sox win their first game of the year.

Fenway Paak! Morse and Kidder Happy Siblings Erika and Julie Red Sox Win!

We went to another game on Saturday with Trish's brother and a friend of his. We then proceeded to spend a relaxing Lazy Sunday with his family before flying back Monday morning.

Thanks to all our friends who participated in the opening day festivities as well as to Grzegorz Duda for inviting me to speak at 33rd Degree. We had a blast!

If you'd like to see more pictures from this adventure, please see Two Opening Days with a Stopover in Kraków on Flickr.

Posted in Java at Apr 14 2011, 09:40:47 AM MDT Add a Comment

AppFuse 2.1 Released!

The AppFuse Team is pleased to announce the release of AppFuse 2.1. This release includes upgrades to all dependencies to bring them up-to-date with their latest releases. Most notable are JPA 2, JSF 2, Tapestry 5 and Spring 3. In addition, we've migrated from XFire to CXF and enabled REST for web services. There's even a new appfuse-ws archetype that leverages Enunciate to generate web service endpoints, documentation and downloadable clients. This release fixes many issues with archetypes, improving startup time and allowing jetty:run to be used for quick turnaround while developing. For more details on specific changes see the release notes.

What is AppFuse?
AppFuse is an open source project and application that uses open source frameworks to help you develop Web applications with Java quickly and efficiently. It was originally developed to eliminate the ramp-up time when building new web applications. At its core, AppFuse is a project skeleton, similar to the one that's created by your IDE when you click through a wizard to create a new web project. If you use JRebel with IntelliJ, you can achieve zero-turnaround in your project and develop features without restarting the server.

Release Details
Archetypes now include all the source for the web modules so using jetty:run and your IDE will work much smoother now. The backend is still embedded in JARs, enabling you to choose with persistence framework (Hibernate, iBATIS or JPA) you'd like to use. If you want to modify the source for that, add the core classes to your project or run "appfuse:full-source".

AppFuse comes in a number of different flavors. It offers "light", "basic" and "modular" and archetypes. Light archetypes use an embedded H2 database and contain a simple CRUD example. Light archetypes allow code generation and full-source features, but do not currently support Stripes or Wicket. Basic archetypes have web services using CXF, authentication from Spring Security and features including signup, login, file upload and CSS theming. Modular archetypes are similar to basic archetypes, except they have multiple modules which allows you to separate your services from your web project.

AppFuse provides archetypes for JSF, Spring MVC, Struts 2 and Tapestry 5. The light archetypes are available for these frameworks, as well as for Spring MVC + FreeMarker, Stripes and Wicket. You can see demos of these archetypes at http://demo.appfuse.org.

For information on creating a new project, please see the QuickStart Guide.

If you have questions about AppFuse, please read the FAQ or join the user mailing list. If you find any issues, please report them on the mailing list or create an issue in JIRA.

Thanks to everyone for their help contributing patches, writing documentation and participating on the mailing lists.

We greatly appreciate the help from our sponsors, particularly Atlassian, Contegix and JetBrains. Atlassian and Contegix are especially awesome: Atlassian has donated licenses to all its products and Contegix has donated an entire server to the AppFuse project.

Posted in Java at Apr 04 2011, 09:38:05 AM MDT 5 Comments

Peter Estin Hut Trip in Colorado's High Country

Last weekend, after returning from our trip to Las Vegas, we packed up our stuff, got a good night's sleep and headed on a hut trip in Colorado's High Country. The name of the hut was Peter Estin Hut and it was a bit of a hike to get to. My friend, Joe, set my expectations correctly when he warned me it'd be a 5 hour death march. It took us 4 hours, 30 minutes and we skied up 2200 vertical feet of switchbacks.

From the Parking Lot Joe and Sean Made it! View off the front port of Peter Estin Hut

The hut itself was great. It was a lot like The Cabin, but bigger, slept a lot more people and was at 11,200 feet. It had an old fashioned cookstove, heat stove (where we melted snow) and a two-room outhouse.

The Roomy Inside Water Source Peter Estin Hut

On Sunday, we hiked up to the top of Charles Peak (12,000') and had a nice (albeit choppy) run down and packed up to hike down.

Livin' it up, hut style We made it! At the top of Charles Peak

This was the first hut trip I'd been on since Last Dollar Hut near Telluride in college. It was definitely a physical challenge, but was a lot of fun thanks to good friends, nice views and "summit push" music from Joe's boombox.

For more pictures from this adventure, check out my Peter Estin Hut Trip photos on Flickr.

Posted in General at Mar 26 2011, 11:20:12 AM MDT Add a Comment

Livin' it up in Vegas at TSSJS 2011

Last Wednesday, Trish and I traveled to Las Vegas for TheServerSide Java Symposium 2011 conference. We had a free room from TechTarget, but opted to upgrade to a suite with a view over the Bellagio Fountains. Trish won a trip to Vegas as a sales award earlier in the year and cleverly exchanged it for cash, so our upgrade was sort of free.

Caesars Pool The Bellagio Fountains

My first talk was on Online Video and my experience at Time Warner Cable. With my former team's iPad app releasing the day before, it was a fun session. The attendance was kind of sparse, but I had some good competition so wasn't surprised.

After I finished speaking, we headed to happy hour and met up with some friends that happened to be in town. We had dinner at the Todd English Pub and headed to the Penn & Teller show at the Rio. We closed the night after Trish had a 45-minute roll at the craps table at O'Sheas.

We slept in on Thursday and I gave my Comparing JVM Web Frameworks talk that afternoon. I made sure to mention some other methods to choosing web frameworks: doing performance comparisons like Peter Thomas has done or choosing Lift because one of its developers says it's the best. While Vaadin did sneak into the #5 spot, I made sure and mentioned that Wicket and Tapestry seem to belong there moreso (based on stats, mailing list traffic, etc.).

Trish took a bunch of pictures during my talk, which had a great turnout and lots of participation.

Getting Intro'd My Intro My Dream on Display

The Problem How do you choose? Choosing a Framework

That evening, we celebrated St. Patty's Day with some college buddies of mine, ate great sushi at Mizuya and experienced the joys of three card poker. Thanks to TechTarget for inviting me to TSSJS 2011; we had an awesome time. You can find all the pictures we took on Flickr.

P.S. If you can't see the presentations in this post (a.k.a. you don't have Flash), you can view them on on Slideshare or download the PDFs.

Posted in Java at Mar 22 2011, 09:04:17 AM MDT Add a Comment

Adding Search to AppFuse with Compass

Over 5 years ago, I recognized that AppFuse needed to have a search feature and entered an issue in JIRA. Almost 4 years later, a Compass Tutorial was created and shortly after Shay Banon (Compass Founder), sent in a patch. From the message he sent me:

A quick breakdown of enabling search:

  1. Added Searchable annotations to the User and Address.
  2. Defined Compass bean, automatically scanning the model package for mapped searchable classes. It also automatically integrates with Spring transaction manager, and stores the index on the file system ([work dir]/target/test-index).
  3. Defined CompassTemplate (similar in concept to HibernateTemplate).
  4. Defined CompassSearchHelper. Really helps to perform search since it does pagination and so on.
  5. Defined CompassGps, basically it allows for index operation allowing to completely reindex the data from the database. JPA and Hiberante also automatically mirror changes done through their API to the index. iBatis uses AOP.

Fast forward 2 years and I finally found the time/desire to put a UI on the backend Compass implementation that Shay provided. Yes, I realize that Compass is being replaced by ElasticSearch. I may change to use ElasticSearch in the future; now that the search feature exists, I hope to see it evolve and improve.

Since Shay's patch integrated the necessary Spring beans for indexing and searching, the only thing I had to do was to implement the UI. Rather than having an "all objects" results page, I elected to implement it so you could search on an entity's list screen. I started with Spring MVC and added a search() method to the UserController:

@RequestMapping(method = RequestMethod.GET)
public ModelAndView handleRequest(@RequestParam(required = false, value = "q") String query) throws Exception {
    if (query != null && !"".equals(query.trim())) {
        return new ModelAndView("admin/userList", Constants.USER_LIST, search(query));
    } else {
        return new ModelAndView("admin/userList", Constants.USER_LIST, mgr.getUsers());
    }
}

public List<User> search(String query) {
    List<User> results = new ArrayList<User>();
    CompassDetachedHits hits = compassTemplate.findWithDetach(query);
    log.debug("No. of results for '" + query + "': " + hits.length());
    for (int i = 0; i < hits.length(); i++) {
        results.add((User) hits.data(i));
    }
    return results;
}

At first, I used compassTemplate.find(), but got an error because I wasn't using an OpenSessionInViewFilter. I decided to go with findWithDetach() and added the following search form to the top of the userList.jsp page:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms"/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

NOTE: I tried using HTML5's <input type="search">, but found Canoo WebTest doesn't support it.

Next, I wrote a unit test to verify everything worked as expected. I found I had to call compassGps.index() as part of my test to make sure my index was created and up-to-date.

public class UserControllerTest extends BaseControllerTestCase {
    @Autowired
    private CompassGps compassGps;
    @Autowired
    private UserController controller;

    public void testSearch() throws Exception {
        compassGps.index();
        ModelAndView mav = controller.handleRequest("admin");
        Map m = mav.getModel();
        List results = (List) m.get(Constants.USER_LIST);
        assertNotNull(results);
        assertTrue(results.size() >= 1);
        assertEquals("admin/userList", mav.getViewName());
    }
}

After getting this working, I started integrating similar code into AppFuse's other web framework modules (Struts, JSF and Tapestry). When I was finished, they all looked pretty similar from a UI perspective.

Struts:

<div id="search">
<form method="get" action="${ctx}/admin/users" id="searchForm">
    <input type="text" size="20" name="q" id="query" value="${param.q}"
           placeholder="Enter search terms..."/>
    <input type="submit" value="<fmt:message key="button.search"/>"/>
</form>
</div>

JSF:

<div id="search">
<h:form id="searchForm">
    <h:inputText id="q" name="q" size="20" value="#{userList.query}"/>
    <h:commandButton value="#{text['button.search']}" action="#{userList.search}"/>
</h:form>
</div>

Tapestry:

<div id="search">
<t:form method="get" t:id="searchForm">
    <t:textfield size="20" name="q" t:id="q"/>
    <input t:type="submit" value="${message:button.search}"/>
</t:form>
</div>

One frustrating thing I found was that Tapestry doesn't support method="get" and AFAICT, neither does JSF 2. With JSF, I had to make my UserList bean session-scoped or the query parameter would be null when it listed the results. Tapestry took me the longest to implement, mainly because I had issues figuring out how it's easy-to-understand-once-you-know onSubmit() handlers worked and I had the proper @Property and @Persist annotations on my "q" property. This tutorial was the greatest help for me. Of course, now that it's all finished, the code looks pretty intuitive.

Feeling proud of myself for getting this working, I started integrating this feature into AppFuse's code generation and found I had to add quite a bit of code to the generated list pages/controllers.

So I went on a bike ride...

While riding, I thought of a much better solution and added the following search method to AppFuse's GenericManagerImpl.java. In the code I added to pages/controllers previously, I'd already refactored to use CompassSearchHelper and I continued to do so in the service layer implementation.

@Autowired
private CompassSearchHelper compass;

public List<T> search(String q, Class clazz) {
    if (q == null || "".equals(q.trim())) {
        return getAll();
    }

    List<T> results = new ArrayList<T>();

    CompassSearchCommand command = new CompassSearchCommand(q);
    CompassSearchResults compassResults = compass.search(command);
    CompassHit[] hits = compassResults.getHits();

    if (log.isDebugEnabled() && clazz != null) {
        log.debug("Filtering by type: " + clazz.getName());
    }

    for (CompassHit hit : hits) {
        if (clazz != null) {
            if (hit.data().getClass().equals(clazz)) {
                results.add((T) hit.data());
            }
        } else {
            results.add((T) hit.data());
        }
    }

    if (log.isDebugEnabled()) {
        log.debug("Number of results for '" + q + "': " + results.size());
    }

    return results;
}

This greatly simplified my page/controller logic because now all I had to do was call manager.search(query, User.class) instead of doing the Compass login in the controller. Of course, it'd be great if I didn't have to pass in the Class to filter by object, but that's the nature of generics and type erasure.

Other things I learned along the way:

  • To index on startup, I added compassGps.index() to the StartupListener..
  • In unit tests that leveraged transactions around methods, I had to call compassGps.index() before any transactions started.
  • To scan multiple packages for searchable classes, I had to add a LocalCompassBeanPostProcessor.

But more than anything, I was reminded it always helps to take a bike ride when you don't like the design of your code. ;-)

This feature and many more will be in AppFuse 2.1, which I hope to finish by the end of the month. In the meantime, please feel free to try out the latest snapshot.

Posted in Java at Mar 15 2011, 05:11:12 PM MDT 1 Comment

WebSockets with Johnny Wey at Denver JUG

This evening, I attended Denver JUG to hear Johnny Wey talk about WebSockets. This month, the location moved and even though I had a nice bike ride to the meeting, I showed up about 20 minutes late. Johnny's talk lasted about 40 minutes, so I missed the first half.

When I arrived, he was talking about workarounds for implementing push applications in browsers. He had a slide that talked about Comet and iframes as the common implementation, and the other major option being ActionScript's XMLSocket. The biggest issues with XMLSocket (according to Johnny) are:

  • Not available on many modern mobile platforms.
  • Flash and managing / detecting plugin versions can add unwanted complexity.
  • Many would consider Flash solutions deprecated.

The biggest issue with implementing push on a client is managing it all, especially if you need to support older browsers. Socket.IO is one possible solution. It rides on the coattails of node.js. Features of Socket.IO include:

  • Abstracts socket methods into a unified API.
  • Open source (MIT) with active community.
  • Multiple server implementations (including Java) with the "reference" implementation developed in node.js.

The client API looks as follows:

var socket = new io.Socket(); 
socket.on('connect', function(){ 
  socket.send('hi!'); 
}) 
socket.on('message', function(data){ 
  alert(data);
})
socket.on('disconnect', function(){}) 

jWebSocket is another solution and it's where a lot of the Java WebSocket development is ending up right now. Highlights about the project include:

  • Open source (LGPL) with relatively active community.
  • Servlet-like API.
  • More "enterprisey" than Socket.IO.

Other options include CometD, which is a Dojo-driven Comet implementation that uses a specification called Bayeux. Jetty and GlassFish both support WebSockets in various forms of functionality and stability. Finally, there's Pusher (a SaaS implementation of push with a RESTful API) and Atmosphere (a container-agnostic framework).

How do you scale web sockets? The same way you make a webapp scale:

  • Go stateless
  • Use short request / response cycle
  • Use the smallest payload possible
  • Cache as much as possible

Scaling challenges with web sockets:

  • Connections have intrinsic state (they never close!)
  • Communications pipeline to your app server
  • Some sort of introspection on LB side (JMX)

There's also some existing controversy in the WebSockets Community, mostly around using Upgrade vs. CONNECT with HTTP. An (IETF) experiment found Upgrade portion of HTTP protocol was often improperly implemented by proxy servers and other network hardware. This seems to have caused Google Chrome to deprecate using Upgrade in favor of CONNECT. CONNECT used in this manner is seen by many as an abuse of the web.

Other useful links that Johnny provided were What can I use… to find out native support across browsers. For example, you can see which browsers support websockets. He also pointed out that websocket.org provides a good intro to WebSockets.

I'm glad I attended Johnny's talk. I've been a little leery of using WebSockets in my applications because of older browsers. Now that I'm aware of frameworks (like Socket.IO) that solve this problem, I'm eager to try it when the need arises.

Related: Dojo/Comet support in Java Web Frameworks

Posted in Java at Mar 09 2011, 07:10:12 PM MST Add a Comment

JSR 303 and JVM Web Framework Support

Emmanuel Bernard recently sent an email to the JSR 303 Experts Group about the next revision of the Bean Validation JSR (303). Rather than sending the proposed changes privately, he blogged about them. I left a comment with what I'd like to see:

+1 for Client-side validation. I'd love to see an API that web frameworks can hook into to add "required" to their tags for HTML5. Or some service that can be registered so the client can make Ajax requests to an API to see if an object is valid.

Emmanuel replied that most of the necessary API already exists for this, but frameworks have been slow to adopt it.

Hi Matt,

The sad thing is that the API is present on the Bean Validation side but presentation frameworks are slow to adopt it and use it :(

RichFaces 4 now has support for it but I wished more presentation frameworks had worked on the integration. If you can convince a few people or have access to a few people, feel free to send them by me :)

The integration API is described here. Let me know if you think some parts are missing or should be improved. We should definitely do some more buzz around it.

In the interest of generating more buzz around it, I decided to do some research and see what JVM Frameworks support JSR 303. Here's what I've come up with so far (in no particular order):

Struts 2 has an open issue, but doesn't seem to support JSR 303. Since I did a quick-n-dirty google search for most of these, I'm not sure if they support client-side JavaScript or HTML5's required. If you know of other JVM-based web frameworks that support JSR 303, please let me know in the comments.

Posted in Java at Mar 08 2011, 11:33:24 AM MST 4 Comments