Matt RaibleMatt Raible is a Java Champion and Developer Advocate at Okta. developer.okta.com

The Angular Mini-Book The Angular Mini-Book is a guide to getting started with Angular. You'll learn how to develop a bare-bones application, test it, and deploy it. Then you'll move on to adding Bootstrap, Angular Material, continuous integration, and authentication.

Spring Boot is a popular framework for building REST APIs. You'll learn how to integrate Angular with Spring Boot and use security best practices like HTTPS and a content security policy.

For book updates, follow @angular_book on Twitter.

The JHipster Mini-Book The JHipster Mini-Book is a guide to getting started with hip technologies today: Angular, Bootstrap, and Spring Boot. All of these frameworks are wrapped up in an easy-to-use project called JHipster.

This book shows you how to build an app with JHipster, and guides you through the plethora of tools, techniques and options you can use. Furthermore, it explains the UI and API building blocks so you understand the underpinnings of your great application.

For book updates, follow @jhipster-book on Twitter.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

Open Source CMS Evaluation - Part II: Customization

This is the third and final post in my quest to find the best open source CMS for my needs. Previous posts include Building a website with an Open Source CMS and Open Source CMS Evaluation - Part I: Installation. Based on these two posts, reader feedback, and my installation experience, the final round of candidates include Joomla, Drupal, Magnolia, OpenCms and MeshCMS. These are listed in the order that I expected the final rankings to be - just to let you know what my feelings were going into this final process. ;-)

The CMS I choose will be used to build Virtuas's website, as well as customers of Virtuas. By using a CMS to produce websites, the design is separated from the content - and the "adding content" process can be much easier for the customer. This greatly simplifies the "creating a website" process for us, and will likely save our customers a fair amount of design costs. In addition, it makes it much easier for site owner's to maintain the site after it's been published.

As for Virtuas's site, it's static right now, and we'd like to change that. We want to ability to show recent blog posts on the front page, as well as make it easy for Practice Leaders to publish articles. In addition, it should be easy for our designers to change the design (1-2 files) and for our marketing team to add press releases and update existing content. Tomorrow, I'll be presenting my choice to the rest of the team, and we hope to design and start publishing content this week. Our goal is to have a new Virtuas site up and running one week from today.

My goal today was to see how easy each CMS was to customize. In addition, I wanted to see how easy it was to publish an article, as well as to aggregate our latest RSS feed titles onto the homepage. To test the design customizability, I tried to reproduce the current Virtuas homepage. Then I published Jeff's Geronimo Article, and attempted to aggregate feeds from Maria's and Bruce's blogs. My main reason for putting the Java CMS'es lower than the PHP ones in my suspected order of finishing is because I don't they don't have the RSS Aggregation feature.

Rather than just jumping in and using each CMS in anger, I tried to start off by reading the documentation for each. My main focus was on how to customize, but I also looked for an RSS Aggregation feature and ease-of-publishing for articles. I read documentation for 15-20 minutes, then dived into creating a custom theme and adding content. I installed each CMS on my PowerBook, and used Safari and Firefox on OS X, as well as Firefox on Windows XP in some cases.

MeshCMS - I spent 40 minutes looking into MeshCMS before I knew it wasn't the one. The main problem I had with it was the upgrade process. To upgrade to a new version, they recommend that you use symlinks to your files and store them in a separate location on the file system. While this may work for some, it seems a little brittle to me. I'd rather use a solution that keeps everything stored outside of the application by default. Creating a new theme was quite easy - but there didn't seem to be any support for multiple menus (i.e. global and local navigation), nor was there any means to customize the menu template.

I did manage to blow up the whole application at one point, simply because I was missing a JSP tag in my template. Since I had this template selected for the administration as well - it hosed the whole application and spit out stack traces for each page. Luckily, renaming the template directory caused MeshCMS change to revert to the default settings and everything was fixed. The interesting thing about MeshCMS is it looks very similar to the SiteMesh+JSPWiki CMS I wrote a few weeks ago. However, mine allowed full menu creation by editing/creating wiki pages.

OpenCms - When I installed OpenCms a couple of days ago, I initially did it on Windows. Everything worked fine and after waiting 18 minutes for everything to import, I was able to browse and edit the default site. However, today was a different story. The version I installed on my Windows XP box no longer works. When I got to http://localhost:8080/opencms/, I get a directory listing with "resources" and "setup" on it. The same day I installed OpenCms on Windows, I installed it on my PowerBook. It took 38 minutes to complete, but nevertheless, it said the process worked. Today I re-ran the setup and now I have the same result as on my Windows XP box. If the setup and installation is this fragile, I'm not interested. Blame me and the fact that I'm a redneck all you like, but the Magnolia installation is still functioning just fine. ;-)

Magnolia - I spent about a half hour with Magnolia before I knew it wasn't for me. While the admin UI is impressive with all it's Ajax goodies, creating a new template is cumbersome and not designer-friendly at all. You have to create a "new node" and then a bunch of "data nodes" under that. The documentation (a QuickStart PDF) is 17 pages long and forgets to mention the "title" data node is needed before the template will show up properly as an option. Once you've created a new template in the admin UI, you have to create the template on your file system - inside the web application. This may make it difficult to upgrade if you're deploying Magnolia as a WAR. The worst part is after creating the template, you have to restart the server. WTF? That seems a bit ridiculous to me. Granted you'll likely be designing your master template in a development environment - but good luck installing Magnolia for a client and having them create a new template.

One of the most interesting things about Magnolia is most of the folks who've recommended it have highlighted that it's "built on the revolutionary Java Content Repository Standard JSR-170". While I can admire the technical merits of this effort, it doesn't necessarily make this a good product. A good product, IMHO, is easy and intuitive to use. The admin interface for Magnolia is not intuitive. I like the fact that I can right-click on a page/node/etc., but on my Mac (with Firefox and Safari), the real context menu shows up on top of the application menu after a second or two. I'm going to pass on Magnolia due to the fact that its not designer friendly, as well as the fact that templates can't be edited in the browser. It looks like something that might be very interesting for developers, but it's simply not friendly for HTML developers.

At this point, it's 10:30 p.m. on Sunday night. I need to make a decision before I go to bed tonight and I'm scheduled to meet with our designer at 7:00 a.m. to start implementing his new design. I haven't started the PHP options, and I've had a couple new ones recommended on my blog while doing this evaluation today. Mal recommended Exponent and Jacob recommended MySource Matrix.

Because I'm down to two choices (and I haven't tried to customize either one), I decided it was worth looking at both of these PHP solutions. Exponent installed easy enough, but MySource Matrix failed miserably. Joomla, Drupal and Exponent all had an easy-to-use web installer that *just worked*. MySource spit out a bunch of permissions errors (even after chmod -R 777 *) and told me I had to run .php files from the command line. Since the other options all installed easily, I decided not to continue evaluating MySource Matrix.

Exponent - I didn't spend very long looking at Exponent. At first, I didn't think it had any documentation b/c it was a bit difficult to find on their site. Maybe it's because they don't have a background set on their site and my browser defaults to gray - making the gray text difficult to read. Even w/o documentation, I was able to navigate around the default Exponent site and figure out how to edit content. It's an easy UI to use, but again I was disappointed to find the "corporate" theme doesn't have a white background. Most good web designers know to set a default background color - and it always annoys me when someone misses this step. It's possible they don't set the background on purpose - like Yahoo does.

The deal-breaker with this CMS was that I couldn't edit any files w/in my browser to change anything (all I wanted was a white background). While it's theme management and templating looks powerful - it's another file-based system where you have to configure everything and then upload it. I agree that this is likely the path that web designers will want to use to get started - but I think it's important that files can be tweaked on-the-fly. Using a good FTP tool is certainly an option, but I'd prefer theme-editing to be part of my CMS. The one thing I did like about this CMS was the clean URLs. Granted, they aren't static-looking by any means, but having a simple ?section=# seems cleaner than the multiple parameters that other systems use.

Drupal - This CMS seems to have a lot of things I want/need as first class citizens. A blog, news feed aggregation and the ability to provide pretty URLs (aliases) for more cryptic CMS-type URLs. I couldn't get the URL aliases to work, but I suspect I was doing something wrong and didn't give it enough attention. I didn't spend a whole lot of time with this CMS, but rather just browsed around the admin interface and read a bit about how to create themes. I installed the PHPTemplate engine, but never installed any themes. When I found out I couldn't edit any of the uploaded templates contents, I started to get discouraged by Drupal. One thing I found disappointing, with both Drupal and Joomla, is they seemed to hard-code my server name into many of their URLs. When I installed these applications on my PowerBook, I used "localhost" for the server name. When testing out things from my Windows box, I couldn't even login to Drupal b/c it kept redirecting me to "localhost". Joomla had a similar problem with localhost, except that it only screwed up stylesheet paths. I was still able to administer the application.

Joomla - My time with Drupal was short-lived, mainly because I was itching to start playing with Joomla - which I've heard a lot of good things about in the past week. Furthermore, it's got a really good-looking administration UI. It's the type of UI that a designer would look at and appreciate. When trying to edit pages from my Windows box - everything worked, but I couldn't save the page. No JavaScript errors or anything, there was simply no reaction. Editing pages and content from my PowerBook solved the problem. I was able to easily create a simple theme that looked like virtuas.com and upload it. The theme isn't perfect, but it was easy enough to create using the Velvet theme from Joomlashack.com as a template. The weird thing about Joomla, at least with the default install, is there's no notion of pages. Everything is some sort of news item. In the pages I created, I was also unable to remove all the authoring notation and other junk that I don't want to show. The admin UI had options to remove the stuff, but even after "applying" the changes, they still showed up in the reader view.

It's now 1:30 a.m. on Monday morning and the last three CMS's definitely didn't get the attention they deserved. Nevertheless, I think they're the best of the bunch. Not only were they much easier (and quicker) to install than the Java options, but their UIs are also good-looking and easy to use. Drupal and Joomla both seem like excellent choices. Drupal seems to be more of what I'm looking for since it has all the features I want, and allows aliasing of URLs to make it appear like a static site. However, Joomla is a lot more eye-catching and that alone makes me want to use it. Neither of these CMS'es seem to have a full-featured blogging engine, at least not one that's as good as Roller.

Conclusion: I'm going to recommend we use Joomla, and look into doing some URL Rewriting to pretty up it's URLs. I doubt there's a whole lot we can do, but I'd like to figure out a way to make them a bit more search engine friendly. Drupal seems like an excellent choice as well, but the fact that I can't edit templates from the UI kinda sucks.

Thanks for listening y'all - all your comments and feedback during this evaluation have been great.

Posted in Open Source at Sep 26 2005, 01:32:58 AM MDT 27 Comments

Open Source CMS Evaluation - Part I: Installation

Today I began my journey in evaluating Open Source CMS applications. The motivation for this adventure can be found in my post titled "Building a website with an Open Source CMS". Basically, I'd like to find a good solution to build small-to-medium size websites. I want full-control over the design and features, and it shouldn't be too hard to configure, install or administer. I received many suggestions on my initial post, and thanks to these comments, I'm considering the following CMS solutions:

The reason OpenCms didn't make this list is because of Tim Howland's first demo (he left the link in a comment). As soon as he dropped into a shell, I gave up. I also tried OpenCMS a couple of years ago and couldn't get it to work. I still have a bad taste in my mouth from that experience.

Of this list, I was able to eliminate 3 options quite quickly: Bricolage, MeshCMS and AtLeap. Bricolage b/c their site was down for days when I first started looking at CMSs. In addition, it's written in Perl - of which I've never written a single line - and I don't feel like I'd be well suited to customize a Perl product. As for MeshCMS and AtLeap, both of these were eliminated b/c of the lack of mailing list traffic. This represents a small community IMO and I'd hate to start using a product that's not well supported or well documented. This is unfortunate b/c I could probably customize these products easier than the rest. Nevertheless, it's my customers that are important, not me.

This leaves me with three Java solutions (Alfresco, Daisy, Magnolia) and two PHP solutions (Joomla, Drupal). I'm not too keen on including Alfresco b/c I've never heard of it, but readers of my last post recommended it, as well as my colleague Bill Dudney (another friend who forgot how to blog). At first glance, there are a couple of major differences between the Java and PHP solutions. The Java ones definitely take the cake on download size:

  • Daisy 1.3.1: 59 MB
  • Magnolia 2.1: 13.4 MB
  • Alfresco 0.6 (with Tomcat): 30.3 MB
  • Joomla 1.0.1: 1.7 MB
  • Drupal 4.6.3: 447 KB

Yes, large downloads is the nature of Java applications - but it's interesting to see the wide discrepancy b/c the 3 Java options listed here. In addition, it's a little disappointing that I can't download Alfresco standalone - why does it *have* to come with Tomcat or JBoss? Looking through the Tomcat directories and files - there doesn't seem to be any Alfresco-specific settings or files. Why can't I just download the WAR?

My goal in this session was to install these different CMSs and see which were the easiest to install - as well as which had the features I need to continue evaluated them. What I found was that Daisy is out of the running. This is primarily because it's a wiki, not a CMS. No wonder the JSPWiki guy was impressed with Daisy a few weeks back. Furthermore, the number of things you have to do to get Daisy installed is very long.

Magnolia, on the other hand, was very easy to install. Drop two WAR files into Tomcat and 10 minutes later you have a CMS. Yeah, that's a little scary - it took 10 minutes (647740 ms to be exact) to startup Tomcat after installing Magnolia. While I appreciate the easy installation, I'm still interested to see what kind of data store Magnolia is using. Is it using an embedded database or what? I didn't have time to look, but rather played around with the admin console a bit. I was very impressed with the admin console - even though I didn't figure out how to edit the homepage. It appears to be using Ajax everywhere. Right-clicking to edit a page even brings up an application menu (rather than a browser menu). Magnolia is impressive at first glance and will be included in Part II of my evaluation.

Alfresco, with a 0.6 release, was almost off my radar. The version number makes it seem like a very immature product - and the fact that it requires JDK 5 makes it seem even more immature. Upon startup, it spit out a number of errors - but they seem to be well documented in its "README_tomcat_linux.txt" file. I used OS X and despite the errors - everything appeared to be working OK. However, when I pulled up http://localhost:8080/alfresco, I immediately understood why Bill likes it. It looks like it's using JSF, and I'm willing to bet it's using MyFaces (he's a committer on the project). The disappointing thing I noticed after pulling up the initial page is that it's a login page. I'd expect to see a reader view initially rather than an admin view. After logging in, the interface seems very nice and easy to work with. However, after 30 seconds of clicking on stuff, I can't figure out how to get the reader view to show up - so I give up. I'm going to let this CMS graduate to Part II, but only because I didn't spend much time with it - and also because the UI looks quite polished.

I installed and played around with Daisy, Magnolia and Alfresco all in a 3-hour period today. First of all, I'd like to give props to all the authors of these OS projects as each was installable according to its instructions and I didn't have to google for a single setting. I installed Joomla and Drupal last week - both in under 10 minutes. Joomla was definitely more impressive - mainly because it's default homepage (note: this can change every hour) looks pretty cool. Drupal, on the other hand, is a bit more plain jane. This is not a bad thing necessarily - as it might be easier to design with a clean slate rather than remove-features, then-design. I did have to install PHP4 on my Mac to run both of these packages, but Server Logistics made that super easy. I also tried to install Joomla 1.0 on my Windows box (which has Apache 4.0.7 and PHP 4.3.3 from BlueGlue), but it failed halfway through the install with errors that my user/pass for the database were wrong. Not a big deal, but frustrating since it works fine on the Mac (and yes, the user/pass I'm using are correct).

Watch this space. Part II will address customizing each CMS to use an already-created theme, as well as playing around with each's features. I'd prefer a package that has built-in support for multi-author blogs - so that might make Magnolia and Alfresco look bad. However, I'm not going to hold it against them since I'm a Roller fan and committer. I don't mind using a 2nd application for blogging - especially if the built-in package is less than full-featured.

Conclusion: Alfresco, Magnolia, Joomla and Drupal are the easiest open source CMS applications to install (of the ones I looked at). Daisy was easy to install according to its instructions (of which there were many), but it's more of a Wiki than a CMS.

Update on Saturday at 1:00 p.m. - Based on reader's feedback to this post, I went ahead and installed three more: Plone, eZ publish (gotta love the domain name) and OpenCMS. All the previous CMS applications I installed on OS X, and I installed these 3 on Windows XP. I installed OpenCMS first, and while the setup was easy, it took 18 minutes to finish its importing of data. My Windows machine is easily 2x as fast as my PowerBook, so my guess is this would've been quite painful on the ol' Mac. Regardless, it's a one time wait-fee, so I can't really ding them for that. I played around with the Admin UI for about 5 minutes and found it extremely easy to use - as well as intuitive. I dig how you can view the content and edit it very easily. I never had to read any documentation to figure out how to work this system.

Next up was eZ publish. It took a bit of fiddling to get the installation to work. I had PHP 4.3.3 installed with Apache 2.0.47, and I upgraded to PHP 4.4 before trying anything. While setting up eZ publish, it told me it didn't work with 4.4, so I backed down to 4.3.3, after which it told me I needed at least 4.3.4. Thankfully, PHP is easy to install and upgrade on Windows and I got it all working fairly quickly. Of the three (and all the CMS's in fact), eZ publish has the nicest installer. The whole application and process gave me the feeling that it was the product I was looking for. It's designed to help you setup a website, and allows you to choose templates, features, etc. along the way. At the end, it stalls for a couple minutes - probably b/c it's creating everything and populating the database. The first thing I didn't like was the URLs (/index.php/feature). I wonder if that can be changed to be more path-based? Having an extension followed by additional slashes just doesn't seem right to me. From there, I logged in as an administrator and started playing around with things a bit. The Weblog feature seems very inadequate and doesn't even seem to support RSS. In addition, I couldn't figure out how to edit any of the templates in my browser. But the worst part was the admin UI was sssllloooowwww. It could have been b/c I had other stuff running, but I don't think so. eZ publish seems like a good system to recommend to friends who want quick and easy websites, but I don't think it's one for me.

The last CMS I installed was Plone. It was easy to install on Windows, but only b/c it has an all-in-one installer. I found it a bit strange that it doesn't install in a web server, but rather contains everything as part of the package. This includes Python, its own web server, and its own database (I'm assuming). It's an easy install, but I like the idea of installing a CMS into an existing web server rather than having a CMS with an embedded web server. The admin UI is simple enough to use, but it's very boxy and seems like it would be difficult to customize into a corporate or small business website. Please let me know if you feel this to be untrue (pointing to existing nice-looking installations would help). I didn't like the fact that it highlights members quite prominently, but that can probably be fixed with some good template-tweaking. The most disappointing thing I found was that I was unable to edit "skins" within the admin console. You have to download a separate package, customize it yourself, and then install it. What a pain, especially since I'd really like to be able to 1) modify template, 2) save and 3) view - just like this site allows me to do. For this reason, as well as the embedded server feature, I'm going to have to pass on Plone as well.

Conclusion 2: Alfresco, Magnolia, Joomla, Drupal and OpenCMS are the CMS applications I will continue to evaluate this weekend. I have to make a decision by Monday morning so I can start building a site with it. If any of these CMS'es don't allow me to customize its templates from a browser, please let me know so I can take it off my list.

Update on Sunday at 9:30 a.m. - I'm going to drop Alfresco and add MeshCMS for reasons stated here.

Posted in Open Source at Sep 23 2005, 05:39:04 PM MDT 38 Comments

Building a website with an Open Source CMS

I think the open source community has done an excellent job of figuring out how to create frameworks for developing web applications. But what about websites. You know, the web presence that every company wants - for minimal cost. For most companies, it's nothing more than 5-10 pages that tells a bit about the company, show some management folks and tells you how to get to their offices.

I've developed many websites over the years, many that were static, but more that were dynamic. The unfortunate thing about all of them is they required someone technical to update them. Often, the client had to contact me if they wanted anything new on the site. I've often thought there was a better solution - and I think I'm at a point where I know what customers want, and I know how to provide it. The solution is a Content Management Solution (CMS). One of the biggest problems with static websites is they're not dynamic enough. A CMS can alleviate this problem by reducing the bottleneck that a traditional "webmaster" creates.

In my mind, there are a couple of things that make a good CMS: 1) it's open source (to minimize costs to the client) and 2) it's easy to customize. On the customization front, my demands are a bit more rigorous - mainly because I know what many folks want in a website. Here are my main criteria for a good open source CMS - when it's used to power a regular ol' client-updateable website:

  • Design customization: I should be able to customize all the (X)HTML that's generated using one or two files (like SiteMesh allows). It should be possible to change the look and feel of *everything* by modifying some CSS. It should also be possible to use Mike Stenhouse's CSS Framework to simplify layout choices for clients. Ideally, a web designer or regular ol' HTML person could do the customization.
  • Static-looking URLs: The site should look like a static site. The URLs should be all lowercase and end with .html. It should be possible to modify all the URLs to look as if all pages are static. Apache's mod_rewrite and the URL Rewrite Filter are great tools for making this happen, but it'd be nice if the administration of the application allowed for setting these rules.
  • It shouldn't look like a CMS: No login links, no registration links, etc.
  • Ability to easily add dynamic content: It should be easy to add dynamic content - such as RSS Feed headlines to pages.
  • Menu Customization: In the application, it should be possible to create menus (both main and local navigation) and configure a page to highlight a menu when a particular page is shown.
  • Versioning of pages: In case someone messes something up, it should be easy enough to revert back to a previous version.
  • Easy to use: It should be possible to train a marketing person (with little technical knowledge) how to use the system in 10-15 minutes.

For technical companies (such as Virtuas), there are a few additional requirements I'd like to see:

  • Articles with syntax highlighting: It should be easy to publish articles with code that's colorized. The Java2HtmlPlugin for JSPWiki is a good example of this. I currently don't know of any for XHTML, XML or scripting languages like Ruby or Python.
  • File Upload: For uploading white papers and other technical publications.

So I've started looking at open source CMS's that fill my requirements. Last weekend, I wrote a solution that fills all of these requirements using SiteMesh, JSPWiki, CSS Framework, Acegi Security and the URL Rewrite Filter. It only took me about 6 hours to complete, but after completing it - I started wondering if I really wanted to start another open source project and maintain it. The answer is no, I don't want to create something new - I want to use something that's already out there. However, since I do have something that satisfies all my requirements, I will use it if I can't tweak an existing OS CMS enough.

Here are a list of CMS's that I'll be looking at in the next week or so. If you're associated with any of these projects, please leave a comment and let me know how many of my requirements you satisfy.

I'm a bit hesitant on Daisy b/c it requires XSLT knowledge for design customization. Magnolia has long URLs and doesn't appear like a static site - and the PHP ones often have .php in their URLs. It should be an interesting investigation to see if these (seemingly) heavyweight solutions can solve a few simple requirements.

Posted in Open Source at Sep 12 2005, 11:48:14 AM MDT 49 Comments

[OSCON] Wednesday Morning

I got a good night's sleep last night so I'd be fresh and ready for the Smackdown today. Matthew Porter, Scott Delap and I visited Jive Software's offices last night and had a great time sipping suds in their beautiful downtown office.

My AppFuse tutorial yesterday was well-received by a packed room of developers. Rather than writing code the whole time and doing a measly 30-slides, I added a bit more meat about Spring, Hibernate and testing. Most of audience was unfamiliar with Spring, so this seemed like the right thing to do. Of course, this led to more talking and less coding, but most of the folks I talked to were nevertheless very happy with the tutorial. If you'd like, you can download my presentation from the event.

Thanks to Rob Harrop and Thomas Risberg for both letting me know about the lack of Spring experience, as well as sitting in on my session. It was pretty cool having these guys in the room, as well as SiteMesh/jMock inventor Joe Walnes. Without these guys, many of the cool features in AppFuse would not be possible.

Now I'm sitting in the beginning Keynote session at OSCON, where they've announced they have a record 2000 attendees this year. In addition, it looks like OSCON is in Portland for the long run - this is the 3rd year it's been in Portland. Rather than moving to a new city like they used to, they've decided to stay b/c conference attendees like it so much.

The "unofficial" tagline of the conference is fun. Open source is fun and exciting - both to develop and use. This is in stark contrast to closed source software that tries to stay stable and boring, with no surprises.

When O'Reilly's CodeZoo launched, it only listed Java open source projects. As of today, they've added python.codezoo.com and ruby.codezoo.com, in addition to java.codezoo.com.

Tim O'Reilly

O'Reilly is not just book publisher or conference producer - but also a company that looks to the future and tries to figure out what's next. To highlight this vision, they've created O'Reilly Radar.

We're currently going through "The Open Source Paradigm Shift". Integration of commodity components has led to a new model where value gets captured. Rather than being at the software level, it's at the services level.

Key Questions for Open Source Advocates

  • Will "web 2.0" be an open system? What do "open services" look like?
  • Data as the "Intel Inside" - will we end up needing a Free Data Foundation in 2010?
  • How does the paradigm shift change our business models and development practices?
  • Who shoujld we be watching and learning from?

Things on O'Reilly's Radar

  • Ruby on Rails: new platform and new language. May well be the Perl of Web 2.0.
  • GreaseMonkey: a Firefox extension that alters websites to fit your view. A website is traditionally closed. GreaseMonkey "opens up" a website and rewrites it for the user.
  • HousingMaps.com: leveraging Google Maps and existing data from a bunch of different webservices to build a better website.
  • Ajax: html, javascript and css. The "css" tag on del.icio.us has gone down as the "ajax" tag has gone up.
  • Findory: Uses the articles you like in blogs and news, and finds similar articles. Similar to Amazon's recommendation system.
  • Internet Telephony: Asterisk in particular. Skype and Broadvoice. Broadvoice is pushing BYOD (Bring Your Own Device).
  • HomeBrew: similar to Tivo.
  • Firefox
  • Opening up hardware, not just software - i.e. Car PC Hacks, Smart Home Hacks.

For more, go to the Visualizing Technology Trends on Thursday afternoon. For the first time, the computer book market has stabilized. This is a good sign that the computer industry is about to start rebounding. As far as the book market goes, it's market share and growth - Java still leads the pack by a pretty wide margin. A large reason of this is due to Open Source Java. SimplyHired.com spiders 7.9M jobs on 755 different job boards. General books on Linux are up, especially non-RedHat distros. Books on RedHat have decreased significantly.

This is an interesting conference to be at when you're a Java developer. For the most part, everyone seems to be Perl fans, followed by Python, and a few Ruby guys. Most of these developers are very vocal about the fact that they don't like Java. Then again, Java is the leader in many areas - and it's the open source way to hate the guy on top.

Kim Polese, SpikeSource

Building on the Architecture of Participation. A transformation from Do It Yourself (DIY) to Do It Together (DIT). Thanks to the architecture of participation, open source has achieved World Domination - as evident by governments mandating it and IBM pouring billions into it.

The architecture is characterized by:

  • Commoditization of software
  • Network-enabled collaboration
  • Software customizability

In Phase I, we built and we built with. Open source had DIY origins. Now we're in Phase II, where increasingly the action is out in the long tail. Countless new building materials are piling up on the long tail. Now it's possible to build just about anything with anything. IT shops are building a phenomenal set of DIY "packages" that combine components from both ends of the curve.

The two problems with this:

  • Velocity mismatch: all components are on different release schedules. Linux, Apache, MySQL - all on a different release schedule. In addition, the ones on the other side (Lucene, Struts, Mambo) are on a different cycle.
  • Dependencies: When one version of one product changes - what happens to all the dependencies?

To solve these problems, companies are developing formalized proceses like review boards, support centers, OSS incubation centers, testing groups and they're certifiying / defining stacks internally. Most of this work is laborius and not related to the core competency of the business.

What's next? Phase III - IT becomes core. They do this by offloading critical but non-strategic work to independent service companies. DIY evolves into DIT with the help of independent service companies. Of course, this is all leading up to the fact that SpikeSource provides these services. It's funny that as soon as Kim said "SpikeSource" - all the presentation screens in the room quit working (not on purpose). A minute later they're back. This goes to show that marketing is not liked by the Open Source Gods. ;-)

"Testing is the single biggest refactoring shift in sofware." - J.P. Rangaswami, CIO, DrKW.

We need testing on a massive skale. For this reason, Murugan Pal and Ray Lane started SpikeSource. They saw the next phase is testing open source software so we can scale testing, together. Solve velocity mismatch and dependency problems with rapid per-defect patch management and dynamic stack configuration.

Testing has always been the software's ugly stepchild. We need to scale open source testing the way we scaled open source development. Some perspective: Microsoft has a 1:1 ratio of developers to QA Engineers. There's no Microsoft for open source software, nor should there be. To solve testing on a massive scale, you need participation plus automation. For models of how both scale, think eBay, Google and Amazon. Their best assets were their customers that supplied data that made their services more useful.

Testing is just one service among many. The Linux distros and middleware core building blocks have been there for awhile. Now we have applications and service companies as well. Who benefits when we have abundant integrated, tested, validated automatically patched stacks? IT and ISVs shift high-value development resources to customer-faces - differentiating features and services. In addition, many other groups benefit and higher quality software gets developed.

Testing will do for open source what it did for chip design a generation earlier. Testing is what catapulted the chip industry forward in the 80s. The new testing tools moved VLSI foward. Countless new IC-powered products were made possible and at much faster development speeds. Solving the testing problem can't be done by one company alone. "Come Test with Us..."

After Kim, another speaker (Andrew from OSDL) began his talk. He talked in a monotone and lacked a presentation. The room quickly began to leak people, me being one of them.

Posted in Open Source at Aug 03 2005, 11:15:08 AM MDT 4 Comments

[OSCON] Tuesday Afternoon

Creating Passionate Users
Presented by Kathy Sierra, OSCON 2005

How do you create passionate users? People will do anything and be enthusiastic about it if they're passionate about it. For example, Nikon.com teaches you how to be a better photographer. In their tutorials, they happen to mention that you might need a better camera to take better pictures. If you're going for passion, you have to provide a continuous path for becoming more knowledgeable about a product - and eventually becoming an expert in something. The beauty is the path or thing you provide doesn't even have to be related to the product. It simply has to provide users with enthusiasm about something you provide, which in turn supplies them with the path for that passionate experience.

"Things that look better actually work better." In other words, aesthetics matter.

When writing a book or documentation - you have to make what you're writing about matters to the user. It has to be so important to them that it gives them a queasy feeling. Naturally, the person won't be interested in what you're talking about - you need to be able to communicate the real core of why a person would have an emotional reaction and why it matters to them. "Well, why didn't you say that?" is the reaction you're looking for. If you're can answer that question w/o answering all the questions leading up to id - you're golden. You're supposed to try and scare them to the point that they're never going to have sex again, and then step back one level. What's the compelling meaningful benefit of the product?

You need to seduce your users and keep them interested and passionate by challenging them to learn more. Flow is the feeling when you have no sense of time - and you need to somehow figure out how to get your users into the flow. As long as you believe you're only one compile away from fixing it - you'll spend hours working on something. This is the flow state and comes from the perfect balance between a challenge and the skill+knowledge to solve that challenge.

One of the challenges to creating passionate users is to establish some sort of next level that your users can get to. First of all, you have to figure out what the next level is, followed by what "new powers" and abilities you can give to your users once they get to that level.

Tips for engaging users

When writing, use a conversational tone. The brain has a conversation with text when you read it and you'll have a 40% better retention rate by reading writing with a conversational tone. Also, use pictures whenever possible since they often make things easier to understand. Don't reveal everything - make your users curious.

What do film makers and novelists do? They tell stories.

Where there is community, there is legend, myth, passion and stories. Where there is passion, there are people. How can you propagate the stories and people from the project?

People don't care about you - all they care about is how they feel about themselves after interacting with your product or service.

This was a great session by Kathy and I was very impressed how she presented it. No laptop, just an overhead projector. Lots of group activities and lots of group discussions. I could easily see Kathy and Bert writing a book on Creating Passionate Users.

Posted in Open Source at Aug 03 2005, 12:30:10 AM MDT 1 Comment

[OSCON] Monday Afternoon

Ruby on Rails - Enjoying the ride of programming
Presented by David Heinemeier Hansson, OSCON 2005

About David: started doing Ruby in June 2003. Involuntary programmer of need, served 5 years in PHP. Spent 7 months in a Java shop.

Prerequisites of play: Ruby 1.8.2, dated December 25th. A database, pick one of 6. The RubyGems miner. Some gems called Active and Action.

Directory structure that Rails creates is more for convenience than anything. By picking conventions for you, it makes things easier. It might feel like flexibility is being ripped away from you - but you can change the defaults. However, by following the default settings, things will just work and life will be much easier for you as a developer.

I did a bit of playing on my PowerBook while listening to David's talk. I have Tiger installed, but found that Ruby 1.8.1 was installed on my machine (in /sw/bin/) thanks to Fink. My running "rm -r /sw/bin/ruby" and restarting iTerm, the default changed to /usr/bin/ruby, which is 1.8.2. From there, I downloaded and installed the Rails Installer.

I hate to admit it, but this talk is pretty boring so far. Probably because I've read David's blog for the past 6 months and watched most of the Rails videos. I haven't really learned a whole lot in the first 45 minutes of this talk. To be fair, the content of the talk seems to be properly targeted - there's been a fair amount of questions and everyone seems to be interested. Almost all of the seats are filled in the room; 3-4x as many folks as Dave Thomas's Ruby talk.

One interesting thing I've learned today is many features of Rails (i.e. Webrick) are actually a part of Ruby, not Rails. In addition, Ruby seems to have frequent releases and more features are added to the language each time. I guess that's the advantage of having a language that's not developed by committee.

When creating model objects in Rails, the default is to use a plural form of the object for the database table name. For example, a comment model will map to a comments table. Dave Thomas did mention in this morning's session that Rails isn't smart enough to figure out "sheep" - it gets maps to "sheeps". Apparently, you can easily override this behavior by specifying use_plurals=false somewhere. Another convention built-in to the framework is that the primary key is named "id" and its an auto-incrementing field.

"The database is a data bucket. I don't want any logic in my database, I want it all to be in my data model."

Rails doesn't handle composite primary keys. Rails is mostly designed for green-field development, where you get to control your database and its schema.

There are a number of key properties you can use in your database tables (a.k.a. your model objects) that will automatically get updated if you name them properly. Their names are created_at (datetime), created_on (date), updated_at and updated_on. There are also a number of time-related helpers, i.e. distance_of_time_in_words_to_now(date) » less than a minute ago.

Rails also has the concept of filters, which you can apply to a group of controllers. To use a filter, you define the filter method in controllers/application.rb and then you have to add a before_filter clause in each controller you want it to be applied. While it's cool that Rails has filters, it would be nice if you didn't have to configure the controllers that filters are applied to in the controller. To me, it seems more appropriate to be able to configure the where the filters are applied externally to the controllers. It seems more natural to me that you'd put something like apply_to_controller => { :controller1, :controller2 } in application.rb.

For doing page decoration with Rails (i.e. SiteMesh), you simply create a decorator in views/layouts. If you want a particular decorator to apply to a particular controller, you just name the file the same as the controller's URL. For example, if you have a posts controller (really a PostController.rb file), you'll create a decorator named posts.rhtml to decorate all the HTML rendered from the PostController - regardless of whether you're rendering from a method or from a view template. To have a decorator apply to all controllers, you can simply create a file named view/layouts/application.rhtml. This seems like something that SiteMesh could easily do as well - for example defaulting to /decorators/default.jsp (or something similar).

One thing I like about Rails is it's flash concept and how easy it makes it to display success messages. In my experience with Java web frameworks - many make this more difficult than it should be.

Testing Rails Applications

When running tests, Rails automates the creation of a test database instance that mirrors the schema of your development database. One slick thing in a Rails project's Rakefile is that you can run all the tests that you've touched in the last 10 minutes. I think one of the most unique thing about Rails/Ruby vs. Java is all that almost all of the files (Rakefile, code generation scripts, etc.) are written in Ruby.

Controller tests have a "mini-language" for simulating a browser when testing controllers. For example:

def test_login 
  get :login
  assert_response :success<
  assert_template "login" 
  
  post :login, :password => "secret!"
  assert_response :success
  assert !session[:authorized]
  
  post :login, :password => "secret"
  assert_response :redirect
  assert session[:authorized]
end

In the Controller tests, you can set cookies, parameters and mimic almost everything the browser can do. You can also test that your model objects have been manipulated appropriately. For example:

def test_create_post 
  post :create, :post => { :title => "This is my title", :body => "1" }
  assert_response :redirect
  assert_kind_of Post, Post.find_by_title("This is my title")
  
  post :create, :post => { :title => "", :body => "1" }
  assert_response :success # something was rendered, regardless of error messages
  assert_equal "don't leave me out", assigns(:post).errors.on(:title)
  #or assert_equal 1, assigns(:post).errors.count
end

The find_by_title method is a dynamic finder, where ActiveRecord creates find_by methods for each attribute of the model object. Another cool feature of testing is you can add a line with "breakpoint" in it - and the test will stop executing there - giving you access to all the variables at that point.

Ajax

The main reason for integrating JavaScript into Rails is so developers don't have to write JavaScript. For most developers, writing JavaScript is a pain because of browser incompatibilities and such. Rails ships with 4 JavaScript libraries, including Prototype and Script.aculo.us. It's easy to include the default JavaScript libraries in Rails:

<%= javascript_include_tag :defaults %> 

Both the link_to and form_tag methods have a "remote" equivalent (i.e. link_to_remote) that allows you to hook into Ajax, and by defining a :complete callback, you can call fade effects and the like. You can override many of the lifecycle stages of Ajax, but the most common is the :complete callback. In a Controller, it's easy to distinguish Ajax calls from non-Ajax calls using:

if request.xml_http_request?
  # do logic, for example rendering partials
end

Partials seem to be a pretty cool feature in Rails. They're actually just parts of a page that you include in a parent page with render :partial => "viewName". The slick thing about partials is you can actually populate their model and return them in a controller after an Ajax call.

The Ajax demos that David just showed are pretty cool. He was able to easily show how to delete a comment in his "weblog app", as well as add a new comment - w/o refreshing the page. The slick part of the add was he was easily able to add the new comment id to the Ajax response header, and then grab it in a callback and use the id to reference a <div> and use the yellow fade technique to highlight and fade the new comment.

That's the end of Dave's talk, and the first day at OSCON. Thanks to Dave and David for showing me the cool features of Ruby and Rails.

Posted in Open Source at Aug 01 2005, 05:02:31 PM MDT 5 Comments

[OSCON] Monday Morning

Facets of Ruby
Presented by Dave Thomas, OSCON 2005

I'm sitting in Dave Thomas's session on Intro to Ruby at the Oregon Convention Center. It looks like someone finally figured out the main problem with conferences - lack of power outlets. Kudos to O'Reilly - they've put power strips at the base of every table in this room. With the high-speed wireless and unlimited power, this conference is getting off to a great start.

Is programming still fun?

Round about 2000, programming started getting tedious for Dave - after having fun for the past 25 years. When we program, we combine all the problems of the artistic side of the race with all of the problems of the scientific side of the race. The only way to be successful at it is to enjoy doing it.

Is programming productive?

It has to be to enjoy it. The most satisfying thing about programming is watching it run. That's why scripting languages are so great - because you can see it run now. Myth: a good programmer can be a good programmer in any language. Language and tools make a difference - a good programmer knows which language to choose for a particular problem.

This session is not going to be a syntax session. Damn, sounds like I won't really learn how to program Ruby in this session.

Ruby, the Language

Born in Japan in 1994. Father: Invented by Yukihiro Matsumoto (Matz). Mother: Ada, Smalltalk, CLU, Perl, Lisp. Grew very rapidly in 2000, outpaced Python in 2000. Became international star in 2004.

Dave and Andy are language freaks and downloaded Ruby 1.4 shortly after finishing The Pragmatic Programmer. It passed the 5-minute test, the 1/2-hour test and Dave ended up playing with it all morning. The first Pickax book was 500+ pages long, and they wrote it because there wasn't much English-language documentation on Ruby. Ever since Rails, Ruby's adoption has grown exponentially.

Ruby is a multi-paradigm language: procedural, object-oriented, functional, meta-programming. You can write procedural code, but you'll be using OO concepts at the same time. You can do all of these at the same time.

Ruby code example:

# Generate Fibonacci numbers up to a given limit

def fib_up_to(max)
  f1, f2 = 1, 1
  while f1 <= max
    puts f1
    f1, f2 = f2, f1+f2
  end
end

fib_up_to(1000)

Methods start with def and end with end. The parenthesis around the method arguments are optional.

Now Dave is ragging on Java Programmers and how they discount Ruby because of its duck typing. In a Java program, most things are dynamically typed too. This is because most objects are stored in collections and whenever you pull things out of a collection - you have to cast from Object to the real type. The argument is that you don't have to have static typing. Dave hates Generics because he thinks they should've just done automatic casting.

The basic gist of Dave's argument is that we use dynamic typing (using casting) in Java all the time and you don't see Runtime exceptions all of the place. So the biggest proponents of static typing are actually using dynamic typing all of the place.

Back to the code: in Ruby, you don't need parenthesis around conditionals (for instance in the while statement above). The main reason we have parenthesis is because of Fortran. There's no reason for them. You can put parenthesis and semi-colons in your code, but you don't need to. In this code example, the variables are scoped for the duration of the method. puts (pronounced "put s") just prints the value of a variable to the console.

class Song
  def initialize(a_title)
    @title = a_title
  end
  def title
    @title
  end
end

Instance variables in Ruby start with an @ sign. The first time you use them, they spring into existence. If you access an instance variable and it's value hasn't been set - it's value is nil. Using the return keyword at the end of a method is optional - the default is to return the last line of a method. You can change the "title" method to use

attr_reader :title 

The attr_reader call is actually a method of Class:class. The attr_reader will dynamically add an accessor (that looks like the title method above). To create a setter, you can use attr_accessor and it'll create both a getter and setter.

Ruby is a single Inheritance language.

class KaraokeSong < Song

  attr_reader :lyric

  def initialize(a_title, a_lyric)
    super(a_title)
    @lyric = a_lyric
  end
end

Unlike Java, the super call can happen in any line, or not at all. To solve the single-inheritance problem, you can use mixins and apply them to any class.

Blocks and iterators are pervasive in Ruby, and look to be very easy to use.

3.times { puts "Ho!" }

hash.each { |key, value|
  puts "#{key} -> #{value}"
end

To do method calls with blocks, you use the yield keyword.

You can use blocks to simplify Resource Management and automatically close resources.

File.open("myfile.dat") do |f|
  name = f.gets
  # whatever
end

When the above code hits end, the file is automatically closed.

Duck Typing

Strongly-typed objects, untyped variables and methods. Types are implicitly determined by the things that an object can do. Duck typing is great for testing, refactoring and maintenance. This is very similar to concepts in Smalltalk. There is a strong commitment to unit testing in Ruby - which makes duck typing even easier to use. Duck typing makes things very easy. For example, you can have a method that takes a file as a parameter - and writes data to it. You can test this method by passing in a String (which also supports << for appending) and verify that your method's logic works.

Duck typing is useful in regular code for reducing coupling and increasing flexibility. Ruby community now differentiates the type (what it can do) of an object and the class (what generated it) of an object.

The Road to Metaprogramming

Metaprogramming is really, really powerful in Ruby. Library writers use it, but most developers don't use it. The ActiveRecord framework is an example of metaprogramming. Rather than being an O/R Mapping tool, it's more of a database table wrapper. The belongs_to, has_one, has_many and other validation_presence_of method calls can be written by you. Allowing you to write DSL (domain-specific languages) that appear to be a part of the Ruby language.

4 steps to metaprogramming:

  1. Classes are open: in Ruby, you can always add methods, attributes, etc. to existing classes. For example, you can add an encrypt() method to the String class. Isn't this dangerous? What if someone changes the logic of + for math expressions. No, because if one of your programmers overrides methods that breaks things - you take them out in the parking lot and beat them with a rubber hose! The language shouldn't prohibit us from doing powerful things.
  2. Definitions are active: code executes during what would normally be compilation. As the class if being defined, the code is being executed. For example, you can check if an environment variable is set - and create methods (i.e. log.debug()) accordingly. This can be great for caching.
  3. All method calls have a receiver: Methods are executed on objects. There's always a current object: self. Methods with no explicit receiver are executed on current object.
  4. Classes are objects too: You can easily add methods to classes.

Many more Ruby features: Reflection and ObjectSpace, Marshalling and Distributed Ruby, Tk, Gtk, Fox, networking, databases, etc. Garbage collection, Threads (like Java green threads), Modules and mixins. ObjectSpace - allows you to reflect on all of the objects that exist at runtime. Marshalling allows you to serialize into binary or text formats. No XML - uses YAML instead. Unlike XML, it's readable and looks more like a properties file. Modules (and their methods) can be easily included into a class simply by using "include ModuleName".

Now Dave is going to write a program to extract book sales ranks from Amazon pages, publish them as an RSS Feed, store them in a database, and access via a web application (using Rails). Since this is likely to involve a lot of live coding, I probably won't blog the code Dave writes.

Web Applications in Ruby can be done with Simple CGI, FastCGI, mod_ruby and frameworks (like Rails). Iowa, CGIKit, Nitro and Ruby on Rails are all web frameworks in Ruby. Iowa is a Ruby implementation of Apple's WebObjects. Dave's quote: "Apple really screwed up with WebObjects, they could've owned the market on web frameworks."

Summary

  • Use Ruby because it is Lightweight. A Ruby download is under 10 MB. Ruby Gems allows easy package management for downloading libraries and documentation.
  • Use Ruby is Transparent. It's nice and easy to read - and it only takes a couple of hours to learn its syntax.
  • Portable - runs on PDAs and Mainframes.
  • Open Source - MIT/Artistic style license. 1.8.3's regular expression engine is LGPL, 1.9's engine will be BSD-style license.
  • Easy to Learn - uses the Principle of Least Surprise. Things seem to work as you'd expect. Dave knows people that've downloaded Ruby and put web applications on line in the same morning - w/o any prior knowledge of Ruby.
  • Fun! It's enjoyable to program in.

Ruby Resources

Posted in Open Source at Aug 01 2005, 11:20:05 AM MDT 9 Comments