Last night I had the pleasure of listening to two great speakers at Denver's
JUG. Not only did Brian and
Lajos know their stuff, but they were very comfortable on stage and interesting
to listen to. Below are my notes from this event. When I did a print
preview of this in Dreamweaver, it came out to be 5 pages - so just to warn
you, this is a pretty long post.
Brian Boelsterli - Nudging your organization towards agility: Ingredients
for an agile execution environment
Even if you have a dream team of developers, it doesn't mean the project will
succeed. Process Matters. A lot of what Brian's Rhythm process was developed
while working at Sprint - where they completed 169 iterations on their way
to production. There are problems with agile methodologies:
team has no rhythm (restart, pause, delay all the time), project plans are
big and heavy and just about worthless, methodology teams get in the way,
specifications are close to worthless, design review meetings are horrible. In
order to combat these problems (regardless of your methodology), Brian has
developed a Rhythm Process that helps agile
methods succeed. This system contains a number of ingredients to solve
problems with agile methodologies.
| Heartbeat/Stacked Iterations |
"...no rhythm" |
Iteration Artifact Review Meeting (iARM) |
"...worthless review meetings." |
| Software Push |
"...software deliveries are atrocious" |
Issue Review Meeting (IRM) |
"...lack of pattern for managing our backlog of defects ..." |
| Iteration Advocate |
"...methodology team gets in our way" |
Software Iteration Plan |
"...worthless project plans" |
| Iteration Artifacts |
"...worthless artifacts" |
Iteration Transition Meeting (ITM) |
"...we are status-based, instead of delivery based!" |
The Heartbeat is the logical representation to the physical
thing called an "iteration".
Starts Thursday morning, ends Wednesday night - based on psychology b/c it
fosters sustainability. By giving 2 days to start a new iteration, they've
found higher productivity because no one wants to work the weekend.
Rhythm advocates weekly iterations, regardless of deliverables. The primary
reason for this is to maintain the heartbeat. Rather than doing XP-type iterations
where you do everything in an iteration, you do Stacked Iterations. In
an iteration, you still do full lifecycle (requirements, development, deploy),
but the Business Analysts are 1 iteration ahead of the developers and the
developers are 1 iteration ahead of the deployment team. In Rhythm, there's
an Iteration Advocate who gathers and manages requirements, Developers do
the development, and the QA team does automation and deployment. I've worked
with this system in the past with Brian and this process works quite
well.
Iteration Advocate: Traditional "Process Mentor",
in disguise. They're responsible for making sure the spin happens and the
work gets done. Responsible for one thing: Ensuring the successful
delivery of each and every iteration. A big reason that Rhythm advocates
this role is because they've found that Project Managers tend to spend 80%
of their time doing PM work. By having this role, the PM can concentrate
on managing the project and doesn't have to be concerned about managing people
and their deliverables. As part of each iteration, artifacts are produced.
This makes it possible to put the project in hibernation at any iteration
transition point.
Software Iteration Plan (SIP): A simple
artifact that includes features and tasks, as well as defects to fix for
each iteration. As part of the weekly planning, this artifact is re-visited
and updated. Depending on how formal your organization is, this document
can be kept on a wiki site, in a word document, etc. Meetings are held every
Wednesday afternoon - where PMs sit in a room and team members come in and
report their status.
Iteration Transition Meeting (ITM): Backbone of the process
- happens every Thursday morning and involves all parties involved in the
project. Similar to SCRUM's Post-Sprint Meeting. It's an official buyer/seller
marketplace for cross-functioning teams to buy/sell iteration artifacts.
- Definers are selling their requirements, analysis and design specs
- Developers are selling their functionality
- Testing are selling their test plans and specs
Participation requires a minimum of one representative from each respective
group. Project Manager conducts, Iteration Advocate facilitates. Brian says
this meeting has virtually eliminated the need to do status meetings.
From
working with Brian, I know that one of the key ingredients to this process
is the Testing Team. The folks that typically fill the QA Developer role
is on a whole new level from most testers I've seen. Rhythm Testers are
automation experts and know how to setup CruiseControl, run Ant/Maven/Unit
tests, as well and configure and deploy to production servers. These folks
are the ones who handle the initial installation and configuration of the
test, development and production servers.
As part of Rhythm, it's also important to plan for "Featureless iterations" -
where nothing is done but run performance tests and regression tests. Done
at both 40% and 60% project completion. This was a good presentation
about an excellent process that I still try to use today. In my experience,
the hardest part to implementing it is finding good QA Engineers that are
automation experts as well. Nice job Brian!
» Download Presentation (PPT)
Lajos Moczar:
Creating A Successful Open Source Strategy
Lajos is a committer on OpenEJB and has written two books: one on Cocoon and
one on Tomcat.
Lajos has been working with open source since 1995 and has a company called
Galatea Information Strategies in Colorado
Springs. Lajos mentioned that he recently became well-known in the open-source
world by writing a paper titled
"Start developing and deploying J2EE apps on the first open source J2EE
server".
This title apparently pissed off the JBoss guys to no end, their PR Firm called
JavaWorld and they renamed it to A first look at Apache
Geronimo. Because of the JBoss controversy, Lajos became inspired to
write The
open source monopoly, which explains why he thinks doesn't think JBoss
is open-source.
NOTE: The big reason Lajos doesn't like
corporate-backed open-source is because he feels that quality app servers
like JOnAS deserve lots of press
too. However,
they don't have the marketing dollars that JBoss has, and are therefore overshadowed.
He also mentioned how dangerous it can be when a project's committers get so
full of themselves that they start veering from a standard (i.e. J2EE) and developing
their own add-ons. If companies and developers use these add-ons, it leads
to Vendor Lock-in, which is the same as the lock-in you find in commercial products
with proprietary APIs.
Bottom Line for open source adoption: Open Source is mature for the enterprise,
as long as you're mature in your approach to it.
The state of OSS: Over the past past 4-5 years, OSS has gone
from youth to adulthood. Now a viable choice for large-scale organizations,
common talk in business and news. Recent report from CSC on enterprise usages
of OSS:
- 70% have Linux and/or Apache
- 42% are using Tomcat
- 14% are using PHP
OSS in young adulthood, where childhood was free software,
not a thread to the mainstream. OSS was born of the hacker culture of the
70s (and earlier). Richard Stallman and GNU dominated open source in the
80s and early 90s.
In Adolescence, open source went a big crazy (late 90s).
The Big Split was when open source was redefined as a methodology
instead of a culture. Eric Raymond, Netscape and the launch of "commercial" open
source. Open source != free software. Open source + the Internet
= Linux, which leads to popular access to open source.
Now in Young Adulthood, where there is an explosion of
choices (in Operation Systems, Containers, Technologies (i.e. PHP,
Perl, Python, etc.), Web Frameworks). 34,000 open-source projects on freshmeat,
86,000 on sourceforge. There's also an explosion of licenses. Explosion of
technological advances in web services, Java and development technologies.
This has all resulted in an explosion of adoption in many large organizations.
Is OSS Mature? Open source faces a crisis of maturity:
- Does it achieve the ideals of its origins and stay as a separate force in
the technology world?
- Does it get subsumed by the increasing trend of corporate open source?
With the "corporate open source" model, the project is essentially
a product, except that it's free and and you can see the source. Are we losing
the intrigue and excitement that the 2:00 a.m. developer experiences? The
"corporate open source" model seems to discount the roots of open source
- when developers stayed up until the wee hours of the morning to develop
something they were passionate about. With the corporate model, it
becomes their job and could lead to a loss of passion.
Some Challenges: As OSS matures, there are challenges
that face OSS implementers, particularly those in advocacy/evangelization
roles. Inherent problems - a lot of releases (difficult to keep up),
is it really free, etc.
More Challenges: Overwhelming choices (choose a languages
(Java, Perl, C#, Python, C++) , choose a direction w/in a language (i.e.
Java 1.4 or 1.5) , choosing a technology direction (J2EE or just part of
it - sans EJB).
After this overview, Lajos went into an example of how open source is typically
adopted in an enterprise. It starts off with a web application pilot,
where a developer downloads Apache + Tomcat, spends a day getting them talking
to each other, and then configures everything using the default settings. From
there, an application is developed w/o code or directory structure standards,
without security and without failover solutions or backups. Nothing
is documented as development ensues. This pilot becomes a production
application and the IT organization has no idea how to support the application,
and new developers have no idea how to maintain it, or start a new project
based on the same architecture and tools. Bottom line: using a pilot
like this is an irresponsible way to introduce OSS to your enterprise.
Typical OSS Introduction problems: The first step towards
successful OS is organizing the problems or "challenges". Use of
technology problems, process problems and choice problems. OSS has the strange
effect of highlighting inherent problems in your enterprise or IT strategy.
These problems can be fixed.
- Technology problems: Technologies come and go, but the core
business stays the same. What is important is that you know how to use and
what you choose. Commons IT mistakes: The latest and greatest, bleeding edge
software, technology for the sake of technology, instead of for the business.
Sounds obvious, but you must always start with one question: what is the
purpose of IT?
- Process problems: OSS highlights already existing process
problems b/c it's very accessible, tends towards permutations in options
and versions and there's less work done in the way of best practices
than we need. You need a solid infrastructure - processes and principles
- to handle the problems with OSS and to insure complete success with
it. More than ever, this is the time for process: no OSS implementation
should be w/o them! Develop, test, upgrade, deploy, etc.
- Choice problems: Factors influencing
choices: technology direction, project viability and health, knowledge
base, license type, corporate sponsorship, "spirit" of founders.
Principles of a Pilot: Your pilot is the most critical step
in OSS implementation. The pilot must demonstrate: sound and viable technology
choices, solid processes and principles, vertical slices.
Starting a web pilot right: Research the options: which version
of Apache/Tomcat/MySQL is the most stable (use mailing list). What is the
proven integration solution, security "best practices", licensing concerns?
Are there documentation concerns? Do you have to pay for it? Is it any good?
When starting with open-source you need to plan,
plan, plan.
- Organize your directories
for repeatability, ease of maintenance.
- Separate program directories
from development directories from deployment
directories
- Document all configurables, installation quirks, directory locations
- Define a solid and repeatable build process
- Never take all the defaults! Determine what to change.
Authentication and Security. Problem: multiple logins
is the curse of the modern enterprise - don't reinvent! Use single sign-on
if possible. Don't have an LDAP server, install one! Kerberos is a good
option if you are starting out: a bit of work, but worth it. Problem:
OSS is not inherently more secure than anything else. Solution: research
for the most low-risk products. Keep currently with he project bug lists
and security announcements. Subscribe to all relevant mailing lists.
Development processes. Problem: Need a development process
for OSS-based applications. Solution: Use CVS with replication (CVSUP).
Use Ant/Maven with a well-documented target/properties schema for builds.
Implement a three-stage process, of development, QA and production. Create
a simple process for tracking application releases and enabling rollbacks.
While listening to Lajos talk about all of this stuff, I couldn't help
thinking to myself that AppFuse solves a lot of these problems for you.
Understand the Pitfalls of OSS:
Version mania and product instability: establish
your own release criteria and testing plans. Always upgrade with a rollback
plan. Once you find stability, keep it!
Lack of product infrastructure: Great products are useless w/o the proper
infrastructure. If any of the following don't exist around an open
source project, it's a warning sign that it might not be ready for prime-time.
- premier documentation (i.e. books)
- training services
- tiered support services
- integration services
- security alerts
- intuitive administration and operator interfaces
- interoperability testing - does the release announcement provide upgrade
paths?
- load testing
- release criteria (i.e. basic interoperability testing, tested functionality,
etc.)
Free software is not really free - there will be costs:
- Training
- Consulting
- Hardware
- Support
Flavor-of-the-month technologies: ask the hard questions about any technology,
jumping on the bandwagon for its own sake will be expensive, older proven
technologies are often better. Ruby on Rails fits into this category.
Corporate open source and branded technologies: beware brand
names backed by VC funding - they could end up being monopolies. Can lead
to Vendor Lock-in. Beware of technologies dominated by a single entity. Standards
are the best way to prevent this from happening. The best solution: community-driven
standards - complete with best practices.
Open source or "free" software requires the same approach and dedication as
does commercial software:
- Software infrastructure
- Training, documentation, support
- Sound implementation, upgrade, support processes
At this point, Lajos gave his recommendations of good open-source projects
that are ready for enterprise adoption. I wasn't fast enough to write
down the ones he mentioned for Security, Integration and End-user applications. He
did say he'd post his slides, so hopefully I can fill this list in later.
- Operation System: RedHat/Fedora, SuSE, FreeBSD, OpenBSD, Mandrake
- Data: MySQL (note the license), PostgreSQL, MaxDB, dbXML, Xindice, eXist,
Castor, Hibernate
- Security
- Web Services: Apache SOAP/AXIS, Tomcat, Jetty, JBoss, JOnAS, OpenEJB,
Resin, Geronimo, LUkAS, ebXML, Jabber, SIM, CryptIM, Magpie (PHP RSS),
Jena, ircd, OpenCMS
- Integration: JMS (OpenJMS, SwiftMQ)
- End-user: Firefox, Open Office
Implement: Once you have organized your choices, divide and
conquer. Determine what you will keep commercial: mainframe apps? High-end
database platforms? Take one slice at a time: database, communications, server
platforms, end-user platforms, web services, networking, etc. Be careful
of tightly integrated components.
This marked the end of Lajos's (very enjoyable) presentation. I've known
of Lajos through his Apache+Tomcat
FlashGuides, so it was very cool to meet
the man behind the knowledge. It's also pretty neat that we're both
doing the same line of work. I hope to sit down and have a beer with
Lajos in the near future. During the QA Session at the end, I picked up a
couple of audience tips: Google has their open-source code at code.google.com and Revolution OS is a good movie to rent.
» Download Presentation (PDF)
This is one of the best Denver JUG meetings I've been to in a long time, and
this summer looks like it'll be packed full of good sessions. July's JUG
Meeting is going to be awesome: Bruce Tate talking about Beyond Java followed
by David Geary talking about Ruby on Rails.
In only a few more hours, we're heading on Raible Family Roadtrip #9. Number 7 was when Julie, Abbie and I traveled up the California coast, and number 8 was when my Dad and I drove my '66 Bus to Denver from San Diego. This time it's going to be much more special. The end destination is my favorite place on earth. We're heading for the cabin, which is a log cabin my grandpa built in 1918. I was born in one corner, my sister in the other, and I spent the first 16 years of my life there.
