This afternoon, I sat in a Keynote by Bob McWhirter at TSSJS 2010. Bob is the Chief Architect of Cloud Computing at Red Hat. Bob is "The Despot" at Codehaus, helped start Groovy and founded Drools. Below are my notes from his talk.
The cloud is not an either/or question. A scorched Earth strategy is not necessary to be in the cloud. The cloud is a continuum of technologies, many of those that you're already familiar with. You're used to web, messaging and storage. You can add just a little bot of cloud to your application at a time.
So what is the cloud?
It's the next logical step in application delivery.
How did we get to the cloud? First, we jammed a lot of servers into a closet (back in the .com days). That's a lot of stuff to deal with if all you want to do is deliver an internet-based service. Then we did some colo. The problem with colo was that it was still our stuff. Then we leased managed servers. With the cloud, we lease what appears to be servers (or services). This is a path we've all been taking to abstract the annoying bits away from hosting an internet-based service.
The typical diagram of a cloud has IaaS, PaaS and PaaS. IaaS abstracts away hardware. PaaS abstracts away services. SaaS abstracts away software.
Tenants of the cloud:
- The illusion of infinite resources.
- On-demand acquisition and provision.
- Someone else's responsibility.
Magicians provide the illusion of pulling a card out of your ear, but they don't actually do it. Because there are limits of what we can do. With on-demand requisitioning, there's no more purchase orders. There's no more waiting for delivery. No waiting for IT to rack it. Or install it. By having someone else manage your hardware, it's not your problem. There's clear lines of responsibility. This intentionally limits visibility into the implementation. Hopefully it's best-of-breed.
The Cloud is really just a mindset. Virtualization makes it affordable to make mistakes. Repeatedly. Each service is managed as an autonomous entity. Services are designed for scalability.
Architecture 101 give us a presentation layer, a business logic layer and a persistence layer. You can scale it by putting many instances on many servers. However, the truth is that most apps are a big ball of mud. You need to get your application in order, then figure out which parts really needs to scale. Is the web tier really where we have trouble? Painting HTML isn't all that hard.
You don't have to put everything in the cloud. The cloud is not a revolution. We still need containers for web things, messaging things, storage of things and other things. Containers still exist, they're just in the cloud.
From container to cloud
When you have an application and a database, there's not much to abstract away into the cloud. However, as soon as you notice your DB is slow, you'll add caching. Once you hit your machine's memory limit, you discover distributed caching systems. Once you've done this, you'll discover that the network isn't all that slow, especially compared to spinning disks. Then you realize that you don't even need the storage and you can keep everything in cache. From there, you move to a RESTful caching service. Then you call it a NoSQL Service so you can be buzzword-compliant.
As far as caching services, there's many available, but a lot of them are startups. However, the cloud does not have to mean external 3rd-party providers. Remember DBAs? It was their job to view the database as a service and to protect it. There's a good chance that we'll end up with DBAs for services. By turning caching into an independent service, it becomes subject to economies of scale.
The cloud is a mindset. It's an SOA approach that allows groups to specialize and optimize for scale.
So how do you implementing a local cloud? We already have great caching technologies. Bob, with his JBoss hat on, recommends Infinispan and its REST API. To get around network latency, you can still run your cloud on your LAN. You're essentially trading disk traffic for network traffic. You can use many hands (e.g. MapReduce) to make things happen quickly.
Besides caching, there's a number of other services that can be put in the cloud. For example:
- Messaging (REST-MQ
- Scheduling
- Security & Identity (OASIS, etc)
- Computation (query & otherwise)
- Transactions (REST-TX)
- Business Process Management
- Telephony (VOIP, SMS)
Your application becomes a stack of services, instead of a stack of software.
Higher-order business-logic services are also subject to cloudification. Ultimately, going into the cloud is not rocket science. The cloud does not turn everything on its head. We don't have to wait for new companies to develop new technologies. Startups may be the best consumers of the cloud, but might not be the best providers. We'll let the startups assume the risks for now. Until the 3rd party services have been proven, it's probably best to build your own cloud of services.
More than anything, the cloud has caused a return to fundamentals. Once your app is a collection of services, you can easily pick and choose what pieces you want to put in the cloud. This is where we're headed, whether you like it or not. As more apps become global-scape, you're going to run into these issues. By adopting a cloud mindset and architecture, your app can be prepared for scalability issues.
For more communication with Bob and his Cloud Evangelism, checkout StormGrind or follow him on Twitter.
Personally, I really enjoyed Bob's talk. It was by far the best keynote today, mostly because he told a story and did it elegantly with a nice-looking presentation. Not only that, but he provided the audience with a practical view of the cloud and ways that we can start using it in our applications today.
Update: You can find the slides from this session on SlideShare.
This afternoon, I attended a session on software quality by Jesper Pedersen at TSSJS. Jesper is a Core Developer at JBoss by Red Hat. He's the project lead of JBoss JCA, Tattletale, Papaki and JBoss Profiler 2. He's also currently the chairman of the Boston JBoss User Group. In this session, Jesper hopes to define basic requirements for a development environment and offer ideas on how to clean up a messy project.
Is software quality a friend or a foe? By implementing software quality processes, are you introducing unnecessary overhead? Development platforms are different today. We write a lot more business-specific code. We depend on standard frameworks for the core functionality. We depend on vendors to implement the standards correctly. We also depend on vendors to provide the necessary integration layer.
Since the platform is now a large slice of the pie, we must make sure we know where the issue is located. We must have proper integration tests; we must manage dependencies. Today, we must treat dependencies as if they are part of the application.
Defining the platform for your project helps narrow down the dependencies for your project. The platform is composed of corporate guidelines, standards, vendors and backend systems that you have to integrate with. Documentation is key for a successful project. Key documents types: User Guide, Developer Guide, API Guide, Architect Design, Implementation and Test.
It helps to define a project-wide configuration management system. Define a code-formatting guide will add consistency in your source tree. Also make sure you have separate build, test and execution environments. Use a Maven repository for your dependencies; both to support your project's artifacts as well as vendor artifacts.
"Maven today is an industry standard." -- Jesper Pederson
Define your tool chain as you would for your application. Back your Maven repository with SCM tools like Subversion or Git. For testing, use JUnit (unit testing), Corbertura (test coverage) and Hudson (continuous integration). Furthermore, you can add Checkstyle and Findbugs to verify coding conventions and find basic issues with code.
For the build environment, you need to make sure your dependency metadata is correct. Also, make sure you use the best practices for your given build tool. For example, with Maven and Ivy, it's a good idea to extract the version numbers into a common area of your pom.xml/build.xml so you can easily view all the versions in use. After you've done this, you can externalize the version information from the environment. Watch out for transitive dependencies as they can be tricky. Make sure you know what gets pulled in. Use enforcers to approve/ban dependencies or turn it off (Ivy). You can also vote for MNG-2315. Finally, snapshot dependencies are evil: define your release process so that releases are easy.
What can you do if your project is already a mess? Signs that your project is a mess: you look at your platform as a big black box, you use different dependencies than your deployment platform or you don't have integration tests for sub-systems or dependencies. To fix this, you can use a tool to get an overview of the entire project. Enter Tattletale.
Tattletale can give you an overview of your dependencies (Ant and Maven integration). It's a static analysis tool that doesn't depend on metadata, scanning your class files instead. Using Tattletale, you can produce a number of reports about your dependencies, what they're dependent on and what's dependent on you.
To maintain the lead in your project, make sure to define a checklist for each stage of your development cycle. Do reviews on documentation, architecture, component design and code. Enforce your rules of your project with your build system.
Jesper's final thoughts:
- Maintaining dependencies for a software project can be a tricky task.
- Using an Open Source platform as the foundation will ease the investigation of issues and increase trust.
- Defining a project-wide tool chain is key.
- Enforce all the rules on the project (better up-front than "fixing it" afterwards)
As Dusty mentioned, this session has a lot of good (basic) information, but there wasn't much new stuff. My team is using many of the technologies and practices that Jesper has mentioned. I guess that's validation that we're doing it right. I've heard of Tattletale, but never had a need for it since I haven't been on any "messy" projects recently.
This morning at TheServerSide Java Symposium I attended James Gosling's keynote. Below are my notes from his talk.
The unifying principle for Java is the Network - it ties everything together. Enterprise, Desktop, Web, Mobile, HPC, Media and Embedded. The most important thing in the Java world is the acquisition of Sun by Oracle. James is showing a slide of Duke in a fish tank with a "Snorcle!" title above it.
Obligatory statistics for Java:
- 15 million JRE downloads/week (doesn't count tax season in Brazil)
- 10 billion-ish Java enabled devices (more devices than people)
- 1 billion-ish Java enabled desktops
- 100 million-ish TV devices
- 2.6 billion-ish mobile devices
- 5.5 billion-ish smart cards
- 6.5 million professional Java developers
Java has become "Learn Once, Work Anywhere". Most college students worldwide have taken a Java course in school. James' daughter is in college but isn't interested in Java, mostly because her dad's name is all over the textbooks.
Java EE 6 was approved September 30, 2009. It was many years in the making; the result of large-scale community collaboration. It was built by hardware manufacturers, users, developers and academia. Because of all the politics involved, many engineers had to become diplomats. Most software engineers are from the wrong Myers-Brigg quadrant for this type of negotiation. Needless to say, the process was interesting.
New and Updated APIs in Java EE 6: Servlet 3.0, JAX-RS 1.1, Bean Validation 1.0, DI 1.0, CDI 1.0, Managed Beans 1.0, JASPIC 1.1, EJB 3.1, JPA 2.0 and many others. Also new is the Web Profile. It's the first Java EE profile to be defined. It's a fully-functional, mid-size stack for modern web application development. It's complete, but not the kitchen sink. It's what most people use when building a modern web application in Java.
Java EE 6 adds dependency injection with DI (JSR-330) and CDI (JSR-299). @Resource is still around, but an @Inject annotation has been added for typesafe injection. It has automatic scope management (request, session, etc.) and is extensible via a BeanManager API.
GlassFish is the world's most downloaded app server (1 million-ish downloads/month). GFv2 was the EE 5 reference implementation. GFv3 is the reference implementation for EE 6. But it's not just a reference implementation, it's a benchmark-winning mission-critical large-scale app server. The FCS was released on December 10, 2009.
Goals of Java EE: ease of use, right-sizing and extensibility. Now Roberto Chinnici (EE 6 spec lead) and another guy are on stage showing a NetBeans and GlassFish demo. With Servlet 3.0, you don't need a web.xml file, you just need a WEB-INF directory. There's a new @WebServlet annotation that lets you specify a "urlPattern" value for the servlet-mapping. A new @EJB annotation allows you to easily inject EJBs into your servlet. Roberto wired in an EJB, hit Ctrl+S and refreshed his browser and it all worked immediately. In the background, NetBeans and GlassFish did the redeployment and initialized the EJB container in milliseconds.
@ManagedBeans and @SessionScope and @Named are all part of CDI. When using @Named, the beans become available to JSTL and you can access them using ${beanName.property}. Interestingly, the CDI annotations are in difference packages: javax.annotation.ManagedBean and javax.enterprise.context.RequestScoped.
As David Geary mentions, it's great to see the influence that Ruby on Rails has had on Java EE.
Long demo of JEE6 in NetBeans. Spent quite a bit of time extolling the virtues of hot deploy. Thanks, RoR!
Now Roberto is showing us the admin console of GlassFish and how modular it is. He's installing a JMS module, but it's interesting to see that there's a Ruby Container installed by default. Apache Felix is the underlying OSGI implementation used by GlassFish. You can telnet into it and see the status of all the bundles installed. After installing the full-profile, Roberto shows that you can restart the server from the console.
Isn't the whole point of OSGI that you don't have to restart anything!?
The GlassFish management console is definitely impressive and visually appealing. Apparently, it's extensible too, so you could easily write plugins to monitor your application and provide memory statistics.
Changing topics, one of the things that nice about Java is its a two-level spec. The important thing in the Java world isn't the language, it's the virtual machine. The magic is in the VM! Scala, Ruby/Rails, Groovy/Grails, Python, PHP, JavaScript, JavaFX and many others. In the same breath of talking about Java.next languages, James mentioned JavaFX Script. It's a new declarative scripting language for GUIs. It's similar to Flash or Silverlight, but it's much better because it has the Java VM under it.
At the current rate that we're going with CPUs and cores, there's a good chance we'll have 5220 cores on our desktops by 2030. If you find the concurrency libraries scary, get over it.
For the rest of talk, James talked about what he's hacking on these days. He's helping build an Audi TTS for the Pikes Peak Road Rally in Colorado. The goal is to figure out a way to keep the vehicle above 130 MPH for the whole race. Sounds like a pretty cool project to me.
I don't think there was a whole lot of new information covered in James' talk, but I really do like Java EE 6's Web Profile. However, I think it's something most of the community has been using for many years with Tomcat + Spring + Hibernate. Now it's simply been standardized. If you happen to work at one of those companies that frowns on open source and smiles at standards, you've finally caught up with the rest of us. 
