Performance Testing Memcached
Earlier this week, co-workers and I did some performance testing with Memcached. We wanted to see how long it'd take to send different sizes of objects over the wire to Memcached on a remote server.
We setup a simple environment with 2 Mac Pros both running 2 x 2.8 Ghz Quad-Core Intel Xeon processors and 12GB of RAM. We used one machine as a Memcached server and one machine running an application with the addition of a new Servlet Filter. The Servlet Fitler read a size parameter and used it to set the size of the object being written and read from the Memcached server. We used JMeter to put load on the box. The following describes the load and the results.
Write Times
Write tests were performed with a single user executing 1000 sequential remove then writes.
| Size in Bytes | Time (ms.) | Min. Time (ms.) | Max. Time (ms.) | Total |
|---|---|---|---|---|
| 20000 | 1.31 | 0 | 7 | 1000 |
| 50000 | 1.834 | 1 | 8 | 1000 |
| 100000 | 2.87 | 2 | 9 | 1000 |
| 500000 | 12.641 | 9 | 283 | 1000 |
Read Times
Read tests were performed using 50 users with 1 sec. ramp times executing 100 reads each.
| Size in Bytes | Time (ms.) | Min. Time (ms.) | Max. Time (ms.) | Total |
|---|---|---|---|---|
| 20000 | 4.8414 | 1 | 375 | 5000 |
| 50000 | 18.343 | 1 | 354 | 5000 |
| 100000 | 46.181 | 2 | 415 | 5000 |
| 500000 | 137.7328 | 6 | 953 | 5000 |
During our tests, Memcached was started using the following settings:
memcached -d -m 2048 -M -p 10171 -vv
If you've done similar performance testing with Memcached, we'd love to see your results.
asking a related question : how come more and more java shops are using memcached as opposed to a terracotta/ehcache cluster ? I understand that memcached can be used if your shop has java AND other technologies whereas terracotta would limit it to a java VM.
BR,
~A
Posted by anjan bacchu on September 19, 2008 at 05:02 PM MDT #
Posted by Bill de hÓra on September 20, 2008 at 08:25 AM MDT #
I'll take a whack at the memcached vs jcs/terracotta/etc solution..
The beauty of memcached is the simplicity. Bringing boxes onto the grid, removing them, adding memory, adjusting hashing schemes, serialization schemes, etc - It's all clear because there is so little magic to it. Additionally, in our experience, java's memory management (GC schemes, etc) is explicitly poor for mixed use caching. Again, memcached having it's 'close to the metal' slab allocator makes understanding what's going on dead simple.
Contrast this with Terracotta+ehcache+whatever, which advertise how 'magical' they are. Between bytecode re-writing, lock configurations, and the general overhead of the java runtime - having a single arg server startup and a half dozen commands in the api can be a breath of fresh air.
Also, I see in-process caching as a deficit instead of an asset. We have enough trouble managing the black magic of garbage collection when it only involves our runtime data. Trying to get a GC setup that performs well for transactional flow as well as a caching scheme seems almost impossible. (I would love to be proved wrong however)
That's just a snapshot of our thinking at the moment. I'd love to hear how other java shops are approaching distributed caching.
Posted by Adam Malter on September 24, 2008 at 12:58 PM MDT #
Interesting then that 2 of the highest trafficked sites in the world (top 20, at least) are looking at ditching memcache for Terracotta. I think you guys love memcache because it works for you and that's fine. I am not gonna argue against the reality that memcache works. In fact we at Terracotta refer people to memcache all the time. If you need coherent / consistent data at in-memory scale across JVMs, Memcache is useless. Moreover, only Terracotta can deliver this. And, as Expedia told me, "Terracotta is over 100 times faster at Memcache in our app." So all the sharding / hashing / partitioning might be unneeded complexity, not to mention our being inside the JVM allows us to optimize things just like hotspot itself does. Did you know that we compile out local synchronization where it proves not necessary, at runtime? We have lock leases so individual JVMs can execute a clustered lock operation w/o actually hitting the network AT ALL?
Rule of thumb: Memcache is easy to tune because it is essentially zero heap-overhead, but incoherent and slow. Terracotta is harder to tune, but totally coherent and very fast. Use Terracotta to avoid the DB altogether. Use memcache to memoize data and computations you could always get elsewhere at higher cost. In fact, consider them complements, not alternatives to each other.
Those above who assert memcache is well suited to Web Apps are on to something, BTW. Web apps can have a need for a simple memoizer / incoherence low footprint cache. Things like HTML caches of servlet output, or product and parts catalogs, etc. can all be memoized into cache format. But, I wouldn't put my users' application workflow into memcache. Otherwise you would end up hacking together something like Facebook did where MySQL log listeners on the DB server programatically shutdown memcaching until writes have been flushed to disk by the DB underneath. ICK!
--Ari
Posted by ARI ZILKA on September 24, 2008 at 11:30 PM MDT #
Hi
We have designed a prototype using Memcached,Hibernate/HibernateSearch/Spring MVC/Mysql.
I am doing performance tests using JMETER.
Having a few problems with what I belive to be concurrency issues.
In a nutshell the JMETER request calls a controller which creates a FullTextSession using the context session and passes the search string to the Hibernate Search API which first establishes the size of the result set and then hydrates the first PAGE_SIZE of the result set and returns the List of results in a model to the view.
The performance does not degrade markedly and when I increase the threads I encounter the following error
After a number of these errors the application degrades and eventually crashes.
Basically I cannot work out if memcached cannot perform or whether there is an issue with the way I am using it. If I leave the JMETER with 4 threads running 5 different search requests it will eventually crash. While I am working suring the day a use 10 to 12 threads to create the fatal errors more rapidly.
I have tried to increase the operationTimeout to 20 seconds which appears to have no effect. I have added another memcached server to the config to distribute the load and it had no effect on my issues
At this point I have the following to say about memcached
1) Simple and easy to use. 2) Integrates well with hibernate as a second level cache 3) Appears to scale well. 4) using very verbose mode -vv while running memcached appears to degrade perfomance.
Any ideas at this point or avenues of investigation would be welcome.
Posted by H W T on October 08, 2008 at 10:15 AM MDT #
Posted by handa fanda on January 27, 2010 at 03:32 PM MST #