Something like this

Ok that rocks, 21KW/hour. But can you really get that savings?  If program 1 and program 2 need to run at the same times: you can’t save anything on them[1] and then the virtualization itself adds cycles for purely overhead and increases memory which burns more power. Suppose we now add K% as the overhead measure – real need R= D+D*K/100. If  K=20, we cut savings in half. But 20 seems wildly overoptimistic to me, so put it at 50.  We’re still only needing 72 servers instead of 200, but they are running flat out so we actually increase power use.

That does not rock.   But here’s the kicker – if we reduce idle power to 25Watts, maybe by turning off idle machines, a 50% overhead rate for virtualization means virtualization increases power use by 20KW. If we can reduce idle power to 1 watt, even at 10% overhead, virtualization increases power use.  So what are the real numbers? I have not seen any published studies (of actual data centers) – maybe there are some.

If you want a copy of the xls, send me an email. Maybe I set it up wrong.

http://www.hpl.hp.com/personal/Lucy_Cherkasova/papers/final-perf-study-usenix.pdf

This claims lower levels of overhead

http://www.engineyard.com/blog/2009/10-years-of-virtual-machine-performance-semi-demystified/

but I’m unconvinced. In particular, I’m curious about memory usage.

This is funny.

The benchmark itself reported its elapsed time by calling a function to find the system time at both the beginning and end of the benchmark.  The elapsed time reported by the benchmark was less than the wall-clock elapsed time.  What we hypothesize is that, due to the unrelenting CPU consumption by the benchmarks, the virtualization layer was unable to update its clock with the virtual CPU clock ticks.  This phenomenon is mentioned in [10] and [11] but we feel that this type of CPU workload severely exaggerates the situation.

http://www.cmg.org/measureit/issues/mit39/m_39_1.html

See the last letter in this

http://serverfault.com/questions/135431/is-virtual-machine-slower-than-the-underlying-physical-machine

While it is obvious that load is critical in any analysis, it may not be obvious how for example memory usage can depend on scheduling. If VM1 and VM2 run serially, memory usage is the max of the two, if they overlap it is the sum – unless it’s ok to slow them both down with more VM operations – which will, of course, increase the time to complete which uses capacity since that time is now not available for a third VM etc.

It’s also important to understand whether overhead is per VM. Suppose that we have 2% overhead per VM, all roughly the same size and 10 VMs. Is this overhead additive? Clearly cpu time is additive and so is I/O time, memory pressure is fuzzier and depends on how many VMs we run at any one time.

Notes

[1] If you have multiple cores, which you do, then if there are enough cores to run VM1 and VM2 in parallel, no problem. And this brings up the question of the relative power use, say of 2 dual core machines versus one 4 core machine or other multiples. Note that it’s hard to get power savings by turning off 4 of the 8 cores of a 8 core machine, but possibly easy to power down the one 4 core machine.

Birnbaum believes that one of the key foundational elements of a stateless computing environment is a networked storage system that enables ubiquitous availability of software. The file paths of the individual applications should be based on clearly defined nomenclature, much like the domain of a web site. All application dependencies should be accessible through the network filesystem, and version numbers should be expressed with the path nomenclature.

Big distributed file system – sure. Why should version numbers be expressed with the path nomenclature (a Plan9 idea, btw)?  Now we go on to the ancient problem of caching distributed data.

The obvious challenge posed by rolling out worldwide network storage infrastructure is scalability. If everyone in a global organization is depending on a network storage solution, then it needs to be fast and consistently reliable. The solution that Birnbaum proposes is regional mirroring and caching. The storage system would be universally synchronized between mirrors that have all the data. Caching can also be used at individual facilities to further improve performance. To achieve this kind of global scalability, he says, the best approach is similar to that of Akamai.

So even with a non-globally distributed file system, the problem of shared access is non-trivial. A global file system makes things quite challenging. Suppose we have a file recording trades and the Singapore, London, NY, and Espanola main offices all are reading and writing at the same time. Caching and cache coherency is an utter nightmare.  Akamai, like Google, solves the problem of massive amounts of distributed data by focusing on “delivery” – otherwise known as “read only content” or “many readers one writer” and with no requirement for true synchronization.  But the ML problem is more difficult even if we ignore multiple writers because, presumably, you want Singapore to actually see every trade made in Espanola even though for Akamai, it’s ok if the cache is not fresh. How to solve multiple readers and writers is something else as well.

These concepts don’t cover a whole lot of new ground yet. Much of this was already possible with conventional thin-client systems. The point at which it becomes immensely valuable, according to Birnbaum, is when all of these technologies are used together with virtualization to abstract the processes away from the hardware. Once this is done, individual operations can seamlessly float around data centers and balance out in a manner that offers a more optimal level of resource utilization.

And this seems to me to gloss over the even harder problem. Imagine a serious Oracle application “seamlessly floating” from some set of machines in one data-center to another set.  I can’t imagine how that works. Imagining little jobs floating is easier, but is that really an interesting problem? And this brings us to the most interesting claim:

He claims that 61 percent of a company’s enterprise server capacity goes completely unused and proposes an automated load balancing solution—

SIXTY ONE PERCENT!  Think of the power use.