Facebook engineers and analysts tap the company’s Hadoop clusters via a SQL-like query language known as Hive. [..]

Separate from its Hadoop work, Facebook built Cassandra, a distributed database also based on a piece of Google’s backend. Google uses a proprietary distributed database known as BigTable that runs atop the Google File System (GFS) system, and it published a paper on the technology in 2006. In echo of the Hadoop project, Facebook leaned on the paper in building Cassandra.

But Cassandra isn’t a pure BigTable mimic. Facebook applied BigTable’s data model to the Dynamo distributed storage system developed by Amazon for its S3 storage service, part of the retailer’s increasingly popular Web Services cloud. Cassandra’s authors included Avinash Lakshman, who helped build Dynamo at Amazon. Register

The Tile-Gx chips have 64-bit processing on their cores, and include floating point math instructions that allow a floating point operating to be done in five cycles instead of hundreds of cycles when done in software.

This is, believe it or not, important for PHP support, Ihab Bishara, director of cloud computing applications at Tilera, tells El Reg.

The Tile-Gx chips might support 64-bit processing, but physical memory addressing on the chips is either 39-bit or 40-bit, which works out to either 512GB or 1TB of maximum main memory. Each core burns less than a half watt of power. From the Register

All that work – to make PHP run faster and use less power.

Something like this

Ok that rocks, 21KW/hour. But can you really get that savings?  If program 1 and program 2 need to run at the same times: you can’t save anything on them[1] and then the virtualization itself adds cycles for purely overhead and increases memory which burns more power. Suppose we now add K% as the overhead measure – real need R= D+D*K/100. If  K=20, we cut savings in half. But 20 seems wildly overoptimistic to me, so put it at 50.  We’re still only needing 72 servers instead of 200, but they are running flat out so we actually increase power use.

That does not rock.   But here’s the kicker – if we reduce idle power to 25Watts, maybe by turning off idle machines, a 50% overhead rate for virtualization means virtualization increases power use by 20KW. If we can reduce idle power to 1 watt, even at 10% overhead, virtualization increases power use.  So what are the real numbers? I have not seen any published studies (of actual data centers) – maybe there are some.

If you want a copy of the xls, send me an email. Maybe I set it up wrong.

http://www.hpl.hp.com/personal/Lucy_Cherkasova/papers/final-perf-study-usenix.pdf

This claims lower levels of overhead

http://www.engineyard.com/blog/2009/10-years-of-virtual-machine-performance-semi-demystified/

but I’m unconvinced. In particular, I’m curious about memory usage.

This is funny.

The benchmark itself reported its elapsed time by calling a function to find the system time at both the beginning and end of the benchmark.  The elapsed time reported by the benchmark was less than the wall-clock elapsed time.  What we hypothesize is that, due to the unrelenting CPU consumption by the benchmarks, the virtualization layer was unable to update its clock with the virtual CPU clock ticks.  This phenomenon is mentioned in [10] and [11] but we feel that this type of CPU workload severely exaggerates the situation.

http://www.cmg.org/measureit/issues/mit39/m_39_1.html

See the last letter in this

http://serverfault.com/questions/135431/is-virtual-machine-slower-than-the-underlying-physical-machine

While it is obvious that load is critical in any analysis, it may not be obvious how for example memory usage can depend on scheduling. If VM1 and VM2 run serially, memory usage is the max of the two, if they overlap it is the sum – unless it’s ok to slow them both down with more VM operations – which will, of course, increase the time to complete which uses capacity since that time is now not available for a third VM etc.

It’s also important to understand whether overhead is per VM. Suppose that we have 2% overhead per VM, all roughly the same size and 10 VMs. Is this overhead additive? Clearly cpu time is additive and so is I/O time, memory pressure is fuzzier and depends on how many VMs we run at any one time.

Notes

[1] If you have multiple cores, which you do, then if there are enough cores to run VM1 and VM2 in parallel, no problem. And this brings up the question of the relative power use, say of 2 dual core machines versus one 4 core machine or other multiples. Note that it’s hard to get power savings by turning off 4 of the 8 cores of a 8 core machine, but possibly easy to power down the one 4 core machine.

TimeKeeper really builds on what our experience with RTLinux taught us about barriers to use. TimeKeeper installs simply – no developer needed, it’s just an app; it requires nearly no configuration; and it is invisible to application code except that it makes sure they get accurate time when they ask for the time.

for i = 1 ... n
      create thread i
             do something
end for

as if that was the only way to express parallel computation. This is such an awkward way of looking at problems.  I think many problems come from the sloppy “non-determinism” of the operating systems and multi-core machines.  One of the few  interesting ideas seen in the last 20 years on parallel processing is the Google map-reduce scheme ( . But what I find impressive is bashreduce. This is really a clever trick and a great validation of the UNIX toolset design (as if it needed another validation).

A parallel computer system which has a primary task processor, a second primary task processor, a secondary task processor acting as a backup for the second primary task processor transfers messages by: sending messages from the primary task processor to the second primary processor with the second primary task processor operating on the messages by initially storing a received message in a queue and thereafter reading the message from the queue for processing in accordance with the task associated therewith and accumulating a count of the messages read from its queue; and sending the same messages from the first primary task processor to the secondary task processor which stores the messages in a message queue for possible use if the second primary task processor fails. If a primary task processor fails after processing a given number of messages, the secondary task processor associated therewith starts processing the messages in its queue but after having discarded the first given number of messages.