blog.richardmcdougall.com

Is Cloud Storage going to disrupt Traditional Storage? – Part2: What is Blob storage?

November 5, 2010 – 5:53 pm
Cloud-scale Blob is the new category of storage designed specifically to meet a very relaxed set of requirements uniquely selected to match the needs of the majority of bytes of data for new media types. Gone are the complex constraints of concise consistency [Read More...]

Posted in NAS, Virtualization | 1 Comment »

VMware performance for Gurus

November 5, 2010 – 9:06 am
I'm running my 11th annual Usenix performance class next week in San Jose - VMware performance for Gurus http://tinyurl.com/2g2t37q

Posted in Benchmarking, Performance, Public Talks, VMware, Virtualization | No Comments »

Is Cloud Storage going to disrupt Traditional Storage? – Part1: The demise of expensive datacenter storage

October 25, 2010 – 5:35 pm
Recently, I've been looking at changes that are unfolding for the ways personal users store and manage data, and server applications use, store and manage data. I see some significant trends that radically change the way new types of storage systems are structured. Read more...

Posted in VMware, Virtualization | No Comments »

Flash & Phase Change Memory Talks

October 26, 2009 – 5:46 pm
  This week, I attended talks at the international high performance transaction workshop. Following are my rough notes from the Flash Memory discussions. First up Steve Kleiman from Network Appliance spoke about their intent to move  flash into the clients that access NAS, so that it can intelligently cache and interact with the backend storage: Netapp is building host-side caches for VMware and HyperV The cache will be a block based read cache  Cache is write through   Andy Bechtolshiem, cofounder of Sun and now at Arista Networks talked about trends affecting system design and how Flash could be leveraged. 3d chip packaging will become common, to solve power and latency fundamentals around higher clock rate memory systems.  This will mean that systems will become very NUMA, since the memory is directly attached to the cpu core. This is the decade where we switch to optical in the datacenter. Cutover is 10gbit (copper), 20 requires optical and will move to volume commodity ...

Posted in Performance | No Comments »

RISC to x86, Physical to Virtual

April 21, 2009 – 1:12 pm
Many of the virtualization customers I speak with have been asking about what database can be virtualized, so I typically present a green, yellow and red set of criteria to help guide the process. We typically look at I/O rates, SMP width and capacity of the source physical system as a guide. Recently, we were able to run a full TPC-C like workload inside a virtual machine, showing that we can run at over 90% efficiency, even given the brutal nature of this workload (it's I/O rate is actually about 6 times higher than what you would typically see in a customer workload). The methodology is that green is considered for the easy databases -- typically 2 CPUs or less and less than 500 disk I/O's per second. These virtualize easy, with minimal best practices required. Yellow is for larger databases, where the I/O rate is higher -- they typically need more ...

Posted in Benchmarking, Oracle, Performance, Solaris, VMware, Virtualization | No Comments »

The Solaris Internals Virtual Shell Game

March 2, 2009 – 12:33 pm
In a stark contrast to recent past when Solaris Internals was hosted on physical machines, we've just experienced one of the core values of virtualization -- encapsulated, portable state allowing geographic mobility. Solaris Internals has over the last two weeks been teleported out of my datacenter (As Jim notes, the "RMCplex", which refers to my rack of managed servers), to the VMware's CTO's iMac, and then to my new physical location -- keeping the the solarisinternals.com site live the entire duration. Yes, I just moved house, with zero downtime! The Physical RMCplex It wasn't long ago that solarisinternals.com was hosted on a triad of physical machines - a SPARCstation 5 as the router, a dual-socket Opteron machine as the ZFS NAS server, and a dual core AMD server. The RMCplex is hosting many sites today: solarisdatabases.com solarisinternals.com pronk*.com joost*.com zygan*.com mail server dns server shared shell environment server The router provides the perimeter security ...

Posted in Benchmarking, Fun, NAS, Performance, Solaris, VMware, Virtualization | 14 Comments »

Saving Joules II

November 6, 2008 – 10:31 am
[ there was a bad youtube server hosting the first video, so here's a resubmit with a pointer to an alternative youtube copy It's been well talked up that consolidating physical systems can save a lot of datacenter engery -- for example, consolidating 20 systems, all with 3-5% utilization can reduce power by up to 10-15x -- or considered in more tractable terms, each server removed is equivalent to taking four cars off the road or planting 55 trees in a year. Taking this concept further, the distributed power management feature in Virtual Infrastructure allows us to further reduce power by another 50%. Since load levels of typical applications aren't constant, there are periods of the day where less servers are required -- and in those periods, power can be saved by moving the remaining work onto fewer servers and powering down the unneeded capacity A colleague put together this entertaining video, in ...

Posted in Benchmarking, Fun, Performance, VMware, Virtualization | No Comments »

VMware Performance Tutorial next week at Usenix LISA

November 5, 2008 – 3:26 pm
Heads up for those interested in understanding VMware performance in more details -- I'm running my performance and tuning of VMware ESX server class at Usenix LISA again this November in San Diego. The session is on Friday, November 14, and is an all-day class. Given our obsession with things that go fast, the track code "F1" is most appropriate ;-) http://blogs.vmware.com/performance/2008/10/one-day-tutoria.html

Posted in Benchmarking, Performance, Public Talks, Virtualization | 3 Comments »

Saving Joules

November 5, 2008 – 1:39 pm
It's been well talked up that consolidating physical systems can save a lot of datacenter engery -- for example, consolidating 20 systems, all with 3-5% utilization can reduce power by up to 10-15x -- or considered in more tractable terms, each server removed is equivalent to taking four cars off the road or planting 55 trees in a year. Taking this concept further, the distributed power management feature in Virtual Infrastructure allows us to further reduce power by another 50%. Since load levels of typical applications aren't constant, there are periods of the day where less servers are required -- and in those periods, power can be saved by moving the remaining work onto fewer servers and powering down the unneeded capacity A colleague put together this entertaining video, in which we measured the performance and watts consumed during typical peaky workload. The scenario is based on the email load curve typically ...

Posted in Benchmarking, Fun, Performance, VMware, Virtualization | No Comments »

IO Challenge Update 5: 102,240 IOPS

May 21, 2008 – 12:49 pm
We're really excited to announce that we hit over 100k IOPS! This was done using 77TB of storage, adding a 3rd rack totalling 495 disks -- enough storage to hold the entire printed library of congress, and all being driven from a single ESX server! Chethan's just written a detailed report over on VROOM... The interesting points about this effort to me are: There was almost no tuning required There is no visible I/O throughput ceiling for ESX -- the throughput is roughly that of 50 4-cpu databases... Even at 100,000 IOPS, the virtualization layer only added 0.1ms of additional latency, which amounts to 1.4% virtualization overhead

Posted in Performance, VMware, Virtualization | 2 Comments »