Data Center Design Evolution

From racks to containers to custom SOCs &FPGAs

The large scale public clouds we see today evolved from the requirements of Internet companies like Yahoo, Google and Microsoft to provide data centers to support their growing services such as email and search.   Initially they were not much more than stacks of standard PC servers in “co-lo” data center facilities.   This rapidly evolved to higher density racks and then to pre-populated containers.   In the case of Microsoft the fourth generation was based on a new technology called the IT PAC, which was a very clever design that provided a very efficient design that can be placed outside and cooled by ambient air.   In term of the standard metric of efficiency PUE (approximately power consumed/power needed by servers) it scored 1.15.

microsoft-data-centersMicrosoft Data Center Evolution

Most of the large data center builder have now moved from commodity systems to custom System on a Chip (SOCs). (see this page for a research case) or  for one industry view.)   Another interesting example is the work by Doug Burger and his team on adding FPGAs to the servers that Bing uses to accelerate the search algorithms.

The Public Cloud

As the Internet companies grew their data centers Amazon realized their infrastructure was robust enough to begin offering them as public “pay as you go” clouds. Amazon first release its storage service S3 in 2006 and the compute platform EC2 in 2007 ( This authors first evaluation can be found here.)   EC3 introduced “Infrastructure as a Service” (IaaS) as a concept where customers could launch and configure virtual machines (VMs) that they could configure and expose as web services.   For web startup companies this idea proved to be a bonanza. Without investing in hardware acquisition and infrastructure management a small startup could deploy a production service with one or two servers and then expand it as the demand grew.

Google and Microsoft took a different approach. In early 2008 both launched clouds based on the concept called “Platform as a Service” (PaaS). These cloud services provided a programming model for building services rather than raw virtual machines. This idea significantly raised the level of abstraction for the programmer. Google’s original App Engine  was based on a Python programming model for building applications that ran on Google’s cloud.   Microsoft launched Windows Azure that provided a C# based framework that allowed the programmer the ability to build agent-like networks of web frontends connected to backend worker processes.

By 2013 both Google and Microsoft realized that many customers really wanted raw IaaS so they introduced that into their offerings.

Higher level services and tools

All the public cloud vendors have now realized that selling raw cycles is not a very compelling business. The original three large cloud companies have since been joined by dozens of others and each is offering a variety of infrastructure, platform and software services.   There are really three core markets

  • The startup community’s need for basic VM and databases to support their cloud services.   These are often back-end to mobile apps so it helps if the cloud provider can provide additional tools to support mobile services.
  • The enterprise customer that need extra capacity on a demand basis.   This huge market is driving the concept of hybrid clouds.   That is, using the public cloud as an extension of the enterprises internal IP infrastructure.
  • The big data analytics customer.   These are companies that are doing large scale data analysis and they need access to large SQL and NoSQL storage and tools that can be used to scale-up analytics algorithms.   This includes the entire Apache Hadoop/Yarn stack of tools.   The best overview of this stack is Geoffrey Fox’s CloudMesh talk.  Other pages here will delve deeper into this topic.

It is the last two categories, enterprise and big data analysis, that are the main focus of these pages because they have the most to do with scientific applications.