This topic is addressed by several of the blog posts and a chapter in the book.  The basic idea is derived from the challenge of building extremely large, persistent cloud applications.   There are several key  requirements the application must have:

• Global scale => distributed => well defined and usable consistency models
• Dynamic scaling to support 1000s of concurrent peak time users.
• You must assume the infrastructure is constantly failing
• But your app must stay up!
• Designed so that upgrade and test occur seamlessly while app is running.
• Security and Privacy not afterthoughts.

A style of application construction has evolved to support this called Cloud Native.  The idea is to build app from the ground up from small, stateless “microservice” containers or functions.   These functions

• Supports scalable parallelism
• Rapid application modification and evolution
• Easily distributed to provide fault tolerance

Examples of applications built this way include: Netflix, Facebook, Twitter, Google Docs, Azure CosmosDB which supports billions of transactions per week, Azure Event hub  which handles trillions of requests per week and  Azure Cortana which deals with million evals/sec.   Azure IoTHub, Skype, Power BI, CRM Dynamics, AWS Kinesis are other examples.

To manage all these thousands of microservices you need a framework that can schedule containers across dozens of data centers, handle fault monitoring and replication, scale up and down when needed, manage network proxies and more.

 

Docker, Swarm, Kubernetes and Mesosphere

To build a substantial microservice application one needs to build on a platform that can manage large collections of distributed, communicating services. Thanks to the open source movement we now have an excellent collection of tools for building and managing microservices.   These are

1. Containers.  This revolutionary technology that allows us an alternative to deploying services as heavy-weight virtual machines.   “Containers” run on a host OS and use basic services such as kernel namespaces and resource isolation provided by that OS. Docker containers contain all the OS components and libraries needed by the app, various user-added files and apps and instructions for what process to run. Docker exploits the host OS namespaces for isolation, control groups for resource management.   It uses the Union File Systems so that containers are built as layers on existing FS. The company Docker.com also provides a registry of dockerized tools and application that can be downloaded. We show an example of using Docker and the registry in another section. 

2. As noted the host OS for Docker need not be very heavy because each container will have its own OS features and libraries required by the application. CoreOS is a basic “stripped down” version of Linux that contains only those tools needed to support Docker containers.

The tools for managing clusters of microservices hosting containerized microservices are many.   The ones we have tested are

1. Swarm. Docker.com provides a tool for managing clusters of servers running Docker container instances. We tested Swarm on Azure using the client Docker-machine and it was relatively easy to bring up a cluster of machines and deploy apps.   Swarm shows great promise but it is still beta quality and we found the documentation still rather sparse.
2. Kubernetes was recently released by Google and is based on the cluster management tools they use. We also found it relatively easy to deploy Kubernetes on Azure using another tool called Weave to support the networking.   Kubernetes has an important feature not present in the other microservice cluster management tools.   In Kubernetes every container lives inside a “pod”.   Pods can contain a single Docker container instance or several instances.   This is important because often a single microservice instance may always collaborate with another microservice instance of the same or different type. For example a Dockerized web server may need a private instance of a Redis cache or SQL database.   These are relatively standard docker components, so there is no reason to deploy them in your web server container.   Because each pod instance runs on a single server, the containers in the same pod share resources such as private directories. Communication within a server is going to be faster and more reliable than communication across the network.
3. Mesosphere. One of the many products from the UC Berkeley AMP lab was a distributed computing operating system (dcos) called Mesos.   It has since been spun off as a startup called Mesosphere.com.   Mesosphere installed easily on Azure and it has proven to be extremely reliable.   Because we use it so extensively we have an addition page about it here. An addition description can be found in this post programming-the-cloud-with-microservices-docker-and-mesosphere
4. Microsoft Azure Service Fabric. Microsoft recently released a development kit for a microservice orchestration framework that they have been using internally for a while. The SDK is easy to install into visual studio and it comes with an emulator so you can start building apps. I have not yet had a chance to try the full deployment.   Mark Russinovich has a nice blog about this and an online video of a talk he gave describing the system.

The first experiments described here are with the DCOS systems as deploy on azure: Performance Analysis of a Cloud Microservice-based ML Classifier.   We later explored Google’s cloud container service which was later released as Kubernetes.  Kubernetes is now well supported on Azure, IBM Bluemix cloud and more.   It is becoming a standard supported by the cloud native foundation.