One area of great frustration encountered by application developers involves the challenge of integrating new algorithms into a code base. There are many reasons for this. For example, the algorithm may be described in a journal article where many details of the implementation are omitted or it is available only in a programming language different from the one being used. The code may have software dependencies that are hard to resolve. The new algorithm may also have hardware dependencies, such as reliance on a GPU to get performance and you may not have access to this hardware. On the other hand, if you are the author of a great new algorithm you may be disappointed that your new invention is not being used for these very same reasons.
About 18 months ago a company called Algorithmia™ was founded in Seattle that provides an elegant solution to these problems. They provide a very simple multi-language API that can be used to invoke any of their catalog of 3,500 different cloud-based algorithms. While we may be getting tired of reading about X-as-a-Service for different versions of X, there is one binding for X that has been around for a while in various forms and, as much as it pains me to do so, it begs to be called Algorithms as a Service. And AaaS is just one of the things Algorithmia provides.
AaaS is indeed not a new idea. Jack Dongarra and his ICL team at the University of Tennessee created NetSolve/GridSove in 2003 to provide scientists and engineers with access to state-of-the-art numerical algorithms running on a distributed network of high performance computers. As cool as NetSolve is, Algorithmia goes several steps beyond this concept.
One of Algorithmia’s cofounders and CEO, Diego Oppenheimer has a deep background in building business intelligence tools. While working on that he developed an appreciation of the power of being able to call out to powerful algorithms from inside a user facing application. This capability allows the application to have access to deeper knowledge and more powerful computational resources than available on the user’s device. A key insight from this experience is that algorithms must be discoverable an invokable from any user application runtime. These ideas are all central to Algorithmia. In the following paragraphs we will look at Algoritmia’s marketplace, explore building a new algorithm and discuss a bit of the system microservice architecture.
Algorithmia is a marketplace.
There are over 50,000 developers that use Algorithmia services and the platform encourages these developers to contribute new algorithms to the collection. Invoking an algorithm is dead simple and it can be done from any programming language that can formulate a JSON doc and send a REST message. We will provide some detailed illustrations at the end of this document.
To use it, you need to set up an account. Doing so will get you a starter award of 5000 or so “credits”. When you invoke an algorithm, credits are deducted from your account. Typically, there is a “royalty” cost of about 10 credits and then the cost is usually around one credit per second of execution. A fun example from their library of deep learning collection is an image colorizer. Input is a PNG file of a black and white image and the returned value is a link to the output colorized image. We took a color image from a visit to Red Square a few years ago. We converted it to a grayscale image and gave that to the colorizer. The result is shown illustrated below. The original is on the left, grayscale in the middle and the colorized image on the right. While it is not as good as the best hand-colored photos, it is not too bad. It lost the amazing color of St. Bazil’s Cathedral which is not too surprising, but it was great with sky and skin tones of those people in foreground. (It seemed to think the bricks of the square would look better with some grass color.)
The Python code to upload the grayscale image and invoke the service was incredibly simple.
import Algorithmia
client = Algorithmia.client(‘youruserkeyfromaccountrecation’)
input = bytearray(open("path_to_grayscale.png", "rb").read())
result = client.algo("deeplearning/ColorfulImageColorization/1.1.6")
.pipe(input).result
path_to_local_copy_of_result_image= client.file(result[‘output’]).getFile()
The cost in credits was 154. The exchange rate for credits is 1$ = 10,000 credits (approximately) so this invocation would have cost about 1.5 cents.
This algorithm is from their extensive machine learning and AI collection. A related algorithm is one that computes the salience of objects in an image. Salience is the degree to which an object in the image attracts the attention of the viewer’s eye. The algorithm is called SalNet and it is based on ideas from the paper, Shallow and Deep Convolutional Networks for Saliency Prediction by Pan et. al. (see arXiv:1603.00845v1).
As with the colorizer, salnet it is easy to invoke.
input = { "image": "data://.algo/deeplearning/SalNet/perm/an-uploaded-image.png" }
result2 = client.algo("deeplearning/SalNet/0.2.0").pipe(input).result
Note that in this case we have loaded the image from one that we uploaded to Algorithmia’s data cloud. In fact, it is the same grayscale image of red square. As you can see below, the algorithm picks out the woman in the foreground and also notices the church in the background.
Salience computation can be very helpful in identifying and labeling objects in an image. Image tagging is also something that Algorithmia supports. Running the same image through their tagger returned the observations that the image was “safe” and that there were multiple boys and multiple girls and sky and clouds and it seem to be near a palace.
There are many other AI related image algorithms such as nudity detection, character recognition, face detection and a very impressive car make and model recognition algorithm. A quick look at https://algorithmia.com/use-cases will show many other fascinating use cases.
Another very cool capability of Algorithmia is its ability to host your trained machine learning model. Suppose you have a model you have built with MsXNet, TensorFlow, Scikit-Learns, CNTK or any of the other popular ML frameworks, you can upload your model to Algorithmia so that it can be available as a service. This is explained in here. We will explore this capability in a later post.
While the main emphasis and attraction of the Algorithmia collection is machine learning and AI, there are many more algorithm categories represented there. For example, there is an excellent collection of utilities for managing data and making certain programming tasks extremely easy: such as extracting text from web pages, Wikipedia search tools, computing the timezone and elevation from lat, lon coordinates.
There is also a large collection of time series analysis algorithms. These include forecasting, outlier detection, Fourier filters, auto-correlation computation and many more.
Algorithmia is cloud of microservices
In an excellent talk at the 2017 Geekwire cloud summit, Oppenheimer described some key elements of Algorithmia’s architecture. In this talk he makes the critically important observation that two phases of machine learning, training and prediction, if used in production require very different execution environments. Training is often done on a dedicated system consuming many hours of compute and as much memory as is available. The result of training is a model codified as data. Prediction (also called Inference) uses the model to make predictions or inferences about a sample case. Prediction can be done on the same hardware platform that was used for the training, but if the model is to be used to make predictions concerning thousands of cases for thousands of concurrent users, one need a completely different design.
Their approach to the scale problem for predictions (and for any high demand algorithm in their collection) is based on serverless microservices. They use a Kubernetes microservice foundation with algorithms deployed in Docker containers. Requests from remote client applications are load balanced across API servers who dispatch requests to container instances for the requested function. The challenge is making the latency from request to reply very low. If a container for an algorithm is already in system memory, it requires very little time to spawn a new instance on Kubernetes. Another technique they use it to dynamically load algorithms into running containers. (We don’t know the exact mechanism Algorithmia uses here, but we expect it is exploiting these facts.)
They have made some very interesting optimizations. For example, if the data used in the computation is stored in one of their cloud regions, the docker instance will be instantiated nearby. Just as important, if an algorithm invokes another algorithm they will attempt to co-locate the two containers and reduce the inter-process latency. Composability of algorithms is one of their guiding concepts.
Turning your own algorithm into a microservice.
The process of turning your own algorithm into a microservice is remarkably simple. From the Algorithmia portal there is a “+” symbol in the upper right-hand corner. This give you a dialog box to fill out. You provide a name of your algorithm, the programming language you are using (from a long list .. but sorry, no Fortran or Julia but there are lots of alternatives), and several other choices including: your source license policy, does your function invoke other Algorithmia functions, does your function invoke things on the open internet?
Answering these questions causes Algorithmia to create a nice GitHub repo for your function. Your next step is to install the Algorithmia command line interface and then you can clone your functions GitHub repo. Once you have done that you can edit the function so that it does what you want. The basic skeleton is already there for you in the “src” directory. Here is the basic skeleton in Python rendered as a hello world function.
import Algorithmia
# API calls will begin at the apply() method,
# with the request body passed as 'input'
# For more details, see algorithmia.com/developers/algorithm-
# development/languages
def apply(input):
return "hello {}".format(input)
You can edit the function directly from an editor built into the Algorithmia portal or, now that you have a clone of the repo you can use your own tools to transform this skeleton into your algorithm. If you have done this work on your clone you need to use the Github commands to push your code back to the master.
We tried this with a small experiment. We built a function called KeyPhrases that takes English language text as input and breaks it down into subjects (s), actions (a) which are like verb clauses and objects (o). The algorithm is not very useful or sophisticated. In fact, it uses another Algorithmia microservice called Parsey McParseface which was originally released by Goolge (see https://arxiv.org/ pdf/1603.06042v1.pdf) . This is truly a deep parser that build a very sophisticated tree. For example the figure below illustrates the tree for a pars of the sentence
Einstein’s general theory of relativity explains gravity in terms of the curvature of spacetime.
Parsey McParseface tree output.
Our function KeyPhrases walks the tree and groups the terms, subjects(s), objects(o) and actions (a) and returns a JSON document with the original string and the list of phrases. It also breaks out separate subphrases with “/” marks. In this case it returns
{"phrases":[
"s: Einstein's general theory /of relativity ",
"a: explains ",
"s: gravity /in terms /of the curvature /of spacetime. "
],
"text":"Einstein's general theory of relativity explains gravity in terms of the curvature of spacetime."
}
A more complex example is
Facebook Incs chief security officer warned that the fake news problem is more complicated to solve than the public thinks.
The phrase output is
['s: Facebook Incs chief security officer ', 'a: warned ', 'o: that the fake news problem ', 'a: is more ', 'o: complicated and dangerous /to solve /than the public thinks ']
This is clearly not as rich in detail as the Parsey output, but it does extract some useful key phrases.
To complete the creation of the microservice for this algorithm one need only issue the git commands
$ git add src/KeyPhrases.py $ git commit -m "added src mods" $ git push origin master
The last push causes a compile step to happen and the microservice is now created. Algorithmia also provides an easy template to add documentation and instructions about how to invoke your function. From the Algorithmia editor there is a function that allows you to “publish” your algorithm. After pushing that button, the KeyPhrase example was put in their library. You can see it here: https://algorithmia.com/algorithms/dbgannon/KeyPhrases (If you use it, remember it has not been tested very well, so it may break.)
Algorithmia as an enterprise platform
The Algorithmia serverless microservice platform is robust enough that they offer it as an enterprise product. This allows enterprises to host their own version on one of the public clouds or on their own clusters or across multiple cloud in a hybrid system. This allows their own internally used algorithm to be hosted and invoked by their in-house analytics tools and pipelines in a totally scalable way. This enterprise version comes with a management dashboard and monitoring tools.
Conclusions
Algorithmia is a fascinating company with very interesting products. It is extremely easy to sign up for a free account and it is fun to use. The team was extremely helpful when we had questions. A Jupyter Notebook with some of the examples mentioned above will be posted very soon. We found experimenting with the various algorithms from an interactive notebook was a pleasure. Creating the hosted version of the KeyPhrases algorithm took less than an hour after the original python code was debugged. In our next experiment we will explore hosting deep learning models with Algorithmia.