Understanding Cloud Host Pricing, Part 1

The pricing schemes of the top cloud infrastructure-as-a-service (IaaS) providers are rather complicated. They cannot be compared directly since their performance characteristics vary. Moreover, the differences in costs of instances, storage, and bandwidth may offset each other. For example, one provider A's storage costs may be greater than provider B but B's instance costs may be greater. Some providers bill for instances by the minute (Azure and Google Computer Engine) whereas others bill by hour (rounded up), such as AWS. Some providers charge for storage by the TB (Azure) whereas others charge by GB/month or sometimes GB/hour (Rackspace). Consequently, what constitutes to the best deal in terms of cloud hosting depends on your specific workloads and storage needs. In this series, we will investigate the various aspects of cloud host pricing from the major providers: Amazon AWS, Microsoft Azure, Google Compute Engine, DigitalOcean, and Rackspace.

Retry Pattern

A common design pattern in fault-tolerant distributed systems is the retry pattern. A given operation may experience a variety of failures:

1. rare transient failures

   (e.g., corrupted packet) can be recovered from immediately and thus should retry immediately

2. common transient failures

   (e.g., network busy) can retry after waiting for a period of time (possibly with exponential backoff

3. permanent failure

   should not retry, bail out and clean up

Of course, the final case is that the operation succeeds and the function must do some work to address that. This is an interesting design pattern not only for distributed systems but also fault-tolerant systems in general. For example, high-performance JavaScript engines have parsers and tokenizers that must be robust to various failures. In fact, it is one example where large systems have used multiple exit points, more complicated control flow for which C-based programs may use gotos.

Parallel Data Types

True parallelism can be achieved via a wide variety of avenues. Instruction-level parallelism arguably offered a huge value at one point when processor architectures bulked up to execute machine instructions in parallel whenever possible including when instructions can be reordered or delay slots can be filled. All this could have been done without an application programmer's involvement or knowledge, hence it promised the benefits of parallelism for most programs out-of-the-box without modifications. The onus, instead, was on the processor architects and compiler developers to take advantage of such parallelism, which sometimes turned out to be a tall order. Task parallelism, in contrast, typically demands a lot more from application programmers since exposing and taking advantage of parallelism becomes very domain-specific. The focus of parallelism for task parallelism depends on balancing the loads of execution threads. Runtime support can help fill in the gaps here by applying advanced scheduling techniques such as work stealing, but ultimately what constitutes a thread (i.e., how to partition the program) and how to mediate communication and sharing between threads is often up to the programmer. Another way to achieve parallelism at a fine-grain level is to implement and expose a parallel data type in a language, runtime, or distributed computing framework. Apache Spark achieves this using its RDD (Resilient Distributed Dataset) abstraction which can copy data from Scala Seq, Java Collection, and Python iterables. Hadoop uses Java Iterable for MapReduce. For modern distributed computing environments, the pipeline itself might be any DAG (more general than map-reduce), but ultimately parallelism stems from the data representation, hence data parallelism. Note that of all the forms of parallelism it is data parallelism that ultimately has demonstrated straightforward and efficient scaling to truly large problems.