iconAnil Madhavapeddy, Professor of Planetary Computing

CIEL: A universal execution engine for distributed data-flow computing

Derek G Murray, Malte Schwarzkopf, Christopher Smowton, Steven Smith, Anil Madhavapeddy, and Steven Hand.

Paper in the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11).

URL (usenix.org)   BIB   PDFpdf

This paper introduces CIEL, a universal execution engine for distributed data-flow programs. Like previous execution engines, CIEL masks the complexity of distributed programming. Unlike those systems, a CIEL job can make data-dependent control-flow decisions, which enables it to compute iterative and recursive algorithms. We have also developed Skywriting, a Turing-complete scripting language that runs directly on CIEL. The execution engine provides transparent fault tolerance and distribution to Skywriting scripts and high-performance code written in other programming languages. We have deployed CIEL on a cloud computing platform, and demonstrate that it achieves scalable per formance for both iterative and non-iterative algorithms.

# 1st Mar 2011 iconpapers bigdata cloud conference distributed fp systems

Related News