home Anil Madhavapeddy, Professor of Planetary Computing  

Towards a frugal userspace for Linux / Dec 2024

All the work we've been doing on biodiversity (such as LIFE) comes at a fairly large computation and storage cost due to the amount of data that we churn through. This gets worse when you consider the exploratory nature of science -- we sometimes just need to mess around with the large dataset to test hypotheses which are often shown to be wrong. So then, when the LOCO conference came around, we wrote up our thoughts on what a frugal Linux userspace might look like.

The key insight is that the Linux kernel already exposes a number of namespace mechanisms (that we use in Docker, for example), and so we explore a new OS architecture which defaults to deterministic, reusable computation with the careful recording of side-effects. This in turn allows Linux to guide complex computations towards previously acquired intermediate results, but still allowing for recomputation when required by the user. We're putting this together into a new shell known as "Shark", and this first abstract describes our early results.

# 1st Dec 2024   iconpapers abstract carbon docker life linux loco shark systems zfs

Void Processes: Minimising privilege by default / Jan 2021

This is an idea proposed as a Cambrige Computer Science Part III or MPhil project, and has been completed by Jake Hillion. It was supervised by Anil Madhavapeddy.

Void processes intend to make it easier for all developers to produce effectively privilege separated applications. The project has two primary goals: show the merits of starting from zero privilege, and provide the utilities to make this feasible for the average developer.

Building void processes involves first reliably removing all privilege from a process then systematically adding back in what is required, and no more. This project utilises Linux namespaces to revoke privilege from an application, showing how this can be done and why its easier in some domains than others. It then shows how to inject sufficient privilege for applications to perform useful work, developing new APIs that are friendly for privilege separation. These elements compose a shim called the "void orchestrator", a framework for restricting Linux processes.   […158 words]

# 1st Jan 2021   iconideas docker idea-done idea-hard linux security systems

Unikernels: the rise of the library hypervisor in MirageOS / Oct 2016

DockerCon talk on unikernels and MirageOS

# 14th Oct 2016   icontalks iconvideos docker mirageos ocaml systems unikernels xen

The functional innards of Docker for Mac and Windows / Jun 2016

I gave a talk at the Functional Works meetup, held in Jane Street London about how Docker for Mac and Windows use OCaml and unikernels under the hood.

# 24th Jun 2016   icontalks iconvideos docker janestreet mirageos ocaml systems unikernels xen

Improving Docker with Unikernels (via Docker) / May 2016

I announce the release of three big components that form the basis for Docker for Desktop: a hypervisor framework called HyperKit, a networking framework for host translation called VPNKit, and a versioned data management store called DataKit.  

# 18th May 2016   iconnotes docker ocaml ocamllabs opensource unikernels

Unikernel Systems is now part of Docker / Jan 2016

Announced that Unikernel Systems is now part of Docker

# 21st Jan 2016   icontalks iconvideos docker startups unikernels

Unikernel Systems acquired by Docker (via The New Stack) / Jan 2016

My startup Unikernel Systems was acquired by Docker, and I'll be joining and setting up a UK branch of Docker along with the rest of my team.

'Just like we did with containers, we are interested is democratizing that technology, making it available and useful to the millions of developers and IT pros out there, said Solomon Hykes, founder and chief technology officer for Docker. 'Unikernels allow you to basically get rid of the operating system, and instead compile into the application the small bits of the operating system it really needs.' -- The New Stack   […111 words]

# 21st Jan 2016   iconnotes docker opensource startups unikernels

Unikernels: Functional Infrastructure with Mirage OS / May 2015

Talk at Esper on functional programming with unikernels

# 12th May 2015   icontalks iconvideos california docker irmin mirageos ocaml unikernels

Jitsu: Just-In-Time Summoning of Unikernels / May 2015

NSDI 2015 talk on Jitsu

# 4th May 2015   icontalks iconvideos california distributed dns docker embedded irmin mirageos ocaml unikernels

Using Docker to bulk-build OPAM packages on Linux / Nov 2013

Now that OCaml 4.01 has been released, there is a frenzy of commit activity in the development trunk of OCaml as the new features for 4.02 are all integrated. These include some enhancements to the type system such as injectivity, module aliases and extension points as a simpler alternative to syntax extensions.

The best way to ensure that these all play well together is to test against the ever-growing OPAM package database as early as possible. While we’re working on more elaborate continuous building solutions, it’s far easier if a developer can quickly run a bulk build on their own system. The difficulty with doing this is that you also need to install all the external dependencies (e.g. libraries and header files for bindings) needed by the thousands of packages in OPAM.

Enter a hip new lightweight container system called Docker. While containers aren’t quite as secure as type-1 hypervisors such as Xen, they are brilliant for spawning lots of lightweight tasks such as installing (and reverting) package installations. Docker is still under heavy development, but it didn’t take me long to follow the documentation and put together a configuration file for creating an OCaml+OPAM image to let OCaml developers do these bulk builds.   […748 words]

# 15th Nov 2013   iconnotes docker ocamllabs opensource unikernels