< back to projects

Personal Containers

(2009 - 2015)

As cloud computing empowered the creation of vast data silos, I investigated how decentralised technologies might be deployed to allow individuals more vertical control over their own data. Personal containers was the prototype we built to learn how to stem the flow of our information out to the ad-driven social tarpits.

I've had a passion for self-hosted, decentralised computing for many years since Nick Ludlam and I set up the recoil.org collective in the late 90s. In late 2008, I'd been working on early cloud computing as part of the Xen Virtualisation project and already seeing the rapid rise of centralised data gathering in the early cloud providers. When I left Citrix in 2009, I joined Derek McCauley and Jon Crowcroft in their new Horizon Digital Economy centre to lead a charge into building more privacy-centred digital infrastructure. I had the huge privilege of receiving a strings-free 5-year postdoctoral fellowship in Cambridge. It's rare to see such long term postdoc opportunities these days, but something I am hugely supportive of for new projects.

My hacking first began with Nick Ludlam in 2008 on a prototype of a lifedb server and app, which we envisioned as a place to aggregate all the messages from disparate sources (for example, to mirror the then-new Twitter service into my IMAP email). I worked on “Privacy Butler: A Personal Privacy Rights Manager for Online Presence” to add a policy engine to this prototype. While the prototype worked well enough for me, it was largely a negative result since it was just too risky to put all that private data in one location (especially aggregated).

Now back at Cambridge in 2010, I began working with Thomas Gazagnaire on a more robust implementation of data aggregation that would have stronger end-to-end security and privacy. We started coding up an implementation in OCaml to followup my Functional Internet work, and built out infrastructure like an OCaml ORM in “Statically-typed value persistence for ML” to make it easier to work with databases. It became obvious pretty quickly that having this much data in one place required end users to become sysadmins, and so I started to lay out a new architecture for this sort of end-user managed data in “Multiscale not Multicore: Efficient Heterogeneous Cloud Computing” .

Our first prototype of a personal container running as a unikernel was published in “Turning down the LAMP: Software Specialisation for the Cloud” , and would form the basis of the MirageOS project. To this day, the MirageOS community remains passionate about decentralised systems from these origins! We explored a number of directions in the early days:

One of the main drivers for personal containers was to drive applications that would otherwise be too invasive from a privacy perspective. Ian M. Leslie and I worked on the "c-aware" project in “Confidential Carbon Commuting: exploring a privacy architecture for incentivising "greener" commuting” to figure out if personal containers could help influence user behaviour to reduce carbon usage. Overall, this project taught us just how much effort it would be to deploy real-world infrastructure in corporate environments like the University of Cambridge. We also struggled to get any users to deploy our prototype servers, something explored more in user studies with colleagues in Horizon Nottingham in “Perceived risks of personal data sharing” .

My work on personal data processing petered out from a research perspective in around 2013 since the underlying infrastructure I had built really started gathering steam with Unikernels and OCaml Labs. We hadn't quite cracked the problem of how to break the cloud hegemony, but (as with XenoServers and Xen), the pieces that succeeded emerged from the research questions we asked. However, I don't consider this project permanently closed by any means -- after all, I've been self hosting my email since 1997! We've been working steadily over the past decade of MirageOS (as of 2021) to build out a really solid, self-hosted protocol stack that will work as a unikernel. I am revisiting the question of decentralisation in the form of physical infrastructure in the Interspatial OS project, and you can read my early thoughts in “An Architecture for Interspatial Communication” .

Related publications

Privacy Butler: A Personal Privacy Rights Manager for Online Presence
Ryan Wishart, Dominic Corapi, Anil Madhavapeddy and Morris Sloman.
Workshop paper in the IEEE International Workshop on Smart Environments (SmartE 2010) on Mar 2010 at Mannheim, Germany.
Statically-typed value persistence for ML
Thomas Gazagnaire and Anil Madhavapeddy.
Workshop paper in the Workshop on Generative Technologies (WGT 2010) on Apr 2010 at Paphos, Cyprus.
Multiscale not Multicore: Efficient Heterogeneous Cloud Computing
Anil Madhavapeddy, Richard Mortier, Jon Crowcroft and Steven Hand.
Conference paper in the ACM/BCS Visions of Computer Science on Apr 2010 at Edinburgh, United Kingdom.
Turning down the LAMP: Software Specialisation for the Cloud
Anil Madhavapeddy, Richard Mortier, Ripduman Sohan, Thomas Gazagnaire, Steven Hand, Tim Deegan, Derek McCauley and Jon Crowcroft.
Using Dust Clouds to Enhance Anonymous Communication
Richard Mortier, Anil Madhavapeddy, Theodore Hong, Derek Murray and Malte Schwarzkopf.
Workshop paper in the Eighteenth International Workshop on Security Protocols (IWSP 2010) on Apr 2010 at Cambridge, United Kingdom.
The Personal Container, or Your Life in Bits
Richard Mortier, Chris Greenhalgh, Derek McCauley, Alexa Spence, Anil Madhavapeddy, Jon Crowcroft and Steven Hand.
Workshop paper in the proceedings of Digital Futures 2010 on Oct 2010 at Nottingham, United Kingdom.
CIEL: a universal execution engine for distributed data-flow computing
Derek Murray, Malte Schwarzkopf, Christopher Smowton, Steven Smith, Anil Madhavapeddy and Steven Hand.
Signposts: End-to-end networking in World of Middleboxes
Andrius Aucinas, Amir Chaudhry, Jon Crowcroft, Sebastian Probst Eide, Steven Hand, Anil Madhavapeddy, Andrew W. Moore and Charalampos Rotsos.
Workshop paper in the proceedings of the SIGCOMM 2012 demo track on Aug 2012 at Helsinki, Finland.
Lost In the Edge: Finding Your Way With Signposts
Charalampos Rotsos, Heidi Howard, David Sheets, Richard Mortier, Anil Madhavapeddy, Amir Chaudhry and Jon Crowcroft.
Evolving TCP. How hard can it be?
Zubair Nabi, Toby Moncaster, Anil Madhavapeddy, Steven Hand and Jon Crowcroft.
Workshop paper in the CoNEXT 2012 Student Workshop on Dec 2012 at Nice, France.
Cost, performance & flexibility in OpenFlow: Pick three
Charalampos Rotsos, Richard Mortier, Anil Madhavapeddy, Balraj Singh and Andrew W. Moore.
Workshop paper in the proceedings of the IEEE Workshop on Software Defined Networks on Jun 2012 at Berlin, Germany.
Exploring compartmentalisation hypotheses with SOAAP
Khilan Gudka, Robert M. Watson, Steven Hand, Ben Laurie and Anil Madhavapeddy.
Workshop paper in the Workshop on Adaptive Host and Network Security (AHANS 2012) on Sep 2012 at Lyon, France.
Confidential Carbon Commuting: exploring a privacy architecture for incentivising "greener" commuting
Chris Elsmore, Anil Madhavapeddy, Ian M. Leslie and Amir Chaudhry.
Workshop paper in the proceedings of the 1st Measurement, Privacy, and Mobility (MPM 12) workshop on Apr 2012 at Bern, Switzerland.
Perceived risks of personal data sharing
Anya Skatova, Jaspreet Johal, Robert Houghton, Richard Mortier, Neelam Bhandari, Tom Lodge, Christian Wagner, James Goulding, Jon Crowcroft and Anil Madhavapeddy.
Conference paper in the proceedings of the Digital Economy 2013 conference on Nov 2013 at Nottingham, UK.
An Architecture for Interspatial Communication
Anil Madhavapeddy, KC Sivaramakrishnan, Gemma Gordon and Thomas Gazagnaire.

Related projects

2002 - 2009 Xen Virtualisation
2003 - 2008 Functional Internet
2012 - 2021 OCaml Labs
2010 - 2019 Unikernels
2012 - 2021 OCaml Labs