Xen Hypervisor

I was on the original team at Cambridge that built the Xen hypervisor in 2002 -- the first open-source "type-1" hypervisor that ushered in the age of cloud computing and virtual machines. Xen emerged from the Xenoservers project at the CL SRG, where I started my PhD and hacked on the emerging codebase and subsequently worked on the development of the commercial distribution of XenServer.

Back at the turn of the century, the Computer Lab SRG faculty at the time (led by my first PhD supervisor Ian Pratt) decided to start the XenoServers project, which would build a public infrastructure for wide-area distributed computing. An EPSRC grant lead to a number of graduate students all surging into the SRG in around 2002 to work on the project, including me.

The later history of Xen is chronicled on the original Xen website, but the early days were a heady mixture of furious hacking to put the various prototypes together. I did a very early port of NetBSD to the PV interface, before the introduction of linear page table checking into the hypervisor defeated my port and Christian Limpach took it over. That early work was recorded in the Xen 2002 technical report. My involvement for a while after was limited, as I also interned with Sandy Fraser in Princeton and set up the Ubiquitous Interaction Devices project with Intel Research.

It was after the open source release of Xen 1.0 and the submission of my PhD thesis that I joined XenSource as an early engineer and began release managing the first commercial distribution of Xen, known as XenServer. This involved building an entire embedded "appliance" that hid the underlying complexities of managing virtual machines. To add to the fun, we also built an entire management toolstack in OCaml, making it one of the largest commercial uses of functional programming back then. Our experiences with building this are published in Using functional programming within an industrial product group: perspectives and perceptions, and the XenServer management stack is still going strong as an open source project.

The nitty-gritty of XenServer engineering has never been captured in an academic paper, but I wrote a few blog posts (on the now-defunct Citrix blog) that are mirrored here:

• Installing Ubuntu on XenServer covers how the then-nascent Linux distribution could be virtualised.
• Shedding light on XenApp on XenServer performance tuning discusses some performance profiling issues with XenServer after the Citrix acquisition of XenSource.
• Peeking under the hood of High Availability illustrates the extremely cool high-availability feature we built into XenServer 5.0, using some fairly complex OCaml hacking under the hood of the management stack.

Once I returned to academia full-time in 2010, much of my later work also improved the Xen toolstack. I laid out the early vision for multiscale computing in Multiscale not multicore: efficient heterogeneous cloud computing and subsequently a prototype from the Unikernels project in Turning Down the LAMP: Software Specialisation for the Cloud. As Xen got itself an ARM port a few years later, my work on Jitsu: Just-In-Time Summoning of Unikernels also fed back to Xen development by highlighting potential efficiencies in the toolstack. I also investigated whether FPGAs would make sense in cloud environments in Reconfigurable Data Processing for Clouds.

In 2021, I largely use Solo5 and KVM as my main hacking and production hypervisor, but I plan to revisit Xen at some point as I begin looking at RISC-V architectures and embedded systems again as part of Interspatial OS.

Activity

• About – Research note (Feb 2025)
• Programming FPGAs using OCaml – Research note (Feb 2025)
• Unikernels – Project (2010–2019)
• Personal Containers – Project (2009–2015)
• Xen 2002 – Note about Xen 2002 (Jan 2003)
• Functional Internet Services – Project (2003–2008)
• Ubiquitous Interaction Devices – Project (2003–2008)
• Started PhD at Cambridge – Research note (Sep 2002)

References