Xen Virtualisation

Back at the turn of the century, the Computer Lab SRG faculty at the time -- led by Ian Pratt -- decided to start the XenoServers project, which would build a public infrastructure for wide-area distributed computing. An EPSRC grant lead to a number of graduate students all surging into the SRG in around 2002 to work on the project, including me.

The later history of Xen is chronicled on the original Xen website, but the early days were a heady mixture of furious hacking to put the various prototypes together. I did a very early port of NetBSD to the PV interface, before the introduction of linear page table checking into the hypervisor defeated my port and Christian Limpach took it over. That early work was recorded in the “Xen 2002” technical report. My involvement for a while after was limited, as I also interned with Sandy Fraser in Princeton and set up the Ubiquitous Interaction Devices project with Intel Research.

It was after the open source release of Xen 1.0 and the submission of my PhD thesis that I joined XenSource as an early engineer and began release managing the first commercial distribution of Xen, known as XenServer. This involved building an entire embedded "appliance" that hid the underlying complexities of managing virtual machines. To add to the fun, we also built an entire management toolstack in OCaml, making it one of the largest commercial uses of functional programming back then. Our experiences with building this are published in “Using Functional Programming within an Industrial Product Group: Perspectives and Perceptions”, and the XenServer management stack is still going strong as an open source project.

The nitty-gritty of XenServer engineering has never been captured in an academic paper, but I wrote a few blog posts (on the now-defunct Citrix blog) that are mirrored here:

• Installing Ubuntu on XenServer covers how the then-nascent Linux distribution could be virtualised.
• Shedding some light on XenApp on XenServer performance tuning discusses some performance profiling issues with XenServer after the Citrix acquisition of XenSource.
• Peeking under the hood of High Availability illustrates the extremely cool high-availability feature we built into XenServer 5.0, using some fairly complex OCaml hacking under the hood of the management stack.

Once I returned to academia full-time in 2010, much of my later work also improved the Xen toolstack. I laid out the early vision for multiscale computing in “Multiscale not Multicore: Efficient Heterogeneous Cloud Computing” and subsequently a prototype from the Unikernels project in “Turning down the LAMP: Software Specialisation for the Cloud”. As Xen got itself an ARM port a few years later, my work on “Jitsu: Just-In-Time Summoning of Unikernels” also fed back to Xen development by highlighting potential efficiencies in the toolstack. I also investigated whether FPGAs would make sense in cloud environments in “Reconfigurable Data Processing for Clouds”.

In 2021, I largely use Solo5 and KVM as my main hacking and production hypervisor, but I plan to revisit Xen in 2022 as I begin looking at RISC-V architectures and embedded systems again as part of Interspatial OS.