I thought I’d kick off my Citrix blog with a question I get pretty often from Linux enthusiasts: how to install unsupported Linux distributions on XenServer 4.1.

The most common solution people find is to use the “Other Install Media” template, insert the distribution installation CD, and find that the mouse cursor doesn’t work when they boot into X11. The reason for this is that they are using the hardware-assisted emulation mode of installing Linux. In this mode (dubbed “HVM”), all input and output is emulated, and in particular the mouse interface uses the USB tablet interface. If the distribution doesn’t include a driver for USB tablets, then no mouse will appear.

Windows guests run at high-speed in HVM mode due to the installation of the XenServer tools which install high-speed drivers, but these are not necessary for Linux distributions since they can be run in para-virtualized mode (dubbed “PV”). This involves obtaining a Xen-enabled PV kernel from the distribution, and modifying the VM record in XenServer to boot into this kernel instead of HVM mode. The XenServer built-in templates for popular distributions such as RHEL, CentOS or SUSE Linux already automate all this and are in PV mode from the installer onwards.

In the remainder of this post, I’ll explain how to take a distribution without direct support (Ubuntu 8.04), get it installed in HVM mode on XenServer 4.1, and convert it to PV mode with a XenCenter graphical console.

• Download the "Alternative Installation CD". The main installation CD uses graphical mode, which won't install as well in HVM mode due to the use of esoteric 16-bit mode instructions for the graphics operations. The 16-bit emulation mechanisms vary between processors (with better support on AMD chips, and a software instruction emulator required on Intel VT chips). However, the Ubuntu alternate CD uses a text-based installer which works fine.

• Create a new VM on the XenServer 4.1 host using the "Windows Server 2003" template. This template is set up with a sensible set of hardware emulation flags and default disks, and so is a good base for the HVM installation of Ubuntu as well. Attach the Ubuntu ISO you just downloaded to the VM, and proceed to install Ubuntu as normal. You should install it onto the first disk, to make the subsequent steps in this guide easier.

• When the installation is finished, reboot the VM (don't forget to detach the installation ISO first). It should boot up in HVM mode into the graphical login screen. The XenCenter display will show it as not being optimized, which is fine. At this stage, I prefer to work via a remote command-line using SSH. Open up a Terminal from Ubuntu, and run "sudo apt-get install openssh-server". Then find out the VM's IP address with "ifconfig eth0", and then connect to it remotely. Alternatively, you can continue to type in the commands directly into the terminal as well.

• On the Ubuntu guest, you now need to install the latest Xen version of the Ubuntu kernel:

  • Install the Linux kernel virtual package with "sudo apt-get install linux-image-xen". This is a virtual package which pulls in the latest Xen kernel and modules, in my case 2.6.24.19.21.
  • You now need to workaround a bug in grub. Due to the switch in recent versions of Linux to work with the hypervisor-independent paravirt_ops interface, update-grub doesn't update the grub configuration with your newly installed Xen kernel. To fix this:

    • Open /boot/grub/menu.lst in your favourite editor.

    • Scroll to the bottom to the kernel list, and find the entry which looks like:

      title           Ubuntu 8.04, kernel 2.6.24-16-generic
      root            (hd0,0)
      kernel          /boot/vmlinuz-2.6.24-16-generic root=UUID=<uuid> ro quiet splash
      initrd          /boot/initrd.img-2.6.24-16-generic
      quiet
      

    • Add a new entry which is similar to this, but change all references to the 2.6.24-16-generic to the Xen kernel. In /boot I have vmlinuz-2.6.24-19-xen, so my new entry looks like:

      title           Ubuntu 8.04, kernel 2.6.24-19-xen
      root            (hd0,0)
      kernel          /boot/vmlinuz-2.6.24-19-xen root=UUID=<uuid> ro quiet splash
      initrd          /boot/initrd.img-2.6.24-19-xen
      quiet
      

    • Also edit the default entry in the menu.lst to match the number of the kernel you just added. I set mine to 3, since it is the fourth entry in the list and the indexing starts from 0.

• When this is done, shut down the guest but do not reboot it just yet. You first need to edit the VM record for your Ubuntu VM to convert it to PV boot mode. From the control domain console of your XenServer:

  • Determine the UUID of the Ubuntu VM by using the xe CLI:
    • xe vm-list name-label=Ubuntu params=uuid --minimal : this will print out the UUID of the VM named "Ubuntu". If you are logged into the control domain, pressing the <tab> key will perform auto-completion of UUIDs in subsequent XE commands, so you don't need to keep typing it in every time!
    • xe vm-param-set uuid=<uuid> HVM-boot-policy= : this will clear the HVM boot mode from the VM.
    • xe vm-param-set uuid=<uuid> PV-bootloader=pygrub : this will switch the VM to using to the pygrub bootloader which starts the guest in PV mode by examining its filesystem for kernel.
    • vm vm-param-set uuid=<uuid> PV-args="console=tty0 xencons=tty" : this configures the kernel boot arguments to display the login console on the correct TTY, so that it shows up in the XenCenter console.
  • Next, you need to flag the root disk of the VM as bootable so that pygrub knows where to look for the PV kernel:
    • xe vm-disk-list uuid=<uuid> and look for the UUID of the VBD for the disk. VBD stands for "Virtual Block Device" and represents how to map the virtual disk into the virtual machine.
    • xe vbd-param-set uuid=<vbd uuid> bootable=true will set the root disk VBD to be bootable.

• You should be all set now! If you boot up the Ubuntu VM, it should start up in text-mode with the high-speed PV kernel. If it doesn't work due to an incorrect grub configuration, you can use the xe-edit-bootloader script in the XenServer control domain to edit the grub.conf until it works.

• The next step is to install the XenServer tools within the guest, so that metrics such as the network interface IP addresses are recorded and reported from XenCenter. To do this:

  • Due to a portability issues with the default shell in Ubuntu (dash), you will need to replace it by: sudo apt-get -y install bash && sudo dpkg-reconfigure dash. We've actually fixed this issue in future releases of XenServer, but for XenServer 4.1 you will need to use bash.
  • Attach the XenServer Tools ISO into the VM, and mount it into the guest with sudo mount /dev/xvdd /mnt
  • Install the tools with sudo dpkg -i /mnt/Linux/xe-guest-utilities_4.1.0-257_i386.deb.
  • The warnings about the VM being unoptimized should disappear, and additional information such as the IP address of the guest should appear in XenCenter.

• In order to access the Ubuntu installation via the graphical console, you need to configure it to run VNC on the external network interface. XenCenter polls the guest to see if it is listening on the VNC port 5900, and offers the option to switch to the graphical console if it finds it. I followed the excellent instructions on this forum post. To summarise them:

  • sudo apt-get install vnc4server xinetd : to install the required packages

  • Edit /etc/gdm/gdm.conf and uncomment the RemoteGreeter=/usr/lib/gdm/gdmlogin line, set the key Enable=true in the [xdcmp] section.

  • Install a new service file for xinetd into /etc/xinetd.d/Xvnc with the following contents:

    service Xvnc
    {
      type = UNLISTED
      disable = no
      socket_type = stream
      protocol = tcp
      wait = no
      user = nobody
      server = /usr/bin/Xvnc
      server_args = -inetd -query localhost -geometry 1024x768  -depth 16 -cc 3 -once -SecurityTypes=none -extension XFIXES
      port = 5900
    }
    

  • The major difference from the forum poster is to run it on port 5900, and not to restrict it to just localhost (since XenCenter also needs to connect to it).

  • Finally, restart the xinetd service by running sudo /etc/init.d/xinetd restart.

Once you're done with this installation, you can shut down the VM and convert it to a template. Any exports or clones will continue to run in PV mode, since the XenServer XVA export format records all of the metadata required to re-create the VM records.

Enjoy the Ubuntu on XenServer experience! Remember to report any issues you have with the in-guest packages on the Ubuntu support forums, or just give them positive feedback.

PS: many thanks to Andrew Peace and Ian Campbell for assistance. May their Linux beards remain long and uncut.