I thought I’d kick off my Citrix blog with a question I get pretty often from Linux enthusiasts: how to install unsupported Linux distributions on XenServer 4.1.
The most common solution people find is to use the “Other Install Media” template, insert the distribution installation CD, and find that the mouse cursor doesn’t work when they boot into X11. The reason for this is that they are using the hardware-assisted emulation mode of installing Linux. In this mode (dubbed “HVM”), all input and output is emulated, and in particular the mouse interface uses the USB tablet interface. If the distribution doesn’t include a driver for USB tablets, then no mouse will appear.
Windows guests run at high-speed in HVM mode due to the installation of the XenServer tools which install high-speed drivers, but these are not necessary for Linux distributions since they can be run in para-virtualized mode (dubbed “PV”). This involves obtaining a Xen-enabled PV kernel from the distribution, and modifying the VM record in XenServer to boot into this kernel instead of HVM mode. The XenServer built-in templates for popular distributions such as RHEL, CentOS or SUSE Linux already automate all this and are in PV mode from the installer onwards.
In the remainder of this post, I’ll explain how to take a distribution without direct support (Ubuntu 8.04), get it installed in HVM mode on XenServer 4.1, and convert it to PV mode with a XenCenter graphical console.
Download the "Alternative Installation CD". The main installation CD uses graphical mode, which won't install as well in HVM mode due to the use of esoteric 16-bit mode instructions for the graphics operations. The 16-bit emulation mechanisms vary between processors (with better support on AMD chips, and a software instruction emulator required on Intel VT chips). However, the Ubuntu alternate CD uses a text-based installer which works fine.
Create a new VM on the XenServer 4.1 host using the "Windows Server 2003" template. This template is set up with a sensible set of hardware emulation flags and default disks, and so is a good base for the HVM installation of Ubuntu as well. Attach the Ubuntu ISO you just downloaded to the VM, and proceed to install Ubuntu as normal. You should install it onto the first disk, to make the subsequent steps in this guide easier.
When the installation is finished, reboot the VM (don't forget to
detach the installation ISO first). It should boot up in HVM mode
into the graphical login screen. The XenCenter display will show it
as not being optimized, which is fine. At this stage, I prefer to
work via a remote command-line using SSH. Open up a Terminal from
Ubuntu, and run "sudo apt-get install openssh-server". Then find
out the VM's IP address with "ifconfig eth0", and then connect to
it remotely. Alternatively, you can continue to type in the commands
directly into the terminal as well.
On the Ubuntu guest, you now need to install the latest Xen version of the Ubuntu kernel:
sudo apt-get install linux-image-xen". This is a virtual
package which pulls in the latest Xen kernel and modules, in my
case 2.6.24.19.21.update-grub doesn't update the grub configuration
with your newly installed Xen kernel. To fix this:
Open /boot/grub/menu.lst in your favourite editor.
Scroll to the bottom to the kernel list, and find the entry which looks like:
title Ubuntu 8.04, kernel 2.6.24-16-generic
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-16-generic root=UUID=<uuid> ro quiet splash
initrd /boot/initrd.img-2.6.24-16-generic
quiet
Add a new entry which is similar to this, but change all
references to the 2.6.24-16-generic to the Xen kernel. In
/boot I have vmlinuz-2.6.24-19-xen, so my new entry
looks like:
title Ubuntu 8.04, kernel 2.6.24-19-xen
root (hd0,0)
kernel /boot/vmlinuz-2.6.24-19-xen root=UUID=<uuid> ro quiet splash
initrd /boot/initrd.img-2.6.24-19-xen
quiet
Also edit the default entry in the menu.lst to match the
number of the kernel you just added. I set mine to 3, since
it is the fourth entry in the list and the indexing starts
from 0.
When this is done, shut down the guest but do not reboot it just yet. You first need to edit the VM record for your Ubuntu VM to convert it to PV boot mode. From the control domain console of your XenServer:
xe CLI:
xe vm-list name-label=Ubuntu params=uuid --minimal : this
will print out the UUID of the VM named "Ubuntu". If you are
logged into the control domain, pressing the <tab> key
will perform auto-completion of UUIDs in subsequent XE
commands, so you don't need to keep typing it in every time!xe vm-param-set uuid=<uuid> HVM-boot-policy= : this will
clear the HVM boot mode from the VM.xe vm-param-set uuid=<uuid> PV-bootloader=pygrub : this
will switch the VM to using to the pygrub bootloader which
starts the guest in PV mode by examining its filesystem for
kernel.vm vm-param-set uuid=<uuid> PV-args="console=tty0 xencons=tty"
: this configures the kernel boot arguments to display the
login console on the correct TTY, so that it shows up in the
XenCenter console.xe vm-disk-list uuid=<uuid> and look for the UUID of the
VBD for the disk. VBD stands for "Virtual Block Device" and
represents how to map the virtual disk into the virtual
machine.xe vbd-param-set uuid=<vbd uuid> bootable=true will set
the root disk VBD to be bootable.You should be all set now! If you boot up the Ubuntu VM, it should
start up in text-mode with the high-speed PV kernel. If it doesn't
work due to an incorrect grub configuration, you can use the
xe-edit-bootloader script in the XenServer control domain to edit
the grub.conf until it works.
The next step is to install the XenServer tools within the guest, so that metrics such as the network interface IP addresses are recorded and reported from XenCenter. To do this:
sudo apt-get -y install bash && sudo dpkg-reconfigure dash.
We've actually fixed this issue in future releases of XenServer,
but for XenServer 4.1 you will need to use bash.sudo mount /dev/xvdd /mntsudo dpkg -i /mnt/Linux/xe-guest-utilities_4.1.0-257_i386.deb.In order to access the Ubuntu installation via the graphical console, you need to configure it to run VNC on the external network interface. XenCenter polls the guest to see if it is listening on the VNC port 5900, and offers the option to switch to the graphical console if it finds it. I followed the excellent instructions on this forum post. To summarise them:
sudo apt-get install vnc4server xinetd : to install the
required packages
Edit /etc/gdm/gdm.conf and uncomment the
RemoteGreeter=/usr/lib/gdm/gdmlogin line, set the key
Enable=true in the [xdcmp] section.
Install a new service file for xinetd into
/etc/xinetd.d/Xvnc with the following contents:
service Xvnc
{
type = UNLISTED
disable = no
socket_type = stream
protocol = tcp
wait = no
user = nobody
server = /usr/bin/Xvnc
server_args = -inetd -query localhost -geometry 1024x768 -depth 16 -cc 3 -once -SecurityTypes=none -extension XFIXES
port = 5900
}
The major difference from the forum poster is to run it on port 5900, and not to restrict it to just localhost (since XenCenter also needs to connect to it).
Finally, restart the xinetd service by running
sudo /etc/init.d/xinetd restart.
Once you're done with this installation, you can shut down the VM and convert it to a template. Any exports or clones will continue to run in PV mode, since the XenServer XVA export format records all of the metadata required to re-create the VM records.
Enjoy the Ubuntu on XenServer experience! Remember to report any issues you have with the in-guest packages on the Ubuntu support forums, or just give them positive feedback.
PS: many thanks to Andrew Peace and Ian Campbell for assistance. May their Linux beards remain long and uncut.