home · projects · papers · blog · gallery · contact
anil madhavapeddy // anil.recoil.org

Test your OCaml packages in minutes using Travis CI

30 September 2013   |   Anil Madhavapeddy   |   tags: ocaml, ocamllabs, and ocamlot   |   all posts

A few months ago, Mike Lin posted instructions of how to bootstrap an OCaml testing environment within the Travis continuous integration tool. I finally got around to integrating his prototype scripts properly using the latest OCaml and OPAM versions during my travels last week to ICFP. It’s been an extraordinarily quick and pleasant experience to add the (free!) Travis test runs to my OCaml programs on GitHub, so here’s how you can do it too. Dave Scott and I have used this for about 15 of our own projects already, and I’m switching the whole of the Mirage repo infrastructure over to it this week.

(edit: I’ve done a followup post about integrating Travis with SSH to make secure deployments easier.)

Getting started

Getting my first Travis build working with one of my OCaml projects took about 2 minutes in total:

First, log into Travis and sign in via Twitter. Click on the Accounts button on the top-right and you should see a list of the all the GitHub repositories that you have access to. Just click the On switch for the one you want to start testing. Nothing will actually happen until the next code push or pull request goes to that repository. Behind the scenes, the On button that you clicked use the GitHub APIs to turn on the Travis post-commit hook for your repository.

Create a .travis.yml file in your main repository (see below or this gist). Travis doesn’t have native support for OCaml, but it isn’t really needed since we can just use C and write our own shell scripts. The env variables define a matrix of the different combinations of OCaml and OPAM that we want to test. Just remove variations that you don’t care about to avoid wasting Travis’ CPU time (open source projects are supported on a fair-use basis by them). Here’s the .travis.yml that I used for my ocaml-uri library:

language: c
script: bash -ex .travis-ci.sh
env:
  - OCAML_VERSION=4.01.0 OPAM_VERSION=1.0.0
  - OCAML_VERSION=4.01.0 OPAM_VERSION=1.1.0
  - OCAML_VERSION=4.00.1 OPAM_VERSION=1.0.0
  - OCAML_VERSION=4.00.1 OPAM_VERSION=1.1.0
  - OCAML_VERSION=3.12.1 OPAM_VERSION=1.0.0
  - OCAML_VERSION=3.12.1 OPAM_VERSION=1.1.0

Now you just need the .travis-ci.sh shell to run the actual tests. Travis provides an Ubuntu Precise/i386 VM that is destroyed after every test run, so we need to initialize it with the OCaml and OPAM binary packages. Since you often want to test different versions of all of these, I created a series of stable Ubuntu PPAs that have OCaml 3.12,4.0,4.1 and OPAM 1.0 and the (currently beta) 1.1 package manager. You can find them all on my Launchpad page, but the below script takes care of it all for you.

# Edit this for your own project dependencies
OPAM_DEPENDS="ocamlfind ounit re"
	 
case "$OCAML_VERSION,$OPAM_VERSION" in
3.12.1,1.0.0) ppa=avsm/ocaml312+opam10 ;;
3.12.1,1.1.0) ppa=avsm/ocaml312+opam11 ;;
4.00.1,1.0.0) ppa=avsm/ocaml40+opam10 ;;
4.00.1,1.1.0) ppa=avsm/ocaml40+opam11 ;;
4.01.0,1.0.0) ppa=avsm/ocaml41+opam10 ;;
4.01.0,1.1.0) ppa=avsm/ocaml41+opam11 ;;
*) echo Unknown $OCAML_VERSION,$OPAM_VERSION; exit 1 ;;
esac
	 
echo "yes" | sudo add-apt-repository ppa:$ppa
sudo apt-get update -qq
sudo apt-get install -qq ocaml ocaml-native-compilers camlp4-extra opam
export OPAMYES=1
opam init 
opam install ${OPAM_DEPENDS}
eval `opam config env`
make
make test

Now just do a push to your repository (a commit adding the Travis files above will do), and you will soon see the Travis web interface update. For example, here’s the output of ocaml-uri that the above example files are for. Of course, you should tweak the scripts to run the tests that your own project needs. Let me know if you make any useful modifications too, by forking the gist or e-mailing me.

Testing pull requests in OPAM

Travis isn’t just for code pushes though; as of a few months ago it can also test pull requests. This is an incredibly useful feature for complex projects such as the OPAM repository that has lots of contributors. You don’t need to do anything special to activate it: whenever someone issues a pull request, Travis will merge it locally and trigger the test runs just as if the code had been pushed directly.

I did do some special scripting to make this work with the OPAM package repository. Ideally, every new package or metadata change in OPAM will attempt to rebuild just that package set (rebuilding the entire repository would far exceed the 50 minute testing budget that Travis imposes). I put together a very hacked up shell script that greps the incoming diff and rebuilds the subset of packages. This is now live, and you can see both successful and failed pull requests (once the request has been merged, there’s a tiny little green arrow beside the commit that was tested).

This is a very unconservative estimate test matrix since a package update will also affect the reverse transitive cone of packages that depend on it, but it does catch several common typos and incompatibilities (for example, packages that use OPAM 1.1-only features by mistake). The longer term plan is to use the OCamlot command line tool to parse the pull request and compute an exact set of packages that need rebuilding.

Deployment

Travis has one last very cool feature up its sleeve. When a project has successfully built, it can run a scripting hook in the VM, which can be used to trigger a further code push or service update. If the service requires authentication, you can encrypt a secret in the travis.yml using their public key, and it will be available in the VM as an environment variable (but won’t be useful to anyone else looking at the code repository).

There are quite a few uses for these Travis deployment scripts: automating the rebuilding of the ocaml.org website infrastructure, rebuilding the central opam-doc cmt file archive, or even autoupdating Mirage microkernels to Amazon EC2 and Rackspace.

So how does this tie into the ongoing work on OCamlot? Quite simply, it’s saved us a ton of frontend work, and lets us focus on the more interesting OCaml-specific problems. Travis is also somewhat reactive (since it only runs in response to pushes or pull requests), and we still need to be able to run complete repository sweeps to look for more systematic breakage. It also doesn’t support any non-Ubuntu operating systems yet. However, Travis takes the burden off us for handling the steadily increasing number of package updates, and can be used to launch further OCamlot jobs on the other architectures and distributions. All in all, I’m very grateful to Mike for taking the trouble to blog about it back in March!

And a little bit of cheeky ARM hackery

I couldn’t resist one final little hack to see if I could actually do some ARM/OCaml testing using the Travis infrastructure. I adapted Ian Campbell’s excellent guide to ARM cross-chroots, and ended up with a rough script that builds an unattended chroot and can run OCaml/OPAM installations there too.

This isn’t something I’m quite comfortable running on all of my repositories just yet though, since an OCaml cross-build using qemu took over 5 hours and was quite rightly terminated when running within Travis. To mitigate this, I’ve built a custom apt-repository for ARMel packages to install a binary toolchain, and you can see the results of Lwt/ARM building successfully. I’ll update on this as I get it working on more packages, as I’m quite interested in getting a solid Xen/ARM toolstack up and running for another exciting project over in OCaml Labs…

 
blog comments powered by Disqus