home · projects · papers · blog · gallery · contact
anil madhavapeddy // anil.recoil.org

Talks from OCaml Labs during ICFP 2014

31 August 2014   |   Anil Madhavapeddy   |   tags: ocaml,ocamllabs   |   all posts

It’s the ever-exciting week of the International Conference on Functional Programming again in Sweden, and this time OCaml Labs has a variety of talks, tutorials and keynotes to deliver throughout the week. This post summarises all them so you can navigate your way to the right session. Remember that once you register for a particular day at ICFP, you can move between workshops and tutorials as you please.

Quick links to the below in date order:

Language and Compiler Improvements

The first round of talks are about improvements to the core OCaml language and runtime.

» Modular implicits

Leo White and Frederic Bour have been taking inspiration from Scala implicits and Modular Type Classes by Dreyer et al, and will describe the design and implementation of a system for ad-hoc polymorphism in OCaml based on passing implicit module parameters to functions based on their module type.

This provides a concise way to write functions to print or manipulate values generically, while maintaining the ML spirit of explicit modularity. You can actually get get a taste of this new feature ahead of the talk, thanks to a new facility in OCaml: we can compile any OPAM switch directly into an interactive JavaScript notebook thanks to iocamljs by Andy Ray.

» Multicore OCaml

Currently, threading in OCaml is only supported by means of a global lock, allowing at most one thread to run OCaml code at any time. Stephen Dolan, Leo White and Anil Madhavapeddy have been building on the early design of a multicore OCaml runtime that they started in January, and now have a (early) prototype of a runtime design that is capable of shared memory parallelism.

» Type-level Module Aliases

Leo White has been working with Jacques Garrigue on adding support for module aliases into OCaml. This significantly improves the compilation speed and executable binary sizes when using large libraries such as Core/Async.

» Coeffects: A Calculus of Context-dependent Computation

Alan Mycroft has been working with Tomas Petricek and Dominic Orchard on defining a broader notion of context than just variables in scope. Tomas will be presenting a research paper on developing a generalized coeffect system with annotations indexed by a correct shape.

 Mirage OS 2.0

We released Mirage OS 2.0 in July, and there will be several talks diving into some of the new features you may have read on the blog.

» Unikernels Keynote at Haskell Symposium

Since MirageOS is a unikernel written entirely in OCaml, it makes perfect sense to describe it in detail to our friends over at the Haskell Symposium and reflect on some of the design implications between Haskell type-classes and OCaml functors and metaprogramming. Anil Madhavapeddy will be doing just that in a Friday morning keynote at the Haskell Symposium.

» Transport Layer Security in OCaml

Hannes Menhert and David Kaloper have been working hard on integrating a pure OCaml Transport Layer Security stack into Mirage OS. They’ll talk about the design principles underlying the library, and reflect on the next steps to build a TLS stack that we can rely on not to been more insecure than telnet.

Hannes will also continue his travels and deliver a couple of talks the week after ICFP on the same topic in Denmark, so you can still see it if you happen to miss this week’s presentation:

» Irmin: a Branch-consistent Distributed Library Database

Irmin is an OCaml library to persist and synchronize distributed data structures both on-disk and in-memory. It enables a style of programming very similar to the Git workflow, where distributed nodes fork, fetch, merge and push data between each other. The general idea is that you want every active node to get a local (partial) copy of a global database and always be very explicit about how and when data is shared and migrated.

This has been a big collaborative effort lead by Thomas Gazagnaire, and includes contributions from Amir Chaudhry, Anil Madhavapeddy, Richard Mortier, David Scott, David Sheets, Gregory Tsipenyuk, Jon Crowcroft. We’ll be demonstrating Irmin in action, so please come along if you’ve got any interesting applications you would like to talk to us about.

» Metaprogramming with ML modules in the MirageOS

Mirage OS lets the programmer build modular operating system components using a combination of OCaml functors and generative metaprogramming. This ensures portability across both Unix binaries and Xen unikernels, while preserving a usable developer workflow.

The core Mirage OS team of Anil Madhavapeddy, Thomas Gazagnaire, David Scott and Richard Mortier will be talking about the details of the functor combinators that make all this possible, and doing a live demonstration of it running on a tiny ARM board!

» CUFP OCaml Language Tutorial

Leo White and Jeremy Yallop (with much helpful assistance from Daniel Buenzli) will be giving a rather different OCaml tutorial from the usual fare: they are taking you on a journey of building a variant of the popular 2048 game in pure OCaml, and compiling it to JavaScript using the js_of_ocaml compiler. This is a very pragmatic introduction to using statically typed functional programming combined with efficient compilation to JavaScript.

In this tutorial, we will first introduce the basics of OCaml using an interactive environment running in a web browser, as well as a local install of OCaml using the OPAM package manager. We will also explore how to compile OCaml to JavaScript using the js_of_ocaml tool.

The tutorial is focused around writing the 2048 logic, which will then be compiled with js_of_ocaml and linked together with a frontend based on (a pre-release version of) Useri, React, Gg and Vg, thanks to Daniel Buenzli. There’ll also be appearances from OPAM, IOCaml, Qcheck and OUnit.

There will also be a limited supply of special edition OCaml-branded USB sticks for the first tutorial attendees, so get here early for your exclusive swag!

» The OCaml Platform

The group here has been working hard all summer to pull together an integrated demonstration of the new generation of OCaml tools being built around the increasingly popular OPAM package manager. Anil Madhavapeddy will demonstrate all of these pieces in the OCaml Workshop, with guest appearances of work from Amir Chaudhry, Daniel Buenzli, Jeremie Diminio, Thomas Gazagnaire, Louis Gesbert, Thomas Leonard, David Sheets, Mark Shinwell, Christophe Troestler, Leo White and Jeremy Yallop.

The OCaml Platform combines the OCaml compiler toolchain with a coherent set of tools for build, documentation, testing and IDE integration. The project is a collaborative effort across the OCaml community, tied together by the OCaml Labs group in Cambridge and with other major contributors.

» The 0install Binary Installation System

Thomas Leonard will also be delivering a separate talk about cross-platform binary installation via his 0install library, which works on a variety of platforms ranging from Windows, Linux and MacOS X. He recently rewrote it in OCaml from Python, and will be sharing his experiences on how this went as a new OCaml user, as well as deliver an introduction to 0install.

» Service and Socialising

Heidi Howard and Leonhard Markert are acting as student volunteers at this years ICFP, and assisting with videoing various workshops such as CUFP Tutorials, Haskell Symposium, the Workshop on Functional High-Performance Computing and the ML Family Workshop. Follow their live blogging on the Systems Research Group SysBlog and leave comments about any sessions you’d like to know more about!

Anil Madhavapeddy is the ICFP industrial relations chair and will be hosting an Industrial Reception on Thursday 4th September in the Museum of World Culture starting from 1830. There will be wine, food and some inspirational talks from the ICFP sponsors that not only make the conference possible, but provide an avenue for the academic work to make its way out into industry (grad students that are job hunting: this is where you get to chat to folk hiring FP talent).

This list hasn’t been exhaustive, and only covers the activities of my group in OCaml Labs and the Systems Research Group at Cambridge. There are numerous other talks from the Cambridge Computer Lab during the week, but the artistic highlight will be on Saturday evening following the CUFP talks: Sam Aaron will be doing a live musical performance sometime after 8pm at 3vaningen. Sounds like a perfect way to wind down after what’s gearing to up to be an intense ICFP 2014. I look forward to seeing old friends and making new ones in Gothenburg soon!


Grepping the source of every OCaml package in OPAM

08 April 2014   |   Anil Madhavapeddy   |   tags: opam,ocaml,ocamllabs   |   all posts

A regular question that comes up from OCaml developers is how to use OPAM as a hypothesis testing tool against the known corpus of OCaml source code. In other words: can we quickly and simply run grep over every source archive in OPAM? So that’s the topic of today’s 5 minute blog post:

git clone git://github.com/ocaml/opam-repository
cd opam-repository
opam-admin make
cd archives
for i in *.tar.gz; \
  do tar -zxOf $i | grep caml_stat_alloc_string; \
done

In this particular example we’re looking for instances of caml_stat_alloc_string, so just replace that with the regular expression of your choice. The opam-admin tool repacks upstream archives into a straightforward tarball, so you don’t need to worry about all the different archival formats that OPAM supports (such as git or Darcs). It just adds an archive directory to a normal opam-repository checkout, so you can reuse an existing checkout if you have one already.

$ cd opam-repository/archives
$ du -h
669M	.
$ ls | wc -l
   2092

Codio, the insanely slick web way to build Mirage unikernels from a browser

26 March 2014   |   Anil Madhavapeddy   |   tags: mirage,ocaml,ocamllabs   |   all posts

I noticed an offhand tweet from Phil Tomson about Codio adding OPAM support, and naturally had to take a quick look. I was really impressed by the whole process, and ended up building the Mirage Xen website unikernel directly from my web browser in less than a minute, including registration!

I notice Codio supports OCaml and opam on the server side now.

— phil tomson (@philtor) March 26, 2014
$ parts install opam
$ opam init -a
$ eval `opam config env`
$ opam install mirage-www -y
$ make MODE=xen

Then have a cup of coffee while the box builds, and you have a mir-www.xen, all from your web browser! Codio has a number of deployment options available too, so you should be able to hook up a Git-based workflow using some combination of Travis or other CI service.

This is the first time I’ve ever been impressed by an online editor, and might consider moving away from my beloved vi…


Easily OPAM switching to any OCaml feature request

25 March 2014   |   Anil Madhavapeddy   |   tags: ocaml,ocamllabs   |   all posts

Gabriel Scherer announced an experiment to host OCaml compiler pull requests on GitHub for six months. There is a general feeling that GitHub would be a more modern hosting platform than the venerable but reliable Mantis setup that has in place for over a decade, but the only way to find out for sure is by trying it out for a while.

One of the great benefits of using GitHub is their excellent API to easily automate workflows around issues and pull requests. After a suggestion from Jeremy Yallop and David Sheets over lunch, I decided to use this to make it easier to locally apply compiler patches. OPAM has a great compiler switch feature that lets you run simultaneous OCaml installations and swap between them easily. For instance, the default setting gives you access to:

$ opam switch
system  C system       System compiler (4.01.0)
--     -- 3.11.2       Official 3.11.2 release
--     -- 3.12.1       Official 3.12.1 release
--     -- 4.00.0       Official 4.00.0 release
--     -- 4.00.1       Official 4.00.1 release
--     -- 4.01.0       Official 4.01.0 release
--     -- 4.01.0beta1  Beta1 release of 4.01.0

I used my GitHub API bindings to knock up a script that converts every GitHub pull request into a custom compiler switch. You can see these by passing the --all option to opam switch, as follows:

$ opam switch --all
--     -- 4.02.0dev+pr10              Add String.{split,rsplit}
--     -- 4.02.0dev+pr13              Add String.{cut,rcut}.
--     -- 4.02.0dev+pr14              Add absolute directory names to bytecode format for ocamldebug to use
--     -- 4.02.0dev+pr15              replace String.blit by String.unsafe_blit
--     -- 4.02.0dev+pr17              Cmm arithmetic optimisations
--     -- 4.02.0dev+pr18              Patch for issue 5584
--     -- 4.02.0dev+pr2               Parse -.x**2. (unary -.) as -.(x**2.).  Fix PR#3414
--     -- 4.02.0dev+pr20              OCamlbuild: Fix the check of ocamlfind
--     -- 4.02.0dev+pr3               Extend record punning to allow destructuring.
--     -- 4.02.0dev+pr4               Fix for PR#4832 (Filling bigarrays may block out runtime)
--     -- 4.02.0dev+pr6               Warn user when a type variable in a type constraint has been instantiated.
--     -- 4.02.0dev+pr7               Extend ocamllex with actions before refilling
--     -- 4.02.0dev+pr8               Adds a .gitignore to ignore all generated files during `make world.opt'
--     -- 4.02.0dev+pr9               FreeBSD 10 uses clang by default, with gcc not available by default
--     -- 4.02.0dev+trunk             latest trunk snapshot

Testing the impact of a particular compiler switch is now pretty straightforward. If you want to play with Stephen Dolan’s optimized arithmetic operations, for instance, you just need to do:

$ opam switch 4.02.0dev+pr17
$ eval `opam config env`

And your local environment now points to the patched OCaml compiler. For the curious, the scripts to generate the OPAM pull requests are in my avsm/opam-sync-github-prs repository. It contains an example of how to query active pull requests, and also to create a new cross-repository pull request (using the git jar binary from my GitHub bindings). The scripts run daily for now, and delete switches once the corresponding pull request is closed. Just run opam update to retrieve the latest switch set from the upstream OPAM package repository.


ICFP 2014 - a call for sponsorship and how you can help

03 March 2014   |   Anil Madhavapeddy   |   tags: icfp   |   all posts

The call for papers for this year’s International Conference on Functional Programming has just closed, with around a hundred cutting edge research papers submitted on the theory, application and experiences behind functional programming. This marks just the beginning of sorting out the program, as there are also over 10 big affiliated workshops that run throughout the week on topics ranging from specific languages (Erlang, Haskell, OCaml), the broader commercial community, and even art and music.

The ICFP conference experience can be a remarkable one for students. Some great ideas have emerged from random corridor conversations between talks with the likes of Phil Wadler, or from rain-soaked discussions with Simon PJ at Mikeller, or in my case, from being convinced to write a book while in a smoky Tokyo bar.

Functional programming worldwide has been growing ever more popular in 2014 (and lucrative). We’re committed to growing the ICFP community, not just in numbers but also in diversity. We had a record number of sponsors in 2013, and sustaining the growth means that we need to reach ever wider to support the activities of the (not-for-profit) conference.

So as this year’s industrial relations chair, I thought I’d throw the gates open and invite any organization that wishes to support FP to get in touch with us (e-mail at avsm2@cl.cam.ac.uk) and sponsor us. I’ve put an abridged version of the e-mail solicitation below that describes the benefits. Sponsorship can start as low as $500 and is often tax deductible in many countries.

I’m writing to ask if you would be willing to provide corporate financial support for the 19th ACM SIGPLAN International Conference on Functional Programming (ICFP), which takes place in Gothenburg, Sweden, from September 1st through 3rd, 2014:

http://icfpconference.org/icfp2014/

Corporate support funds are primarily used to subsidize students – the lifeblood of our community – and in turn serve to raise the community profile of the supporting companies through a high-profile industrial recruitment event.

Last year, unprecedented levels of support from you and folks like you at over 25 companies and institutions made it possible for students from all over the world to attend ICFP 2013 in Boston. The Industrial Reception, open to all attendees, was by all accounts a roaring success. All 2013 sponsoring companies had the opportunity to speak to the gathered students, academics, and software professionals.

This year, let’s build on that success and continue to grow our community, and bring even more students to ICFP 2014 in Sweden!

Your generosity will make it possible for students from all over the world to attend ICFP, the premier conference in functional programming. There, they will meet luminaries in the field, as well as people who’ve built a successful career and/or business on functional programming. They will return home inspired to continue pursuing functional programming in the confidence that exciting future careers await them.

This year, we’re continuing similar system of levels of financial support as last year. Our goal is to enable smaller companies to contribute while allowing larger companies to be as generous as they wish (with additional benefits, in recognition of that generosity).

The support levels, and their associated benefits and pledge amounts and benefits are as follows (costs in US dollars).

Bronze: $500: Logo on website, poster at industrial reception, listed in proceedings.

Silver: $2500: As above plus: logo in proceedings, logo on publicity materials (e.g., posters, etc.)

Gold: $5000: As above plus: named supporter of industrial reception, opportunity to include branded merchandise in participants’ swag bag.

Platinum: $10000: As above plus: named supporter of whole event, logo on lanyards, badge ribbon, table/booth-like space available (in coffee break areas), other logo on lanyards, badge ribbon, table/booth-like space available (in coffee break areas), other negotiated benefits (subject to ACM restrictions on commercial involvement).

If you are interested, please get in touch with me or any of the organizing committee. If you’re interested in helping out ICFP in a non-financial capacity (for example as a student volunteer), then there will also be plenty of opportunity to sign up later in the year.


Unikernels, and the Rise of the Virtual Library Operating System

13 January 2014   |   Anil Madhavapeddy   |   tags: ocaml,ocamllabs.mirage   |   all posts

The Communications of the ACM have just published an article that Dave Scott and I wrote providing a broader background on the concept of Unikernels that we’ve been working on since about 2003, when we started building Melange and the Xen toolstack. You can read either the print article (requires an ACM subscription) or the open access version on the ACM Queue. There’s been some interesting discussion about it already online:

Two of the most interesting bits of feedback for me personally came from Butler Lampson (via Jon Crowcroft) and Robert Harper, two computer scientists who have made key contributions to operating systems and programming languages and provided some broader perspective.

Butler Lampson points out (edited for the web):

I found the Mirage work quite interesting: a 21st century version of things that we did at Xerox in the 1970s. Of course the application domain is quite different, and so is the whole-program optimization. And we couldn’t afford garbage collection, so freeing storage was not type-safe. But there are lots of interesting parallels.

The “OS as libraries” idea was what made it possible to fit big applications into the Alto’s 128k bytes of memory:

Lampson and Sproull, An open operating system for a single-user machine, ACM Operating Systems Rev. 11, 5 (Dec. 1979), pp 98-105. ACM.

The use of strong type-checking and interfaces for an OS was pioneered in [Mesa](http://en.wikipedia.org/wiki/Mesa(programming_language%29) and [Pilot](http://en.wikipedia.org/wiki/Pilot(operating_system%29):

Lauer and Satterthwaite, The impact of Mesa on system design, Proc. 4th ICSE, Munich, Sep. 1979, pp 174-182.

Redell et al, Pilot: An Operating System for a Personal Computer, Comm. ACM 23, 2 (Feb 1980), pp 81-92 (from 7th SOSP, 1979). ACM

Robert Harper correctly points out some related work that was missing from our CACM article:

Both Ensemble and FoxNet made strong echoes throughout the design of Mirage (and its precursor software such as Melange in 2007). The Mirage command-line tool uses staged-computation to build a concrete application out of functors, and we are making this even more programmable via a new combinator-based functor types library that Thomas Gazagnaire built, and also experimenting with higher kinded polymorphic abstractions.

My thanks to Butler Lampson and Robert Harper for making me go re-read their papers again, and I’d like to leave you with Malte Schwarzkopf’s OS Reading Group papers for other essential reading in this space. Many more citations immediately relevant to Mirage can also be found in our ASPLOS 2013 paper.



all posts