Jane Street and Docker on moving to OCaml 5 at ICFP/SPLASH 2025 / Oct 2025

(This is part 3 of my ICFP25 series: see also 1. Chairing PROPL25, 2. OxCaml tutorial, 4. post-POSIX IO, 5. What I learnt)

It's been about six years since we wrote the papers on parallelism and effects, and four years since we helped to release upstream OCaml 5.0 with multicore support, a mammoth effort that took up years of work for my OCaml Labs and Tarides crew. After the release came out, I focussed on building applications using OCaml 5 for my own work on planetary computing, for example on using the new features with the fledgling Eio library to get some experience with direct-style OCaml programming.

Meanwhile, big OCaml users have also been adapting their codebases to shift from OCaml 4 to 5. Jane Street have expanded their tools and compiler team and driven through their production switch to the multicore runtime, and Docker for Desktop is progressing with their switch to direct-style code via Eio for hundreds of millions of users! Read on to learn more...

The Path to OCaml 5 in Production at Jane Street

Although it was the last talk of the entire conference week, I'm discussing this first as it was the most exciting thing I learnt at ICFP! At the REBASE ^[1] workshop, Yaron Minsky announced that Jane Street's production servers are now running on the OCaml 5 runtime!

That was the good news (our runtime is trading trillions of dollars, wow). The bad news is that there was a bumpy road internally for Jane Street to go from the version that was first released (OCaml 5.0 on Dec 2022) to their current tree. Ron gave a really good roundup of some of the effort that went into release engineering their internal rollout. Since the entire OCaml runtime was rewritten as part of the multicore runtime upgrade, Jane Street encountered GC pacing issues and other unexpected changes in resource usage as a result of the new runtime behaviours. This took some significant design and engineering effort and to fix, which Ron covers in detail in his talk.

Diagnosing the OCaml 5 performance bumps

So what was the root cause of this bumpiness? I think a lot of it was just normal release engineering; we clearly signalled when releasing OCaml 5.0 that it was not yet feature complete, and that the 4.x runtime would continue to be supported for some years.

The developer team released OCaml 5.0.0 in December 2022. OCaml 5.x features a full rewrite of its runtime system for shared-memory parallel programming using domains and native support for concurrent programming using effect handlers.

Owing to the large number of changes, especially to the garbage collector, OCaml 4.14 (the final release in the OCaml 4.x series, originally released in March 2022) remains supported for the time being. Maintainers of existing codebases are strongly encouraged to evaluate OCaml 5.x and to report any performance degradations on our issue tracker. -- ocaml/ocaml README

The latest OCaml 5.4.0 release that came out just a couple of weeks ago is the first release with full feature parity with the OCaml 4.x LTS branch. Features such as statmemprof, the MSVC port, GC compaction, thread sanitizer, RISC-V and S390X architecture support all had to be engineered back in. OCaml development in recent years has been very developer intensive because of the need to not only reintroduce these features, and also keep up with the torrent of new features being introduced to OCaml 5.x. All in all, I think it's been a very successful few years for the language to have kept steadily improving!

Should we maintain multiple language runtimes in OCaml?

However, Ron's talk was an excellent chance for us to reflect on what we might have done differently with the benefit of hindsight. David Allsopp posits that we should have maintained the OCaml 4 and 5 runtimes simultaneously:

One of the early ideas was to merge just the runtime changes as a separate runtime, leaving all the language changes to a subsequent update. The main thing here would have been to upstream the immense changes to the allocator and garbage collector along with the domains and fibers machinery, while not yet exposing it.

I remember the concern being that having essentially a runtime variant (not unlike the debug runtime) might lead to very slow uptake at actually testing it and possibly a maintenance burden. i.e. we were concerned at maintaining two runtimes. This would probably have resulted in something like OCaml 4.15.0, with an experimental official multicore-aware runtime. -- Reflections on ICFP 2025, David Allsopp

I agree with this. Although we weren't sure in 2021 that it would be possible to have two simultaneous runtimes^[2] it was clear by the time we were engineering the 5.0 PR that this would be possible. Still, a few years to stabilise multicore performance vs the decade-old 4.x runtime isn't bad going at all, so I don't have deep regrets about the approach we did take!

We need continuous continuous performance engineering

The other area where all of us agreed more effort is necessary is on continuous performance engineering. In the runup to 2022, Tom Kelly, KC Sivaramakrishnan and I set up Sandmark, which was a body of microbenchmarks and some macrobenchmarks. Tarides setup a continuous benchmarking service around this, and I hosted a bunch of carefully tuned machines with specific BIOS settings in the Cambrige Computer Lab.

Roll on six years, and the maintainence cost of this service becomes clear. It's quite a bit of effort to maintain the old machines (one is running Ubuntu 16.04! I'm not telling you which one) to keep consistency in previous results. New machines all come with a proportional configuration effort, and their results have to be interpreted. Operating systems have to be upgraded, and tuned afresh. Continuous benchmarking is itself a continuous process of engineering, and should be treated as such!

Our discussions at ICFP centred around the idea that we should not only maintain this benchmarking infrastructure, but add an incentive to macro projects to submit representative tests of their performance. Rocq, Why3, Semgrep, or Frama-C, for example, should all have test cases run within this infrastructure that move beyond microbenchmarks to realistic performance patterns that will show up issues with GC pacing in a way that microbenchmarks do not.

The challenge with doing this so far has been the difficulty of getting many of these big projects to compile on a random OCaml trunk snapshot (essential to test them against a pre-release OCaml compiler). Solving this will take some thought (particularly around ppx usage), but the effort seems worthwhile as we move into a new phase of OCaml 5 engineering now that feature parity has been achieved. Stay tuned for more on this from Tarides!

Functional Networking at Docker

I also had the opportunity to share both a retrospective on the work Dave Scott and I have been doing on Docker Desktop for some years, and also the efforts from new OCamlers like Patrick Ferris and Ryan Gibb on helping us to port some aging OCaml code over to OCaml 5.

We got a paper accepted to ICFP on the topic, and so I had a lot of fun presenting "Functional Networking for Millions of Docker Desktops" to the mainline ICFP audience! I first discussed the past; how we joined Docker and came up with HyperKit and VPNKit to solve scaling problems that Docker was facing early in its growth.

Our experience report makes the broad case for library operating sytems and functional programming being a good fit, especially with strict languages like OCaml which offer thin interfaces to the OS interfaces.

Our use of library-oriented programming to deliver Docker for Desktop is [...] a very useful way to build the "invisible systems glue" code that is pervasively needed in many systems programming tasks.

There are an ever-growing number of hardware and software interfaces to access the outside world, most obviously with GPUs for machine learning workloads but also FPGAs and new storage and persistent memory devices. These usually require significant retrofitting to work with existing codebases, and so building translation adapters like VPNkit and using library VMMs like Hyperkit will become more common in the future. -- Functional Networking for Millions of Docker Desktops, 2025

And looking into recent specifics, Patrick Ferris's contributions to VPNKit also allow this codebase to move to OCaml 5, and take advantage of direct-style IO! In a nutshell, it lets old code like this:

module Make_packet_proxy
(I: Mirage_flow.S) (O: Mirage_flow.S) = struct
 let run incoming outgoing =
  let rec loop () =
   I.read incoming >>= function
   | Error err -> fail "%a" I.pp_error err
   | Ok `Eof -> Lwt.return_unit
   | Ok (`Data buf) -> begin
      O.write outgoing buf >>= function
      | Ok () -> loop ()
      | Error err -> fail "%a" O.pp_error err
     end
  in loop ()

...migrate to direct-style code Eio like this:

module Proxy = struct
 let run incoming outgoing =
  try
    while true do
      Eio.Flow.copy incoming outgoing
    done
  with
  | End_of_file -> ()
  | Write_error err ->
      fail "%a" pp_write_error err
  | Read_error err ->
      fail "%a" pp_read_error err
end

The old code^[3] had monadic concurrency, functors for parameterising the IO drivers, and error handling duplicated across OCaml exceptions and the concurrency monad. The new code uses direct control flow constructs like while, and also one form of error handling. This is all still a work-in-progress, but looking a solid approach with no blockers except hacking time to get it merged.

Read the ICFP paper to learn more about this, or browse my slides! It's exciting to see production code here get simpler and faster as we move to OCaml 5. The reasons why this happens are explored further in my VMIL keynote talk that I gave the next day, where I make a case for runtimes focussing on post-POSIX IO! And beyond that, we have OxCaml waiting in the wings for even more performance gains. OCaml is living in exciting times.

(This is part 3 of my ICFP25 series: see also 1. Chairing PROPL25, 2. OxCaml tutorial, 4. post-POSIX IO, 5. What I learnt)

1. As far as I can tell, REBASE is SPLASH's equivalent of the venerable CUFP series that I helped run earlier in the century.

   ↩︎︎

2. Our 2020 parallelism paper proposed two minor GC strategies, one of which broke the C FFI and so didn't make the cut in the end due to the amount of ecosystem churn it would cause.

   ↩︎︎

3. A fun historical note is that I gave one of the first talks about VPNKit in Jane Street London about a decade ago! Back then we had a deep discussion about whether to use Lwt or Async, and it looks like we'll now meet again via OxCaml.

   ↩︎︎