Anil Madhavapeddy

It's time to go post-POSIX at ICFP/SPLASH 2025 / Oct 2025 / DOI

This is part 4 of 5 of a series of posts about ICFP 2025.

After the excitement of presenting my Docker experience report, I went straight into giving a keynote talk at VMIL 2025. This talk bubbled up intrusive thoughts I've had resulting in the past 25 years: every system I've worked on, ranging from Xen to Docker on all seem to boil down to "make shared memory go fast".

I'd started to believe it was time for change in the way we approach IO about 12 years ago when I talked about wierd IO behaviour to a packed audience at FOSDEM, and now I believe it's even more true in 2025.

So I made one key argument to the audience: it's time to accept that standards such as POSIX are now holding back the development of good language runtimes, and we need to embrace the diversity of highly concurrent, shared-memory interfaces. And unfortunately, there's no portable subset amongst these, and so this may require a rethink of our frontend language interfaces as well.

After explaining io_uring on Linux, and then the Windows and macOS variants, I showed how we're trying to support these in our OCaml 5 Eio library. While Eio has plenty of really cool features, the defining one for me is that it abstracts away IO operations (like Flow.copy) sufficiently that the backend can do highly parallel dispatch to the kernel over shared memory interfaces like uring.

Crucially, we treat the shared memory interface in Eio as the first-class citizen, with the POSIX(ish) backends relegated to a compatibility role. This way, future interfaces to the programmer can be "parallel first".

POSIX shouldn't be relegated just yet

The discussion with the audience after the talk was just fantastic. Stephen Kell, who has thought deeply about the interaction between the kernel and userspace asked whether I was being too harsh on POSIX, which has served us faithfully for many years. And indeed, I agree with Stephen! POSIX gives us a fine boot layer, and a fine interaction layer (for terminals anyway), and a great single threaded interface. Where it falls over is the highly concurrent and parallel world of high performance computing, where must align our data paths and not have any interference from third party code.

So perhaps we need to restructure runtimes to explicitly have a "boot phase" (POSIX) where they are establishing their resources, and then switch into a "steady phase" (uring and friends) where they are blasting data at high speeds. These are really quite distinct modes of operation, and both are useful. Note that "high speed" here applies to embedded systems as well; if I do less work on those systems in the CPU, then I'll get better energy usage, so low-overhead mechanisms like uring are useful there too.

One other thought I had from Stephen's excellent question was the important role of POSIX in portability for many years. Moving forward into the next decade, what standards body will help developers write portable software for all these different IO interfaces? It seems inevitable that this will fragment into per-language specifications instead of the operating system (or C-level) interfaces we've had for the past 50 years.

Building more uring-based low-level examples

I was also slightly surprised by how few people had used io_uring, in a room full of language VM developers. One of the most useful things I did when developing the OCaml uring bindings was to build a parallel file cp, but I never got around to any real networking code.

So I've started to build a "raw" OCaml 5 + uring HTTP server that is completely non-portable, but serves to show how it works at the lowest level. This should also give us a nice benchmark against which to test higher level interfaces. My plan is to make this work with OCaml 5 first, and subsequently add OxCaml support. Sadiq Jaffer pointed me to the magic caml_alloc_local FFI function added in OxCaml that allows allocation directly into the OCaml stack from C, which should be all I need to make the shared memory interface never allocate into the heap.

Patrick Ferris also spent some ICFP time hacking on integrating OxCaml into the OCaml uring bindings:

The idea being that a completion queue entry (a notification that some operation has completed) could be fully represented using 64 bits (two 32-bit, unboxed values). -- Patrick at ICFP 2025

Having had my regular morning coffee with Simon Peyton Jones back in Cambridge, he then pointed out that Haskell also has a uring backend PR lingering for years, and so perhaps we should do the same exercise there too, to understnad it all! I can't say no to Simon, but if a Haskell expert is interested please do get in touch so I don't have to inflict my OCaml-style Haskell on the world...

3rd Nov 2025: Add link to Patrick's uring bindings.