Checking out the latest features in OxCaml 5.4.0+ox9

Anil Madhavapeddy

doi:10.59350/r852a-fja33

This website runs on OxCaml, a set of performance-oriented extensions to the OCaml language developed at Jane Street. I last upgraded our monorepo back in the summer, and David Allsopp has just released a major OxCaml 5.4.0-ox9 release. This upgrade shifts the base version of the fork over to a newer release of OCaml (yay!), as well as many improvements to their extensions.

If you're new to OxCaml then you may want to start with the ICFP tutorial first, and there's a quick recap and installation instructions. The rest of the post goes through the changes that affected my code, grouped into reducing allocations, kinds, modes, effect handlers, the C interface and performance, followed by a peek at the emerging features behind extension flags and some advice on whether to dive in yourself.

This website's development dashboard, using the OCaml console library, running OxCaml 5.4.0+ox9

1 A recap of OxCaml: tis a speedy camel

For those arriving fresh, OxCaml is a fork of the OCaml compiler (oxcaml/oxcaml) and used in production at Jane Street. The extensions are designed to be conservative, so normal OCaml code mostly works, and are enabled when more control over allocation, memory layout, or data-race-free concurrency is needed.

I like it because, unlike Rust, we can benefit from the ergonomics of having a garbage collector by default, and selectively write high-performance code where necessary. The main extensions are:

Layouts and kinds, which let you declare unboxed types such as float#, int64_u and unboxed records r# that are stored flat with no pointer indirection, and polymorphic code that's explicit about which of these representations it accepts.
Modes, which annotate a value with dataflow properties. These include local for stack allocation, unique for values with a single mutator, and portable and contended for values that can cross between parallel domains safely.
Checked performance claims, such as [@zero_alloc] on a function to guarantee no allocations, along with SIMD vector operations and the Flambda 2 optimiser.
Parallelism, via the parallel and await libraries and the capsule abstraction, which depend on the mode system for their safety guarantees. We did a tutorial at ICFP about these last year.

OxCaml generally tracks upstream OCaml, rebasing irregularly and tagging numbered snapshots in between. The 5.2-based series has run for some years now (from minus25 to minus40), and the process of rebasing to the 5.4-based series has now reached ox9. While the compiler itself advances regularly, the supporting OxCaml libraries must also match or else we can't really take advantage of the new features.

These support libraries like Base, Core, ppx_jane etc, are distributed through an oxcaml opam overlay whose guard packages make sure that an OxCaml opam switch installs the right packages. This is a complex packaging endeavour, and that's why I'm excited that David Allsopp has merged a usable 5.4.0-ox9 release that we can all use in public that also includes the corresponding Jane Street libraries!

2 Installing OxCaml 5.4.0-ox9 yourself

The instructions on oxcaml.org still refer to 5.2.0+ox and will no doubt be updated soon, but I did the following to directly install the 5.4.0-ox9 libraries.

opam update --all
opam switch create 5.4.0-ox9 oxcaml-compiler.5.4.0-ox9 \
  --repos ox=git+https://github.com/oxcaml/opam-repository.git,default
eval $(opam env --switch 5.4.0-ox9)
opam install -y merlin ocaml-lsp-server ocamlformat utop core_unix parallel

The ox repository must be listed before default so that patched versions of tools such as merlin, ocamlformat and ppxlib take priority over the upstream ones. If you already have an older OxCaml switch, there's no upgrade path within it as the libraries for minus39 conflict with any other compiler release. It's easier just to create a fresh switch for the new 5.4.0 series.

I went through all of the changes while updating my Claude OCaml Marketplace skills for OxCaml 5.4.0-ox9 (changeset). What follows are interesting new features released in the past three months that I reviewed while also updating my monorepos to deploy the compiler. I've left out everything that arrived with upstream OCaml 5.3/5.4 (effect handler syntax, Stdlib.Iarray and so on), so look at the upstream release notes for those first if you're unfamiliar with the OCaml ecosystem.

3 Reducing allocations

The headline feature of OxCaml is to execute programs faster by doing less work on housekeeping (like allocating and deallocating memory) and more on useful compute.

3.1 Unboxed numbers and mixed records

OxCaml has several mechanisms to reduce boxing of integers and floats, which are extremely beneficial to improving data processing performance. The # suffix is used for the unboxed version of primitive types, such as float#, int8# and the unboxed version r# of a record r.

In addition to these unboxed variants of standard types, there are also custom types for 32-bit, 64-bit and native unboxed integers and a 32-bit float which used to also use the int64#, int32#, nativeint# and float32# syntax. These four are now standalone types called int64_u, int32_u, nativeint_u and float32_u (#6663), which is a breaking (but good) change.

type sample = { id : int64_u; weight : float#; tag : int8#; label : string }
let s = { id = #7L; weight = #0.5; tag = #3s; label = "a" }

type boxed = float# box                         (* equal to float *)

type m = { f : float; u : float# }              (* no [@@flatten_floats] *)
let unboxed () : m# = #{ f = 1.0; u = #2.0 }

type node = { next : #(node option * float#) }
let rec cycle = { next = #(Some cycle, #4.0) }  (* through an unboxed field *)

5.4.0-ox9 also makes unboxed types much more usable in general, as seen in the above snippets:

There was no way to name the boxed form of an unboxed type, but now the predefined 'a box exists (#5342).
A record mixing float and float# fields was rejected unless annotated [@@flatten_floats]. The attribute is now optional, and without it the record also generates an unboxed version (#6106).
A recursive value couldn't pass through an unboxed field ("not allowed as right-hand side of let rec"), which made cyclic structures built from flat records very difficult to express (about as awkward as they are in Rust!). This now works in 5.4.0-ox9 (#6847).
Unsigned types are now builtin, with uint8_u through to uint64_u with unsigned division and remainder primitives. I couldn't find literals or arithmetic operations yet though (#6534).

3.2 Non-allocating optional types

The builtin 'a or_null type is a non-allocating replacement for option, where This x is represented as x itself and Null as a special immediate value.

type 'a maybe = Nope | Yep of 'a [@@or_null]
type callback = Unset | Set of (unit -> unit) [@@or_null]
type flag = Off of unit# | On of string [@@or_null]

let run = function Unset -> () | Set f -> f ()

5.2.0-minus39 introduced [@@or_null] on user-defined variants, which allows an unboxed optional type. It required the payload to have exactly one type parameter, so only generic option-like types qualified. As shown above, the payload can now be any type in 5.4.0+ox9 (#5830), and the null constructor can also carry a void payload (#6383), and the resulting type has the same kind as the built-in or_null.

In practice this means an optional callback or an optional integer can have a descriptive type of its own without requiring an allocation due to the boxing. I've started using this quite a bit in my zero allocation httpz library that drives this site.

3.3 Atomic record fields

An OCaml record field can be marked [@atomic] (since OCaml 5.4) so that the field is accessed with atomic reads and writes. This allows it to be safely shared between parallel domains without being wrapped in a separate Atomic.t.

An [@atomic] field couldn't previously appear in a record that also held unboxed fields, which meant that (e.g.) a lock-free counter alongside a float# in a record had to be boxed in a separately allocated Atomic.t. Block indices also didn't support atomic fields either, but both now work (#6423, #6667).

This means that shared mutable state can be inline in a flat record, which allows for some tightly laid out concurrent-friendly datastructures.

One behaviour that's changed is that a functional update { c with … } used to copy all atomic fields with an ordinary read. This is now rejected unless the atomic field is given explicitly { c with the_atomic=... } (#6453).

open Stdlib_stable

type counter = { mutable hits : int [@atomic]; rate : float# }
let hits : (counter, int) idx_atomic = (.hits)

let bump (c : counter) = Idx_atomic.incr c hits
let swap (c : counter) = Idx_atomic.compare_and_set c hits 0 1

let reset (c : counter) = { c with hits = 0 }
(* { c with rate = #1.0 } is rejected: it would read [hits] implicitly *)

4 Kinds

OxCaml kinds describe how a type is represented in memory (e.g. a boxed pointer, or a flat float, etc) along with whether it can cross modes. Kinds allow polymorphic code to be more explicit about which representations work, instead of assuming everything is a pointer.

Roughly, the kind system adds an extra level to the type system: terms are classified by types, and types are classified by kinds. [...] Kinds capture properties of types along several different dimensions. For example, kinds can be used to identify which types have values that are passed in floating point registers, or are safely ignored by the garbage collector. -- OxCaml Kinds, Jan 2026

Kind annotations used to be accepted without any check that they were accurate. Writing ('a : any) on a parameter used as (e.g.) an 'a list compiled silently with the kind narrowed to what the body needed. The 5.4.0+ox9 compiler now warns by default when an annotation is weaker than the inferred kind (#5362) or when a modifier has no effect. This lets annotations be trusted as documentation, so some of my older code raised warnings when upgrading (but everything was easily fixed to the more specific kind annotation).

The remaining changes remove gaps in the compiler implementation.

[@@@implicit_kind] was ignored in structures; it now applies there as it does in signatures (#6114).
Modifiers on abstract kinds are now accepted (#5874).
The new modifier addressable marks kinds whose boxed form has interior pointers (#6712).
A constructor whose arguments are all void used to be accepted, but now needs [@immediate_all_void_constructor] (#6256). This is because the representation of such constructors is planned to change in a future release and this attribute will make the upgrade easier.

type ('a : any mod separable) cell = 'a array   (* [any] alone: warning 181 *)

module Buffer = struct
  [@@@implicit_kind: ('elt : bits64)]
  let singleton : 'elt -> 'elt array = fun x -> [| x |]
end

module type S = sig
  kind_ k
  val get : ('a : k separable). 'a array -> 'a
end

type handle : bits8 addressable
type t = A of unit# [@immediate_all_void_constructor] | B of int

5 Modes and stack allocation

A mode is an annotation on a value that tracks a dataflow property, like whether the value is allocated on the stack (local), has a single owner (unique) or can safely cross to another domain (portable). The OxCaml type checker enforces that these properties are used consistently.

The change in 5.4.0-ox9 that broke my code was a bugfix for a leak (#6396). Previously, if the last expression of a function was a call that returned a local value (e.g. fst p on a local pair), then the enclosing function was also made local-returning as well which never released the caller's stack frames. This is now an error unless the call is wrapped in exclave_, bound to a variable, or annotated [@nontail]. The fix itself was mechanical in the few places in my monorepo where the ox9 compiler rejected old code.

Implicit mode inference, by contrast, became more permissive and usable:

A mode on a parameter used to apply to every axis; e.g. (f @ stateful) made f nonportable. The implied axes are now inferred (#6745) using fancy type math.
A partial application is now constrained only by used parameters (#6966).
A coercion can mode-cross the result of a function (#6407).

Together these remove a number of mode annotations and eta-expansions that were previously quite verbose and also sometimes breaking existing OCaml code. A handy new default warning also reports modalities that have no effect (#5532).

let first (p : (int * int) @ local) = fst p [@nontail]  (* a tail call is rejected *)
let pair x y = exclave_ stack_ (x, y)

let needs_portable (f : (unit -> unit) @ portable) = ()
let h (f @ stateful) = needs_portable f     (* portability is inferred, not fixed *)

let g : int -> int = fun x -> x + 1
let g' = (g :> int -> int @ portable)       (* coercion mode-crosses the result *)

type r = { mutable n : int @@ global }      (* warning 220: redundant modality *)

6 Effect handlers

Effect handlers let a computation suspend itself and hand control to a handler further up the stack, allowing direct-style concurrency libraries such as Eio to schedule lightweight fibers without resorting to monadic code. The effect handlers in OxCaml expose a different interface to those I helped engineer into OCaml 5.

Previously, Effect.perform was the only way to perform an effect and was mostly dynamically typed. A second interface in OxCaml now threads a zero-size token from the point where a handler is installed to each perform (#6281).

This helps with the JavaScript compilation (where, until the wasm stack switching extension is more widely adopted, there is a regional CPS transform to turn effects into JavaScript). The old effect function still works but raises an alert.

There's also a new Stdlib.Dynamic (#6412). to allow fiber-local storage. A dynamic variable is bound for the extent of a function call, and is inherited by child fibers, and is safe to read in parallel.

type _ Effect.t += Ask : int Effect.t

let body (h : Effect.Handler.t @ local) () = Effect.Safe.perform h Ask + 1

let answer =                                (* 42 *)
  Effect.Deep.Safe.try_with body ()
    { effc = fun (type a) (e : a Effect.t) ->
        match e with
        | Ask -> Some (fun (k : (a, _) Effect.Deep.continuation) ->
                   Effect.Deep.continue k 41)
        | _ -> None }

let depth : int Dynamic.t = Dynamic.make ()
let current () = match Dynamic.get depth with Null -> 0 | This d -> d
let nested () =                             (* 1; an unbound variable reads Null *)
  let r = Dynamic.with_temporarily depth 1 ~f:current in
  r

I haven't ported my local OxCaml Eio fork to the new functions yet, but it looks reasonably straightforward.

7 Foreign function interface

One of the more complex areas to deal with in OxCaml is the C interface, since the homogeneous value layouts are more flexible with modes.

A C stub declared external could previously not satisfy a [@@zero_alloc] signature. This now works (#6651), so allocation-free interfaces no longer need a wrapper function around each stub!
Reading memory at a raw address now has compiler primitives to avoid Obj.magic (#6350).
C code that called back into OxCaml had no means of releasing stack allocations made by that callback. They can now open and close a region (#6142), and can also donate memory it has laid out as OCaml blocks to the major heap (#6248). This last feature looks really useful for my DuckDB/Clickhouse bindings!

module Trunc : sig
  val to_int : float -> int [@@zero_alloc]
end = struct
  external to_int : float -> int = "caml_int_of_float" [@@noalloc]
end

external peek : ('a : any). int64_u @ local -> 'a @ local
  = "%unsafe_get_ext_ptr" [@@layout_poly]

#ifdef OXCAML_RUNTIME
caml_region_t region = caml_region_begin();
value v = caml_callback(f, Val_unit);      /* may return a stack-allocated value */
intnat n = Long_val(Field(v, 0));
caml_region_end(region);                   /* frees the callback's stack allocations */
#endif

8 Performance and optimisations

The GC performance seems to have generally had a big boost due to elite Jane Street optimisation ninjas beavering away.

The garbage collector has much better pacing now. Previously, the major collector had no idle phase but now after sweeping it waits until allocation passes a threshold before starting again (#6484 and upstream). This reduces GC activity significantly in programs with small or steady-state heaps. Pacing also accounts better for large off-heap data, which we encounter when mapping in large IO libraries in Bigarrays.

There seem to be tons of new optimisations like continuation lifting and specialisation that are now enabled at -O2/-O3 codegen. x86-64 output also looks better through conditional moves, and immediate and address folding. For larger binaries, frametables are about 40% smaller (#6399), and large links, debug information and incremental rebuilds are faster. Things are generally snappier overall in my builds; fast compile time is one of OCaml's best features and OxCaml is still pretty good even in release builds!

9 Emerging features

The following need an extension flag or are documented as experimental, but I couldn't resist taking a peek to see where the language is heading. I'm still not using most of these things in my production binaries though.

9.1 Layout polymorphism

This is a huge quality-of-life feature to allow multiple representations without resorting to ugly ppx_template extensions. On 5.2.0+minus39, a let poly_ binding type-checked, but actually using it at an unboxed layout killed the compiler with an internal error. Both now compile and run (#5870, #6287)!

(* -extension layout_poly_alpha *)
let poly_ id x = x
let poly_ pair x y = #(x, y)

let demo () =
  let #(s, n) = pair "a" #1L in     (* string, int64_u *)
  #(s, id n, id #42.0)              (* id at bits64 and at float64 *)

9.2 Mode polymorphism

A let-bound function has so far had only one mode signature, and so different mode use needed two copies or a (extremely ugly) ppx template. The good news is that inference can now generalise over modes (#5483) but the bad news is that there's no syntax yet!

(* -extension mode_polymorphism_alpha *)
let id x = x                        (* 'a @ [< 'm] -> 'a @ [> 'm] *)
let keep (r : int ref @ unique) : int ref @ unique = id r
let both (r : int ref) = (id r, id r)
(* without the extension [id] has one mode and [both] is rejected *)

9.3 Preemption

5.2.0+minus39 had a fledgling Preemption effect, but no actual handler interface for scheduling when a fiber should be forced to yield. A fiber run under a preemptible handler in 5.4.0+ox9 now seems to receive that effect when its tick callback asks for one (#5913).

This makes time-slicing of compute-bound fibers expressible in a user-level scheduler (in native code only). It feels a little weird to have this feature after spending so many decades working in cooperative-style threads, but I guess preemption has its place too!

let preemptions = ref 0
let run work =
  Domain.Tick.with_ ~interval_usec:1000 (fun _ ->
    Effect.Deep.Preemptible.match_with work ()
      { retc = Fun.id; exnc = raise;
        effc = (fun (type a) (e : a Effect.t) ->
          match e with
          | Effect.Preemption ->
            Some (fun (k : (a, _) Effect.Deep.continuation) ->
              incr preemptions; Effect.Deep.continue k ())
          | _ -> None);
        tickc = (fun () -> Effect.Preempt) })

9.4 AVX 512 operations

In the land of Tessera, we really do make heavy use of as many vector operations as we can get our mittens on, and Mark Elvers has also been heavily using AMX on Intel's Endeavour cluster.

Excitingly, OxCaml has a 512-bit vector type now behind a beta flag, with mask register types including load and store primitives, and also via C stub support (#6293). There don't seem to be any intrinsics yet, so I'm not sure how to use these without going through C, but I guess that's why it's still beta!

(* -extension simd_beta *)
type state = { lanes : int64x8#; live : mask#; name : string }

There have also been bugfixes as vector registers were not preserved across a stack check, resulting in silent miscompilation of SIMD code that's now been fixed (#6334).

10 Should you use OxCaml yourself?

This is all very bleeding-edge stuff, so unless you're deep into OCaml or very willing to deal with build issues, I'd wait a little while. If you really want to have a go, then I recommend my agent OxCaml skills (ocaml-dev/oxcaml) as they really do make navigating all the new extensions much much easier. You can also track Thicket to track changes more regularly.

I'll document in the coming few weeks how my own OxCaml monorepo works. While upgrading to this latest release, I also had to port odoc, and lots of other compiler-specific tooling also needs more work due to the internals changing rapidly. My monorepo also has extensive forks of existing ecosystem libraries with their signatures altered to support the portable and local modes, as well as importing in cool new libraries and adapting those too.

None of these were particularly bad blockers, and the OxCaml guard packages continue to do their job of keeping different releases apart to avoid build failures from mixing and matching the wrong ecosystems. The OxCaml compiler itself is rock solid, and the resulting binaries are extremely fast (otherwise I wouldn't be putting all this effort into it!). I'm very pleased overall with the output of OxCaml, and the emerging mode/kind/layout polymorphism will hopefully result in the classic OCaml ergonomics being restored in the early part of 2027.