Updating to the OxCaml 5.2.0-minus39 opam packaging

This website is written using the OxCaml language extensions from Jane Street, so I can experiment with zero allocation frameworks written in OCaml. Since David Allsopp released oxcaml-minus39 a few weeks ago, I've updated to that. This first required figuring out the packaging of some of the third-party dependencies that I require. Using OxCaml is more complex than vanilla OCaml since many existing packages fail to compile due to it requiring additional type annotations or eta-expansions. However, there's a neat set of guard packages present in the OxCaml opam overlay that make it straightforward to find compatible versions.

My own need was for Eio, as Eio <=1.3 didn't build under OxCaml. The Jane Street overlay shipped an eio.1.3+ox fork built from a patched branch. Thomas Leonard and I fixed the OxCaml build issues in Eio 1.4, but found ourselves blocked from using it due to the system of guard packages.

I figured I'd dig into how this works and make it easier to contribute fixes to Jane Street for third-party packages like mine. I've opened PR#59 on OxCaml's repo, and here's an explanation of how the OxCaml guard machinery works, how to let newer releases through, and what's new in ox-minus39 itself.

1 How the package guards work

The OxCaml opam-repository is an opam overlay that you add alongside the main ocaml/opam-repository. It supplies both the OxCaml compiler itself as well as +ox forks of upstream packages that don't yet compile with the extended compiler.

This overlay repository seeks to ensure that opam can never resolve around one of the patched packages (e.g. if upstream releases a newer version), and install the unpatched upstream version instead. The OxCaml maintainers need to explicitly test the new version and permit it through the OxCaml package guards.

David Allsopp did this by adding a couple of opam meta-packages per forked package: For eio, for example, we get:

• oxcaml-eio.guard, meaning "the patched eio is not installed here".
• oxcaml-eio-patches.enabled meaning "the patched package is installed here".

The two packages conflict with each other, and a third oxcaml-patch-guards package uses a disjunction of dependencies to require one or the other. Crucially, the guard packages' conflicts: field lists the upstream versions that are forbidden if we go down the 'unpatched' package route:

conflicts: [ "oxcaml-eio-patches" "eio" {< "1.4"} ]

This means that every eio version before 1.4 was incompatible with OxCaml. The solver is then free to look for an eio 1.4+ release in upstream opam-repository and select that if available.

2 Allowing newer packages through the guards

The nice property of this design is that retiring an OxCaml fork does not require deleting the +ox package, as someone on an older OxCaml compiler still needs it.

For mdx, for example, it just requires patching the guard package to allow the new version through:

--- a/packages/oxcaml-mdx/oxcaml-mdx.guard/opam
+++ b/packages/oxcaml-mdx/oxcaml-mdx.guard/opam
@@ -6,7 +6,7 @@ authors: "David Allsopp"
 license: "CC0-1.0+"
 homepage: "https://oxcaml.org"
 bug-reports: "https://github.com/oxcaml/opam-repository/issues"
-conflicts: [ "oxcaml-mdx-patches" "mdx" ]
+conflicts: [ "oxcaml-mdx-patches" "mdx" {< "2.6.0"} ]
 messages: ["WARNING! An older version of OxCaml is being installed" {oxcaml:version = "archived"}]
 depends: [
   ("oxcaml-merlin" {post} | "oxcaml-merlin-patches" {post})

So this now has mdx working, but then I needed to add the unix dependency that mdx 2.6 now needs in the dune rules. That fix shipped in the dune 3.24.2 release, so I've added that into the diff too.

Is this a good system then? It's certainly an ingenious use of the opam solver, but the packaging encoding itself is difficult to parse at first glance (although made much easier with agents; I used my local deepseek agent to help me out. I'm hopeful that as Ryan Gibb advances his package management calculus, we'll be able to improve the syntax UI (and the error messages!) in time.

3 What's new in OxCaml minus39?

I updated my Claude OCaml Marketplace skill, which you can find here. A few highlights from minus31-39 that caught my eye are:

• Domain preemption has an initial implementation, so a tight compute loop can be interrupted at a poll point instead of starving everything else sharing the domain.
• Arrays of unboxed elements now have their own packed representations (int8 array, float32 array, vec512 array etc), so an array of small numbers isn't a word per element.
• The borrow_ operator lets you pass a unique value where an aliased one is wanted and still own it uniquely afterwards.
• Runtime metaprogramming's quotes and splices (<<e>> and $(e)) only need -extension-universe beta, and [%eval] is now an ordinary Eval.eval function.
• There's an LLDB language plugin to ease debugging oxidised binaries.
• Implicit kinds are enabled so fewer kind annotations need writing out by hand.
• Mode syntax improved so that (e : @ modes) parses directly, removing the need for the : _ @ modes workaround.
• Any two-constructor variant can now be [@@or_null] giving custom Nope | Yep of 'a types the same non-allocating null encoding as the built-in 'a or_null.
• -O4 exists, being -O3 plus the mysterious "reaper" pass, which I need to investigate.
• Some things were deleted, such as the block-index array syntax (.(0), .:(0), .L(i)), float record indices, and the _internal kind escape hatches. I never used these so wasn't affected.

This website is now running on minus39 using my bleeding edge monorepo. If you're interested in trying OxCaml yourself, the compiler's stable, but the packaging is still fluid and needs some expertise in how opam works for you to add your own overrides. Hopefully this post will help you navigate that a bit more easily!