Arise Bushel, my sixth generation oxidised website / Jan 2025

This website has been through quite a few iterations over the years. The first version in 1998 was written in Perl and hosted on OpenBSD; the second was rewritten in 2000 when I got commit access to PHP; the third rewrite became a hybrid OCaml/PHP/Perl special in 2004 in Blosxom; then the forth rewrite around 2013 got turned into a unikernel in MirageOS; then the fifth in 2019 then transitioned to an OCaml static site generator hosted on a prerelease multicore OCaml webserver. So the sixth generation now needs something to continue the grand Rube Goldberg tradition of helping me learn the latest and greatest in systems technology.

And so here it is! The site is now written in a bleeding-edge unreleased variant of OCaml with extensions based around Rust-like type system features activated, including rather exciting data-race freedom work that just won a best paper award at POPL 2025. It's normally difficult to work on continuously moving compilers, but Diana Kalinichenko did a tremendous amount of work into making it usable with opam out of the box, and this post documents the journey to getting this website live.

Getting the oxidised compiler

Firstly, we did some groundwork a few months ago by adding support into the opam-repository for bootstrap versions of dune, menhir and ocamlfind. These are used to build the Jane Street version of the OCaml compiler, which is published as an opam-repository#with-extensions.

The extensions there are straightforward for those familiar with opam. On a clean system you can run:

$ opam init
$ opam switch create 5.2.0+flambda2 \
 --repos with-extensions=git+https://github.com/janestreet/opam-repository.git#with-extensions,default
$ eval $(opam env)

This creates a new opam switch known as 5.2.0+flambda2, and we can then verify it's running the variant compiler.

$ ocaml
OCaml version 5.2.0+jst
Enter #help;; for help.
# let () =
    let local_message : string @@ local = "Hello, World" in
    print_endline local_message
  ;;
Error: This value escapes its region.

That last bit is the new region magic which I'm keen to start experimenting with for this website! But before that, we need to get the rest of the ecosystem packages needed for the website working under this compiler.

Installing ecosystem packages

I decided to build the new site based on a content manager I've been designing (and scrapping) for a few years, codenamed Bushel. The basic idea behind Bushel is to extend Markdown sufficiently with rich contextual data (such as contacts, papers, projects, ideas and so on), and allow for cross-referencing to other sites that also follow the Bushel protocol. I'll talk about that in more detail in future posts, but for now that means that I need a more dynamic website than the static one I used for the past few years.

Since the Jane Street compiler doesn't yet support the effect system from OCaml 5, I couldn't use my own Eio-based webserver. So after some discussion with Michael Dales who is also porting his site to OCaml, I took the opportunity to learn the the excellent Dream server, which is based on Lwt. I also used Daniel Bunzli's cmarkit library for Markdown parsing, and my own Jekyll_format and yaml libraries.

Amazingly, all of these libraries worked out of the box on the Jane Street compiler, except for one snag: the parsetree internals have changed in their branch. This means that PPX extensions will not work out-of-the-box. Thankfully, there is an abstraction library called ppxlib which has been ported to the variant compiler, and the differences in the parse tree were easy to fix up (thanks Nathan Reb and Patrick Ferris for your recent ppxlib work!)

After forking and fixing just two libraries that were using ppx (and not part of the Jane Street core libraries that were already ported), all I had to do was to pin them and add them to my development environment.

opam pin add ppxlib 0.33.0+jst
opam pin add dream-httpaf 1.0.0~alpha4
opam pin add hpack https://github.com/avsm/ocaml-h2.git#js-extensions-fixes
opam pin add lwt_ppx https://github.com/avsm/lwt.git#js-extensions-fixes

And this then installs the overridden version of packages that I needed, with the pins making sure that the right dependencies were also present. After that, it was plain sailing! I've now compiled up a native code version of my webserver code, deployed it into a Docker container, and deployed it on Linux.

In the future, I hope to use dune package management to ease the deployment of the site, but it didn't work in its current preview form due to a problem with depopts. Just teething problems with a preview, so I'll post more about that when I get it working! I also have a half-finished port of the variant compiler to OpenBSD, so that I can shift my website back to its familiar home rather than running on Linux.

I haven't yet actually taken advantage of any of the new extensions in the Jane Street variant, since I wantd to get this site up and running first. I'll tidy up the code, open source it in the coming weeks, and then we can dive into some region extensions and see how far I get!