Got back to Cambridge mid-week and back on the keyboard to start preparing for Michaelmas term, which creeps up fast! Tessera v2 beta2 turns out to smooth space at the cost of time, so beta1 may be as far as the v2 series goes without a retrain. Evidence TAP gets a splendid explainer from Sam and some new Associate Scholars at Pembroke, along with a TDM reservations library and some thoughts on working across OCaml and OxCaml. Elsewhere there's Scrutineer's git workflow and an invitation to a bug sprint, rather a lot of the Windows tax, why AI disclosure is a social problem rather than a technical one, a fourth Recoil host running OpenBSD, and the usual fun links.
1 Tessera v2 beta2 and the temporal axis
Mark Elvers and I published more Tessera v2.0 beta2 to source.coop this week to test out some fixes to the improved embeddings.
The v1 embeddings sometimes show a "checkerboard" where Sentinel-1 and Sentinel-2 coverage is thin. This happens in areas where there are few satellite observations for the model to infer embeddings from. v2-beta2 tried to smooth them over by applying MODIS corrections during inference to augment the data, since MODIS is coarser spatially but has more consistent coverage. This actually works pretty well, and Mark Elvers measured a 35% reduction in the boundary artefact.
Unfortunately, this does regress another aspect of the embedding temporally.. The v2 model wasn't trained on MODIS, and so inferring with it shifts the embeddings year-on-year in ways the model does not anticipate. Because MODIS observations themselves vary year to year, this breaks the cross-year alignment v1.1 added. James G. C. Ball caught this by testing them on the v2 Trentino embedding as while intra-year cross-validation is neutral, training on 2018 and testing on 2019 dropped the macro F1 (from 0.586 -> 0.511) wherase the v2-beta1 embeddings instead improve (from 0.576 to 0.626).
So it looks like, somewhat reluctantly, that the v2-beta1 embeddings may be the best we can do in the v2 series without retraining a model with more MODIS observations or (maybe) doing some extensive fine tuning. Opinions from other testers are still incoming, so do get in touch if you have a view. We're still looking for GPU resource to train the v2.5 model on corrected data rather, so get in touch if you can help with those too!
Meanwhile, the wall-to-wall inference v1.1 run for 2017 to 2025 is underway after last month's GPU shortage and should finish in a few weeks. I released GeoTessera 0.10.2 as a point release to fix v1.1 Zarr URL resolution that broke in last week's Source Cooperative migration, and am working on a robustness pass for recoverable Zarr writes and improved Windows support for Zarr.
2 More polite crawling for the Evidence TAP
Sam Reynolds has written a splendid piece for the Pembroke blog on what the Evidence Traceable AI Pipeline is for (as part of his introductory post for College!).
For the last three years, we have been working with colleagues in Computer Science on an AI pipeline that finds, classifies, extracts data and summarises findings from the academic literature using self-hosted open-access Large Language Models, benchmarking performance against our hard won, human created database. Underpinning this are pioneering agreements with the largest scientific publishers, brokered with the help from the University Library, to download millions of academic papers. We are also developing tools that let decision makers interact with this extracted information. This is the foundation for the Evidence Traceable Accountable Pipeline (E-TAP) project here at Pembroke, generously funded by the MacArthur Foundation, through which we hope to generalise this approach for other fields. -- Sam Reynolds, Aug 2026
I'm delighted that Sam Reynolds and Mélanie Gréaux have now joined Pembroke as Associate Scholars alongside Sadiq Jaffer. Having the conservation evidence, education experts and the computer scientists all in College will make for some fun socials, especially as many other colleagues from other departments have expressed interest in the project as well!
2.1 Text and Data Mining Reservation
On the coding side, I released a TDMRep 1.0 library to opam. Our download engine is crawling papers at some scale now, and publishers have adopted a standard to express their responses to AI crawlers via the W3C TDM Reservation Protocol. My tdmrep library gives us the ability to read this metadata from OCaml code, and the Taposaur crawler declares its intentions (non-commercial, research use only) when requesting papers as well.
I also pulled out json-pointer 1.0 into my tangled repos. This is a query syntax for JSON that's seeing some adoption (e.g. /users/0/name), and I'm using this to parse Semantic Scholar fulltexts in the Evidence TAP corpus.
Since we're using these libraries in my OxCaml httpz stack now, I've also started extracting that code out of our internal monorepo. E-TAP is a "live" service now, with the stack fetching papers and categorising them, so I'm getting more handson experience with OxCaml in production.
2.2 Working on both OCaml and OxCaml
One challenge is that a lot of our high-performance infra (both for ETAP and Tessera) is built in OxCaml, which is a fast moving target as Jane Street release compilers quickly and with breaking changes. There's also some very ugly (but performant) edges to the language as it evolves, such as the use of ppx to get around the lack of layout polymorphism. While this makes code hard to edit sometimes, I've found that coding in OCaml and then agentically translating to OxCaml works very well, since the type system catches layout issues very reliably.
I got my oxcaml/opam-repository#59 merged, which makes it much easier to mix OCaml and OxCaml packages through the guard packages.
However, this approach does neccessitate having full control over dependencies, since writing a
parser in OxCaml is very different from normal OCaml and many dependencies need annotations. My
httpz library has matured in recent months from a parser into
a full HTTP stack with a zero-allocation fetch for HTTP clients, and
proffer, a portable HTTP server layer. I've been experimenting with various
strategies to packaging this up which I'll share in the coming weeks.
I've also started httnope, to build a conformance corpus that supplies adversarial peers to httpz clients or servers and checks the observable effects on fresh connections. This is more useful than testing against a well-behaved HTTP peer since all the "interesting" failures all found in behaviours no reasonable HTTP server would send (but bad attackers probably will). I'm bootstrapping this using LLMs parsing RFCs, but Thomas Gazagnaire has also pointed me to some of his work on an FSM library that he's having good success with, so I'll work more on this...
3 Scrutineer's git workflow for security scanning
The Scrutineer security scanner deployment from last week now has a private Git remediation workflow I added. A gated patch attempt is exported as a cherry-pickable commit on a private Git remote, so I can review with:
$ git fetch scrutineer '+refs/findings/*:refs/remotes/scrutineer/findings/*'
$ git cherry-pick scrutineer/findings/F-63
...over an ssh tunnel. Triage decisions go back as Git notes under refs/notes/scrutineer/triage/<principal>. This is a local-only prototype for now, but I'll continue to gain experience over the next few weeks.
There are quite a few bugs the scanners are finding across a variety of repos, but I released the ones found in cstruct via #324 which corrects indexing and subview offset handling. Thomas Gazagnaire reviewed it and I cut 6.3.0. That only leaves about 150 verified bugs to triage elsewhere...
I also got a couple of fixes merged upstream into Scrutineer (#949 and #950). Separately, we've been invited to a week-long bug sprint as part of Patch the Planet, the Trail of Bits initiative that pairs security engineers with maintainers. Their projects already include Python, PyPI, cURL and Go, so OCaml would be in decent company! I'll email the OCaml security team and a few others next week to arrange something for October.
4 The Windows tax
Unfortunately there comes a time in every open source library's life when it has to pay a Windows tax, and productivity comes to a grinding halt. Since GeoTessera users are typically ecologists rather than computery people, I have to figure out new bugs whenever I add a new feature (in this case, Zarr).
Every time I SSH'd into my Windows box this week I got random errors, and it turned out to be RedirectionGuard, a newish mitigation that the builtin OpenSSH server now activates by default. It stops privileged processes following symlinks created by unprivileged users. Unfortunately for some reason my opam install has Flexlink installed via a symlink, which maddeningly lead to obscure toolchain errors only when I was remotely connected and didn't manifest when I was at my keyboard. Argh!
The work in geotessera#403 now makes the Zarr stores handle Windows paths consistently. On Linux or macOS I can reproduce a path bug in seconds, but for Windows I have to go through a painful CI loop and also understand a very complex path mechanism. I kind of assumed that Python would have good support for abstracting all this, but it feels very similar to OCaml in its difficulty writing portable code...
I also dove into Eio while I was in front of my Windows box, and opened #929 to fix anonymous-pipes and then also worked on improving Forester support in #931. Fixing that was a bit of a rabbithole#932 but has improved the state of Eio on Windows quite a bit. Just need process and pty support next I think.
5 AI disclosure is more of a social issue than technical
I removed the AI disclosure attributes and opam fields across my monorepo this week, and deleted the ai-disclosure skill from my OCaml Claude marketplace. That reverses what I proposed in April and May.
Disclosure of how people code is a technical answer to a social question. When I wrote the proposal you could still imagine agent-written code as a distinguishable subset of a codebase. But today agents are absolutely everywhere, and the number of models have exploded. Tracking this in detail doesn't seem like a winning strategy.
The opam-repository policy that Romain Calascibetta started is a much better social solution. I wrote my thoughts on OCaml Discuss; we ask only that a human is around to answer our reviewers when they have a question. The opam-repository is where we need to aspire to build community within OCaml, which is only getting harder as automated code generation removes the necessity of collaboration we used to have.
To put this in context, some of the PRs we get to opam-repository are a bit surreal these days. #30634 proposed a realdentalcosts library with "stdlib-only OCaml URL helpers", from an organisation that only appeared on GitHub on a few days ago and was spraying packages across several ecosystems at once. Sacha Ayoun dubbed this LLM-oriented SEO.
Meanwhile, #30643 went the other way with a very pleasant interaction with a vscoqbot bot account for rocq-prover/vsrocq. Enrico Tassi replied within the hour with "sure, tell me what needs to be fixed" and we got the minor issues sorted quickly.
6 A fourth Recoil host at Mythic Beasts
I've been provisioning a fourth Recoil host at Mythic Beasts and putting OpenBSD back on it, so our routable IPv4 allocation can finally host throwaway services I've been using exe.dev for. OpenBSD's vmm suits this well since it brings some sorely needed software diversity, and its single-vCPU hypervisor is just fine for this.
Mythic Beasts remain a joy to host with since there's always a real human at the other end who replies (thanks Pete!) quickly. They're plugging a USB stick into the machine so I can netinstall OpenBSD myself, and quickly sorted out routing the custom /28 from our block as well.
7 Fun links
- There's a cool workshop called TerraBytes on next week (Sep 8th) I'm going to catch some talks online for.