.plan-26-46: I bless the rains down in Madagascar

Tessera reaches Madagascar, gave an IUCN mitigation webinar, the Haven forest protocol soft-launches, started working a UK hydrology model, and making OxCaml more deployable with self-hosted ATProto pipelines.

https://anil.recoil.org/notes/2026w46image

This week I discuss Tessera traveling to Madagascar, and how I learnt how to count birds properly, and then cover matters of teaching, and the Haven forest protocol soft-launching, and also worked on making OxCaml more deployable. The usual fun links too, it's been a busy old week!

1 On matters of geospatial Tessera

By far the most motivational news of the week came from my good friend and colleague Julia P.G. Jones, who reported a successful deployment of my Tessera Android app all the way over in Madagascar!! Julia headed out there with Miranda Lam last week, and we had some very last-minute remote debugging on her phone. I wasn't at all sure it would work, so seeing it in action and helping with their navigation locally on the ground is an absolute highlight of my week!

I'll write up more about the Android application and Kotlin libraries next week, as it's the first time I've built an agentically-assisted mobile application. Many thanks to Alastair Tse for lending his expertise to help me select the right frameworks, and also happy birthday Miranda :-)

1.1 A local perma-caching Zarr proxy

We serve Tessera embeddings as Zarr v3 shards over HTTP, which works very well until offline access is needed or we're re-reading the same shards for the hundredth time (something that's causing significant stress on our Cambridge outbound bandwidth lately!). The classic way to solve this is by using an HTTP caching proxy which can intercept requests before they go to the network.

Therefore I built zarr-perma-proxy in my ox monorepo, which acts as a specialised 'permanent' disk cache for immutable public HTTP objects. The proxy understands the pattern of Zarr v3 HTTP requests, so a cache miss downloads the complete shard with a single upstream request. After that, every byte range request (for an individual chunk) is served from local disk.

The upstream shard files sitting on the remote server aren't decoded or processed, and since those stores are immutable there's no cache revalidation necessary. I've still got to make the on-disk cache format compatible with geotessera so it can directly read the files, but this works pretty well for now. I also opened a PR to geotessera that adds --proxy, GEOTESSERA_PROXY and a proxy= argument, so that existing Python readers can route their Zarr reads through the permaproxy.

This has two main uses for me so far:

  • On my laptop, it keeps a permanent local copy of whatever portion of the Tessera embeddings I'm using, for demos and offline analysis.
  • On our big petabyte storage in Cambridge, it keeps a copy of whatever embeddings people in the group are pulling. Everyone in our group working on habitat mapping, like Aneesh Naik, Michael Dales and James G. C. Ball, will need to do a global run of their habitat models as they approach maturity on smaller regions.

1.2 RRRRR you ready to RRRRumble?

For the R users among you, my prototype geotessera-r library should be ready for a first release soon. Thanks to feedback, I fixed some quadratic behaviours for larger ROIs and also added parallelism via mirai. It now needs R 4.4 or later, but I think that's mostly ok for our userbase.

I'd be very grateful for more testers. If you use R for your geospatial work, please do give it a go and file issues with anything that seems unidiomatic or broken!

1.3 Icechunk at Pembroke high table

Sadiq Jaffer, Srinivasan Keshav and I had a lovely Pembroke High Table with Joe Hamman, the founder of Earthmover, who built Icechunk. I discovered over dinner that he's also one of the founders of CarbonPlan, whose website has an incredibly cool design with interactive graphs (well before coding agents made that sort of thing blasé). CarbonPlan was a big inspiration for our work in the Cambridge Centre for Carbon Credits a few years ago.

Joe convinced me that Icechunk really is the way to go for our future geospatial workflows. The only thing that kept me pinned to Zarr is the lack of a standardised on-disk format for Icechunk, but after discussing it with Joe I think I'll have a go at auto-formalising a specification of the Icechunk format in Lean 4, so that we can build an OxCaml implementation that's interoperable with the reference one. This would add a host of new features (such as transactional updates), and also make the Zarr proxy above much more feature-rich.

Sadiq gives an impromptu Tessera demo for Joe Hamman before Pembroke high table
Sadiq gives an impromptu Tessera demo for Joe Hamman before Pembroke high table

1.4 Inferring Tessera tiles on Intel Xeon CPUs

Mark Elvers has also been following up his Intel AMX experiments from the spring. We're investigating whether a few Intel AMX cores can keep pace with a small GPU for Tessera inference. Intel has since given us time on their Endeavour cluster, and Mark has got the v2 pixel student model running well over twice as fast there (from 56 to 136 tiles an hour)!

Given the current GPU shortage, this could well be what we end up doing in 2027 for the next iterations of the Tessera models...

1.5 A hydrology model for next year's drought

The WMO reported in September that El Niño is firmly established and set to become a very strong event, raising the risks of extreme weather into 2027. So I've been working with Sadiq Jaffer, Pedro Sousa, Rich Turner and Tom Kelly on a data-driven hydrology model of the UK, to help figure out if we can do an improved forecast for the UK's drought situation next year.

There's plenty of excellent prior art in this space, of course, and our hope is that Tessera will be particularly useful for downscaling. Tom has been writing up notes on his blog that are an excellent primer if (like me) you are new to the field:

Also on my reading list is Google DeepMind opening the datasets behind their AI weather models. These include SatWeave, a global mosaic of geostationary satellite imagery, and PARDIG, which gives global precipitation estimates at 0.1° and 15-minute resolution learnt from sparse spaceborne radar measurements.

This also puts the Environment Agency hydrology data I gathered for the UK label map to good use. My agentic crawling of the UK data continues (politely, respecting rate limits) apace, with over 250 separate data sources now included! I'm going to sift through them in more detail next week to figure out which would be useful towards the hydrology model.

1.6 IUCN webinar on impact mitigation

David Coomes and I co-presented at the quarterly IUCN impact mitigation and ecological compensation webinar on Thursday, with about 100 people online. We covered how Tessera can help with biodiversity monitoring, including several of the ongoing projects in David's group on habitat monitoring.

The audience was really engaged, with questions ranging from the quite technical (enquiring about the model architecture), to a variety of downstream heads. One lovely followup was from Thomas Hodgson, who was inspired by the webinar to spend his evening building a birdsong embedding explorer. He left an audio recorder in his garden for a month last year, and has compressed them into embeddings and a 3D UMAP atlas of each clip. It's quite relaxing clicking on one of the dots to hear the birdsong, which should put you in the right frame of mind for the statistical thoughts in the next section!

2 On matters of conservation science

2.1 Counting birds with citizen science data

We had a brilliant CRI seminar from Prof Ali Johnston of St Andrews on "Counting birds with citizen science data", all about eBird and the huge efforts to turn tens of millions of birdwatchers' checklists into actionable insights for conservation practitioners.

Ali Johnston kicking off her seminar on counting birds with citizen science data
Ali Johnston kicking off her seminar on counting birds with citizen science data

The talk focused on the statistical corrections needed to make sense of the 'raw' crowd data. While birds are the most surveilled taxa, the tracking records vary in quality from rigorously designed surveys like the BTO's Breeding Bird Survey through to a casual list that's scribbled in a notebook while on a walk (these are my sort of records!).

The eBird database from Cornell is somewhere in between these, as its digital checklists record enough about the effort that the individual involved puts in (e.g. how long, how far, whether everything seen was reported) to allow much of the bias to be modelled out afterwards. This is where Ali, the statistics expert, comes in!

The spectrum from rigorous survey design to casual birdwatching, with eBird checklists in between
The spectrum from rigorous survey design to casual birdwatching, with eBird checklists in between

Once those bias corrections are in place, the results she showed are remarkable. The example above shows weekly relative abundance maps for the Barn Swallow across its entire global range. This is much more detailed in both space and time than a conventional AoH map is. The guidelines for doing this properly are in Johnston et al (2021).

Weekly relative abundance of the Barn Swallow from eBird, versus the conventional range map inset
Weekly relative abundance of the Barn Swallow from eBird, versus the conventional range map inset

This is of course directly relevant to our habitat mapping efforts with Tessera. Michael Dales timed his Cambridge visit to coincide with the seminar and has written up his conversations with Ali and her students. It's particularly intriguing to see that 'complete' eBird checklists are a good source of absence data, which is the missing thing in most occurrence datasets.

2.2 Samosas, Themis and a Haven for nature

This week's samosa social at Pembroke had a treat for the early comers, as they got a five minute talk from Carl Edward Rasmussen about his Themis Mechanism for equitable global emission reduction.

Thomas Swinfield also brought up the parallels of Themis with his work on forest conservation in Canopy PACT, the charitable spinout from 4C. Canopy PACT hit a big milestone this week, as the Haven scientific protocol that we've been working on with Isometric has soft-launched. Haven exists to protect forests by bringing scientific rigour, buyer demand and ecosystem alignment to nature-based reductions credits, at a time when confidence in those credits (and so the financing for forest conservation) has fallen. I'll write more about Haven once it's fully launched, but just look at the Madagascar video above for why I care so much about ensuring equitable livelihood benefits!

Carl Rasmussen explaining the Themis Mechanism to the samosa social early comers
Carl Rasmussen explaining the Themis Mechanism to the samosa social early comers

3 On matters of teaching

Jon Sterling gives his 1A Discrete Maths course to follow straight on from my Foundations of Computer Science introduction to our Cambridge computer science first years. It has been delightful to see his course receive much positive feedback after he uploaded the opening of his first lecture on why we still teach this material "by hand" in the generative AI era:

I cannot make any promises about what kind of a job you will have when you leave here. But what I can promise is I will move Heaven and Earth to ensure that you have the opportunity to learn this astonishingly beautiful and inspiring topic in the same way that I did, not because I believe that only the past can beckon, but rather because I know that the future will need people with the skills that you will get here. And because I want you to know joy in thinking. -- Jon Sterling, Weeknotes 2026-W46

3.1 Discussing the future of the CST Tripos

We had our Computer Science departmental annual report this week, in which we considered the future of Part II and Part III. One concern is that increasing the number of Part III students would roughly double the effort needed for undergraduate project supervision.

Some suggestions included letting students elect at the start of Part II whether to graduate after three or four years (with one dissertation in their final year), or instead following Engineering with smaller one-term Part II projects, or dropping the Part II project altogether (something I'm very much against as the experience of building something end-to-end is super valuable).

We're seeing a fall in our applications as many other departments are internationally. I suspect we're going to move to a longer four-year programme that caters to those who really want to learn computer science, and that this will justify the extra supervision efforts required.

Open questions about Part II and III at the department annual report
Open questions about Part II and III at the department annual report

3.2 Forester 6 out and Eio Windows improvements

I was also delighted to see Forester 6.0 released after a long development cycle, since it's what I've been using for my lecture notes this year. Jon describes the Zeno's paradox of getting the release out, with the many steps required to pass the release checklists such as Windows CI in the opam repository.

Getting tools like Forester working smoothly on Windows is my main motivation for all the Eio Windows work I've been doing recently. This week I opened PRs to enable MDX on Windows, which is helping to spot differences between POSIX backends in areas such as symlinks and junctions support. I think we're fairly close to Forester fully working now, so I'll try compiling my full lecture notes on Windows soon!

Elsewhere in Eio, there were more sandbox escapes fixed, and I reviewed fixes for error handling, exceptions escaping, Unix sockets cleanups and collection of system information.

This was a fitting moment for David Allsopp to say thank you and goodbye to AppVeyor after 11 years of it running the OCaml Windows CI. I'd quite forgotten about Travis CI too!

4 Making OCaml and OxCaml more deployable

As we're using OxCaml more and more for our services, I need a way to quickly deploy binaries to various machines without having to set up a compiler toolchain on each of them.

4.1 Self-hosted ATProto pipelines on Tangled

To that end, I've spent a lot of time this week extending last week's custom spindle into an end-to-end, completely self-contained ATProto infrastructure that lives in my monorepo. The idea is to use the Tangled lexicons to drive our own self-hosted runners, and so be ready for when private repository support lands on Tangled.

This is all a work in progress, but the rough shape is:

  • OxCaml libraries for the ATProto plumbing, and a spindle that listens for pushes and pull requests and hands the builds off to our OCluster pool, so nothing from a repository runs on the spindle host itself.
  • Automated odoc generation for every package in the monorepo.
  • Publishing of signed OS packages (deb, rpm and apk) with their repository indexes, to make it easier to deploy binaries of our services without needing an OxCaml toolchain on the target machine.
  • Disposable macOS VMs as a CI target alongside the Linux and Windows workers.

You can watch me testing things on the pipelines page. I'll blog properly once it's live.

The self-hosted spindle running pipelines for the ox monorepo, with links to the published packages and docs
The self-hosted spindle running pipelines for the ox monorepo, with links to the published packages and docs

4.2 Towards a merge queue deployment prototype

This also links back to my thoughts from 2023 on moving the opam repository to merge queues. Raphaël Proust and some of the other opam-repository maintainers have expressed interest in seeing a demo of how this could work. Mark Elvers built a prototype of this for me earlier in the year using day10 on a self-hosted runner with GitHub's built-in merge queue to gate PRs to an opam overlay repository, so now I'm trialling it on ATProto.

I was also reminded of how useful this would be while merging many PRs into the opam-repository this week. These were mostly new releases (including Forester 6.0) but also a very long run of bounds fixes from Thomas Gazagnaire as he prepares Alcotest 1.10. We have to make the human labour in verifying such changes much easier, which I believe merge queues are a step towards. Mark has since gotten his day10 builder working on Windows and the BSDs, running in parallel with the existing opam-repo-ci jobs. This should also make merge queues much faster to use, I hope!

Paper of the week comes via Justin Cormack, about an extinct 'vampire camel' that once stalked the North American plains. Thirteen million years ago, the scary sounding Vampyrocamelus welshi was a tall, skinny, giraffe-like camel wandering the Great Plains with a set of hooked fangs. Something for next Halloween...

%rc
I also had a flying visit from @gtyson, who has just been appointed to a professorship at Oxford AKA The Other Place. Congratulations Gareth! Elsewhere in the group and beyond:

Also, to follow up on last week's OpenUK award for the project, Amanda Brock sent on a nice picture of the award receipt. I'm still buzzing from all of this!

The Tessera team collecting the OpenUK award
The Tessera team collecting the OpenUK award

References

[1]Madhavapeddy (2026). .plan-26-31: Sorting out Tessera and Evidence TAP infrastructure. 10.59350/30e5y-n8p97
[2]Madhavapeddy (2025). Foundations of Computer Science. 10.59350/qms3q-ymn65
[3]Madhavapeddy (2026). .plan-26-44: Commons People. 10.59350/94p1z-tk116
[4]Sousa et al (2026). Earth observation embeddings are effective sub-grid descriptors for probabilistic weather downscaling. arXiv. 10.48550/arXiv.2608.12271
[5]Madhavapeddy (2026). Streaming millions of TESSERA tiles over HTTP with Zarr v3. 10.59350/tk0er-ycs46
[6]Madhavapeddy (2026). Using Forester to turn Foundations of CS into interactive evergreen lectures. 10.59350/sjcvd-hb857
[7]Kratzert et al (2019). Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets. Copernicus GmbH. 10.5194/hess-23-5089-2019
[8]Johnston et al (2021). Analytical guidelines to increase the value of community science data: An example using eBird data to estimate species distributions. 10.1111/ddi.13271
[9]Shalev et al (2026). PARDIG: Global Precipitation Estimates Learned from Sparse Satellite Observations. arXiv. 10.48550/arXiv.2611.13941