# .plan-26-32: Finally a use for serverless and found the Forester for the trees *2026-08-09 — note* Two paper updates this week! The first is that our [GETMobile article on micro-NPUs](https://anil.recoil.org/papers/2026-benchmarking-npus) got published in the [ACM SIGMOBILE magazine](https://anil.recoil.org/notes/sigmobile-npu). This is a shorter, more digestible magazine version of the [paper from last year](https://anil.recoil.org/papers/2025-npu-bench). Great work by [Josh Millar](https://profiles.imperial.ac.uk/joshua.millar22)\! Then [Ryan Gibb](https://ryan.freumh.org) gave an excellent SRG [practice presentation](https://amok.recoil.org/@avsm/117048921944660817) of his forthcoming ICFP 2026 paper on [formalising package management calculi](https://anil.recoil.org/papers/2026-package-calculus). His slides were dense with cool math, which emerged from his use of [Slipshow](https://docs.slipshow.org/en/stable/). The ICFP camera ready is also now available, with the coveted artefact badges! The official ICFP DOI will be live once the conference is done in a few weeks.
Behold the coveted full spectrum of quality badges from SIGPLAN.
Behold the coveted full spectrum of quality badges from SIGPLAN.
 The rest of the week went on [converting the TESSERA embeddings to Zarr](#tessera-embeddings) with my first use of AWS Fargate's serverless offering, [rebuilding the Evidence TAP paper infrastructure](#evidence-tap) around Django and Capnp, a summer internship wrapup [on hunting and ranger patrols](#on-hunting-and-ranger-patrols), planning the port of [Foundations of CS to Forester](#teaching), and some [OxCaml and Prometheus hacking](#oxcaml-hacking). The usual [fun links](#fun-links) are at the end. ## Tessera embeddings I've spent the week on a series of syncs over to [source.coop](https://source.coop) from our old AWS bucket, and also from the Cambridge cluster, in order to converge all our [TESSERA](https://anil.recoil.org/projects/tessera) embeddings in one place. Because there was so much data involved, it turns out that doing a conversion of the npy format to the [Zarr v3 layout](https://anil.recoil.org/notes/tessera-zarr-v3-layout) would have taken about 4 months (due to our network link) for all eight years of embeddings if we did it from Cambridge\! Therefore [Mark Elvers](https://www.tunbury.org/) and I [investigated](https://www.tunbury.org/2026/08/03/week-31-2025/) the cheapest way of running the computation required to run AWS VMs. A naive computation using off-the-shelf on-demand EC2 VMs revealed it would cost around \$40000, which seems a little high for our university budgets! However, after chatting to my old friend and colleague Andrew Peace (who works in the AWS S3 team) over coffee, he mentioned that spot instances can be quite cheap. I found [AWS Fargate Spot](https://aws.amazon.com/blogs/aws/aws-fargate-spot-now-generally-available/) which allows for serverless containers to run whenever there is spare capacity in that region. This required bundling up the [geotessera](https://anil.recoil.org) Zarr PR using `uv` and a Dockerfile. S3 secrets weren't required as Fargate has builtin support to 'grant' the container magic access to S3 without requiring an environment secrets dance. Once done, the conversion (one container per UTM zone) ran in parallel with very little drama.
The usage graph of the Fargate spot over a few days, about $500 (credit: Mark Elvers)
The usage graph of the Fargate spot over a few days, about $500 (credit: Mark Elvers)
The total cost? Around \$557 including tax according to Mark! He had to run one or two zones on a non-spot Fargate, but the cost overall was massively lower than managing VMs on-demand ourselves. So perhaps EC2 should be for 'off demand' computing now... After the conversion to Zarr was done, I spot checked the tile coverage by comparing the 1.1 and 1.0 embeddings, and you can see the results of having an expanded landmask below. We probably won't backfill these for 1.0, preferring instead to get global coverage for [v1.1](https://anil.recoil.org/notes/tessera-v11-out) (which our collaborators at dClimate are working hard on thanks to another AWS Open Data grant they got a few months ago). I still also need to work on the meta-Zarr-index on source.coop, since there's no established standard for that; STAC might work but this is more fine-grained than that. The per-store layout at least follows the [shared geo-embeddings convention](https://anil.recoil.org/notes/tessera-embeddings-convention) we moved to earlier in the year, so it's just the index across stores that's missing. It's on my list for next week to figure this out.
Checking for missing tiles in the manifests
Checking for missing tiles in the manifests
A nice overview of earth embeddings from [Konstantin Klemmer](https://konstantinklemmer.github.io/) was published this week, titled simply "[Earth Embeddings](https://doi.org/10.1109/MGRS.2026.3710416)". This is a great overview of geospatial earth embeddings from a broader perspective, including relating them to developments in LLMs. Konstantin gave an EEG seminar on [global geographical location encoders](https://www.cst.cam.ac.uk/seminars/list/211927) back in 2024 and is moving to UCL in London, so I hope to hear more on this work as he settles in\! I also spent some pleasant time discussing [habitat mapping](https://anil.recoil.org/projects/life) with both [Michael Dales](https://mynameismwd.org) and [Aneesh Naik](https://aneeshnaik.github.io/) who are [making steady progress](https://digitalflapjack.com/weeknotes/2026-08-10/) on working through various strategies into a more unified map. I'm having a go in the background at mirroring OpenStreetMap's web UI to make it easy to visualise the habitat labels using their standard tag mechanism, rather than us reinventing the wheel. ## Evidence TAP I had a splendidly fun catchup with [Sam Reynolds](https://samreynolds.org) to understand more about his [Conservation Copilot](https://anil.recoil.org/projects/ce) (which he's been coding himself!). I was very impressed by his progress, as he's built a sophisticated layer over OpenRouter to handle many of the corner cases around the pipeline of evidence synthesis. On my side, I've been doing a giant overhaul of all the paper storage and download infrastructure, since there are a great many local projects that need some of the insights, especially for conservation purposes. ### Switching to Django and Capnp The index website I [built last week](https://anil.recoil.org/notes/2026w31) is working well, but I needed something more sophisticated for managing the ingestion pipeline. The low-level logic for downloading each paper is [working well](https://tangled.org/anil.recoil.org/taposaur/tree/main/taposaur) using my OCaml HTTP client (which offers useful low level control over how to [handle each service](https://tangled.org/anil.recoil.org/taposaur/tree/main/lib/services) based on the good work [Robin Message](https://www.lambdacambridge.com/robin-message) did earlier in the year). But coordinating all this is quite difficult, since the filesystem layout contains roughly ~25 million files (and rapidly growing) and Ceph is too slow to spider incrementally with that many files. Therefore I've prototyped a Django and Celery based task queue, and exposed the paper downloader as a [Capnp OCaml service](https://capnproto.org/) which the website can issue requests to. This fits into the broader strategy in ETAP for building 'living evidence databases'. The idea here is that (for example) [Alec Christie](https://profiles.imperial.ac.uk/a.christie) can also run a downloader node at Imperial for _their_ living evidence database, using their own library credentials. However, multiple LEDs can share indices about what they have, including paper metadata and checksums, even though the fulltexts are closed. This will help build a network of users (initially just two, but we hope it'll grow) of libraries that are curating their own papers and allowing individual evidence synthesis groups to manage large groups of DOIs and filter them for quality control and AI slop. I'll share more on this next week as my OCaml spider is churning away migrating to the new layout, which will take a few days of reindexing across all the papers\!
The classifier also drops back to Core, so I need to mirror that next to avoid hitting their API
The classifier also drops back to Core, so I need to mirror that next to avoid hitting their API
## On hunting and ranger patrols I caught up with Smera Sachin on her last day interning at the [CCI](https://www.cambridgeconservation.org), where she's spent the summer working with [Charles Emogor](https://charlesemogor.com) on [validating anti-poaching predictions with ranger insights](https://anil.recoil.org/ideas/validating-anti-poaching-predictions). She's been picking up where Hannah McLoone left off last year, but with hunter GPS tracks alongside the ranger ones across two Nigerian parks. Charles has also been speccing out the next phases of the work since he got a followon [AI4PG fast grant](https://anil.recoil.org/notes/2026w30) a fortnight ago, so there is plenty more to come. Smera will write up her work later this summer before she heads back to Harvard and I'll link to it then.
Charles and Smera at the CCI on her last day here!
Charles and Smera at the CCI on her last day here!
## Teaching After a nice coffee in the sun at [Pages](https://www.pagescambridge.com/) with [Jon Sterling](https://jonmsterling.com), I decided it's time to port my [1A Foundations of CS](https://anil.recoil.org/notes/focs) course over to [Forester](https://forester-notes.org) format. Jon already does this for his own 1A course on [Discrete Mathematics](https://www.jonmsterling.com/00XY/) so it would be really nice to harmonise with this to reduce sticker shock for the 1As from having lots of different formats for the basic material. This would complement and not replace the nice [course PDF](https://www.cl.cam.ac.uk/teaching/2425/FoundsCS/focs-202425-v1.6.pdf) we already generate. [Jon Ludlam](https://jon.recoil.org) also brilliantly covered the course while I was sabbatical last year, and [converted the online slides to be interactive](https://www.cl.cam.ac.uk/teaching/2526/FoundsCS/materials.html) using `js_of_ocaml` to embed a compiler directly into the presentation material. More on this next week as I finish working my way through the Forester syntax! I'm also shamelessly using this as an excuse to test out [Forester 6 beta](https://www.jonmsterling.com/0LVR/) since I need a test subject for my forthcoming Eio Windows PRs. ## O(x)Caml hacking Most of my OCaml hacking was related to the Tessera and Evidence TAP infrastructure mentioned above, but I did find a bit of time for some open source work as well. ### Switching to oxcaml-minus39 I got [this website](https://anil.recoil.org/notes/bushel-lives) converted to the latest OxCaml, and wrote up [how the opam guard packages work](https://anil.recoil.org/notes/oxcaml-opam-guards) since finding compatible versions of third-party libraries is the fiddly bit. Since doing this, my [monorepo](https://github.com/avsm/oxmono) is much simpler in the `minus39` branch as the opam packaging is robust enough that I don't need to vendor the whole monorepo. ### OCaml Prometheus refresh [Mark Elvers](https://www.tunbury.org/) needs a Prometheus that works with eio, and so [Thomas Leonard](https://roscidus.com) and I (with helpful review input from [Romain Calascibetta](https://github.com/dinosaure)) spent some time on building a transition library. Since the core of the library has to stay independent of any concurrency library (and indeed, of Unix, so that it can run as a [unikernel](https://anil.recoil.org/projects/unikernels)), we're working on two releases. The first release is a minor [1\.4](https://github.com/mirage/prometheus/releases/tag/v1.4) that provides a simple compatibility upgrade path for existing users: > This release is not a breaking change, but instead prepares for deprecation of Lwt in the core package so a future release can remove Lwt from the prometheus core. Users of the core's Lwt-typed functions should migrate to Prometheus\_lwt now. Code using the new interface will keep working unchanged in future releases. > > Add synchronous variants of the timing helpers to the core. Metric recording is synchronous and these will become the only core timing helpers once the Lwt-typed ones move to prometheus-lwt. The idea here is simply to add the future interface as an alias of the existing one, so that users who `opam upgrade` and get the new release will see deprecation warnings with instructions on what to do. Once that's finished, I'm going to finish up the forthcoming prometheus 2.0 which [moves Lwt out of the core](https://github.com/mirage/prometheus/pull/65) (into the module signatures created in `prometheus.lwt` in the 1.4 release) and subsequently integrate Mark's [PR that adds Eio](https://github.com/mirage/prometheus/pull/60) as an independent package. It's quite a lot of work doing all this refactoring, but worthwhile I think! The core `prometheus` library remains entirely OS independent, using the [mirage-mtime](https://github.com/mirage/mirage-mtime) library to obtain a portable clocksource for example. ## Fun links - David Runciman (who I last had lunch with back in 2009 when we were on stage together for the [800th anniversary of Cambridge](https://www.cam.ac.uk/about-the-university/history/800th-anniversary)) is in a brilliant episode of the [Radical podcast](https://www.bbc.co.uk/sounds/play/p0nz487k) talking about lowering the voting age to 6 (yes that's right not 16, but 6). I explored [inversely proportional voting](https://anil.recoil.org/notes/ecology-at-aarhus) last year when thinking about what fair structures might look like if we adjusted for expected lifespan (i.e. younger people get more than one vote). David did make some good points about why relaxing suffrage would be problematic, as well as the difficulty of radical migrations in any voting policy. In particular, the young people most likely to vote might be those already politically active, and so it's essential to include compulsory voting alongside any such change to ensure that a fair slice of the population is represented. - Hope you've got your [solar eclipse kit](https://www.bbc.co.uk/news/resources/idt-2a268141-80ca-480d-bd04-6cf688bcc3dd) at the ready for next week\! - The irrepressible Bill Thompson is hosting [Puntcon 2026](https://thebillblog.com/about-bill/puntcon/) next Sunday in Cambridge. I sadly can't make it as I'll be in Ireland, but you should go if you get a chance and are in town\! - My beloved Pembroke got [shortlisted for the RIBA Stirling Prize](https://www.ribaj.com/riba-awards/2026/east/haworth-tompkins-pembroke-mill-lane-education-cambridge/) so here's a glamour shot of Mill Lane looking good in the sun.
The view from the new Mill Lane end of Pembroke. There's a six-rope indoor climbing wall in that church spire now...
The view from the new Mill Lane end of Pembroke. There's a six-rope indoor climbing wall in that church spire now...
Synopsis: GETMobile micro-NPU article and package calculus published, TESSERA embeddings conversion using Fargate Spot, and starting to shift to Forester for teaching. Words: 1987 Discussion: - Mastodon: - Twitter: ## Related - [Updating to the OxCaml 5.2.0-minus39 opam packaging](https://anil.recoil.org/notes/oxcaml-opam-guards) (note, 2026-08-08) - [GetMOBILE article on benchmarking micro-NPUs now out](https://anil.recoil.org/notes/sigmobile-npu) (note, 2026-08-05) - [.plan-26-31: Sorting out Tessera and Evidence TAP infrastructure](https://anil.recoil.org/notes/2026w31) (note, 2026-08-02) - [Benchmarking Ultra-Low-Power -NPUs](https://anil.recoil.org/papers/2026-benchmarking-npus) (paper, 2026-08-01) - [.plan-26-30: Bananas ripen, models open, and OxCaml releases](https://anil.recoil.org/notes/2026w30) (note, 2026-07-26) - [Package Managers à la Carte: A Formal Model of Dependency Resolution](https://anil.recoil.org/papers/2026-package-calculus) (paper, 2026-07-01) - [Tessera v1.1 released, with smoother and temporally stable embeddings](https://anil.recoil.org/notes/tessera-v11-out) (note, 2026-06-12) - [TESSERA now supports the Zarr geo-embeddings convention proposal](https://anil.recoil.org/notes/tessera-embeddings-convention) (note, 2026-03-27) - [Streaming millions of TESSERA tiles over HTTP with Zarr v3](https://anil.recoil.org/notes/tessera-zarr-v3-layout) (note, 2026-03-14) - [Benchmarking Ultra-Low-Power μNPUs](https://anil.recoil.org/papers/2025-npu-bench) (paper, 2025-11-01) - [Presenting our Ecology of the Internet ideas at Aarhus 2025](https://anil.recoil.org/notes/ecology-at-aarhus) (note, 2025-08-22) - [Validating predictions with ranger insights to enhance anti-poaching patrol strategies in protected areas](https://anil.recoil.org/ideas/validating-anti-poaching-predictions) (idea, 2025-06-01) - [Arise Bushel, my sixth generation oxidised website](https://anil.recoil.org/notes/bushel-lives) (note, 2025-01-29) - [Foundations of Computer Science](https://anil.recoil.org/notes/focs) (note, 2025-01-03) - [TESSERA, a pixelwise geospatial foundation model](https://anil.recoil.org/projects/tessera) (project, 2025-01-01) - [Conservation Evidence Copilots](https://anil.recoil.org/projects/ce) (project, 2024-01-01) - [Mapping LIFE on Earth](https://anil.recoil.org/projects/life) (project, 2023-01-01) - [Unikernels](https://anil.recoil.org/projects/unikernels) (project, 2010-01-01) --- Canonical: https://anil.recoil.org/notes/2026w32 Type: note Tags: tessera, ocaml, conservation, academia, ai, teaching