I've completed a working cut at a streaming interface for TESSERA embeddings, and it's unexpectedly addictive to zoom around the world staring at false colours! The amazing thing about this interface is that it's entirely browser based, using JavaScript, WebGPU and WASM to perform all the analysis on the client side. The Zarr embeddings are chunked and served over HTTP, using range requests to retrieve the minimum amount of data.
\n\nThe video shows classification workflows, but I've also got solar panel detection working using Sadiq's patch based embeddings. You can try it for yourself when I release this properly next week!
\nThere are obvious limitations to how much we can do in the browser; for most serious work we will need a server running, but my goal here is to see if we can embed TESSERA into the living dashboard that Shane Weisz is working on. I've also just received a drop of areas-of-habitats from Michael Dales which I'll have a go at integrating next week.
\nWhat's blocking everyone from using Zarr and TESSERA? Well, we need to transcode a petabyte of embeddings from the old numpy format into Zarr, which is a difficult parallelisation problem. I'm making steady progress on an OxCaml pipeline for this with a from-scratch OxCaml-Zarr implementation that Mark Elvers helped me kick off. Mark also published his ocaml-tessera pipeline which I'm going to import to OxCaml next week as well, so that we can do both model training and tile inference in OCaml!
\nFor more production-oriented usecases, Srinivasan Keshav has published a server with his excellent Tessera Embeddings Explorer. We've been building our implementations independently and swapping ideas for user interfaces and analyses, which has been a very productive way of experimenting for different users. His implementation is in use for several downstream tasks projects and should be what people use, while mine is heading towards more dynamic mobile/browser workflows. You can grab that code at https://github.com/ucam-eo/tee, complete with convenient Dockerfile.
\n![\"%rc\"](\"/images/wg28-26-2.webp\" "\\"Viana")
\nI got invited to WG2.8 again, so Simon Peyton Jones and I trooped off there from Cambridge earlier in the week. I had to leave early due to some family matters, but I got to present 'planetary programming' to the assembled gods of functional programming!
I began my talk by presenting the streaming browser demo above (which, thanks to the perma caching httpz oxcaml proxy I hacked up lets the remote tiles be cached on my laptop so the app works offline too). I used the opportunity to posit a high-level programming design problem I've encountered when coding with TESSERA...
\nThere's a contradictory tension in the styles of programming that we\nconventionally embark on with the workflows needed by machine learning. In our\nFAIRground paper we describe a purely functional Python\nvariant that represents conventional 'forward programming'. You can do lots of\nnice things when the language is pure, such as enabling incremental live\ncomputations. This forward programming style would be good for building a global computational wiki for example.
\nHowever, when we program with observational embeddings like TESSERA, we're\ndoing 'backwards programming'. The units we're dealing with are 128-dimensional\nself-supervised representations that have been learnt from primary satellite\ndata, and the job of the program is to help cluster these higher dimensional\nstructures into some semblance of useful meaning. We do this via downstream\nclassifiers, segmenters and regression tasks. This is a very different\nprogramming style from forward programming even though it requires a similar\namount of CPU.
\n![\"%rc\"](\"/images/wg28-26-3.webp\" "\\"There")
\nThe ultimate goal of both of these scientific programming styles is to establish causal relationships; that is, we want to form (or reinforce or falsify) a theory of how the world works that can be tested by the scientific method. So I wondered: how do we combine these three styles into a programming language? This is a very general question, but I figured there was no better place to ask than a room full of people who have designed dozens if not hundreds of languages between them.
![\"%c\"](\"/images/fwd-back-causal-ss-1.webp\" "\\"Three")
ppx_template for now, but other features like float16 (useful for TESSERA) could be done fairly easily.It was a short trip to Portugal in the end, but massively energising. I do love hanging with functional programmers!
\nTwo big perspective papers on global biodiversity are out in PNAS this week, which I wrote up separately in a detailed note.\nTo follow up on these, Cyrus Omar is chairing this year's PROPL (to be held at PLDI this summer), and we've been discussing doing something different this year to tie into a more 'action-oriented' workshop that combines the learnings from the last couple of years with the call to biodiversity action above.
\nMeanwhile, to followup on the first TESSERA hackathon over in India a few weeks ago, Aadi Seth and Srinivasan Keshav have put together a call for students to get involved. If you're interested then apply here and get going with TESSERA!
\n![\"%rc\"](\"/images/iitm-campus-1.webp\" "\\"Life")
\nAnd not to be left behind by their Delhi counterparts, KC Sivaramakrishnan announced that applications are now open for the FP Launchpad in IIT-Madras.\nThis should be of interest to computer scientists who want to get involved in environmental work; as I mentioned before, one of the illustrative projects to kick off the FPL is programming TESSERA embeddings more ergonomically:
\n\nA programmable public infrastructure for environmental planning, combining\nTESSERA's satellite-derived representations with CoRE Stack data and\ncompositional functional models in O(x)Caml to support auditable indicators\nand scenario analysis for India’s water and habitat systems.\n-- FP Launchpad Charter, 2026
\n
So it's action stations at both the IITs and I'm looking forward to working with them from Cambridge! This is a nice followup to our Cambridge VC visiting India and kicking off a cricketing tour!
\nFollowing the CACM Docker article, there's been lots of positive online discussions about the article. Hackernews leads the way with typically split opinions. Some loved it, some hated it, some thought it should be replaced with a very small shell script, and others reminisced over our use of SLIRP. Overall though a lovely discussion and vibe.
\n ![\"%c\"](\"/images/hn-docker-ss-1.webp\" "\\"Docker")
I also read two interesting papers while researching more background for the package calculus that Ryan Gibb has been working on:
\n![\"%rc\"](\"/images/wg28-26-1.webp\" "\\"The")
\nWe had a delightful visit from Professor Yifang Ban from KTH, who delivered this week's EEG seminar on EO-AI4GlobalChange. We went to the pub after, and discussed a rather staggering number of downstream tasks that Prof Ban works on:
\n\nIn this seminar, Professor Ban will discuss recent research at the\nintersection of EO and AI, with a focus on deep learning methods for\nmonitoring environmental change at scale. She will present selected results\nfrom EO-AI4GlobalChange, a collaborative research project developing novel,\nglobally-applicable deep learning approaches for analysing multi-sensor,\nmulti-modal EO data. The talk will cover examples including 2D and 3D urban\nmapping, urban change detection, wildfire detection and near-real-time\nmonitoring, flood mapping, and multi-hazard building damage detection.
\nThe seminar will also briefly introduce PANGAEA, a global benchmark for\nGeospatial Foundation Models, and discuss insights from the systematic\nevaluation of widely used foundation models across multiple geospatial\ndomains. Finally, Professor Ban will briefly outline the objectives of the\nrecently established AI4EO Working Group within Group on Earth Observations\n(GEO), which aims to advance GEO’s vision of Earth Intelligence for All\nthrough AI-driven Earth observation research, innovation, and collaboration.\n-- Yifang Ban, EEG Seminar, March 2026
\n
But most importantly, we had excellent fish and chips to celebrate her first visit to Cambridge!
\nLast summer I spoke at the US-UK Forum on Measuring Biodiversity at the National Academy of Sciences, jointly organised with the Royal Society. Two companion papers from that forum have just been published in PNAS to turn the Washington meeting into an agenda for transforming how the world monitors global biodiversity.
\n![\"%c\"](\"/images/nas-rs-2.webp\" "\\"Throwback")
\n"Nine changes needed to deliver a radical transformation in biodiversity measurement" (pdf) lead by Bill Sutherland tackles the flood of new data from emerging technologies and makes concrete recommendations for ensuring biodiversity measurement can turn into effective action on the ground. Then "From data to decisions: Toward a Biodiversity Monitoring Standards Framework" (pdf) lead by Andrew Gonzalez introduces the BMSF as a structured, auditable "chain of evidence" that connects ethical principles through to standardised data collection, curation, analysis and reporting into a single federated framework. The first paper identifies what needs to change, and the BMSF paper provides a blueprint for how to implement those changes.
\nThe BMSF is particularly relevant to tying together my group's work on planetary computing, and so I've been thinking hard about what to organise this year to bring more computer scientists into the fold. Data processing might sound boring at first glance, but the scales and diversity of information involved in biodiversity is mindbogglingly complex. We are, after all, trying to describe all life on earth...
The key numbers in global biodiversity are pretty stark: roughly 1 million species threatened with extinction, a 70% decline in monitored wildlife populations since 1970,\nand insect populations declining at around 10% per decade.
\nAgainst this backdrop, the paper makes nine recommendations towards accelerating action:
\nCapitalise on novel technology to integrate data sources. This is partly where my work comes in! Geospatial foundation models like TESSERA, eDNA, acoustic monitoring and citizen science are generating data at scale. The challenge is combining these heterogeneous sources, which is where models like TESSERA fit by making downstream task combinations way more efficient. What's missing is effective labeling infrastructure to combine the observational knowledge from space with human expert knowledge on the ground (more on that later).
\nAgree standard methods for data collection. Inconsistency in methods hinders all major uses of biodiversity data, but it's not practitioners that are at fault: there's just a lot of different levels of biodiversity that need to be monitored. Consider how different even a medium sized back garden is from one end to the other, and then scale this out to a tropical rainforest. The paper calls for a tiered and modular monitoring workflow, which is described in detail in the companion BMSF paper.
\nEnsure new technologies are calibrated with existing data. As we race to upgrade technology, monitoring results from traditional field surveys (that might date back decades) need to be migrated to newer AI-driven workflows. But we must keep the primary observations distinct from the statistical derivations or else we poison our sources and confound long-term trends.
\nFill data gaps using emerging technologies, especially in the tropics. Biodiversity data is geographically biased toward charismatic species in wealthy countries. Invertebrates, soil biota and tropical ecosystems are massively underrepresented; just check out Shane Weisz and his living RED list dashboard to see how poorly some taxa are represented in the overall picture.
\nCreate living databases of trusted information to reduce AI poisoning. Our CE evidence TAP pipeline of the scientific literature shows how we need institutionally federated networks of living evidence with humans-in-the-loop who can continuously gather, screen and index literature. The alternatives of depending on blackbox and unaccountable LLMs is rapidly descending on us, and so we are building out dynamic meta-analysis and active vetting of literature databases to spot AI poisoning to not only reject fabricated papers but also to invalidate downstream damage in the citation networks.
\nEnsure data generation is valued. Last summer, David Coomes and I spoke at an IUCN meeting about how vital field-based data collection is for ground truth, especially as self-supervised AI advances. Incentives for data sharing need reform in the majority world where data is currently scarce, for example via governance for wholesale sharing, academic credit through coauthorship, recognition of data production as a research output, and sufficient resourcing for archival. My principles for collective knowledge capture much of this from a technical perspective, but of course regulatory and financial reform is also necessary to push incentives in the right direction.
\nEnsure respectful incorporation of Indigenous Knowledge. When I spoke at the CE conference recently, I carelessly used the term 'indigenous knowledge' to refer generically to grey literature, and E. J. Milner-Gulland quite rightly told me off afterwards! Indigenous knowledge systems are distinct in that they hold generations of environmental insight accumulated through direct interaction with ecosystems. I need to learn more about this all as it also came up at the Indian AI summit as well in the conversations about sovereignty. The paper stresses Free, Prior and Informed Consent and the CARE principles for Indigenous data governance. After our paper was published, WarīNkwī Flores also pointed out their work on sovereign data supply chains which is very complementary.
\nEnsure measurements enable quantification of effectiveness of actions. Monitoring outcomes without a focus on evaluating the impact of interventions is like redecorating while the house burns. High spatial and temporal resolution data (for instance, dynamic ground truth combined with geospatial models) could estimate credible counterfactuals (what would have happened without the intervention). This in turn enables evidence-based policy at much larger scale than we currently do. Most ecological mitigation recommendations in the UK are not evidence based, which means there's a lot of paperwork without much point!
\nIncrease the resilience of global datasets to technical and societal change. This is a problem across society with the pace of AI, but it's particularly urgent for nature where business as usual results in mass extinctions if we take action in the wrong place. There are short term measures such as federation that will help, but I'm fascinated by the potential of our work on building an ecology for the internet that draws on ecological theory to make digital infrastructure more resilient. This is a pretty far-out prospect right now, but coding models have advanced massively in just one year since I wrote that paper, and there's a growing connection between biodiversity science and computer science. There will be a workshop on "Rewilding the Web" in Edinburgh (28-29 May 2026) exploring exactly these ideas of applying ecological insights to digital infrastructure resilience, so I hope to see you there if you are into this!
\nThe BMSF paper then provides a more concrete implementation to follow the flow of data. This is probably of more relevance to computer scientists as it covers a lot of practical ground for which we already have many piecemeal solutions in the tech world. The paper describes an iterative cycle:
\n![\"%c\"](\"/images/msf-fig.webp\")
The framework is federated by design, in a manner quite compatible with Ostrom's guidelines as locally generated data flows into regional and national hubs which apply consistent analytical workflows, producing GBF-compatible indicators for international reporting. Countries or organisations can start with simple methods and progressively adopt more sophisticated approaches as their capacity grows.
\n\n\nFor monitoring frameworks like REDD+ MRV, a centralized, top–down body like\nFAO makes sense given the direct link to the UNFCCC and IPCC and the coupling\nof climate observing systems to carbon stock and emissions models and\nassessments.
\nHowever, the multifaceted nature of biodiversity and the existing landscape\nof key organizations make a federated governance model, with national\nownership at its heart, more appropriate and likely to succeed for the BMSF.\n-- Sec 2.4, From data to decisions: Toward a BMSF
\n
The paper also provides a handy worked example for forest connectivity\nindicators under the maximum GBF Target 3, showing how each step produces a\ncertified evidence chain for a full provenance chain from remote sensing to the final reported indicator value.
\nThe BMSF maps on pretty well to the general principles for collective knowledge systems that I recently sketched out:
\n"Permanence" maps to the BMSF's recommendation to use DOIs for datasets, version-controlled code pipelines, and long-term archiving. Rec 9 from the first paper makes the case that biodiversity data must survive political shocks, institutional closures and format obsolescence.
\nThe BMSF's requirement for persistent identifiers at every step of the chain fits in nicely with the "DOIs for all" approach that Rogue Scholar and Zenodo have been pioneering for blogs and datasets (my own site uses both).
\n"Permission" connects to the BMSF's Ethics step and its emphasis on FAIR and CARE principles, indigenous data sovereignty, and tiered access.\nAs I noted in my collective knowledge principles, biodiversity data is a prime example of where everything cannot be open: location data for endangered species could be exploited by poachers, and indigenous communities have sovereign rights over their own traditional knowledge.
\nThe BMSF's federated model explicitly supports this as not all collected data needs to be shared openly, and the principle of "guardianship" enables tiered access levels. This is the kind of spatial permissioning that current Internet protocols handle poorly.
\n"Placement" maps to the BMSF's federated, nationally- and regionally-owned architecture. The tiered design allows countries to operate at their own level of capacity while still contributing to global assessments.
\nWe're feeling this with TESSERA's multi-petabyte embeddings as well; they physically can't move casually between continents and networks, and so the processing must be distributed and federated rather than centralised. Aadi Seth has suggested a mirror in India as our first federated node!
\n"Provenance" is at the centre of the BMSF as it is designed as an auditable chain of evidence where every indicator can be traced back through analytical methods, data processing, licensing and raw observations.
\nOur fifth recommendation calls out for hallucination-free AI systems for evidence synthesis, and the BMSF's Trust+Analysis phases provide the architecture to make provenance tracking systematic.\nThe same provenance protocols we need for the broader Internet (tracking where knowledge came from and whether to trust it) apply even more to biodiversity data that informs actions affecting the abundances of millions of species.
\nThere's also clear need for revamping peer review in this world, something that BON-in-a-Box is tackling. Our PROPL 2025 paper on programming the BON lays out some projects in this space, and the talk below is an excellent overview of the space for those computer scientists interested in learning more.
\n\nSeveral strands of the research in the EEG connect directly to what these papers are calling for!
\n ![\"%rc\"](\"/images/tee-ss-1.webp\" "\\"The")
\nTESSERA provides an open and robust geospatial foundation model to satisfy the first recommendation and support diverse downstream tasks from crop classification to canopy height estimation.
Srinivasan Keshav and I are just starting work on a labelling framework that fits within the BMSF architecture, so that the ground-truth data used to finetune TESSERA for biodiversity applications can map to the standardised collection, curation and provenance requirements laid out in the framework. Since TESSERA itself is fully open and reproducible, this is a big step toward making AI-driven biodiversity assessments auditable end-to-end. There is also the intriguing possibility of using the ATProto to anchor the federation while re-using identity infrastructure across social networks, but we need to carefully consider issues of privacy. More on this as it develops!
\nOur paper pipeline for analysing the academic literature at scale for\nConservation Evidence is a working prototype of the "living databases"\nthat the fifth recommendation calls for.
\nDynamic meta-analysis, hallucination-free screening with humans in the loop,\nand continuous evidence ingestion are all on the table here, and the BMSF\nprovides the architecture within which these tools can interoperate. We're just\nbeginning a new project on this that also includes Education as well\nas conservation, which I'm very excited about!
\nThe most fun and somewhat far out thing is our ecology of the internet work, which tries to draw a connection\nbetween ecological resilience principles and the digital infrastructure that\nbiodiversity data depends on. This is a deliberately circular argument to see\nwhere the thoughts go: we need resilient digital infrastructure to monitor\nbiodiversity, and ecological theory can teach us how to build that\ninfrastructure.
\nThe upcoming Rewilding the Web workshop in Edinburgh will bring\ntogether ecologists, philosophers and technologists to push this further. I'm also working with\nCyrus Omar to figure out how to connect all these into a global live wiki, and Jon Ludlam is hacking on a prototype already!
\n\nThese two perspective papers represent the published output of the conversations we had at the NAS last summer, and my own agenda is now a bit clearer. I want to connect models like TESSERA and our Cambridge TAP to federated, auditable monitoring systems that can transform the flood of biodiversity data into credible evidence that's urgently needed to halt and reverse nature loss. Stay tuned for some ideas I have about venues where we can do this together!
A busy week on the socials as the paper Dave Scott, Justin Cormack and I wrote\nlooking back at the Docker adventure made the cover of the Communications of the CACM. Lots of really positive coverage about it online,\nand the print issue should be with you all this coming week!
\n ![\"%c\"](\"/images/cacm-docker-cover-1.webp\" "\\"Cover")
In TESSERA land I spent most of my week heads down on getting full Zarr streaming support. This has been extraordinarily successful; I now not only have a significant chunk of embeddings streaming over HTTP using Zarr, it also meant that I could build an entire classification and segmentation pipeline that runs entirely in my browser! I prototyped a user interface that can run Sadiq's solar farm CNN entirely in my browser using wasm and WebGPU. There's a nice roundup for Zarr visualisation options that greatly helped me put this together.
\n![\"%c\"](\"/images/tessera-zarr-stream-1.webp\" "\\"A")
Here's a screenshot to whet your appetite; I'll write a full blog about this shortly and post a hosted URL. The reason it's taking a bit longer is that Mark Elvers and I have been busy rearranging storage in our cluster; exposing all the embeddings as Zarr means transcoding 100s of terabytes of data, which is killing our poor internal network.
\nWhile working on this, I also found this overview of hierarchical JEPA to be very good.
\nThe OxCaml TESSERA pipeline continues to come online and Jon Ludlam also got odoc OxCaml docs building from my monorepo which is a HUGE help to developing all the code.
\nWhile preparing my wg2.8 I also needed an HTTP caching proxy that would permanently persist Zarr tiles on my laptop for demos, so I wrote a HTTP perma proxy over httpz/OxCaml as well which is working for me.\nAlso spent a bit of time reviewing OCaml relocatability in opam as that would be extremely useful for rapid development setup of oxmono.
\nI also researched Huggingface support for Zarr which is still not quite there but it seems to be of interest to HF.
\nRyan Gibb followed up on his FOSDEM talks extravaganza with a preprint on the package management calculus we've been working on for a while. Lots of discussions and online interest in places like HN and Lobsters. I thought one of the most telling things about how subtle this area is was from HN. Someone confidently pointed out that Rust has semver:
\n\n\nThe rust ecosystem standardised on semver. This means it is perfectly allowed to use 1.2 in place of 1.1. While you can specify upper bounds for the dependency ranges, that is extremely uncommon in practice. Instead the bounds are just "1.1 or newer semver compatible" etc.
\n
...but the reality that Ryan points out when you look at the crates registry is:
\n\n\nIn https://github.com/rust-lang/crates.io-index I count just under 7000 upper bounds on dependency ranges that aren't\njust semver in disguise (e.g. not ">=1.0.0, <2.0.0"):\n$ rg --no-filename -o '"req":"[^"]<[^"]"' . | grep -Ev '< ?=? ?([0-9]+(.0){0,2}|0.[0-9]+(.0)?)"' | wc -l\n6727\nSo it's definitely used. One person's non-breaking change is another's breaking change https://xkcd.com/1172/
\n
This repeats itself all over the place. Table 1 of the paper covers a lot of the edge cases that we had to categorise when working through all the package ecosystems.
\nThere was also a brief discussion on Lobsters about narrow waist effect in technology.
\nMy paper of the week is "Building Bridges: Safe Interactions with Foreign Languages through Omniglot" from OSDI25, which builds out a generic FFI with zero-copy. Seems very useful for OxCaml as well; I'm going to see Mae Milano at next week's WG2.8 and hopefully learn more!
\n ![\"%c\"](\"/images/omniglot-fig1.webp\" "\\"Building")
Next week I'm in Portugal at WG2.8, so less hacking than usual!
I am beyond excited to be on the cover of the CACM March issue with "A Decade of Docker Containers", coauthored with Dave Scott and Justin Cormack:
\n
\n\nFor the past decade, Docker has provided a robust solution for building,\nshipping, and sharing applications. But behind its simple "build and run"\nworkflow lie many years of complex technical challenges.\n-- A Decade of Docker Containers, Communications of the ACM, Mar 26
\n
Docker was such a whirlwind ride that we never got to write any academic papers about some of the technical systems magic that went into it. Today's article, along with the ICFP experience report from last year form a companion pair to delve into the tricks required to scale the system to millions of daily users.
\nWe cover the technical origins in Linux, the library VMM layers needed to hide Linux on macOS and Windows. And then we discuss where Docker is going next, with the giant AI coding wave making it incredibly important to sandbox agents running pretty much everywhere now.
\n\nThe video accompanying the article was recorded in my office by the wonderful Rosie Powell, with thanks to Pembroke College. And the pixel cover art of container ships that the CACM commissioned is fantastic!
\n![\"%rc\"](\"/images/cacm-docker-cover-5.webp\" "\\"The")
\nFirstly, a huge thank you to Solomon Hykes, the project founder and the person who invited us to join forces in the early days of Docker. We all holed up in a French farm and hacked like mad (photo from Solomon) and came up with the first iteration of Docker for Desktop in a few days!
We didn't actually realise that's what we'd call it back then. The project was originally codenamed Pinata and even had a CLI tool of the same name for quite a while! In order to get a feel for whether or not it would be popular, we took a leaf out Gmail's launch and send out limited invite codes. There was nothing to have been worried about as it took off fast (except the traditional HN disdain) with positive reviews.
\n\n\nDocker For Mac is a game changer. I’ve been able to cope with the previous tools but the experience has been rough to say the least.\n-- Docker For Mac Beta Review, Noah Zoschke, Apr 2016
\n
After the desktop beta came out in 2016, we also open sourced quite a few components, some of which are now features implemented into macOS and Windows. Some tricks like VPNKit are now adopted widely in other ecosystems, which is nice to see.
\nContainer management and cloud computing are obviously worth vast amounts of money now, but the giant whale and plush toys and crazy antics at Dockercons are what I'll remember most fondly. Throughout all the ups and downs, Docker's been (I strongly feel) a strong force for openness in preventing any single entity capturing the full workflow of how we manage software, and therefore contributing to building a vibrant and diverse ecosystem.
\nToday, it's still entirely possible for a small player to quite simply spin up their own selfhosted infrastructure and interoperate with the behemoths. That's important; heck I use swarm mode on my own ##selfhosting to this day!
\nWe're seeing a big change in open source community building happening this year. The vibe coding onslaught is calling into question how we'll make open source friends in the future, and it looks like we're falling back to reputation networks for contributors. I hope that we see more Docker-style communities spring up than boring corporate driven artificial ecosystems!
\n![\"%rc\"](\"/images/cacm-docker-cover-3.webp\" "\\"Stealing")
\nThe other great thing to see in recent years is the new generation of maintainers hacking on adjacent technologies among my colleagues and students. Mark Elvers is advancing Windows support with HCS, Ryan Gibb just uploaded his latest work on formalising dependency management and Patrick Ferris has been hacking on shell integrated provenance. And "old" maintainers like Thomas Gazagnaire who I cofounded UnikS with are taking Docker and OCaml into space!
![\"%rc\"](\"/images/cacm-docker-cover-2.webp\" "\\"At")
\nCombining these advances with agentic coding results in radically different coding methodologies, but using the same lower level interfaces that Docker's built on today. Evolution is happening fast, and more accessible than ever thanks to Docker's open source roots.
Here's to the coming century of containerisation; enjoy reading the article and do let me know if you have any comments or queries!
\n![\"%c\"](\"/images/cacm-docker-cover-4.webp\" "\\"Remembering")
Most of the week was taken up by hopping over to New Delhi to host a TESSERA hackathon and also to attend the AI Impact Summit. I redeyed back to host Shriram Krishnamurthi in Cambridge as he does his UK tour.
\nThe best news of the week was that the TESSERA paper got\naccepted into CVPR 2026, out of a whopping 16000+\n(!) submissions. This has been a giant amount of work for the whole team, but\nparticular props to lead author and PhD student Frank Feng who has lead the whole\neffort with perseverance and a big smile the whole time!
\nIt looks like CVPR is in Denver right before PLDI in Boulder (where I have an\nOxCaml tutorial to help hold) so I guess a chunk of my June will be spent in\nColorado this year.
\nI also spent some time porting Mark Elvers OCaml Zarr implementation over to\nOxCaml, and also started adding Zarr zone support\nto geotessera so we can start\nconverting the registry over.
\nRobin Message and Sadiq Jaffer have restarted the literature downloader, and Robin has\nbeen manually classifying DOI prefixes into a two-level tree so we can easily\ndispatch download logic on a per-publisher basis (we have individual agreements\nvia the University library with many publishers).
\nI'm surprised that DOIs are not a two-level tree to start with, as now with no\ncentral source of detailed DOI prefix metadata if a journal is sold to another\npublisher (as just happened with JFP), you either\nhave to forward a portion of your DOI space or continue to resolve old journal\narticle DOIs forever.
\nI also started migrating a lot of datasets over to our new Ceph cluster,\nincluding full syncs of GBIF, OpenAlex, and Crossref. This should set us up\nnicely for Shane's dashboard using locally hosted\ndatabase for fast queries. On the queue once the storage settles is also\niNaturalist open data, and\nto mirror the TESSERA embeddings to our Ceph so that local Cambridge users\nsuch as Andres Zuñiga-Gonzalez can access them more easily to do global analyses directly without a full\nlocal copy.
\nI also had a really fun discussion with Jon Sterling over High Table dinner\nat Pembroke about whether it was a good idea for me to get into Lean to start\nto specify the semantics of URI resolution.
\nJon published a design for canonical URLs in Forester last year,\nand as I'm getting slightly obsessed with managing Atom, RSS and JSONFeeds at the moment (the /network\nview above is powered by this) this seems relevant to both that and also the literature downloader. In return for Jon's help, I will happily\ncode up an OCaml monorepo script for him!
\n![\"%rc\"](\"/images/week26n8-3.webp\" "\\"Showing")
\nShriram passed through Cambridge on Friday on his UK lecture tour, so I leapt at the chance to host him after leaping off the redeye from India. I last chatted to Shriram at ICFP over the summer, and this time we got hear him speak about The Human Factors of Formal Methods in the Logic & Semantics seminar here in the CL.
The talk was fantastic and I can't recommend watching it enough; I have so many\npapers to follow up on now:
\nShriram used the Stroop effect in his talk, which naturally led to Neel Krishnaswami and me wondering if LLMs could learn the Stroop effect too! I found one paper on this topic:
\n\n\nMoreover, as in humans, age is a key determinant of cognitive decline: “older” chatbots, like older patients, tend to perform worse on the MoCA test. These findings challenge the assumption that artificial intelligence will soon replace human doctors, as the cognitive impairment evident in leading chatbots may affect their reliability in medical diagnostics and undermine patients’ confidence.\n-- Age against the machine, 2024
\n
I find the analogy between human age and 'model age' a bit incongruous, since of course models dont age -- there are improved training regimes. So the basic takeaway is that human cognitive impairment is decreasing as frontier LLMs advance.
\nWhen chatting about how to teach our agents OxCaml better, Shriram pointed me to his 2017 paper on Teaching Programming Languages by Experimental and Adversarial Thinking:
\n\n\nIts essence is to view programming language learning as a natural science\nactivity, where students probe languages experimentally to understand both\nthe normal and extreme behaviors of their features. [...] The approach is\nmodular (with minimal dependencies), incremental (it can be introduced slowly\ninto existing classes), interoperable (it does not need to push out other,\nexisting methods), and complementary (since it introduces a new mode of\nthinking).\n-- J. Pombrio et al 2017
\n
There's obvious parallels here to how the OCaml to OxCaml translation process\nworks, whereby we typically add in mode annotations once the OCaml version is\nworking. The only practical twist is that shifting to OxCaml also requires\nporting code to Base/Core as well, since the stdlib doesn't have mode\nannotations.
\n![\"%rc\"](\"/images/week26n8-4.webp\" "\\"Ran")
![\"%rc\"](\"/images/week26n8-1.webp\" "\\"Most")
\nExtremely random feature: I added finger support to my website, so you can just do finger @anil.recoil.org (it is installed by default on macOS) to see my latest weekly.
I need to get TESSERA Zarr in shape. This will fix so many infrastructure\nissues with using the embeddings! I'm also going to vibe code up a cool\nwebsite for the project, using the feed aggregation logic from my own website\nand these Threejs Claude skills\nI just stumbled across.
\nI'm also off to WG2.8 the week after, so I need\nto figure out what functional programming goodness I will present there!
\n![\"%c\"](\"/images/week26n8-2.webp\" "\\"Thanks")
Very little sleep this week as I hopped over to India's AI Impact Summit for a slew of events to followup my earlier meetings at OpenUK and the Turing. The Indian government knocked it out of the park with the first summit held in the majority world: there were over 200,000 people registered and the keynote for Sarvam AI's launch had more people attending than the entire French AI summit last year! The venue was the enormous Bharat Mandapam, which was opened just a couple of years ago in the G20.
\nI found the summit a fantastic networking event, although unlikely to result in any significant policy shifts aside from establishing India as a serious target region for growth. I was lucky enough to meet Yann LeCun and get a bunch of technical insights into TESSERA from him, which was my personal highlight!
\n![\"%c\"](\"/images/aisummit-gen-2.webp\" "\\"The")
![\"%rc\"](\"/images/aisummit-gen-3.webp\" "\\"The")
\n![\"%rc\"](\"/images/aisummit-gen-5.webp\" "\\"The")
\nThrough a series of misadventures not suitable for the public web, I ended up\nin the rooftop VIP section of the summit and got to take in the breathtaking\nviews of the whole crowd. The size of the conference centre is incredible, with a\nset of conference halls all connected by a lovely garden through to the gates\ninto Delhi proper.\nThe main expo I visited was the "world hall", where every country had an\nexhibition (some projects were more credible than\nothers).
I didn't see any really mindblowing demos in the stalls, but perhaps that's\nbecause my baselines have shifted in the last year. What was remarkable\nwas the breadth of solutions on offer: pretty much every aspect of Indian\nsociety seemed to be covered, from urban living to rural food security.
\nThere was little 'eat our own dogfood' on display,\nas the security lines were long and manual. The payment\nsystem was actually really cool: UPI allows\na vendor to display a QRCode to receive payment with 0% overhead. The user scans the QRCode on their\nmobile, pays online, and then shows the proof to the vendor. The vendor doesn't need to have any electronic\nequipment at all. This had nothing to do with AI, but reminded me strongly of our work on spotcodes over two decades ago!
\n![\"%rc\"](\"/images/aisummit-gen-1.webp\" "\\"Most")
\n![\"%rc\"](\"/images/ai-hackathon-2.webp\")
\nWith 200,000 people pouring through the expo, it was basically impossible to have any coordinated meetings as even getting from one end to the other took ages. However, this meant that a lot of attendees vibe coded up alternative (and more usable) interfaces, and were showing off QRCodes which I scanned in order to access their versions. One enterprising chap had a UPI payment link to an MCP endpoint that I could point my Claw to help figure out where to go next; I didn't buy this, but I appreciated the hustle!
![\"%c\"](\"/images/aisummit-gen-4.webp\" "\\"The")
![\"%rc\"](\"/images/aisummit-aia-1.webp\" "\\"Yann")
\nOne of the evening events was hosted by the good folk at the AI Alliance, a not-for-profit aiming to boost open models.\nThe guest of honour Yann LeCun, who turns out to be the nicest person! He gave a layman's summary to the audience about why LLMs are an evolutionary dead end, and then started talking about how self-supervised methods such as Barlow Twins and JEPA are the future. Barlow Twins are, of course, the exact training mechanism we've been using to train TESSERA, and so I bounded up to him to discuss it in more detail!
Yann had made an argument on stage that the only fair way to train language models that are open is for every country to contribute a large corpus of country-specific data (not necessarily open), and to combine these at the global level via one level of federated learning into a 'united language model'.
\nWhile this may work ok for LLMs, I thought it was even more perfect for satellite data! We already have a base set of public data, in the form of Landsat and Sentinel 1/2/3. Meanwhile, many countries have their own geostationary task satellites hovering over them, often with much higher resolutions and interesting instruments. I even heard rumours that India has commissioned a Lidar survey of the entire country, but this may be a future project as all I could find concretely is one of major river systems.
\nSo while open frontier LLMs may be a lost cause in the short term, it strikes me as a real opportunity that TESSERA may be the perfect way to trial Yann's idea of a global training cooperative. The incentives are all there, and geospatial foundation modeling seems to be maturing rapidly.
\n![\"%c\"](\"/images/aisummit-aia-2.webp\" "\\"With")
![\"%rc\"](\"/images/ai-hackathon-1.webp\")
\nI skipped the 'government day' of the summit (with all the glitzy tech CEO talks) in order to satisfy demand for TESSERA talks. I advertised a hackathon with OpenUK on the day before the summit, and we had a full house of signups by the end of the day! I posted a trip report for this separately.
There was so much discussion at the hackathon that I trotted along to the IIT-Delhi campus the morning after to give a detailed talk on the bigger picture of TESSERA (similar to the talk at ARIA last week). The audience was highly engaged and I went well over time answering questions. A number of students were interested in followup opportunities to work in this area, and I pointed them to KC Sivaramakrishnan and his FP Launchpad which is taking off in April. We've got TESSERA and OCaml playing well together now, so there's a really fun opportunity to combine functional programming with planetary computing now!
\nBoth speeches were charming, and it was good to see the emphasis on openness. Kanishka Narayan made a wry observation that Britain might not lead on raw engineering resource, but it does have 'the best technical taste', which I thought was quite an apt claim!
\nMost of the conversations I had here were all about landuse and datacenter growth. There seems to be massive investment within India for datacentre capacity, so questions of water usage and landuse are obvious barriers. I'm looking forward to working with the CoRE Stack team to help map out some of these challenges throughout India.
\nThere was also a lot of interest in datacentres in space, so I took the opportunity to explain what we're doing with our startup Parsimoni lead by Thomas Gazagnaire. The idea of having a multi-tenant 'Docker in space' was received well by everyone I mentioned it to, and Thomas has been finding a lot of similarity to our earlier unikernel work as he builds SpaceOS out in California.
\nThe summit declaration that came out today is remarkable in actually being signed by the US, UK and China unlike last time.\nSome snippets of interest to me from the statement are:
\n\n\nWe take note of the voluntary and collaborative International Network of AI for Science Institutions as a platform to connect scientific communities and pool AI research capabilities across regions among participating institutions, in order to accelerate the impactful adoption of AI.\n[...]
\nWhile encouraging international collaboration on meaningful skilling and reskilling AI initiatives, we take note of the voluntary guiding principles for reskilling in the age of AI and the playbook on AI workforce development, which would support participants in preparation for a future AI driven economy.\n[...]
\nWe take note of the Global AI Impact Commons as a voluntary initiative that provides a practical platform to encourage and enable the adoption, replication, and scale-up of successful AI use cases across regions.\n-- AI Impact Summit Declaration, Feb 2026
\n
The word "safety" is notably missing; it's all about rapid progress and equitable access now. This year will not be about whether or not AI adoption should happen; it's now a race to defend ourselves against AI poisoning and whether we take the red pill or the blue pill and embrace an open future. It is for this reason I'm very grateful to OpenUK for all their work on helping make sure the UK takes the red open-source pill. It'll be a harder road, but a worthwhile one.
\nAn excellent recap of the outcomes of the summit can be found in this hour long segment on Indian TV with none other than my colleague Neil Lawrence!
\n\nI also contributed to the very comprehensive OpenUK AI Openness Summit Report which is available here and very comprehensive. I'm not entirely sure what happened to the ATI report I contributed to in November; I suspect it's been filed away Indiana Jones style in some vast document repository underground...
\nThanks for the hospitality New Delhi! It was an exhilarating whirlwind to be at the summit. Well done Abhishek Singh and other organisers.
\n(Updated 23rd Feb 2026 with a link to the OpenUK report. March 6th 2026 with a typo from Sam Reynolds.)
We held the first TESSERA hackathon over the Indian AI Impact summit in Delhi today, thanks to sponsorship\nfrom OpenUK. Despite only announcing it the weekend\nbefore, we had a full house of active participants (some of whom came up all\nthe way from Bangalore for the day to attend!).
\nThis is the first TESSERA hackathon I've taken part in outside the UK (where\nwe've had several really fun ones with the IUCN and\nUNEP-WCMC). It was especially packed because of the\nmassive buzz in Delhi with the summit going on; every hotel in the city was\nbooked out and traffic was gridlocked as 200,000 people descended. We held the\nhackathon in the lovely India International Centre\ncampus in the centre of Delhi which was a peaceful locale amidst the chaos\noutside.
\n![\"%c\"](\"/images/tessera-hackathon-3.webp\" "\\"The")
![\"%rc\"](\"/images/tessera-hackathon-1.webp\" "\\"The")
\nI organised the hackathon with Aadi Seth from IIT-Delhi, who leads the CoRE Stack. We first learnt about how the CoRE stack works; my notes follow:
![\"%rc\"](\"/images/tessera-hackathon-4.webp\" "\\"I")
\nTo get people going in TESSERA I first of all explained the basic ideas behind the model, on the same lines that Srinivasan Keshav and David Coomes did in our Ecosystem Resilience Workshop. The two main 'getting started' ways right now are:
app.ipynb in VSCode or Jupyter (as you prefer, but I tend to use VSCode as it works better with uv). Then pick an area of interest and try out the labeling workflow directly on your laptop.docker compose up.While this all went right, the wifi at the venue was pretty slow, so downloading embeddings all the way from Cambridge was pretty slow. I think there are two avenues we need to explore quickly:
\nAfter this, we had a brief period to try out some labeling, but the conversations about exactly what we hack on together will continue on our EEG Zulip channel! There was a lot of discussion about how to store labels, as many of the groups there had ground truth information that they weren't quite sure how to share. I think it would be very valuable to have a general geospatial labeling service that could then export to various specific services like OpenStreetMap or for downstream training.
\nThanks OpenUK and the Impact Summit for facilitating this, and for IIT-Delhi and the other participants for the fascinating discussions and hacking!
\n![\"%c\"](\"/images/tessera-hackathon-2.webp\" "\\"The")
\n![\"%c\"](\"/images/tessera-hackathon-5.webp\" "\\"The")
The week was dominated by having to sort out yet another 300TB of SSDs to grow our Ceph cluster, as the TESSERA embeddings being generated on Vultr are even more optimised and being spit out at a rate of 4TB per day. But I did get around to playing with Lego as well!
\nI released a minor release of geotessera 0.7.x to speed up the startup time of the library, now that we have so many embeddings generated (doi). Srinivasan Keshav has been knocking up a beautiful TESSERA Embeddings Explorer which is now Dockerised and so is running on my laptop, perfect for demos on the fly! He noticed the TEE startup time was slow and reported it, and I undid a hack I put in to support Windows better by picking a simpler way to manage coordinate transforms.
\nAlso saw an excellent post about what makes TESSERA a blueprint for other EO models from Rakshith Santish. I'm glad he enjoyed the ablations!
\nMark Elvers and I are getting more comfortable with using Ceph to manage all the TESSERA embeddings, and Malcolm Scott helped us to wire up a bunch of old machines donated by Jane Street that we can use to distribute them. We now have multiple large storage blobs that we're consolidating, ranging from TrueNAS mirrored storage (from the 4C days) over to a backup Ceph cluster at Scaleway (kindly sponsored by Tarides) and also the OCaml infrastructure and documentation generation that sucks up our maintenance time.
\nMeanwhile, use of oxmono for our OxCaml infrastructure is working extremely well, and I'm solely using that with worktrees and branches to deploy services now. The OxCaml compiler is rock solid and with a monorepo, managing packages is a breeze. Jon Ludlam is also using it to fix OxCaml doc generation which will be extremely useful for agentic coding (since the agents can read the generated docs to determine if the interfaces are 'good' or contain lots of hidden modules and other bad practise).
\n![\"%rc\"](\"/images/tap-lego-1.webp\")
\nThe only real downside with the monorepo is that a dune exec takes about 3s to initialise on my laptop from scanning all the dune files. I'll investigate some ways to speed up this, as the Dune cache doesn't help much until all the dune files have been parsed.
It was also good to see other OCaml projects progressing: Mark is getting more familiar with OxCaml performance and the OCaml TIFF library in Outreachy is going well. I hope to use this soon in TESSERA-oxcaml...
\n![\"%rc\"](\"/images/tap-lego-2.webp\")
\nWe're just finalising a grant that's been awarded to do the exciting next phase of the CE "Cambridge Evidence TAP" project by generalising it to other fields (such as education, public health and climate adaptation). While waiting for the grant paperwork to finish, my colleague from Education Jenny Gibson had the brainwave of inviting Gina Gomez de la Cuesta from Play Included to come along to facilitate a brainstorming session about what we'll work on when the project starts.
![\"%rc\"](\"/images/tap-lego-3.webp\")
\nThis involved getting our friends from across the university and CSaP together. It was me and Sadiq Jaffer from Computer Science. Lynn Dicks and Sam Reynolds from Conservation, Jenny Gibson from education, Mélanie Gréaux from the WHO, Rob Doubleday and Nicola Buckley from the Judge, and Alex Marcoci from CSER. I've got to say that the use of Lego to break the ice among so many fields, and also to help us form thoughts about a very complex and nuanced area was just brilliant.
![\"%rc\"](\"/images/tap-lego-5.webp\")
\nAnd after a long week, it was just nice to kick back and play with LEGO. I'm really looking forward to doing more work with our friends from other departments: working with conservation has gotten me out and about finding hedgehogs, so now learning about play in education, development and learning will hopefully help me understand how to be a better teacher! I also highly recommend contacting Gina at Play Included if you would like to try this yourself for one of your own projects.
After last weeks talk at ARIA, I enjoyed hosting the founders of a new focussed research organisation called 'Echo Labs' funded by ARIA who are doing very ambitious things with AI and biodiversity. More on what they're up to after they officially launch, but I was hugely impressed with their drive and focus to deliver near term impact. I hope we will continue to build collaboration with them as part of our Centre for Landscape Regeneration.
\n![\"%rc\"](\"/images/cci-valentines.webp\")
\nWhile they were visiting the CCI, there was also a dramatic unveiling of the new CCI logo, so everyone poured into the coffee room. Well done on a successful launch; the new logo is very practical and inclusive and pretty and I'll be using it everywhere once I get my mittens on the high res versions!
Shane Weisz continued his storming first year PhD by giving a fantastic EEG seminar on his PhD work to date on speeding up RED List assessment. He's off to the inaugural Conservation Technology Conference in Peru next week to speak about this work there as well, which should be most exciting (and hopefully filled with much birding). We decided to codename the overall dashboard effort as "Enki".
\n\nOn the Enki storage topic, I've been working on syncing GBIF locally as Parquet files (about 9TB) so we don't need to hammer their API. But I noticed that the AWS Open Data hosting hadn't been updated for six months, but a single Bluesky post was enough to get GBIF's attention and they fixed it within days. Props to them for being so responsive, but it's also interesting that almost noone else seems to locally copy the data regularly.
\nSimon Peyton Jones pointed me to his friend Julian Allwood's new book Promise the Earth: A Safe Climate in Good Faith which I picked up from the CUP bookstore. It's an intriguing book in that it's written by an engineer and a theologian, and makes t\nhe argument that restraint is the only way forward.
\n\n\nThis brilliant book makes the case that rational self-interest alone will not\nbring about the radical changes to the world economy required to protect our\nchildren from the climate breakdown that is coming. The role of faith and\ncompassion has historically played a major role in the way that people\ncooperate and plan for the future; this persuasive book makes the case that\nit is needed more than ever.\n-- Review by Professor Mark Miodownik MBE FREng, 2025
\n
Jon Ludlam finished off undergraduate exam questions for Foundations of Computer Science and I am going to do a review on them -- the only teaching thing I am doing this sabbatical year.
\nOn Monday, I'm jetting off to the India AI summit in\nDelhi and to see some relatives for the week.\nI'll be back in Cambridge on Friday to host Shriram Krishnamurthi who is passing through on sabbatical. Very excited to see him again to continue our ICFP conversations about teaching!
\nAlso the third outing of PROPL has been accepted to PLDI 2026 this summer, so I'll be helping the new chair Cyrus Omar prepare the PC invitations and roundup the deadlines for submissions.
\nSome fun links:
\nchasewnorton, zero-day vibing like mad with 100 PRs on a codebase no living being understands.This week started off with my conducting the PhD viva for Madeline Lisaius, which was a very enjoyable 4.5 hours of discussion with her and external examiner Kevin Tansey who came up from Leicester for the day. I can't comment on the result until it is officially ratified by the examiners board here, but you can judge for yourself from the expressions below how it went! Maddy is off to Paris next for a new role working on CLAY2, and we will miss her here in Cambridge!
\n![\"%c\"](\"/images/maddy-viva-1.webp\" "\\"A")
After that I went to prepare for a big talk at ARIA after being invited by Ilan Gur from meeting him a few months ago. I went down with Srinivasan Keshav and Sadiq Jaffer and I gave the talk while Keshav demoed his increasingly brilliant TESSERA Embeddings Explorer and Sadiq showed off his solar panel CNNs and predicting what would grow in the newest season of Clarkson's Farm (seriously, it works great).
\n![\"%rc\"](\"/images/aria-visit-2.webp\" "\\"Sadiq")
\nMany of the program directors were present, and I appreciated the chance to throw out some of the bleeding edge stuff we've being doing, and we got a lot of useful feedback and also excitement about usecases. I gotta say I’m really digging the general sense of optimism and sense of momentum from all the ARIA staff I talk to. There’s a "can do" attitude that's sometimes been missing from the general discourse in the UK in recent years, and I greatly enjoyed all the discussions across programs.
![\"%rc\"](\"/images/aria-visit-1.webp\")
\nIt was also delightful to see top functional programmer Kathleen Fisher there briefly (who is also their new CEO!. On the train back, I also took the opportunity to refresh Kathleen's wikipedia article.
Nature sent along a reporter to the Conservation evidence conference conference, and she published an excellent piece in Nature on how biodiversity has an evidence problem and approaches to fix it (the latter being most important!).
\n\n\nWilliam Sutherland, a conservation scientist at the University of Cambridge\nwho leads the Conservation Evidence project, says that his team is now\ndeploying artificial intelligence to improve the speed and the thoroughness\nof the project's process. The ambition is for users to be able to interrogate\nthe data set with a specially designed 'conservation chatbot', meaning\npractical questions would be answered with a narrative summary and links\nprovided to sources of evidence. The data set would constantly be updated to\ntake account of new studies and retractions, with humans overseeing the\nprocess. The concept is described in a preprint that was published last year.\n-- Biodiversity conservation has an evidence problem — it’s time to fix it, 2025
\n
As another followup to the conference, I also got invited by Stephanie Day to give evidence to the the Parliamentary Office of Science and Technology (POST). POST is an office of both Houses of Parliament, which provides independent and balanced analysis of research evidence related to public policy issues. We had a lively roundtable with a breadth of experts discussing the pros and cons of evidence-based approaches and data accessibility for both open models and also conventional publishing, and I'm sure this will make for a good briefing report when completed.
\nI've got an upcoming article (on the front cover!) of CACM, so Dave Scott and I\nwere actually video interviewed in my office by a wonderful videographer Rosie\nPowell who did the trek up three floors\nof stairs with all her kit! More on the article itself when it's out in a few\nweeks. It was a set of good news for papers all around, with another article\naccepted to Nature Communications as well after 5 (!) rounds of reviews and\nprobably over 100 pages of responses.
\n![\"%c\"](\"/images/cacm-interview-1.webp\" "\\"Dave")
The week would also not be complete with doing more work on the TESSERA storage array. This time we were busy juggling machines to cope with the influx of embeddings from Vultr which are filling up storage at an alarming rate. There was a bit of drama last month when one of our machines failed; while we didn't lose anything, it took a long time to sync data back. Now, we're building a much more resilient Ceph cluster to avoid a dependency on any one host.
\nI also managed to do a lot of hacking on oxmono, including porting compression libraries like Brotli, Zstd and Snappy over to OxCaml in preparation for my pure OCaml Parquet implementation. OxCaml is pretty solid so far, although it's inscrutable enough due to being a moving target that Claude's a necessity to work through all the annotated interfaces.
\nI've got a battery of meetings and followups from ARIA, including a bunch of biodiversity related meetings and also a storage array to fix up!\tI'm in Cambridge, although the wet and windy weather isn't inspiring to go out and do my running routine...
\nFun links:
\nI helped CE host a gathering of evidence champions for biodiversity at Pembroke in late January. The first day was a remarkable closed meeting to sit in on, with the science leads of all five UK nature conservation bodies and the JNCC present! Sadiq Jaffer and I got to present TESSERA to them and discuss more broadly how we could apply machine learning to accelerate nature recovery and preservation across the UK. Read CE's blog series (parts 1, 2, 3) with more details from me next.
\n![\"%c\"](\"/images/ce-champions-1.webp\" "\\"The")
The first day's meeting of chief scientists was convened by Julia P.G. Jones who recently became co-chair of the new Chief Scientists Group. Julia wrote a fantastic roundup blog post about the goings on:
\n\n\nThe presentation resulted in a fascinating discussion about how such\ninnovations may influence the work of the SNCBs in future. The next step is\nto train and validate downstream models combining existing ground-based data\nwith the embeddings. The hope is that this would allow us to interpolate\ndatasets between sampled locations and then ask larger-scale questions about\necosystem change and the impact of interventions for a wider range of\noutcomes than is currently possible. Such understanding is crucial for so\nmany applications.\n-- Could geospatial foundation models help improve conservation effectiveness?
\n
![\"%rc\"](\"/images/ce-champions-2.webp\" "\\"Julia")
It was a real honour to be able to directly discuss our research with this\ngroup, with me personally finally getting to meet Dr Sara McGuckin, from the\nNorthern Ireland Environment Agency (NIEA); just round the corner from where I\nwent to school! I showed her Patrick Ferris's work on mapping NI\ntrees as well and really want to\nget around to doing a TESSERA version soon.
\nThe second day was held in the new Pembroke Auditorium, which was as always\na gorgeous venue. The district heating system failed on the day, and it was\nunfortunately close to freezing. Luckily the hardy conservation community didn't\ncomplain even once; this was just another day in the outdoors for them. Bill even\nrallied everyone with his famous bell!
\n\nThe CE blog published a comprehensive roundup of all the talks with a summary of takeaways for funders. All of the talk recordings are available on CE's Youtube channel and I also mirrored them ad-free on our Watch EEG channel if you are more Fediverse oriented.
\nI'm going to highlight three talks here, but only because they're relevant when viewed together -- if you're interested in this then all of the talks were of high quality and worth your time to watch!
\nBill's talks are always a must-watch, and this one combined nuggets of evidentiary falsification (bird nest box designs turn out to be entirely suboptimal) vs a basic optimism that evidence based conservation is steadily becoming the norm rather than the exception.
\n\nJulia's theme was on choosing the right design for the vastly growing quantities of biodiversity data. She focussed on remembering that "assumptions are everything" and the importance of upfront design combined with with post-project evaluation to avoid a causal soup of data that's difficult to reconnect.
\n\nAnd last and least, I got to talk about some of our recent advances in using AI for conservation, such as TESSERA and the Conservation Evidence TAP. Before that though, I opened the talk with my thoughts on the basic choice facing conservation this year. AI is here to stay, for good or for evil, and the entire field has to cope really quickly. The easy (blue pill) would be to simply use off-the-shelf chatbots for decision-making, but be unable to reproduce or trace the provenance of outputs from the black boxes. The harder (red pill) is to engineer open and traceable processing pipelines which can filter out AI poison and give sovereign decision making capability to governments. And of course, all the work we're doing is as open as we can make it. Watch the talk to learn more!
\n\nThe conference was a very intense two days indeed, but it was brilliant to see the Auditorium so full of enthusiasm and desire for effective action.
\n![\"%c\"](\"/images/ce-champions-3.webp\" "\\"The")
\n![\"%c\"](\"/images/ce-champions-4.webp\" "\\"My")
\n![\"%c\"](\"/images/ce-champions-5.webp\" "\\"Did")
Since helping with the OxCaml tutorial last year at ICFP,\nI've been chomping at the bit to use it for real in our research infrastructure\nfor planetary computing to manage the petabytes of TESSERA embeddings we've been generating.
\nThe reason for my eagerness is that OxCaml has a number of language extensions\nthat give giant leaps in performance for systems-oriented programs, while\nretaining the familiar OCaml functional style of programming. And unlike Rust,\nthere's a garbage collector available for 'normal' code. I am also deeply sick\nand tired of maintaining large Python scripts recently, and crave the modularity and\ntype safety of OCaml.
\nThe traditional way I learn a new technology is by replacing my website infrastructure with the latest hotness. I switched my live site\nover to building with OxCaml last year, but never got around to deeply\nintegrating the new extensions. Therefore, what I'll talk about next is a new\nwebserver I've been building called\nhttpz\nwhich goes all in on performance in OCaml!
\n(Many thanks to Chris Casinghino, Max Slater, Richard Eisenberg, Yaron Minsky, Mark Shinwell, David Allsopp and the rest of the Jane Street\ntools and compilers team for answering many questions while I got started on all this!)
\nhttpz is a high-performance HTTP/1.1 parser that aims to have no major heap allocation, and very minimal minor heap allocation, by using OxCaml's unboxed types and local allocations.
\nWhy is this useful? It means that the entire lifetime of an HTTP connection\ncan be handled in the callstack alone, so freeing up a connection is just a\nmatter of returning from the function that handles it. In the steady state, a\nwebserver would have almost no garbage collector activity. When combined with\ndirect style effects, it can also be written without\nlooking like callback soup!
\nI decided to specialise this library for HTTP/1.1 for now, and so settled on\nthe input being a simple 32KB bytes value. This represents an HTTP request with\nthe header portion (HTTP body handling is relatively straightforward for POST\nrequests, and not covered in this post).
\nGiven an input buffer like this, what can we do with OxCaml vs vanilla OCaml\nto make this go fast?
\nThe first port of call is to figure out the core types we're going to use for our\nparser. If you need to get familiar with OCaml's upstream memory representation then\nhead over to Real World OCaml.
\nIn my usual OCaml code, I use libraries like cstruct\nthat I originally wrote back in 2012 to manage non-copying views into bytes buffers. Cstruct\ndefines a record that has four words (the box, and three words for the fields):
\ntype buffer = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t\ntype Cstruct.t = private {\n buffer: buffer;\n off : int;\n len : int;\n}\nThe idea is to use the record to get narrow views into a larger buffer, and that these\nsmall views can just live on the minor heap of the runtime which is fast to collect.\nOxCaml advances this by providing unboxed versions of small\nnumbers\nthat live in registers or on the stack, via a new syntax int16#.
Instead of Bigarrays, we're now going to switch to use bytes instead, but the\nbasic idea is the same. Since httpz's buffer is a max of 32KB, 16-bit integers\nalso suffice for all positions and lengths!
type Httpz.t = #{ off : int16# ; len : int16# }\nThere are actually two new features here: the first is that records can be unboxed with the #{}\nsyntax, and the contents themselves are of a smaller width. Let's have a closer look\nat the difference between the Cstruct boxed version and this new OxCaml one:
My first port-of-call is usually to use utop interactively to poke around\nusing the Obj module. This isn't quite so easy in OxCaml since the unboxed\nrecords use a special layout:
# type t = #{ off : int16# ; len : int16# };;\ntype t = #{ off : int16#; len : int16#; }\n\n# let x = #{ off=#1S; len=#2S };;\nval x : t = #{off = <abstr>; len = <abstr>}\n\n# Obj.repr x;;\nError: This expression has type t but an expression was expected of type\n ('a : value)\n The layout of t is bits16 & bits16\n because of the definition of t at line 1, characters 0-41.\n But the layout of t must be a sublayout of value.\n\nThat failed, but it did reveal that we have this intriguing int16 pair layout\ninstead of the normal OCaml flat value representation! Let's use the compiler\nto figure this out...
\nI next built a small\ntest program and inspected the lambda intermediate language from the compiler. To avoid dependencies, I just bound the raw compiler internals directly by checking out the oxcaml source code.
\nexternal add_int16 : int16# -> int16# -> int16# = "%int16#_add"\nexternal int16_to_int : int16# -> int = "%int_of_int16#"\n\ntype span = #{ off : int16#; len : int16# }\n\nlet[@inline never] add_spans (x : span) (y : span) : span =\n #{ off = add_int16 x.#off y.#off; len = add_int16 x.#len y.#len }\n\nlet () =\n let x = Sys.opaque_identity #{ off = #1S; len = #2S } in\n let y = Sys.opaque_identity #{ off = #100S; len = #200S } in\n let z = add_spans x y in\n Printf.printf "off=%d len=%d\\n" (int16_to_int z.#off) (int16_to_int z.#len)\nThis introduces enough compiler optimisation barriers such that\nthe addition is not optimised away at compile time. We can compile this\nwith ocaml -dlambda src.ml and see the intermediate form after type checking:
(let\n (add_spans/290 =\n (function {nlocal = 0} x/292[#(int16, int16)] y/293[#(int16, int16)]\n never_inline : #(int16, int16)\n (funct-body add_spans ./x.ml(6)<ghost>:196-294\n (before add_spans ./x.ml(7):229-294\n (make_unboxed_product #(int16, int16)\n (%int16#_add (unboxed_product_field 0 #(int16, int16) x/292)\n (unboxed_product_field 0 #(int16, int16) y/293))\n (%int16#_add (unboxed_product_field 1 #(int16, int16) x/292)\n (unboxed_product_field 1 #(int16, int16) y/293)))))))\nYou can see the unboxing propagating nicely here through the intermediate code!
\nThe next step is to verify what this looks like when compiled as optimised native\ncode. I used ocamlopt -O3 -S on my arm64 machine which emits the assembly code\nafter all the compiler passes, and found:
In the entry point:\n orr x0, xzr, #1 ; x.#off = 1\n orr x1, xzr, #2 ; x.#len = 2\n movz x2, #100, lsl #0 ; y.#off = 100\n movz x3, #200, lsl #0 ; y.#len = 200\n bl _camlX__add_spans_0_1_code\n\n_camlX__add_spans_0_1_code:\n add x1, x1, x3 ; len: x.#len + y.#len\n sbfm x1, x1, #0, #15 ; sign-extend to 16 bits (int16# semantics)\n add x0, x0, x2 ; off: x.#off + y.#off\n sbfm x0, x0, #0, #15 ; sign-extend to 16 bits\n ret\n\nWe can see from the assembly that there's no boxing, and no heap allocations,\nand the sbfm instruction maintains\nthe 16-bit semantics via sign extension.
\nLet's double check that the normal boxed OCaml does do more work and that isn't\njust the flambda2 compiler doing its magic. Here's a boxed version of the benchmark using\nplain OCaml:
\ntype span = { off : int; len : int }\n\nlet[@inline never] add_spans (x : span) (y : span) : span =\n { off = x.off + y.off; len = x.len + y.len }\n\nlet () =\n let x = Sys.opaque_identity { off = 1; len = 2 } in\n let y = Sys.opaque_identity { off = 100; len = 200 } in\n let z = add_spans x y in\n Printf.printf "off=%d len=%d\\n" z.off z.len\nCompiling this boxed version with ocamlopt -O3 -S and looking at the assembly shows\nmuch more minor heap activity:
_camlY__add_spans_0_1_code:\n sub sp, sp, #16\n str x30, [sp, #8]\n mov x2, x0\n ldr x16, [x28, #0] ; load young_limit\n sub x27, x27, #24 ; bump allocator: reserve 24 bytes (3 words)\n cmp x27, x16 ; check if GC needed\n b.cc L114 ; branch to GC if out of space\n L113:\n add x0, x27, #8 ; x0 = pointer to new block\n orr x3, xzr, #2048 ; header word (tag 0, size 2)\n str x3, [x0, #-8] ; write header\n ldr x3, [x1, #0] ; load y.off from heap\n ldr x4, [x2, #0] ; load x.off from heap\n add x3, x4, x3 ; add them\n sub x3, x3, #1 ; adjust for tagged int\n str x3, [x0, #0] ; store result.off to heap\n ldr x1, [x1, #8] ; load y.len from heap\n ldr x2, [x2, #8] ; load x.len from heap\n add x1, x2, x1 ; add them\n sub x1, x1, #1 ; adjust for tagged int\n str x1, [x0, #8] ; store result.len to heap\n ...\n ret\n L114:\n bl _caml_call_gc ; GC call if needed\nThe OCaml minor heap is really fast, but it's nowhere near as\nfast as just passing values around in registers and doing\ndirect operations, which the unboxed version lets us do!
\nMy benchmark above used direct external calls to compiler primitives,\nbut OxCaml exposes normal modules for all these special types so\nwe can just open them and gain access to the usual integer operations:
\nmodule I16 = Stdlib_stable.Int16_u\n\nlet[@inline always] i16 x = I16.of_int x\nlet[@inline always] to_int x = I16.to_int x\n\nlet pos : int16# = i16 0\nlet next : int16# = I16.add pos #1S\nThere's more than just integer operations in OxCaml. Hot off the press in the\npast few weeks have been unboxed character operations as well, so we don't need\nto use an OCaml int (this is unboxed as well, but I presume the compiler can\noptimise and pack 8-bit operations much more effectively if it knows that we're\noperating on a char instead of a full word).
\nThe httpz parser tries to use these, but the support for untagged ints isn't fully\ncomplete yet (thanks Max Slater for\nthe pointer).
\nHTTP date timestamps use unboxed floats as well.
\nOnce we've declared these unboxed records, they're fully nestable within other unboxed records.\nFor example, HTTP requests with multiple fields remain unboxed:
\ntype request =\n #{ meth : method_\n ; target : span (* Nested unboxed record *)\n ; version : version\n ; body_off : int16#\n ; content_length : int64#\n ; is_chunked : bool\n ; keep_alive : bool\n ; expect_continue: bool\n }\nFunctions can therefore naturally return multiple values without allocation by using unboxed tuples in the return value of a function:
\nlet take_while predicate buf ~(pos : int16#) ~(len : int16#)\n : #(span * int16#) =\n let start = pos in\n let mutable p = pos in\n while (* ... *) do p <- I16.add p #1S done;\n #(#{ off = start; len = I16.sub p start }, p)\n\nlet #(result_span, new_pos) = take_while is_token buf ~pos ~len\nVanilla OCaml did some unboxing of this use of tuples, but not with\nrecords (which would land up on the minor heap). With this OxCaml code,\nit's all just passed directly on the stack through function call traces.
\nWe can then also mark parameters to demand that they won't escape a function, enabling stack\nallocation more explicitly:
\n(* Buffer is borrowed, won't be stored anywhere *)\nlet[@inline] equal (local_ buf) (sp : span) (s : string) : bool =\n let sp_len = I16.to_int sp.#len in\n if sp_len <> String.length s then false\n else Bigstring.memcmp_string buf ~pos:(I16.to_int sp.#off) s = 0\nIf a function needs to return a local value, then it uses a new exclave_ keyword. For example, in the HTTP request parsing we look up a stack allocated list of headers:
val find : t list @ local -> Name.t -> t option @ local\n\nlet rec find_string (buf : bytes) (headers : t list @ local) name = exclave_\n match headers with\n | [] -> None\n | hdr :: rest ->\n let matches =\n match hdr.name with\n | Name.Other -> Span.equal_caseless buf hdr.name_span name\n | known ->\n let canonical = Name.lowercase known in\n String.( = ) (String.lowercase name) canonical\n in\n if matches then Some hdr else find_string buf rest name\n;;\nNotice that it's a recursive function as well, so this is a fairly natural way\nto write something that remains heap allocated. You can learn more about this\nfrom Gavin Gray's OxCaml tutorial slides.
\nA nice quality of life improvement is that OxCaml allows stack-allocated\nmutable variables in loops, eliminating the need to allocate ref values. This\nallows parsing code to have local mutability:
let parse_int64 (local_ buf) (sp : span) : int64# =\n let mutable acc : int64# = #0L in\n let mutable i = 0 in\n let mutable valid = true in\n while valid && i < I16.to_int sp.#len do\n let c = Bytes.get buf (I16.to_int sp.#off + i) in\n match c with\n | '0' .. '9' ->\n acc <- I64.add (I64.mul acc #10L) (I64.of_int (Char.code c - 48));\n i <- i + 1\n | _ -> valid <- false\n done;\n acc\nWhereas in conventional OCaml there might be a minor heap allocation for the\nreference:
\nlet parse_int64 buf sp =\n let acc = ref 0L in (* Heap-allocated ref *)\n let i = ref 0 in (* Heap-allocated ref *)\n let valid = ref true in (* Heap-allocated ref *)\n while !valid && !i < sp.len do\n let c = Bytes.get buf (sp.off + !i) in\n match c with\n | '0' .. '9' ->\n acc := Int64.add (Int64.mul !acc 10L) (Int64.of_int (Char.code c - 48));\n i := !i + 1\n | _ -> valid := false\n done;\n !acc\nThe toplevel Httpz.parse function has a pretty simple signature from a user's perspective:
\nval parse : bytes -> len:int16# -> limits:limits ->\n #(Buf_read.status * Req.t * Header.t list) @ local\nThis function receives some a bytebuffer and resource limits and returns an unboxed local tuple of the connection status, parsed (unboxed) request and a stack-local list of header spans that represent the offsets within the input buffer of what was passed.
\nI should probably make the input buffer local too; one nice aspect of OxCaml is how easy it is to incrementally add type and kind annotations and lean on the compiler type inference to help guide where to fixup callsites.
\nThere are lots and lots of other new features in OxCaml which I've started integrating, but require careful planning of layouts.\nFor example, I wanted to use or_null to have a non-allocating\nversion of option, but you often end up with long compiler errors about value inference failures, so I ended up just allocating\na local type instead. Something to investigate more in the future as I get familiar with OxCaml.
\nI also ran into issues using mutable fields in unboxed records and found this is documented:
\n\n\nWe plan to allow mutating unboxed records within boxed records (the design\nwill differ from boxed record mutability, as unboxed types don’t have the\nsame notion of identity).
\n
It's also difficult right now to strip away the OxCaml extensions and go back\nto normal OCaml syntax. Chris Casinghino pointed me to the OxCaml ocamlformat fork which\nhas a --erase-jane-syntax, but it requires some build system work to\nintegrate and seems to lag a little behind the new features (like unboxed small\nliterals). For now, I've decided to just focus on using OxCaml exclusively and\nsee how it goes for a while.
Finally, the tooling is still a fluid story. Arthur Wendling and Jon Ludlam are making\nfast progress on getting odoc working in the\nmainline tool, but it's not quite there today.
\nWhile I built small scale examples to test out the architecture, I leaned heavily\non Claude code to build out the majority of the parser so I could rapidly experiment.\nTo do this, I synthesised a set of OxCaml specific Claude skills\nin my Claude OCaml marketplace which you can add to your own projects as well. Browsing the skills is a pretty nice way of getting familiar with the different features.
\nI generated those skills via a combination of summarising the OxCaml source trees and cribbing from the ICFP 2025 tutorial, and then getting CC to verify that the example code actually compiled. All automated and very easy to refresh every time a new compiler drops from Jane Street.
\n![\"%c\"](\"/images/claude-oxlocal-1.webp\" "\\"The")
Ultimately, none of this matters if the runtime performance isn't there!\nLuckily, the HTTPz parser is incredible in a synthetic benchmark (just passing\nbuffers around) as opposed to a network benchmark, using Core_bench to measure\nperformance. What's impressive isn't the straightline throughput, but the\nmassive drop in heap activity which greatly increased the predictability and\ntail latency of the service. And with all the extra typing information, I\nexpect that straightline performance will only increase (and this is before\nI've looked at the SIMD\nsupport).
\n| Metric | \nhttpz (OxCaml) | \nTraditional Parser | \n
|---|---|---|
| Small request (35B) | \n154 ns | \n300+ ns | \n
| Medium request (439B) | \n1,150 ns | \n2,000+ ns | \n
| Heap allocations | \n0 | \n100-800 words | \n
| Throughput | \n6.5M req/sec | \n3M req/sec | \n
I then glued this together using Eio into a full\nwebserver.\nIt works, and serves traffic just fine and in fact you are reading this web page via it right now!
\nThe current Eio/OxCaml does a data copy right now since Eio uses Bigarray, but I had a catchup coffee\nwith Thomas Leonard and Patrick Ferris where I agreed to treesmash my local eio into\nswitching entirely to bytes from the io-uring layer up. Sadiq Jaffer informs me\nthat his compactor doesn't trigger automatically, so any bytes above a 4KB\nthreshold are allocated using mmap and so are fine to pass to the kernel for\nzero copy receive.
\nThe key OxCaml feature to make this io_uring integration awesome is a new FFI\nfunction that allocates an OCaml value directly into the caller's OxCaml stack\nrather than the heap. This means that we should be able to come up with a scheme\nby which io_uring requests are routed directly to an OCaml continuation that's woken\nup directly with a buffer available to it on the stack. True zero-copy to the kernel\nawaits, which should also help speed up Docker's VPNKit hugely\nas well.
Keen readers may note that my OxCaml repo links here go to a new monorepo I've\nsetup for the purpose of hacking on real code in production outside of Jane Street's\nwalls.
\nI'll blog more about this next week, but for now I hope you've enjoyed a little\ntaste of what the OxCaml extensions offer in real world code. Stay tuned also for\neven more performance improvements, and for native TLS with an OxCaml port of\nocaml-tls from Hannes Mehnert soon!
I decided to spend this week on as much focussed hacking as I could, and in particular finished up switching my website to a new webserver in OxCaml. This attracted a lot of attention, so I spent a surprising amount of time answering questions on the socials about it! If you're reading this site, then it also works...
\nOne funny thing that happened right after deploying it was that I noticed tens of thousands of concurrent connections opened. It turns out that Moltbook had a sub-molt used by agents that track Hackernews somehow, and a bunch of them had decided to mine my website for ... something. The switch to agents dominating the Internet are arriving rapidly!
\nThe actual development of httpz is happening in a new oxcaml monorepo I've opened. I've not abandoned my older way of publishing to opam as well, but Jon Ludlam and Thomas Gazagnaire are refining that approach. When coding in OxCaml, I need to fork almost every library, so a monorepo is the only way to go!
\nOn a more human note, it was delightful to see Jon Sterling discuss his Research Group Manual which codifies many of the reasons why I started a blogging tradition here in my own group.
\nIn the land of OCaml, I proposed dropping the minimum supported version of opam to general support. The general drag of maintenance of the CI infrastructure in my group is becoming a problem; just moving to opam 2.5 or debugging arm64 issues was a giant amount of work for Mark, so we have to keep on top of deprecating old things somehow. On the other hand, we're also extending some cool uses of Capnp capabilities throughout the infrastructure, which makes CLI usage of all these services easier and easier.
\nI also took the opportunity to upload all of the OCaml workshop talks to <watch.ocaml.org>, so they're available for your browsing pleasure.
\nI caught up with Thomas Leonard and Patrick Ferris to discuss what to do about the number of Eio issues piling up. We're all generally happy with how stable and usable it is (I'm using it everywhere), but we'll get together after our current set of projects to do a collective push to merge our branches together. I'm pretty happy with this model of development: get some experience using it, and then make a bunch of changes after "learning by doing".
\nCongratulations also to Ryan Gibb for a tremendous showing at FOSDEM, delivering three talks to packed rooms! They're all online to watch and I want to particularly highlight how much I enjoyed his package management calculus one. We're working on getting this submitted to a PL conference next!
\n\nThe live streaming support from FOSDEM was fantastic this year and I got to see everything live while also sitting in a chilly picnic in Cambridge over the weekend.
\n![\"%rc\"](\"/images/running-into-anna.webp\" "\\"Anna")
\nIt was also delightful to run into Anna Lapwood in the Mill and catch up; first time I've seen her since she left Pembroke to travel the world!
Some fun links:
\nJust short weeknotes this week as I've been travelling back to Belfast for various matters and haven't had much computer time.
\nOne exciting development in the week was that Shane Weisz continued the conversation with the IUCN Red list team about his developing dashboard, which went extremely well. There's so much excitement on both sides about how all this is going!
\nI also spent a chunk of time wrestling with understanding Zarr so I can port TESSERA to use this instead of Numpy arrays. There's been a long and helpful thread on our Zulip about this with a lot of people chiming in. Michael Dales has also been a source of coordinate inspiration from his experiences, so I'll put proper thoughts together on this soon.
\nOn the OxCaml front, Mark Elvers has knocked up some OCaml Zarr so I'll be porting those to OxCaml soon and taking the TESSERA support in OCaml for a spin.
\nThere was quite a lot of TESSERA things going on alongside this.
\nFirst a preprint Earth Embeddings as Products: Taxonomy, Ecosystem, and Standardized Access that's a nice summary of how to use geoembeddings like TESSERA:
\n\n\nGeospatial Foundation Models (GFMs) provide powerful representations, but\nhigh compute costs hinder their widespread use. Pre-computed embedding data\nproducts offer a practical "frozen" alternative, yet they currently exist in\na fragmented ecosystem of incompatible formats and resolutions. This lack of\nstandardization creates an engineering bottleneck that prevents meaningful\nmodel comparison and reproducibility. We formalize this landscape through a\nthree-layer taxonomy: Data, Tools, and Value. We survey existing products to\nidentify interoperability barriers. To bridge this gap, we extend TorchGeo\nwith a unified API that standardizes the loading and querying of diverse\nembedding products. By treating embeddings as first-class geospatial\ndatasets, we decouple downstream analysis from model-specific engineering,\nproviding a roadmap for more transparent and accessible Earth observation\nworkflows.
\n
Then Sadiq Jaffer and Frank Feng did a great podcast on Satellite Image Deep learning where they go through the journey of how we trained the model. I can't believe it's barely been a year!
\n\nTESSERA support also got merged into Torchgeo for those looking to work on customising the model itself. Most users don't have to use this as they can just use our pregenerated embeddings.
\nAfter almost a decade of working with David Allsopp in both the University and later Tarides, he finally "graduated" and went off to join Jane Street where...we will continue to work together on OxCaml and OCaml.
\nGood luck to David as he no doubt enjoys the ridiculously nice Jane Street office, where I would overdose on the fresh fruit juice machine and be on a perpetual sugar high!
\nSome fun links:
\nWhile my colleagues have been busy with start-of-term matters, I've been\nenjoying being able to get some research done while on sabbatical! I got back to\nCambridge from India and settled back to hacking on my research projects.
\nI caught up with the AMD and Vultr team, as they have been sponsoring TESSERA\nwith giving us access to an AMD MI325X 6x node, which has been tremendous help\nin getting TESSERA embeddings going. Before Christmas, our DAWN access expired and we had literally no\nGPUs to continue working on TESSERA, so Vultr stepping in at extremely short\nnotice has been an incredible godsend for our plucky geospatial project.\nThe team from Vultr, especially Kasia Hilborne, have been delightfully nice to\nwork with as well!
\nFollowing up with the extremely random Jensen Huang chat last year, the nVidia team visited and we also\ndiscussed future collaborations around TESSERA. And then I went to the pub\nwith the AMD crew, so this was pretty much full GPU hustling coverage for the\nweek except for the absence of anyone from Intel in my immediate vicinity.
\nI got invited by Sam Reynolds to go speak at the AI4Nature launch down at the IET in London.
\n\n\nAi4Nature is a cross-sectoral collaborative initiative to advance responsible\nAI applications in biodiversity data and nature recovery and enhancement,\nbuilding sustainable global influence through collaborative standards\ndevelopment, education, and knowledge sharing.\n-- AI4Nature, 2025
\n
![\"%c\"](\"/images/ai4nature-2.webp\" "\\"Pretty")
I talked about how modern AI initiatives might help, similarly to the red pill and blue pill talk last week at the Conservation Evidence conference. The talk was recorded, but the recording just focussed on me and didn't show my slides, so you'll have to follow along with my slides by hand if you watch it!
\n\nYou can read a full roundup on the AI4Nature journal along with all the other talks. There were an interesting mix of urban developers, remote sensing experts and conservation ecologists there, all wondering how to balance the delicate mix of nature and human needs. I sent several people there our just-out LIFE recipes paper and others the FOOD biodiversity explorer.\nThanks to Sam for the invite to a great event!
\n![\"%c\"](\"/images/ai4nature-1.webp\")
Dominic Orchard and I went along to the CUP bookshop to pick up our PhD buddy Tomas Petricek's latest book on "Cultures of Programming". Although it was closed on first visit, I nipped along later and found the last copy, which I'm now reading with great enjoyment.
\n![\"%c\"](\"/images/cup-visit-1.webp\" "\\"Dominic")
Fun links:
\nI was at the launch of AI4Nature a few days ago, and met a lot of\npeople looking for practical advice on integrating remote sensing into their biodiversity\ndecision making. So it's good timing that our latest paper just came out,\nlead by the inestimable Alison Eyres to give some "recipes" on how to use our global LIFE metric! Read it here.
\nAs a reminder, the LIFE metric is one we published last year to calculate\nthe extinction costs of different landuse actions (either conversion or restoration) and\nto allow these costs to be compared in a "common currency" anywhere in the world.
\nTo achieve this, the metric tries to be representative of geographic, taxonomic, and habitat diversity, allows disaggregation into scores for groups of species, and be interpretable on a ratio scale (i.e. a two-fold difference in LIFE scores corresponds to a two-fold difference in estimated extinction effects). Finally, in order to enable real-world impact, the LIFE maps must be accessible, actionable and usable. The latter point is what our latest paper covers, by providing some useful recipes for the a decisionmaker to follow! Co-author Michael Dales also jotted down his notes on the paper as well.
\n ![\"%c\"](\"/images/2025-life-uses-ss.webp\" "\\"The")
The paper walks through how to apply LIFE across different\nconservation and development contexts, ranging from the local to the global. Here are the highlights from each:
\nThis case study integrates LIFE with the Global Forest Change forest loss data to quantify biodiversity harms as they happen.
\n\n\n[...] our analyses demonstrate that forest loss has a greater per km2 impact on extinction in some countries than others. The impact in terms of extinction risk arising in Peru, Indonesia and Papua New Guinea are disproportionately large compared to the extent of forest loss in these regions.
\n[...] LIFE identifies areas of high conservation concern that would not necessarily be detected through species richness alone, where the loss of a widespread species is weighted equally to that of a narrowly endemic or threatened species.
\n
This enables monitoring of extinction risk impacts from deforestation as it happens, and helps focus on biodiversity hotspots separately from forest carbon. While other metrics like the countryside species-area relationship could also do this, they're not as readily available as LIFE since we've done all the large-scale computing needed and provided precomputed maps for this usecase.
\n![\"%c\"](\"/images/life-uses-1.webp\" "\\"Monthly")
The biggest driver of land-use change is the global food system, and so we used LIFE to assess the extinction risk impacts of apple consumption in the UK (2019-2023) and how these varied depending on which country the apples were sourced from. This builds on our earlier work showing food impacts can vary by three orders of magnitude depending on the source.
\n![\"%c\"](\"/images/life-uses-2.webp\" "\\"The")
The UK apple example shows how even domestically produced foods have hidden biodiversity footprints depending on sourcing. More generally, it could be used to evaluate the consequences of policies promoting lower-yield domestic agriculture, and to track how food-related biodiversity impacts change over time due to post-Brexit trade shifts or random trade tarriffs. You can see more on this in our interactive food explorer.
\nThis hypothetical scenario covers a company that has converted a forest to agricultural land for coffee production. The company wants to use the LIFE metric to assess its impact and then select the most suitable restoration site to compensate for biodiversity loss by restoring biodiversity to a "pre-impact" baseline. This could then be used to calculate financial contributions towards a no-net-loss biodiversity scenario for that (hopefully essential) development.
\nThe pixel matching mechanism to calculate our baselines were chosen from pixels that are currently agricultural land but were historically forested. This is similar to what we did for our carbon credit tropical moist forest, except that we also account for restoration being highly uncertain and so adding an ex-ante multiplying factor.
\nThey key differentiator for using LIFE here, vs other metrics, is that LIFE can be disaggregated to track the fate of individual species, making it well suited to capture local biodiversity values and transparently assess net losses and gains based on local knowledge.
\nThis fourth scenario uses LIFE to inform prioritisation of site-based conservation actions within the World Land Trust's portfolio of interventions:
\n\n\nWorld Land Trust (WLT) is an international conservation charity that protects\nthe world’s most\nbiologically significant and threatened habitats.
\nWorking through a network of partner organisations around the world, WLT\nfunds the creation of reserves and provides permanent protection for habitats\nand wildlife. Partnerships are developed with established and highly\nrespected local organisations who engage support and commitment among the\nlocal community.
\n-- Who We Are, World Land Trust, 2025
\n
For each project, we estimated extinctions that could be averted\nunder an extreme counterfactual that all habitat was assumed to be\nconverted to agriculture. We multiplied each pixel level\nLIFE-convert value by its area and summed all pixels within the\nproject. We then explored the additional species-level insights by\ndisaggregating the toplevel LIFE metric.
\n![\"%c\"](\"/images/life-uses-3.webp\" "\\"Potential")
LIFE doesn't aim to replace all other metrics here, but provides an excellent baseline that can be disaggregated:
\n\n\nDespite inherent uncertainties, species-level information is valuable for\ncomparing sites. It enables practitioners, policy makers and funders to\nbetter understand why a metric identifies a site as important and provides\npersuasive evidence for conservation investment by highlighting key\nthreatened species.
\n
And last but not least, my favourite rainforest in Africa is the Gola rainforest.\nWe combined LIFE with counterfactual methods to evaluate a real conservation project there that the RSPB has been running for decades.\nWe try to answer the crucial question of whether the intervention actually had a net-positive biodiversity impact via an ex-post evaluation of what's happened in the past decade.
\n![\"%c\"](\"/images/life-uses-4.webp\" "\\"The")
The use of LIFE lets us go beyond broad species richness metrics:
\n\n\nThe Gola region hosts several narrow-ranged, highly threatened species (e.g.\nDiana Monkey and the Pygmy hippopotamus), which would not be highlighted by\nanalyses focused solely on species richness, including those based on PDF or\ncSAR (without rarity weightings)
\n
We're hopefully clear about LIFE's limitations in the paper as well:
\n\n\nLike all global metrics, LIFE's broad applicability relies on assumptions and\nsimplifications. It should be used cautiously, and alongside local knowledge\nand ground-truthing, especially for restoration, offsetting, or fine-scale\nanalysis, and in poorly studied areas.
\n
The computational infrastructure behind LIFE, lead by Michael Dales, is available via our\nquantifyearth GitHub organisation, with the\nheavy lifting done by the Yirgacheffe\nlibrary for processing large-scale raster data. That's what Michael presented back\nat PROPL earlier this year.
\nWith LIFE now demonstrated across these five use cases, we're really excited to see how\nothers apply it to their own conservation challenges. The combination of LIFE\nwith planetary computing infrastructure means we can provide\ndecisionmakers with extinction risk information at much quicker scales and\nspeeds than possible before. But of course, this requires turning metrics into\non-the-ground action, so please do reach out if we can help with that!
I ended up the India family trip with a successful navigation of getting everyone back to Ireland in one piece, and then with just a few hours was back in Cambridge to help host the big Conservation Evidence conference at Pembroke!
\nLuckily, I had pretty much no actual work to do beyond show up and smile, as the catering staff at Pembroke are beyond outstanding as are the Conservation Evidence team.
\n![\"%c\"](\"/images/pemb-aud-1.webp\" "\\"The")
You can see my talk here or read the roundup:
\n\nI also wrote up some notes on our new paper on five ways to use the LIFE metric for conservation decision-making, which just came out in Biological Conservation. Well done Alison Eyres!
\nI've been working with KC Sivaramakrishnan for a while on helping him craft the FP Launchpad and he announced it to the public!
\n\n\nThe Functional Programming (FP) Launchpad at IIT Madras aims to build\nresearch and educational capacity for crafting efficient, reliable and\ntrustworthy software with mathematical guardrails. As AI coding agents make\nsoftware increasingly accessible and enable its production at unprecedented\nscale, they also introduce far-reaching consequences for civic society. In\nthis context, the centre is founded on the belief that discovering future\nsoftware engineering best practices requires sustained feedback loops between\nresearch and real-world systems, as well as educational structures that\ncultivate a new generation of maintainers for foundational software.\n-- FP Launchpad manifesto, 2026
\n
All three of the illustrative projects are of huge interest to me:
\n\n\n\n
\n- An open-source, verifiable voting system for general elections, built on top of Shakti RISC-V processor, MirageOS unikernels and O(x)Caml.
\n- A programmable public infrastructure for environmental planning, combining\nTESSERA’s satellite-derived representations with CoRE Stack data and\ncompositional functional models in O(x)Caml to support auditable indicators\nand scenario analysis for India’s water and habitat systems.
\n- A formally verified runtime system for O(x)Caml in O(x)Caml guaranteeing memory\nsafety and functional correctness properties. The focus will also include\nformal verification of tools for debugging and observability, such as\neBPF-based tracing, continuous runtime contract checking, and deterministic\nrecord and replay.
\n
The official launch of the centre is in early April, so I'll report back in a few months when I head over to Chennai!
\nFun links:
\nI got a few questions about the dev setup I used for my AoAH sprint last month.\nI've cleaned this up and published\nclaude-ocaml-devcontainer,\na Devcontainer of everything you need to do OCaml\nor OxCaml development in Claude Code in a sandboxed Docker container. This means you\ncan (reasonably safely) run it in unattended mode with permissions bypass enabled.
\nA devcontainer can either be used in an editor that supports it like VSCode, or directly from the CLI (which is what I do for Claude Code). Adding it to your project is as simple as:
\n$ mkdir .devcontainer\n$ cd .devcontainer\n$ curl -OL https://raw.githubusercontent.com/avsm/claude-ocaml-devcontainer/refs/heads/main/.devcontainer/devcontainer.json\nEdit the JSON file to add any other post-installation or extensions that you might need for that project. It's intended to be customisable.
\nTo spin the devcontainer up, I use the npx CLI:
\n$ npx @devcontainers/cli up --workspace-folder .\n$ npx @devcontainers/cli exec --workspace-folder . bash -l\nThis will mount your current project into /workspace in the dev container. The set of network domains that can be accessed are limited to a select few, but I'll parameterise this into the project metadata in the future.
When you're in the workspace container, you have two preinstalled OCaml switches:
\n$ opam switch\n# switch compiler description\n 5.2.0+ox ocaml-variants.5.2.0+ox 5.2.0+ox\n→ default ocaml.5.4.0 default\nAnd claude maps its config from your home directory, so you can start up sessions once you've authenticated as normal, and everything runs in the container reasonably sandboxed.
Since compiling up O[x]Caml can take a while, it's a prebuilt image, but you can also clone the repository and customise the Dockerfile in .devcontainer to your heart's content.\nThe default disk space in a GitHub Action wasn't sufficient, but it turns out that the default ubuntu-latest has a ton of pre-installed packages that you can just delete to double your disk space.
One pretty cool thing about the action is that it doesn't use qemu to build the\nmultiarch images. Instead, the\nmultibuild.yml\ndispatches separate builds to the native arm64 and amd64 hosts, and then\ncombines them together. This is faster and more reliable than the conventional\npath of going through CPU emulation for the non-native host.
\nI also modify my own devcontainer to mount some limited SSH keys and a\n.gitconfig so I can commit from within the container, which allows for more\nunattended feedback loops.
After an amazingly fun trip to Udaipur and lots of family chitchat, we got over to Vijayawada and then Hyderabad, while visiting family temples and villages en route.
\n![\"%c\"](\"/images/anil-india-1.webp\" "\\"The")
\n![\"%c\"](\"/images/anil-india-2.webp\" "\\"But")
I wrote my reading list in my happy new year 2026 post. And just to hit the ground running, Frank Feng and I got the TESSERA embeddings for 2025 kicked off on the second day of the new year, since all the Sentinel-1/2 data was ready for us to generate the annualised embeddings.
\nLooking forward to the new year. If 2025 is anything to go by, 2026 is going to be a wild ride with AI developments...
Happy new 2026 everyone! I've had a wonderful journey through Udaipur and the lake palaces, and\nam currently in Vijayawada exploring my ancestral village\nfor the next few days. I've caught up on some reading, so here are my book thoughts (and podcasts for the first time)\nfrom the past year!
\nI thoroughly enjoyed listening to the BBC Reith\nlectures this year by Rutger\nBregman, after hearing about them from Carl Edward Rasmussen. I picked up a copy of Bregman's new book\nMoral Ambition to read in my\ntravels; it was a light read and I got through it all on the flight to India.
\n ![\"%lc\"](\"/images/2026-books-ma.webp\")
I enquired online if anyone else had read it, and Laura James nailed it\nwith her assessment that it "looks like a classic airport book for business folk having a career crisis". Well, working in the climate and biodiversity space feels like a constant crisis, so I dove straight into it and ended up bounding off the plane with a spring in my step!
\nBregman has written this book for a rather narrow audience. The key idea he pushes is that ambitious and idealistic young people shouldn't waste their lives on "bullshit jobs" (a term coined by the late David Graeber, author of the fantastic history of debt). He argues that it's possible to be both ambitious and moral in our careers:
\n\n\nSometimes it seems 'ambition' has become a dirty word, incompatible with an\nidealistic lifestyle. Many people are more preoccupied with the kind of work\nthey do than the impact their work has [...] or 'think global, act local' as\nif achieving little is somehow a virtue.
\n-- Moral Ambition, Ch2, Rutger Bregman
\n
The book follows a style fairly typical of this class, whereby a number of anecdotes about morally ambitious characters from history (such as Ralph Nader before he became a perennial presidential candidate) are recounted. I particularly liked the breakdown of 'illusions' that cause inaction, such as the illusion of awareness that we've run across in our research on food consumption impacts :
\n\n\nPsychologists speak of the "belief-behaviour" gap. Take people who think it's\nawful how animals are treated but still eat meat; progressives who think\nplanes are too polluting but fly all the same; church-goers who scarcely give\nto charity, despite the scriptures' call to tithe.
\n-- Moral Ambition, Ch4, Rutger Bregman
\n
And then the illusion of good intentions that we've seen repeatedly in our work on conservation evidence and in climate policy. There is a huge lack of evidence driven decision making, possibly due to the extremely difficult job of robustly evaluating counterfactual decision making:
\n\n\nA global analysis of 1,500 climate policies put in place between 1998 and\n2022 found that only 63 of them – a mere 4 per cent – led to a significant\ndrop in emissions.
\n-- Moral Ambition, Ch4, Rutger Bregman
\n
And something I've run across all the time in the open source community is the 'illusion of purity', whereby "activists continued to write each other off for all manner of minor missteps and mistakes" (ch4).
\nThe most disappointing chapter of the book was where he covered the Effective\nAltruism movement in a strangely wavering tone. He tries to\neven-handedly cover the very strange\ncommunity that built up around EA, and also criticise the pooling of wealth, but didn't really stimulate any new insights or\ncritical thought in me. He does take a pragmatic stance on the status quo, which is understandable given the emphasis on action over awareness. I can't say that I disagree here either, given the number of absolutely amazing philanthropists I've been lucky to work with on my own research:
\n\n\nThe truth is that money and moral ambition need each other. Philanthropy\ndoesn't have to get stuck in vanity and paternalism. It can lead to real\nsystemic change, as long as you prioritise wisely and keep an eye out for\ndamaging side effects. What's more, private individuals are in a unique\nposition to support unpopular causes – when government and business steer\nclear.
\n-- Moral Ambition, Ch8, Rutger Bregman
\n
Pushing on beyond that, though, I greatly enjoyed the author's own ambition in setting up a School for Moral Ambition and promoting the idea of creating moral ambition circles. This is pretty close to the sort of thing we think about day-to-day in our university environment anyway, so I hope to find Carl Edward Rasmussen and set one up in Cambridge when I'm back next year (if there isn't one we can already join).
\n ![\"%rc\"](\"/images/2026-books-twu.webp\")
For something less heavy, I weaved through "This Way Up: When Maps Go Wrong" by the famed Map Men. This is an irreverent walk through all the extremely weird ways humans have come up with to project a 3D topology onto a 2D surface.
\nThis included the fun\npsychogeography movement from\nFrench surrealists who cut up a map of\nParis and rearranged it\nwith parts of Paris that were "stimulating" and "worthy of study" with giant\narrows between them to represent the teleportation to the most exhilarating\nareas. Perhaps we need to draw a similar 'a bicycling birders guide to\nCambridgeshire' in 2026!
\n ![\"%rc\"](\"/images/2026-books-ww.webp\")
I'm only halfway through "When there are wolves again"\nby EJ Swift, but it's SO GOOD so far that I had to include it.\nIt combines some of my favourite topics about restoring depleted\necosystems and exploring space in one book. Something I've often discussed with\nDavid Coomes are the challenges around reintroducing wolves to Scotland; a real\nsource of tension between conservationists (who want to get rid of deer) and\nthe farmers (who quite reasonably don't want their livestock eaten by roving\nwolves).
\nThis also has a storytelling format of having two narrators chatting to each other,\nand maintains a really narrow and hopeful focus. You might notice I'm dosing up\non optimism to charge myself up for 2026, so thank you Jon Crowcroft for recommending\nthis one to me!
\n ![\"%rc\"](\"/images/2026-books-sf.webp\")
We can't be too optimistic and lose touch with reality, so a brilliant novel that I lost behind my sofa but found again\nin November is "When the Sparrow Falls"\nthat has has an entertaining (but getting awfully close) take on AI agents.
\nIn the fictional Caspian republic, the rest of the world has been infected by AI governance, one\nsmall nation resists being overtaken and outlaws digital technology. Anyone has to be checked\nto see if they're human at the borders, and the books opens with the autopsy of an unfortunate\nhigh-ranking government official revealing that they're an AI. The rest of the book is\ndark humour, bleak worldbuilding, and mystery at its finest. Not a cheerful book though...
\n ![\"%rc\"](\"/images/2026-books-pg.webp\")
Thank you Isobel Cohen for recommending "Prisoners of Geography" which is\na decade old now, but a fascinating view on geopolitics just before Trumpian\npolitics took over. Tim Marshall argues that no matter how much technological\nprogress is made, a dominant force in politics will always be geography.
\nI don't actually agree with it at all though. This summer I went to Botswana where I found a country that had a terrible\noutlook when it achieved independence: no capital\nreserves, surrounded by hostile apartheid regimes, and with mineral wealth that\nusually lead to invasion or takeover. But today, Botswana has one of the highest life expectancies\nand most stable governments in Africa.
\n\nThe same is also true in Costa Rica, having built up a solid ecotourism industry. But I did find the framework of Tim Marshall's argument to be well worth reading, and I've queued up his 2023 book on the politics of space to read this year.
\n ![\"%rc\"](\"/images/2026-books-ch.webp\")
I used to live in Chennai way back in 1990 after the Gulf War chucked us out of\nKuwait, and KC Sivaramakrishnan gave me a copy of Chennai: A\nBiography when I visited\nhim earlier in the year.
\nMadras is a city of so many "firsts": the first Indian corporation, the first\narmy regiment. But I had no idea that the bookshop that I used to hang out in 36 years ago\nfinding all the maths books I could devour, called\nHigginbotham's, is actually the oldest\nbookshop in all of India! A\nbrilliantly engaging piece of history from V.\nSriram who apparently leads heritage\nwalks through the city that I must catch on my next visit.
\n ![\"%rc\"](\"/images/2026-books-sc.webp\")
I greatly enjoyed tuning into "Solving for Climate" from my favourite climate optimists Hannah Ritchie and Rob Stewart! They've got a nicely balanced set of science and society oriented episodes, with some of my favourite being:
\n ![\"%rc\"](\"/images/2026-books-st.webp\")
I might be biased as I've appeared on a previous season, but Signals and Threads has really developed into a solid series that manages to make listening about tech not boring (a crime that about 99% of other podcasts I've attempted to listen to have committed). After working on it all day, do I really want to listen to more tech while out for a run?
\nYaron Minsky has hosted some unexpectedly fun episodes this year:
\n ![\"%lc\"](\"/images/2026-books-kb.webp\")
I've not actually read this yet, but thank you Sara Biswas for an effervescent review that got me really excited to pick this up on the way home! Katabasis is a novel published this year by RF Kuang (who studied at Cambridge) with a hilariously dark plot about two graduate students (and magicians) who must venture into hell to save their thesis advisor in order to get letters of recommendation. I shall reserve judgement until after I have read it, and indeed after I have completed all the recommendation letters I need to get done despite allegedly being on vacation. But with a setting like this, I can feel it's going to be good!
\nThat's a wrap for my abridged reading recommendations for the year! I have a\nmonster pile of unread books to attempt to catch up on this year, so I'll try\nto blog about what I'm reading more regularly than annually.
Agentic programming has been getting a hilariously bad rap in the OCaml community recently, but it's definitely here to stay despite the security and legal concerns. I realised that to form a useful opinion on all this, I needed to really get into using Claude with OCaml for real outputs and not just toy code. So this holiday month, I'm going to release a new useful OCaml library per day until Christmas using Claude Code: the advent of agentic humps is here!
\nBonsai_term terminal UI to Sortal via Async.Mosaic terminal UI to Sortal via Eio.![\"%c\"](\"/images/aoah-ss-2.webp\" "\\"Claude")
I'm working through a large backlog of ideas that I'll figure out as each days goes on. Ideas thrown on the pile by colleagues include TCP connection reuse and pooling library with TLS support, HTTP cookie jar handling using Eio, Batteries-include HTTP(S) client library with redirect/cookies, digest vast amounts of Git and summarise it (see a preview), Zulip bindings using Requests and Eio, Kitty graphics protocol to show graphics in your terminal, client bindings for the JMAP protocol, client bindings for the Immich self hosted photo service, client bindings for the Peertube video service, generate image srcsets in various resolutions for websites, DOI resolution of papers to structured metadata, and a Parquet library in pure OCaml. I'm also working on an io_uring OxCaml webserver if I can get the Linux kernel not crashing on me before Santa visits...
My overall goal is to accelerate the heck out of how I manage the growing data in this website. I've been building it as homebrew infrastructure for the past twenty five years, and now I want it to move from ad-hoc scripts to principled data management. I am also using the libraries to do data processing in the day job for the remote sensing of nature or evidence synthesis. I'll edit the above list every day to link to what I actually did.
\nI've picked these choices fairly carefully as they're not "core" libraries that\nare difficult to write and require functional ingenuity, but are instead\nproblems that involve a fair amount of boilerplate code that is typically quite\ntedious to write in OCaml. Hand writing code might be on the ropes, but not\nquite out of action just yet! But first, let's establish some groundrules for if\nthis is a good idea or not.
\nThere's a definite gag reflex involved with releasing so much code: by prioritising quantity over quality, aren't I just contributing to the world of AI slop? However, the hypothesis I am exploring is that the software engineering process fundamentally changes when using agents towards specification driven development, which has always been the holy grail of functional programming.
\nThere's been extensive discussion recently about the role of LLMs in open source elsewhere that informed my thinking. I liked Thibaut Mattio stating how he's approaching his own agentic software development:
\n\n\nAI writes a significant amount of the initial code, and I review, revise, and iterate on a large portion of it. That’s how I work these days. But the architecture, design, and core logic are very much the result of deliberate iteration and manual refinement.\n-- Thibaut Mattio, OCaml Discuss, 2025
\n
Bryan Cantrill came up with a superb set of principles for LLM Usage at Oxide. In particular, he separates out using LLMs for reading, writing and coding. I totally agreed with him that I hate people sending me LLM-generated writing for me to review; I would rather get the raw prompt and use my own LLM+context rather than read through other people's slop.
\n\n\nLLM-generated prose undermines a social contract of sorts: absent LLMs, it is presumed that of the reader and the writer, it is the writer that has undertaken the greater intellectual exertion. (That is, it is more work to write than to read!) For the reader, this is important: should they struggle with an idea, they can reasonably assume that the writer themselves understands it — and it is the least a reader can do to labor to make sense of it.\n-- Using LLMs at Oxide, RFD0576, Dec 2025
\n
However, there is an undeniable (and growing) power in the ability to generate code at scale using LLMs. I've been doing a lot of this with Python in recent months, but I find myself increasingly frustrated by the lack of typing guardrails involved with agentic coding there.
\nI believe that a strongly typed, modular language like OCaml could become one of the best languages for agentic coding in the longer term, with advances happening rapidly to cure the data deficiency problem for relatively obscure languages with smaller corpuses. Also, with OxCaml on the horizon, getting help with increasingly complex (but rewarding) code annotations such as modes and kinds sems essential.
\nAfter reflecting on the recent discussions, I decided on these for my little\nDecember experiment:
\nAnd overall, this process should not help me learn more about agentic workflows\nbut also contribute to the wider discussion, so I'll capture what I learn in\nthis blog series at the end.
\nSome non-rules:
\nIf anyone else wants to join in the Advent of Agentic Humps, ping me on\nwhatever communication medium you like. Just remember the groundrules: don't waste\nother maintainer's time without their permission first.
I'm somewhat frazzled after managing to get through 25 days of agentic coding. It hasn't actually been that much physical work, but I\nunderestimated just how much my brain would be in full gear multitasking across\nso many terminal windows and ideas. My outbound queue that I didn't manage to write\nup is enormous: I have sessions running with OxCaml experiments, io_uring webservers, implementations of ATProto, a pure OCaml Parquet, and some even stranger ideas I won't go into now!
\nIn the past, my computer systems brain was limited by the speed of coding, but now it feels like we're entering a different age. I'll reserve my longform thoughts on all of this for the new year as I need to head into Christmas festivities, but I wanted to leave you all with my Claude Code OCaml marketplace in case you want to try this stuff for yourself!
\n ![\"%c\"](\"/images/aoah-plugin-ss-1.webp\")
Claude has made it very simple to distribute plugins. I just created a GitHub avsm/ocaml-claude-marketplace and kicked off Claude's own development plugin to develop plugins with. I then fed it all my skills and asked it to generalise them a bit. They'll likely need some work to generalise beyond me, but feel free to send in PRs!
\n![\"%c\"](\"/images/aoah-plugin-ss-4.webp\" "\\"Just")
![\"%c\"](\"/images/aoah-plugin-ss-2.webp\" "\\"It'll")
![\"%c\"](\"/images/aoah-plugin-ss-3.webp\" "\\"And")
I'll be back in the new year with more thoughts, but in the meanwhile, I hope\nyou all have a very good holiday and a chance to recharge! It's going to be a\nmad 2026...
", "url": "https://anil.recoil.org/notes/aoah-2025-25", "title": "AoAH Day 25: Claude OCaml Marketplace for all your festive coding needs", "summary": "Wrapping up 25 days of agentic coding with a Claude Code OCaml plugin marketplace to share the skills and tools developed throughout the series.", "date_published": "2025-12-25T00:00:00.000000Z", "date_modified": "2025-12-25T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ai", "ocaml", "oxcaml", "llms", "aoah" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-24", "content_html": "My original purpose for starting this AoAH series was to build a\nfeed aggregator for my group website, so I had to finish up with something to show!
\nI'm not sure if taking the longest way around\nwas wise here but I ended up building tuatara, an\naggregator to pull together all my colleagues' writing into one place.\nThey're a quirky bunch with many diverse homegrown feeds in various\nstates of brokenness, so it's difficult to build a one-size-fits-all tool.
\nSo given it's the end of the year and I'm sozzled on Christmas eve on\nmulled wine, I decided to make Tuatara mutate its own code by linking with my Claudeio library to\nforce it to evolve and modify itself as it runs across feed errors. Every deployment of\nTuatara is meant to be slightly different.
\nThe initial generation of the code was pretty straightforward, using Sqlite to\nstore a database with all the posts and importing metadata from my previously\ncreated Sortal contacts manager.
\n> tuatara import-sortal\nSortal Import Results:\n\n Total contacts scanned: 420\n Contacts with feeds: 15\n Feeds imported: 16\n Feeds skipped (already exist): 0\n\nRun 'tuatara fetch' to download posts from the imported feeds.\nBut when we actually get the feeds, I rapidly realised that there are lots of\nparsing quirks needed:
\n> tuatara fetch\nFetching Anil Madhavapeddy...\n 340 posts (0 new)\nFetching David Allsopp...\n Not modified\nFetching Jessica Man...\n Not modified\nFetching Jon Ludlam...\n 28 posts (0 new)\nFetching Jon Sterling...\n Not modified\nFetching Mark Elvers...\n Not modified\nFetching Martin Kleppmann...\n Error: Feed parse error: document MUST contains exactly one <feed> element at l.0 c.0\n URL: http://feeds.feedburner.com/martinkl\nFetching Onkar Gulati...\n Error: Not_found\n URL: https://onkargulati.com/feed.xml\nFetching Patrick Ferris...\n Error: Feed parse error: <entry> elements MUST contains at least an <author> element or <feed> element MUST contains one or more <author> elements at l.1460 c.7\n URL: http://patrick.sirref.org/weeklies/atom.xml\nFetching Richard Mortier...\n 79 posts (79 new)\nFetching Ryan Gibb...\n 38 posts (38 new)\nFetching Sadiq Jaffer...\n 10 posts (10 new)\n\nTotal: 127 new posts (3 errors)\nEither we skip content, or talk to the people involved to fix their feeds, but\nit's Christmas eve so that's unlikely. And anyway, we want to be liberal in\nwhat we accept so why\ncan't I fix my own software first?!
\n![\"%c\"](\"/images/aoah-tuatara-ss-1.webp\" "\\"Like")
The non-obvious and probably-terrible answer here is to use our fancy coding\nmodels to force the Tuatara source code to heal itself. I added an --evolve\nflag to allow tuatara to invoke Claude/OCaml upon errors and patch its own code.\nTurns out self help works, as Martin's blog feed was just fine!
Fetching Martin Kleppmann...\n Error: Feed parse error: document MUST contains exactly one <feed> element at l.0 c.0\n URL: http://feeds.feedburner.com/martinkl\n\nInvoking Claude Code to fix parse error...\n\nThe feed from `http://feeds.feedburner.com/martinkl` is an **RSS 2.0 feed** (it\nstarts with `<rss version="2.0">`), but tuatara was incorrectly detecting it as\nan **Atom feed**. This caused the Syndic Atom parser to fail with "document\nMUST contains exactly one \\<feed\\> element".\n\nThe root cause was that the `detect_feed_type` function prioritized the HTTP\n`Content-Type` header over the actual content. FeedBurner (and other feed\naggregators) often serve RSS feeds with an incorrect `application/atom+xml`\ncontent-type header.\n\nThis is a generic fix that will work for any feed aggregator or CDN that\nmis-labels RSS feeds as Atom (or vice versa), so no domain-specific quirk was\nneeded.\nBut the parsing drama continued, as Michael Dales uses the wrong date format in his\nfeed (tsk tsk I'd send the RFC police out if it wasn't Christmas), but Tuatara\nevolves a quirk that gets past it:
\n\n\nThe quirk module converts ISO 8601 dates (2025-10-22T12:24:00-00:00) to RFC\n822 format (Wed, 22 Oct 2025 12:24:00 GMT) which is what Syndic's RSS2 parser\nexpects.
\n
And Onkar Gulati and Patrick Ferris both have an empty author field which would\nordinarily give us a dreaded Not_found exception:
\n\nFetching Patrick Ferris...Error: Feed parse error:
\nelements MUST contains at least an element or element MUST contains one or more elements at l.1460 c.7 URL: http://patrick.sirref.org/weeklies/atom.xml
But never fear, the inexorable --evolve flag figures it out and patches its own code!
There were some non-trivial quirks as well; Andres Zuñiga-Gonzalez uses Quatro for his website which puts the entire HTML blob into the summary field, but the evolution managed to use html5rw to parse its way out of this. This sort of fix is very hard to generalise, so it's actually quite useful for the tool to fix itself on demand for our small group.
\nThen I needed a quick way to do a clean frontend output so I can visualise the\nJSONfeed. Claude has a /plugin frontend-design skill that is built in, and\nprompting it to give me a few designs let me integrate a --html output.
And because it's Christmas, I added some snowflakes as well. Yay!
\n ![\"%c\"](\"/images/aoah-tuatara-ss-4.webp\" "\\"Ho")
The paper I enjoyed writing the most this year was Steps towards an Ecology for the Internet for\nAarhus 2025. In the back of my head since has been a desire\nto start figuring out what self-evolving software actually might be. It's a\nstrange, and probably impractical idea, but I'm delighted that I took a tiny\nstep towards it with this project.
\nBack in March, I had the honour of being invited to a Bellairs meeting to discuss a heady combination of semantics and computational science. Jon Sterling demonstrated his wonderfully organised Forester website. And I... showed how my mismash of semi-structured writings can kind of be connected together in a vaguely coherent way to build my website. Next year will have me thinking much harder about the implications of self-evolving code, of how radically transformative to global biodiversity semi-structured agentic processing might be, and other heavy matters. But to close this year, I'm disproportionately pleased to have gotten my tiny website under control a little!
\n![\"%c\"](\"/images/aoah-tuatara-ss-2.webp\" "\\"Sitting")
As I noted in my letter to the ACM, it's important that we can use AI for things that boost the\nhuman condition; I really enjoy reading my colleagues' long form thoughts much\nmore than doomscrolling on the web, and so making it easier to gather their\nthoughts digestibly and easily is a nice end to my agentic humps\neffort. Tomorrow on Christmas I'll publish all the skills I used so others can try them out.
Yesterday's monopam workflow used git submodules to combine\nvendored packages, but was awkward to use for crosscutting changes involving\nlots of vendored git repositories. Today I asked what agentic code development\nwould look like if we could unify all code into a single git repository,\nwhere upstream packages become branches instead of submodules. I've\nopen-sourced the unpac CLI to\nexplore this, and have begun using it myself.
\nCoding agents work best when all relevant code is locally available so\nthey can grep and make cross-cutting changes. I first noticed this when building Bonsai terminal UIs and Mosaic, where I had to manually assemble monorepos just to get the agent working.\nThings come to a crashing halt when package management\ngets involved; the tool calls for web search are far slower and unreliable.\nThis means that the agent doesn't really have a good view on what third-party\npackages might be useful to solve a problem, leading to the common complaint\nthat LLMs reinvent the wheel.
\nTo fix this, unpac parses package metadata and materialises it into a git branch\nstructure in a single repository to make vendoring, patching, and updating\na native git workflow. Local changes can later be exported into git patches for\nsending upstream, but in the meanwhile our agents can work on a single\nrepository.
\nThe secret sauce to working on so many branches is to use git worktrees, which allow multiple\nbranches to be checked out simultaneously from one git repo! I'll explain how unpac works next, and you can\nbrowse a working unpac tree. You do end up with a\nlot of git branches, which got me banned from GitHub back when I announced Docker DataKit. Luckily this time around I am hosting on Tangled where I host my own Git remotes and so don't have to worry about third-party SLAs!
\n ![\"%c\"](\"/images/aoah-unpac-ss-6.webp\" "\\"All")
unpac organises code and dependencies using a lot of unrelated git branches, with\ncareful merging across them. The main branch only holds the unpac metadata about which projects exist and which opam remotes to use.\nWhile this defaults to the upstream opam-repository, I'm also using this with the oxcaml/opam-repository and my aoah-opam-repo overlays (maintained via the opam metadata skill) to help track community forks.
[opam]\ncompiler = "5.4.0"\n\n[[opam.repositories]]\nname = "opam"\npath = "/workspace/opam/opam-repository"\n\n[[opam.repositories]]\nname = "aoah"\npath = "/workspace/opam/aoah-opam-repo"\nEach third-party opam package then has three branches in my unpac repo:
\nupstream/opam/<pkg> holds the unmodified upstream code and historyvendor/opam/<pkg> is upstream history relocated to a vendor/opam/<pkg>/ prefix using git-filter-repo.patches/opam/<pkg> is the vendor branch with any local changes applied.The projects you are working on are all in a project/<name> orphan git\nbranches, independent of all the others. Adding a dependency to a project is a mere matter of\nmerging the vendor branch for a given dependency into the project branch.
This merging will materialise the dependency code in the project branch\nconflict-free under vendor/opam/. This allows us to build a monorepo of OCaml\ncode that's maintained its git history across all the different developers,\nwhile also allowing local commits to be held and rebased. The agent has a ton\nof context available to it now without having to go to the outside world!
Before showing off the unpac CLI, it's worth explaining git worktrees as I'd\nnever used them before today. Normally, you can only have one git branch\nchecked out at a time, but worktrees free us of that restriction:
\n\n\nA git repository can support multiple working trees, allowing you to check\nout more than one branch at a time. With
\ngit worktree adda new working\ntree is associated with the repository, along with additional metadata that\ndifferentiates that working tree from others in the same repository. The\nworking tree, along with this metadata, is called a "worktree".This new worktree is called a "linked worktree" as opposed to the "main\nworktree" prepared by git-init or git-clone. A repository has one main\nworktree (if it's not a bare repository) and zero or more linked worktrees.\nWhen you are done with a linked worktree, remove it with
\ngit worktree remove.\n-- git-worktree documentation, 2023
Creating them is pretty straightforward using git worktree add. This creates\na new checkout with the .git entry being a file containing an entry\nlike:
gitdir: /workspace/git/worktrees/tuatara\nWithout worktrees, agents fall over themselves switching branches due to the\nrequirement that all files be committed before switching. With worktrees, you\ncan have uncommitted stuff in multiple branches, meaning we can simultaneously\nview upstream code while making patches, compare vendor and patches branches\nside-by-side with diff, and work on multiple projects from the same repository.\nI used to have these affordances back when I used Mercurial and Perforce (with\ndifferent distribution models, admittedly), so it's great to have it back!
\nThis elaborate git schema is all very good, but not something I'd want to manage\nmanually. The unpac CLI takes care of all the gruntwork involved,\nincluding integrating an opam solver to create 100s of branches with one\ncommand. Let's take a look at an example project:
\n$ unpac init\n\n# Vendors in packages from opam including dependencies\n$ unpac add opam eio --solve\nWe now have a bunch of vendor branches in our local repository, and need to\ncreate a project to use them:
\n# Create a new project branch\n$ unpac project new myapp\n\n# Merge in the patch branches of these dependencies into project/myapp/vendor\n$ unpac opam merge eio --solve myapp\n\n# Hax0r like it's 2026 on your project\n$ cd project/myapp\n![\"%c\"](\"/images/aoah-unpac-ss-1.webp\" "\\"The")
We can now do a simple dune build in the project/myapp directory since all our code is present in one working branch.\nA typical unpac project looks like:
.\n└── dune-project\n├── vendor/\n│ ├── eio/ # vendored eio source\n│ ├── lwt/ # vendored lwt source\n│ └── ...\n├── src/ # your project source\nDune automatically discovers and builds the vendored packages. No special configuration is needed beyond standard dune files.
\nHowever, some packages don't build with dune since the upstream projects don't use dune but have other build systems (quite reasonably! Choice is important). In the past, I have maintained over 50 dune ports via opam overlays, mostly by hand. However, I can now easily use my coding agent to do all the porting automatically.
\n![\"%c\"](\"/images/aoah-unpac-ss-4.webp\" "\\"Claude")
You can see some of the diffs in the patch branches in my working tree: patches/logs or patches/cmdliner or patches/bos for example. Since the agent has a clean local interface to work with, it can keep its commits neatly organised.
\nAn unpac vendor status command neatly summarises the status of which packages have been patched, and which project they're merged into:
$ unpac vendor status\nPackage Patches Merged into\n----------------------------------------------------------------------\nangstrom 0 -\nasn1-combinators 0 myapp\nastring 0 -\nbase64 0 myapp\nbigstringaf 0 -\nbos 1 -\nbytesrw 0 -\nbytesrw-eio 0 -\nca-certs 0 -\ncheckseum 0 -\ncmdliner 1 tuatara\nconpool 0 -\ncookeio 0 -\ncsexp 0 myapp\ncstruct 0 -\ndecompress 0 -\ndigestif 0 myapp, tuatara\ndomain-local-await 0 -\ndomain-name 0 myapp\nYou can see here that cmdliner has been patched and merged into the tuatara\nproject, but bos has been patched and is unmerged. This is all calculated\ninternally via git commands, so there's no separate metadata store to get out\nof sync.
I haven't completed porting all the third-party packages to use dune just yet,\nbut I've left it running overnight. When that's done, the big feature we gain\nis that a dune build can seamlessly cross-compile binaries since all the OCaml\ncode and C bindings are in one place. This is what MirageOS\n4 does, and we can reap the benefits now for\nconventional binaries too. Windows builds should also be a lot easier as long\nas the dune rules don't have too many Unixisms.
\nThe unpac CLI was self-explaining enough that another agent session could import\nan OCaml project by analysing that project and running the sequence of unpac\ncommands.
\n![\"%c\"](\"/images/aoah-unpac-ss-2.webp\" "\\"Importing")
In order to reduce the load on external git clones, unpac also supports having a local "git branch cache" which pulls remotes just once, and then all unpac invocations pull from that local store. As an experiment over the holidays, I've left a session doing a slow clone of all opam git remotes, to see how well git scales to a few thousand branches.
\nWe do end up with 100s of local branches, and so an unpac push command checks which ones need pushing and takes care of it for you.
![\"%c\"](\"/images/aoah-unpac-ss-3.webp\")
You can browse one of my working unpac repositories on\ntangled/anil.recoil.org/unpac-work\nto get a sense of the structure.
\nI'm still working on the pulling/rebasing functionality, but the basic idea is\nall the same: pull from the outside world into a pristine branch, relocate the\ndirectory, and then have local patch branches.
\nThe current unpac focuses on opam, but Ryan Gibb has been leading the research on a generalised packaging language that can describe package management across ecosystems. Imagine something like this in a future unpac:
\n# Works with opam, npm, cargo, pip...\ndependencies = [\n { source = "opam", name = "eio", version = ">=1.0" },\n { source = "npm", name = "d3", version = "^7.0" },\n { source = "cargo", name = "tokio", version = "1.0" },\n]\nI've already had need for this last week when I vibespiled 50 HTTP libraries across 10 languages into an OCaml implementation. I really want to be able to more easily draw from other language ecosystems, and unpac's git branch model works regardless of the package manager (hence the opam/ suffix for vendor/ branches).
![\"%rc\"](\"/images/aoah-unpac-ss-5.webp\" "\\"I")
Unpac's branching model actually doesn't work hugely well with GitHub due to the storage limits on an account being hit pretty fast, but it's peachy when used with self-hosted Git services. I'm sure we could do something with Git object alternates as well to improve on this in the future.
\nThere's quite a lot of work required to make unpac production grade, but I'm astounded by how quickly I could put this prototype together in a day. Sketching out CLI tools and cram tests is extraordinarily fun as well, as I could specify my desired user interface and then engage in a Socratic dialogue with the agent to refine the specification.
\nI'm also having subversive thoughts now about issue management. I've been a fan for many years of Jane Street's Iron code review system. However, despite having talked to Stephen Weeks and Yaron Minsky extensively about it over the years, I've never found the bandwidth to build an equivalent for open source. But with coding agents being able to interpret natural language alongside code, it seems like a really obvious extension to also store issues within branches as well as code, and to unify our agent context horizon. Something for the 2026 queue!
\nI'd love to hear any feedback on unpac's model from other projects. I wouldn't use the tool I've released just yet as it's only about 18 hours old, but I'll work on it more in the new year as well and do a proper release once it's self hosting. Many thanks to Ryan Gibb and Patrick Ferris (who came up with the name) for several design discussions that lead to this post.
\nSome useful comments about this post:
\nFirst, Török Edwin from the XenServer team reports that:
\n\n\nXAPI has 2 invalid commits in its commit history (invalid email address\ncontaining a duplicate of the author name and email), so it is impossible to\npush its history into a brand new GitHub repo, you can only do that by\nforking the original repo through the API. Although I see you are not using\nGitHub, so you might be fine. Still might be a good idea to run
\ngit fsckon\nthe final repo, XAPI may not be the only project with some invalid commits in\nits history, and that could create problems later (e.g. iftangledstarts\nrunninggit fsck)".
Something to investigate for the new year: antagonistic git remotes! Then, Michael Dales notes that:
\n\n\nFor any production system I’ve had to deliver, I’d always vendor in third\npart code using git submodules (and those taken from a local mirror of each\nrepo). You just can’t use external dependencies if you need to know you can\nship updates to customers at the drop of a hat, and you never know when an\nopen source project might go away.
\n
And then Dave Scott reminded me how much this proposed branch scheme makes it easier for shipping\nproducts to comply with open-source licenses. In Docker for Desktop we have an elaborate license gathering\nscript for the 'about' box\nto credit contributors.
There's outrage in the computer science community over a new feature rolled out\nby the ACM Digital Library that generates often inaccurate AI summaries. To make things worse, this is hidden behind a 'premier' paywall, so authors without access (for\nexample, having graduated from University) can't even see what is being said.
\nThe summaries themselves are deeply average. Looking at one of my\nrecent papers, it somehow expands a carefully crafted\ntwo paragraph summary into a six paragraph thing that says roughly the same\nthing. This seems like exactly the wrong place to apply LLM technology to as\nit's replacing a carefully peer-reviewed paragraph with a longer slopful version.
\n![\"%c\"](\"/images/acm-slop-1.webp\" "\\"The")
The ACM stands for the dissemination of knowledge accessibly. I could imagine cases where summarising abstracts would be useful: for example, into foreign languages for which no such abstract exists, or really nice audio transcriptions for assistive usage. However, putting it behind a paywall and distracting from peer-reviewed human-created content is really, really bad.
\nI dug in a bit deeper to find out more, and discovered this statement:
\n\n\nCurrently, we offer written summaries of individual articles as well as podcast-style audio summaries of conference sessions. We will soon add chat-style interactivity to our content and search functionality. All summaries on the Digital Library are clearly labeled as AI-generated. When citing ACM content, authors should always cite the original article, not an AI-generated summary.
\nAI can make mistakes, and it cannot replace the experience of reading an article in full. But we do believe it will help you find, understand and use ACM content both more quickly and more deeply.
\nThese tools were designed in consultation with a diverse group of Digital Library stakeholders and will continue to evolve as Artificial Intelligence advances. We are continuously tuning our Foundational Model to optimize readability and we conduct regular audits for hallucinations and other errors. We are very interested in your thoughts and suggestions- please leave them by clicking the "Feedback" button on the far right of this page. If you find a problem with a specific AI-generated summary, please return to that summary and click the Feedback there.\nArtificial Intelligence Tools in the ACM Digital Library, undated.
\n
I have many questions here: who is this diverse group of stakeholders, what foundation model is being used, what tuning happened, what audits, and what is happening with the corrections from authors. Are we suddenly using the world's scholars to create a fine-tuning label database without their permission? There's a definite lack of transparency here.
\nLuckily, Jonathan Aldrich is on the ACM Publications Board, which must be a thankless job. He acknowledged this very graciously yesterday during the outrage:
\n\n\nI also owe the community an apology; I was told about this feature (though I'm not sure I was told it was going to be the default view). I should have recognized the potential issues and been loudly critical earlier, before it went live. But I will do my best to get it fixed now.\nJonathan Aldritch, Mastodon sigsocial, 17th Dec 2025
\n
This got me thinking about what the ACM should be doing instead of this. Putting these abstracts up represents not only a step in the wrong direction, but also a high opportunity cost of not unlocking some other positive activity that can leverage AI for social good.
\nWe're at a real crossroads with scientific communication and scholarly publishing, but I firmly believe that the ACM can correct itself and make a real difference.
\nLooking through the ACM digital library footer, I see news channels using X, LinkedIn and Facebook. The only open protocol listed is email, although I did discover an (unlisted on the ACM website) Bluesky account.
\nNone of these platforms are conducive to longform, thoughtful community conversations. Let's look at ACM's mission statement:
\n\n\nACM is a global scientific and educational organization dedicated to advancing the art, science, engineering, and application of computing, serving both professional and public interests by fostering the open exchange of information and by promoting the highest professional and ethical standards.\n-- ACM's Mission, Vision, Core Values and Goals, 2025
\n
The platforms the ACM has chosen for communicating with the scholarly community are algorithmic engagement factories. There are countless papers on the ACM Digital Library itself recording the harms and spread of misinformation.
\n\n\nWhile early ads were found to be effective in creating brand awareness and positive attitudes, recent Internet advertising has been described as nonsensical, uninformative, forgettable, ineffective, and intrusive.\n-- The Effects of Online Advertising, Communications of the ACM, 2007
\n
Instead, the ACM should focus on using models that encourage scholarly discourse, such as standards-based mechanisms like an Atom/RSS feed for their news (which can be consumed widely and accessibly), and consider increasing engagement on non-advertising-driven platforms such as Bluesky. A poll at the start of the year from Nature revealed that 70% of respondents use that platform. I suspect it's less for computer science, but the ACM setting a direction would also go a long way to give community direction.
\n(Update 22nd Dec 2025: Stefan Wagner and Vadim Zaliva both point out that the ACM also already runs a Mastodon instance, which is easily bridged to Bluesky. A fine alternative!)
\n ![\"%c\"](\"/images/acm-slop-3.webp\" "\\"The")
I've been working on collective knowledge principles to boost the conservation evidence project. As part of this process, I've downloaded tens of millions of fulltext papers to help figure out where living things are on the planet. By far the most difficult task here was getting access to even the open papers. At the recent COAR meetup, half the conversations were around the difficulty of obtaining knowledge even before curation.
\nIncredibly, while just browsing around the ACM DL in order to research this article, I got blocked from the entire library. This was after opening about 10 browser tabs: not an unusual amount of human traffic!
\n![\"%c\"](\"/images/acm-slop-2.webp\" "\\"I'm")
Contrast this to the Public Library of Science (PLOS), which has a allofplos repository that allows me to download the entire fulltext paper repository by running a single line of Python: pipenv run python -m allofplos.update. The script not only downloads papers, but does a bunch of important bookkeeping:
\n\nThe script:
\n\n
\n- checks for and then downloads to a temporary folder individual new articles that have been published
\n- of those new articles, checks whether they are corrections (and whether the linked corrected article has been updated)
\n- checks whether there are VORs (Versions of Record) for uncorrected proofs in the main articles directory and downloads those
\n- checks whether the newly downloaded articles are uncorrected proofs or not after all of these checks, it moves the new articles into the main articles folder.\n-- AllOfPLOS README, 2016
\n
PLOS has been doing for years, so why hasn't the ACM done this yet for its open access papers? I applaud the ACM's recent shift to open access by default but this is pointless if not accompanied by an investment in the dissemination of knowledge.
\nOne of the most exciting things about Bluesky is that it allows for the reuse of the identity layer to build other services. Right now we are seeing that AI poisoning of the literature is upending all kinds of evidence-driven social norms, and is a huge threat to rational policymaking for all of society.
\nThe ACM is uniquely positioned in computer science as the body that could build a reasonable reputation network that not only identifies academics, but also enforces provenance tracking of whether papers and artefacts did in fact follow a rigorous methodology. LLMs are now amazingly good at constructing fake papers, and so capturing the peer review process and building up a defence against "knowledge from thin air" will be one of the great challenges for the remainder of this decade.
\n ![\"%c\"](\"/images/davidparkins-ai-poison.webp\" "\\"AI")
My December adventures in agentic programming have been eyeopening in just how quickly I can build hyper-personalised views on a large body of knowledge. While many computer science scholars tend to view LLMs skeptically, there is a good use of agentic summaries of papers: by allowing readers to summarise papers directly for themselves when supplying the LLM with other information about what they already know.
\nBryan Cantrill explained this best in his principles for LLM Usage at Oxide. He separates out using LLMs for reading, writing and coding. I totally agreed with him that I detest people sending me LLM-generated writing, but he teased out a good explainer as to why:
\n\n\nLLM-generated prose undermines a social contract of sorts: absent LLMs, it is presumed that of the reader and the writer, it is the writer that has undertaken the greater intellectual exertion. (That is, it is more work to write than to read!) For the reader, this is important: should they struggle with an idea, they can reasonably assume that the writer themselves understands it — and it is the least a reader can do to labor to make sense of it.
\n
\n-- Using LLMs at Oxide, RFD0576, Dec 2025
And that, dear reader, is why the ACM redistributing AI summaries is a bad idea. It breaks the social contract with the reader that the ACM Digital Library is a source of truths which the scholars who contributed it do understand. We might not agree with everything on the library, but it's massively dilutive to have to sift through AI-generated writing to get to the original bits.
\nIf the ACM itself deliberately introduces errors into its own library, that's a massive self-own.\nInstead, if the ACM Library exposed a simple text-based interface that allowed my agents to do the papers summaries just for me, then that personalisation makes it useful. I find deep research agents surprisingly useful when exploring a new field, but primarily because I can guide their explorations with my own personal research intuition, not someone elses.
\nMy appeal to the ACM is to not try to build differentiated paid services using AI. Let the rest of the profit-driven world do that and peddle their slop; at least they are earning money while doing so (hopefully)! The ACM needs to be a force for creative disruption and discovery, and help defend and nurture the joys inherent in computer science research.
\nThis means taking a critical view at how AI is impacting all aspects of our society, but not just rolling out bland services: instead, deploy AI technologies that enhance the human condition and allow us to be even more inquisitive with our time on earth. The recent Royal Society meeting on Science in the Age of AI put it very well:
\n\n\nA growing body of irreproducible studies are raising concerns regarding the robustness of AI-based discoveries. The black-box and non-transparent nature of AI systems creates challenges for verification and external scrutiny. Furthermore, its widespread but inequitable adoption raises ethical questions regarding its environmental and societal impact. Yet, ongoing advancements in making AI systems more transparent and ethically aligned hold the promise of overcoming these challenges.\nScience in the Age of AI, Royal Society, 2024
\n
This is a well balanced view, I feel. There are huge challenges ahead of us, but also huge opportunities for new discoveries!
\nIn the meanwhile, I remain blocked from the ACM Digital Library for unknown\nreasons, so I guess it's time to start the Pembroke Christmas feasting a few\nhours early! Anyone want to head down to the pub now?
Over the past three weeks, I've accumulated dozens of OCaml repositories as\npart of this series. Keeping them coordinated has become a real challenge;\nwhen I fix something in one library, dependent packages need updating, and\nagents working on one repo have no visibility into related code. Ideally,\nI could have all my code in one place and see what agents can do with a lot of local context.
\nToday I'm switching tacks to address this with a monorepo workflow built around dune's\nexcellent vendoring support.\nI last visited this when building RWOv2 and its monorepo when I built a duniverse tool that turned into the opam-monorepo plugin that MirageOS\nnow uses. Let's see what happens in today's agentic world instead!
\nI also wanted to explore the small group dynamic around vibecoding tools. For today's tool, I first asked Mark Elvers to spend a few hours sketching out the sort of tool he might want, and then Jon Ludlam has been using Claude to build up complex odocv3 rules. The way we work together with agentic code is quite different from when we've handcrafted a project, with the code itself now being more throwaway as we pass the baton among each other. I'm lightheartedly calling this 'vibrating' amongst each other to reflect the new speed of agentic iterations, and to differentiate from the more thoughtful process of pair programming. Today's tool monopam helps to manage OCaml monorepos for cross-cutting code and documentation.
\nThe OCaml libraries I've built are designed to be standalone, but obviously have interdependencies among each other. Requests depends on conpool for connection management and HTTP cookie logic from cookeio. The codec libraries like yamlt, tomlt, and init all have optional dependencies on bytesrw for serialisation. Meanwhile, html5rw depends on langdetect and has optional dependencies on the wasm and JavaScript compiler stack.
\nHere's the full inventory of what I've built in the last few weeks:
\n| Day | \nLibrary | \nDescription | \n
|---|---|---|
| 1 | \nocaml-crockford | \nCrockford Base32 encoding | \n
| 2 | \nocaml-jsonfeed | \nJSONFeed 1.1 implementation | \n
| 3 | \nxdge | \nXDG directories with Eio capabilities | \n
| 4 | \nclaudeio | \nClaude OCaml/Eio SDK | \n
| 5 | \nocaml-bytesrw-eio | \nBytesrw/Eio adapter | \n
| 6 | \nocaml-yamlrw | \nPure OCaml Yaml 1.2 parser | \n
| 7 | \nocaml-yamlt | \njsont codecs for Yaml | \n
| 8 | \nsortal | \nContacts management CLI | \n
| 11 | \nocaml-punycode | \nPunycode RFC3492 implementation | \n
| 11 | \nocaml-publicsuffix | \nPublic suffix list for cookies | \n
| 11 | \nocaml-cookeio | \nHTTP cookie handling | \n
| 12 | \nocaml-conpool | \nTCP/TLS connection pooling | \n
| 13 | \nocaml-requests | \nHTTP client library | \n
| 14 | \nocaml-karakeep | \nKarakeep bookmark API | \n
| 15 | \nocaml-html5rw | \nHTML5 parser and validator | \n
| 16 | \nocaml-json-pointer | \nJSON Pointer RFC6901 | \n
| 16 | \nodoc-xo | \nodoc extras for notebooks | \n
| 17 | \nocaml-jmap | \nJMAP email client | \n
| 18 | \nocaml-tomlt | \nTOML 1.1 codecs | \n
| 19 | \nocaml-zulip | \nZulip bot framework | \n
| 19 | \nocaml-init | \nINI file codecs | \n
| 20 | \nocaml-langdetect | \nLanguage detection | \n
And the Claude skills I've developed along the way:
\n| Skill | \nPurpose | \n
|---|---|
| claude-ocaml-metadata | \nAutomate opam package setup | \n
| claude-ocaml-internet-rfc | \nFetch and integrate IETF RFCs | \n
| claude-ocaml-tidy-code | \nRefactor generated OCaml | \n
| claude-ocaml-to-npm | \nPublish js_of_ocaml to NPM | \n
So far, I've been publishing each of these as individual Git repositories, but maintaining an overlay opam repo that a user can add to gain access to consistent metadata that makes the dev packages installable. Unfortunately, incorrect interdependencies are already creeping in; Thomas Gazagnaire asked me today why my yamlt library depends on webassembly, and I'm sure it shouldn't -- I've clearly got a stray missing dependency somewhere in my metadata.
\nWhen an agent works on just one repository, it has no visibility into\nhow changes might benefit (or break) dependent code. It also can't make fixes\nfor documentation across repositories. I noticed quite often in the past month that I was\ncloning source packages temporarily into my workspace for the agent to access, and then\ndeleting them. All this motivates me to investigate alternatives to having lots of small git repos for my day-to-day agentic development.
\nDune has a fantastic but underappreciated feature: it automatically discovers and builds any OCaml code in subdirectories. As David Allsopp explained back in 2018, you can simply clone dependencies into your project tree and dune will build them together.
\nNote that this only works if all the packages contain dune files. Since OCaml is all about choice, there's no hard mandate to use one build tool: it's perfectly fine to use ocamlbuild or Makefiles, as long as your libraries install a findlib META file. Dune will also gain support for opam package installation next year to help make this even easier.
\nYears ago I built a tool called duniverse to automate this vendoring workflow. It worked, but required a lot of manual repository management. With agents now doing the heavy lifting, though, I thought it might be easier now and so decided to revisit it.
\nToday's work ended up extending Mark Elvers initial foray into monorepos to release monopam: a little CLI tool that reads opam metadata from a local repository (like aoah-opam-repo), resolves the dependency graph, materialises the sources as git submodules, and produces a single dune workspace that builds everything together. For now, it depends on an opam local switch to work, but if someone wants to try it with dune package management I'd love to hear how it goes.
\nThe aoah-opam-repo contains all the packages I've built during this series, maintained using the opam metadata skill. Let's turn it into a unified source tree using monopam:
\n$ monopam --opam-overlay aoah-opam-repo -o aoah-vendor --submodules\nScanning opam overlay at aoah-opam-repo\nFound 21 repositories to process\nInitialized empty Git repository in aoah-vendor/.git/\nUsing git submodules for vendor dependencies\nCloning into ...\nremote: Enumerating objects: 14, done.\nremote: Counting objects: 100% (14/14), done.\nremote: Compressing objects: 100% (11/11), done.\nremote: Total 14 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)\n# ...etc\nOutput written to aoah-vendor\n opam-repository/ - opam package definitions\n vendor/ - source code\n setup.sh - run to pin packages and install deps\nThis solves the opam constraints, finds a cut of dependencies, and then git\nsubmodule adds the lot of them into my target repository. At this point, we run\nsetup.sh which creates an opam local switch and then dune build just works\nusing all the locally cloned repos.
.\n├── _opam\n├── dune\n├── dune-project\n├── opam-repository\n│ ├── packages\n│ └── repo\n└── vendor\n ├── dune\n ├── ocaml-bytesrw-eio\n ├── ocaml-claudeio\n ├── ocaml-conpool\n ├── ocaml-cookeio\n ├── ocaml-crockford\n ├── ocaml-html5rw\n ├── ocaml-init\n ├── ocaml-json-pointer\n ├── ocaml-karakeep\n ├── ocaml-langdetect\n ├── ocaml-publicsuffix\n ├── ocaml-punycode\n ├── ocaml-requests\n ├── ocaml-tomlt\n ├── ocaml-yamlrw\n ├── ocaml-yamlt\n ├── odoc-xo\n └── xdge\nThe directory structure is straightforward: we have our opam repository, a local switch\nand the source code all in one place now, and buildable in a single dune\ninvocation.
\nWith all the code in one place, agents can now spot opportunities that span\nmultiple packages. The first thing I did was to build a full documentation set\nacross all my packages.
\n![\"%c\"](\"/images/aoah-monopam-ss-1.webp\")
Jon Ludlam has been doing excellent work on odoc3, the modern documentation generator for OCaml. odoc is a composable documentation generator that has a number of mini-commands that can be called in sequence to build fragments of HTML. Jon's been adding support into dune build rules to build a fully cross-referenced documentation site across an entire dune workspace.
\nThis is where the monorepo approach obviously shines, since we could generate a single site for all my code with types linking directly to their definitions across opam packages. The interactive notebooks I built earlier could reference any type across the whole codebase.
\nI first pinned Jon's odoc branch and then built the unified docs with the right rules.
\n$ opam pin add dune https://github.com/jonludlam/dune.git#odoc-v3-rules\n$ dune build @doc\n$ open _build/default/_doc/_html/index.html\nThis generated a working doc page, that also included cross-referenced links\nacross packages. But even more cool is that if a package doesn't exist in the\nlocal monorepo, it also does a best-effort link straight to the central doc\nrepository on OCaml.org.
\nThere were a few integration issues that may be bugs in the dune rules. For instance:
\n> dune build @doc\nFile "/Users/avsm/src/git/knot/aoah-vendor3/_opam/lib/angstrom/META", line 1, characters 0-0:\nError: Library "angstrom-unix" not found.\n-> required by library "angstrom.unix" in\n /Users/avsm/src/git/knot/aoah-vendor3/_opam/lib/angstrom\n-> required by alias vendor/ocaml-karakeep/doc\nThis is a package that's present in the local tree, but not installed in opam.\nAfter I opam installed it, the doc generation worked. This probably shouldn't\nbreak a local docs build, so I commented on the GitHub PR.
\nAfter this, there were genuine bugs in my own documentation, as evidenced by\nwarnings emitted by dune. The agents fixed problems and added cross-references across\nthe documentation, and I could do a single git status to see all the affected\npackages.
Changes not staged for commit:\n (use "git add <file>..." to update what will be committed)\n (use "git restore <file>..." to discard changes in working directory)\n (commit or discard the untracked or modified content in submodules)\n modified: vendor/ocaml-claudeio (modified content)\n modified: vendor/ocaml-init (modified content)\n modified: vendor/ocaml-json-pointer (modified content)\n modified: vendor/ocaml-requests (modified content)\n modified: vendor/ocaml-tomlt (modified content)\n modified: vendor/ocaml-yamlrw (modified content)\n modified: vendor/ocaml-yamlt (modified content)\n modified: vendor/xdge (modified content)\nI then prompted the agents to find opportunities for optimisation across\nall the packages. Running this in a fixpoint ended up allowing for backwards\nand forwards cross-references: for example, it could add "related libraries"\nsections, and also normalise error handling and logging interfaces where\nthere were inconsistencies.
\n![\"%c\"](\"/images/aoah-monopam-ss-2.webp\" "\\"Docs")
\n![\"%c\"](\"/images/aoah-monopam-ss-3.webp\" "\\"And")
Once I had a consistent monorepo, I could commit the changes and distribute\na batch easily. For example, I uploaded my test odocv3 monorepo\nand commented on ocaml/dune#12995.
\nOn the other hand, monopam's git submodule workflow is awkward to use due to how separate\nsubmodules are from the main git repository. I had to individually commit and push each of the\nchanges, and I couldn't get a unified git diff or make commits across the vendored\nrepositories. I have a scheme in mind to improve this, which is a topic for tomorrow's post!
\nSocially speaking, I'm reasonably convinced a monorepo workflow of some sort is the\nfuture for agentic coding. They just work so much better with local tool calls that can\nrapidly scan a lot of data instead of making remote calls (which are awkward from a permissions\nperspective as well). We'll still need to figure out how the dynamics of 'vibrating' patches\nacross each other goes; it's early days for the dynamics of agentic pair programming.
With language detection now working in OCaml, I completed vibespiling the Nu HTML Validator from Java to OCaml. This is the official W3C validator used to check HTML5 conformance, and it's a substantial codebase with thousands of validation rules. I set out to see what a few days of agentic processing would do to transform the complex Java codebase into a more functionally structured pure OCaml codebase.
\nThe result is a pure OCaml HTML5 conformance checker that integrates with the parser I built last week, all published as ocaml-html5rw. Having the logic in pure OCaml meant that I could also compile it into standalone JavaScript and WASM. Dynamic conformance checking works even better than server-side filtering since live JavaScript executing on the page (and modifying the DOM) can also be checked. I published this to NPM using a new Claude skill, and coded a live panel overlay to live debug HTML5 issues that I use on my own website now.
\n ![\"%c\"](\"/images/aoah-html5v-ss-1.webp\" "\\"My")
I'm going to talk about my results in reverse today, since I thought the outcomes were so unexpectedly useful. I took yesterday's session and asked Claude to build an ocaml-to-npm skill, which I used to publish the html5rw JavaScript and wasm to npm.
\nThis runs the HTML5 validation OCaml code by serialising the live DOM tree and then collecting the various validation errors along with the source. This is sufficient to populate an overlay panel that can not only list the errors, but also highlighting the offending DOM node with a red box. This spotted lots of dynamic errors in my website!
\nPublishing on NPM is quite convenient as there are several CDNs that serve the JavaScript directly. I integrate this into the development version of my blog as simply as:
\n<script src="https://cdn.jsdelivr.net/npm/html5rw-jsoo@1.0.0/htmlrw.js"></script>\n<script>\nfunction validateWithPanel() {\n const result = html5rw.validateAndShowPanel(document.documentElement, {\n // Annotation options\n annotation: {\n addDataAttrs: true,\n addClasses: true,\n showTooltips: true,\n tooltipPosition: 'auto',\n highlightOnHover: true\n },\n // Panel options\n panel: {\n initialPosition: 'topRight',\n draggable: true,\n collapsible: true,\n groupBySeverity: true,\n clickToHighlight: true,\n showSelectorPath: true,\n theme: 'auto'\n }\n});\n</script>\nI'll probably wrap this in a webcomponent in the\nfuture as Arthur Wendling did with x-ocaml but for\nnow this is already useful on my own website. If anyone has any pointers for\nwhat the right CSS patterns are for adding these debug overlay panels to\nwebsites with minimal intrusion, I'd be grateful. I'm extremely unfamiliar with how\nmodern frontend programming works...
\n ![\"%c\"](\"/images/aoah-html5v-ss-2.webp\" "\\"Did")
And of course, if you do prefer to stick to the server-side, then you get fast native code OCaml via a command-line binary provided by the package:
\n$ dune exec -- html5check test.html\ntest.html:126.73: error [no-p-element-in-scope]: No “p” element in scope but a\n“p” end tag seen.\ntest.html:113.72: error [missing-alt]: An “img” element must have an “alt”\nattribute, except under certain conditions. For details, consult guidance on\nproviding text alternatives for images.\ntest.html:120.27: error [duplicate-id]: Duplicate ID “duplicate-id”.\ntest.html:123.36: error [disallowed-child]: Element “div” not allowed as child\nof element “span” in this context. (Suppressing further errors from this\nsubtree.)\ntest.html:152.8: info [multiple-h1]: Consider using only one “h1” element per\ndocument (or, if using “h1” elements multiple times is required, consider using\nthe “headingoffset” attribute to indicate that these “h1” elements are not all\ntop-level headings).\nThe reason this took a few days of background vibespiling comes down to the sheer size of the problem. The Nu Validator is a mature Java application that is built around Java's SAX event model, which I last used in 2000 when I worked on Chello's website. Looking through the validator Java code brought back "fond" memories of building factories of factory makers. In the Nu validators, there are lots of rules checkers that iterate through an HTML5 parse tree and extend a base Checker class:
public final class TableChecker extends Checker {\nprivate Table current;\nprivate final LinkedList<Table> stack = new LinkedList<>();\n\n@Override\npublic void startElement(String uri, String localName,\n String qName, Attributes atts)\n throws SAXException {\n if ("http://www.w3.org/1999/xhtml".equals(uri)) {\n if ("table".equals(localName)) { push(); } else\n if (current != null) {\n if ("td".equals(localName) || "th".equals(localName)) {\n current.cell(atts, localName);\n }\n // ... more element handling\n }\n } } }\nSince the number of rules was massive, a single run of the agent wasn't enough. Instead, I knocked up a Claudeio wrapper that ran the agent iteratively exhorting it to continually sample the rules and iterate on a good architecture. This is only possible since the validator has a massive test suite with expected outputs. The goal of the agent was therefore to try OCaml code architectures that maximised the number of passing rules, and then combining them towards getting 100% pass rate.
\nAfter a few days, this converged and hit 100% pass rate with a bit of human prompt massaging from me. The OCaml version replaces inheritance with first-class modules instead, and each checker implements the Checker.S signature:
\nmodule type S = sig\n type state\n val create : unit -> state\n val reset : state -> unit\n val start_element : state -> element:Element.t -> Message_collector.t -> unit\n val end_element : state -> tag:Tag.element_tag -> Message_collector.t -> unit\n val characters : state -> string -> Message_collector.t -> unit\n val end_document : state -> Message_collector.t -> unit\nend\nThis gives us the same flexibility to compose checkers, but with abstract state\ntypes rather than hidden mutable fields scattered across a class hierarchy.
\nI extended the visual HTML test suite generator I built a few days ago to include the thousands of validation tests, and the library now outputs one epic HTML file that lists each of the thousands of tests.
\nHaving these tests was essential when doing refactoring, as a small change in one checker affected others. Without the comprehensive test oracle, the agents would quickly diverge out of control.
\n ![\"%c\"](\"/images/aoah-html5v-ss-4.webp\" "\\"It's")
The table checker is one of the more complex validators, tracking cell spans,\ndetecting overlaps, and validating that headers attributes reference valid\nth elements. The Java version spreads this across multiple files, but the\nOCaml version consolidates everything into a single module with explicit types.
type cell = {\n mutable left : int;\n mutable right : int;\n mutable bottom : int;\n headers : string list;\n element_name : string;\n}\n\ntype row_group = {\n mutable current_row : int;\n mutable insertion_point : int;\n cells_in_effect : ((int * int), cell) Hashtbl.t;\n mutable cells_on_current_row : cell array;\n row_group_type : string option;\n}\n\ntype table = {\n mutable state : table_state;\n mutable column_count : int;\n header_ids : (string, unit) Hashtbl.t;\n cells_with_headers : cell list ref;\n mutable current_row_group : row_group option;\n (* ... *)\n}\nHowever, the agent didn't fundamentally change the algorithmic structure; we still have the same basic state machine but with much more succinct variant types.
\nOne significant quality of life improvement came from refactoring how the error messages are collected for rendering. The Java code uses string formatting throughout to directly output messages, but the OCaml error_code.mli module defines a polymorphic variant hierarchy that's exhaustively checkable:
\ntype table_error = [\n | `Cell_overlap\n | `Cell_spans_rowgroup\n | `Row_no_cells of [`Row of int]\n | `Column_no_cells of [`Column of int] * [`Elem of string]\n]\n\ntype attr_error = [\n | `Not_allowed of [`Attr of string] * [`Elem of string]\n | `Missing of [`Elem of string] * [`Attr of string]\n | `Bad_value of [`Elem of string] * [`Attr of string] *\n [`Value of string] * [`Reason of string]\n | `Duplicate_id of [`Id of string]\n (* ... *)\n]\nThis allows clients to pattern match on specific classes of errors easily:
\nmatch err with\n| `Attr (`Duplicate_id (`Id id)) -> handle_duplicate id\n| `Img `Missing_alt -> suggest_alt_text ()\n| `Table `Cell_overlap -> report_overlap ()\n| _ -> default_handler err\nThis also means we can add new error categories without changing existing code,\nand the compiler tells us if we miss any cases. This is a pretty classic\nusecase for OCaml that both Yaron Minsky and I talked about (in Caml\nTrading and\nXenServer), but sometimes it's good to remember why I love\nOCaml so much!
\nWhile the OCaml code generated is by no means sparkling clean, it is useful and\noperational already. The typed error hierarchy is perhaps the biggest win, as\nit lifts up the abstraction level to be more idiomatic to OCaml style and\neventually makes it easier to perhaps jump over to Haskell or Lean for even\nmore purity and formal specification work. This is by far the biggest agentic\ncoding translation I've attempted so far, to the point where it used up all my\nClaude 20x Max credits in a matter of days. I have two accounts now!
\nWhat also surprised me was how little the agent struggled with the architectural\ntransformation across programming languages. Given examples of OCaml first\nclass modules (from Real World OCaml and the Jane Street OCaml code), it\nproduced well-structured code.
\n![\"%c\"](\"/images/aoah-html5v-ss-3.webp\" "\\"Claude")
I still have no idea how I'm going to maintain this code in the long term, but\ndo let me know if the HTML5 checker is useful to you. One little Claude trick\nthat remains handy is that after a prompting session, I prompt the agent to fix\nits own skill based on the feedback its received this session. That helps to\ngeneralise the skills as more projects get to using it.
I took a break from yesterday's bot hacking to continue the HTML5 parsing in OCaml adventure. Vibespiling seems to have taken off, with Simon Willison reporting that there's a Swift version now as well. I got curious about how far I could push the vibespiling support: could we go beyond "just" parsing to also do complete HTML5 validation? The Nu HTML Validator is where I went next, which is a bunch of Java code used by the W3C to apply some seriously complex rules for HTML5 validation.
\nI decided to split this work into two days, and started with a simple problem: HTML5 validation includes the need for automated language detection to validate that the lang attribute on HTML elements matches the actual content. This is important for accessibility, as screen readers use language hints to select the correct pronunciation.
The W3C validator uses the Cybozu langdetect algorithm, so I vibespiled this into pure OCaml code as ocaml-langdetect. However, I decided to push harder by compiling this to three different backends: native code OCaml, JavaScript via js_of_ocaml and then into modern WebAssembly using wasm_of_ocaml. As a fun twist, I got the regression tests running as interactive "vibesplained" online notebooks that can do language detection in the browser.
\n ![\"%c\"](\"/images/aoah-langdetect-ss-1.webp\" "\\"The")
Language detection via n-gram analysis is surprisingly simple. The algorithm first trains profiles for each language, by analysing a corpus of text and counting the frequency of sequences of 1-3 characters. This creates a statistical fingerprint of the language.\nThen, when given unknown text it extracts its n-grams and compares against all trained profiles using Bayesian probability. The language whose profile best matches the text wins.
\nIt turns out that n-gram frequencies are remarkably\nstable across different texts\nin the same language. To pick an obvious example, the word "the" appears\nfrequently in English texts, giving bigrams "th" and "he" high frequencies.\nSimilarly, "qu" is common in French, "sch" in German, etc etc. The algorithm\nuses multiple trials with randomized sampling to avoid overfitting to any\nparticular part of the text. Each trial adjusts the smoothing parameter\nslightly using a Gaussian distribution, all of which should be straightforward\nto implement in OCaml.
\nI grabbed the validator/validator/langdetect directory and vibespiled it from Java to OCaml, which is straightforward now with all the earlier Claude skills I've developed this month. The major hurdle to leap that's different from the other libraries is where to stash the precomputed ngram statistics for all the different languages. I wrote ocaml-crunch for Mirage back in 2011 which just generates OCaml modules, but it's still surprisingly difficult to be more efficient and store precomputed data. Jeremy Yallop noted back in March that his modular macros project should support this sort of usecase but it's not quite ready yet. Similarly, using OCaml Marshal requires stashing the marshalled datastructure somewhere, which is hard to do portably.
\nWithout a clear optimisation strategy, I prompted the agent to just precompute the profiles directly into OCaml\ncode. The initial port worked immediately thanks to the clear structure of the\nJava code it was being vibespiled from. The static library was 115MB, but I\ndidn't really notice as the regression tests all passed. The language profiles\ncontain 172,000 unique n-grams across 47 languages, and the naive approach of\ngenerating one OCaml module per language with string literals duplicated\nn-grams across profiles.
\nThe native code library provides a straightforward interface to query the ngrams via a cmdliner binary:
\n$ dune exec langdetect\nHello Thomas Gazagnaire, I'm finally learning French! Just kidding, I don't know anything about it.\nen 1.0000\n\n$ dune exec langdetect\nBonjour Thomas Gazagnaire, j'apprends enfin le français ! Je plaisante, je n'y connais rien.\nfr 1.0000\n\n$ dune exec langdetect\nHello Thomas Gazagnaire, I'm finally learning French! Just kidding, I don't know anything about it.\nBonjour Thomas Gazagnaire, j'apprends enfin le français ! Je plaisante, je n'y connais rien.\nen 0.5714\nfr 0.4286\nBut then, when I compiled it to JavaScript using the dune\nstanzas\nthe massive size was a little too big, with very long compilation times. The\nfix was simply to pack everything into a shared data structure across all\nlanguages, looking something like this:
\n(* Shared string table for all 172K unique n-grams *)\nlet ngram_table = [| "the"; "th"; "he"; ... |]\n\n(* Flat int array: (ngram_index, frequency) pairs for all languages *)\nlet profile_data = [| 0; 15234; 1; 8921; ... |]\n\n(* Offsets: (lang_code, start_index, num_pairs) *)\nlet profile_offsets = [|\n ("en", 0, 4521);\n ("fr", 9042, 3892);\n ...\n|]\nThis reduced the binary from 115MB to around 28MB, which is a reasonable\nreduction without having to resort to compression. Many n-grams appear in\nmultiple languages (consider Latin alphabet characters) so deduplicating into a\nshared string table eliminated quite a bit of redundancy.
\nAt this point, I prompted the agent to build me a full Javascript based\nregression test that took the native code, and gave me a browser based\nversion instead.
\nOne minor hiccup was that the regression tests failed due to JavaScript\nintegers overflowing vs native integers, but the fix was simple and the\nregression tests in the browser made debugging them easy for the agentic loop. Without them, there would have been a lot of human cut-and-pasting which is quite tedious!
\n ![\"%c\"](\"/images/aoah-langdetect-ss-2.webp\" "\\"The")
With the JavaScript size under control, I turned to WASM compilation via wasm_of_ocaml. The first attempt failed with a cryptic error about exceeding operands and a parse error. It turns out WASM's array_new_fixed instruction has a limit of 10000 operands, and our profile data array had 662,000 elements.
The solution was to chunk the arrays and concatenate at runtime, which incurs runtime overhead but is a common enough solution. The generated code now includes 74 chunks for the profile data and 20 chunks for the n-gram string table, but clocks in at around 20MB and could probably be reduced further with some browser compression.
\n ![\"%c\"](\"/images/aoah-langdetect-ss-3.webp\" "\\"Wasm")
Now, the browser tests include the ability to switch between Wasm and\nJavaScript in the same test HTML. There was no real performance difference here, but the dataset is small. The most observable difference is that the wasm needs to be served via a web server and not local filesystem, as otherwise browsers reject it. The browser also must serve .wasm files as mime type application/wasm or it's promptly rejected.
The OCaml langdetect-js package provides a browser-ready API using Brr callbacks from HTML to register them with the JavaScript:
// Detect language\nconst lang = langdetect.detect("Hello, world!"); // "en"\n\n// Get probability scores\nconst result = langdetect.detectWithProb("Bonjour le monde");\n// { lang: "fr", prob: 0.9987 }\n\n// Get all candidates\nconst all = langdetect.detectAll("这是中文");\n// [{ lang: "zh-cn", prob: 0.85 }, { lang: "zh-tw", prob: 0.12 }, ...]\nI also published this to npm so that the JavaScript is conveniently available via a CDN like jsDeliver. Publishing to npm required putting a npm branch in the repo with the npm package.json. I followed the convenient guide by Simon Willison to get a minimal package.json for the project.
This was a good intermediate port to work on since it let me exercise Webassembly a bit more, and understand the tradeoffs in OCaml compilation to these other backends. The process of getting the agent to systematically port first to native code (from Java), and then compile to JavaScript and debug platform-specific issues like the integer overflows, and then go to wasm was quite good.
\nThe agent was particularly helpful for the tedious work of generating the\nchunked array code and debugging the Unicode normalization edge cases.
\nFor future hacking, there are several language optimisations coming up in OxCaml that should make this even more efficient; support for compile time metaprogramming (so I could for example compute a perfect hash statically for all the ngrams), and also for smaller integer sizes so I dont need to use a full 31-bit range for the ngram values. However, I couldn't quite get the wasm_of_ocaml constraints on the oxcaml branch working so I ran out of time today to get this going. Package management takes me out of the flow zone yet again!
\nNow that langdetect works, we'll go onto the full HTML5 validator in Day 21!
", "url": "https://anil.recoil.org/notes/aoah-2025-20", "title": "AoAH Day 20: Human language detection in native code, JS and wasm", "summary": "Porting the Nu HTML Validator's language detection to OCaml, then optimizing from 115MB to 28MB and fixing WASM array limits for browser deployment.", "date_published": "2025-12-20T00:00:00.000000Z", "date_modified": "2025-12-20T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai", "web", "wasm" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-19", "content_html": "After building tomlt yesterday for TOML 1.1 parsing, I proceeded to integrate it with my group's Zulip chat server. I then discovered that Zulip actually uses Python's configparser INI format for its .zuliprc files rather than TOML, woops! But this gave me the perfect opportunity to attempt to quickly replicate the tomlt experience with a third config format codec library for Windows-style INI files as well.
So today I released both ocaml-zulip for Zulip API integration and ocaml-init for INI file codecs that are compatible with Pythonic features such as variable interpolation. Along the way, I developed a new regression test mechanism by writing a Zulip bot that tests the Zulip API using OCaml Zulip!
\nZulip is an open-source "async first" messaging app that strikes a nice balance between immediate and thoughtful conversations:
\n\n\n\n
\n- In Zulip, channels determine who gets a message. Each conversation within a channel is labeled with a topic, which keeps everything organized.
\n- You can read Zulip one conversation at a time, seeing each message in context, no matter how many other conversations are going on.
\n- If anything is out of place, it's easy to move messages, rename and split topics, or even move a topic to a different channel\n-- Why Zulip?, 2024
\n
Zulip itself is fully open source and has a pretty straightforward REST API to communicate with the server, and so I deployed my requests library as well the various API codecs to interface with it. I used the Zulip Python SDK and the Zulip JavaScript library to give me two API specifications. Unlike previous libraries I've vibecoded, there's no language-agnostic test suite so I needed to get a bit more creative to verify correctness by building a live bot to test itself.
\nZulip's .zuliprc file looks like this:
[api]\nemail = bot@example.com\nkey = your-api-key-here\nsite = https://your-domain.zulipchat.com\nThis is classic INI format as used by Python's configparser module. It's\nsimpler than TOML but isn't fully compatible as it has quirks like case-insensitive\nkeys, multiline value support via continuation lines, and basic variable\ninterpolation with a %(name)s syntax.
I couldn't find a feature complete implementation of Python's module, so I quickly\nreused the tomlt approach to build ocaml-init with bidirectional\ncodecs. The resulting API is unsurprisingly extremely similar to manipulate this format file:
\ntype server_config = { host : string; port : int; debug : bool }\n\nlet server_codec = Init.Section.(\n obj (fun host port debug -> { host; port; debug })\n |> mem "host" Init.string ~enc:(fun c -> c.host)\n |> mem "port" Init.int ~enc:(fun c -> c.port)\n |> mem "debug" Init.bool ~dec_absent:false ~enc:(fun c -> c.debug)\n |> finish\n)\nThere's a bool codec in this library that follows Python's configparser\nsemantics exactly, accepting yes/no/true/false/on/off/1/0 as boolean values.\nThis was important for compatibility with existing Zulip configuration files.
With configuration parsing sorted, I turned to building the actual bot\nframework. Our research group at eeg.zulipchat.com\nhas been wanting an Atom feed bot to post updates from our blogs' Atom/RSS sources,\nso this seemed like a good excuse to knock up a bot.
\nI prompted the agent to follow the basic Python\nbotserver considerations but adapted to\na more Eio and OCaml idiomatic style. This resulted in a nice design where a\nbot handler is just a function:
\ntype handler =\n storage:Storage.t -> identity:identity ->\n Message.t -> Response.t\nThe Zulip library provides modules for storage for persisting state (on the Zulip server side), identity containing functions to access the bot's email and user ID, and the incoming Message.t. The handler returns a Response.t which can be a reply in the same context (DM or channel/topic), or a post to a specific channel, or a direct message, or an indication that the bot's ignoring the event.
There's an echo bot handler that's executable that shows the API quite simply:
\nlet echo_handler ~storage ~identity msg =\n let bot_email = identity.Bot.email in\n let sender_email = Message.sender_email msg in\n\n (* Ignore our own messages *)\n if sender_email = bot_email then Response.silent\n else\n (* Remove bot mention and echo back *)\n let cleaned_msg = Message.strip_mention msg ~user_email:bot_email in\n if cleaned_msg = "" then\n Response.reply (Printf.sprintf "Hello %s!" (Message.sender_full_name msg))\n else\n Response.reply (Printf.sprintf "Echo: %s" cleaned_msg)\nAfter this running the bot in an Eio environment is a single function call:
\nlet () =\n Eio_main.run @@ fun env ->\n Eio.Switch.run @@ fun sw ->\n let config = Zulip_bot.Config.load ~fs:(Eio.Stdenv.fs env) "echo-bot" in\n Zulip_bot.Bot.run ~sw ~env ~config ~handler:echo_handler\nOne nice payoff from this advent series is seeing how\nthe libraries begin to compose. The Zulip OCaml package depends on\nRequests for HTTPS communication back to the server:
\nlet create ~sw env auth =\n let session =\n Requests.create ~sw\n ~default_headers:(Requests.Headers.of_list [\n ("Authorization", Auth.to_basic_auth_header auth);\n ("User-Agent", "OCaml-Zulip/1.0");\n ])\n ~follow_redirects:true\n ~verify_tls:true\n env\n in\n { auth; session }\nThis shows how the session abstraction in Requests can persist the common auth\ntokens required, making subsequent API calls very syntactically succinct. The\nbot framework also uses XDGe for configuration directory\nresolution, jsont for JSON parsing of API responses, and of\ncourse Eio throughout for async operations. The dependency graph is starting to\nlook like actual infrastructure!
\nRemember that problem I mentioned earlier about Zulip lacking a\nlanguage-agnostic test suite? My cunning solution was recursive; let's just\nbuild a Zulip bot that can test itself!\nI built a regression test bot that exercises as\nmuch of the Zulip API as possible when triggered via a direct message:
\nlet make_handler ~env ~channel =\n fun ~storage ~identity:_ msg ->\n let content = String.lowercase_ascii (Message.content msg) in\n let sender_email = Message.sender_email msg in\n (* Only respond to DMs containing "regress" *)\n if Message.is_private msg && String.starts_with ~prefix:"regress" content\n then (\n let client = Storage.client storage in\n let summary = run_tests ~env ~client ~channel ~trigger_user:sender_email in\n Response.reply summary)\n else Response.silent\nWhen someone sends the bot a DM with "regress", it runs through a comprehensive\ntest suite covering user operations, channel management, message\nsending/editing, reactions, message flags, typing indicators, presence updates,\nand alert words. I duly started the harness and DMed Vicuna, and the bot\nimmediately spat out a number of failures resulting from errors in the codecs.\nHowever, as the logs show, the errors also included useful information about\nwhere the protocol decoding had gone wrong.
\n![\"%c\"](\"/images/aoah-zulip-regress-1.webp\" "\\"The")
The bot posts a summary to a test channel showing which tests passed or failed,\ncomplete with timing information. This turned out to be more useful than\ntraditional unit tests since it exercises the actual API against a real Zulip\nserver. After one round of fixes, the bot successfully posted its results to\na Zulip channel recording success!
\n![\"%c\"](\"/images/aoah-zulip-regress-2.webp\" "\\"The")
One pattern I've been developing across these libraries is to expose cmdliner\nterms that compose together to make it easy to build CLI tools that expose\nthe configuration needed by a library along with a manual page.
\nThe Zulip_bot.Cmd module provides a config_term that bundles common bot configuration parameters:
let config_term default_name env =\n let fs = env#fs in\n Term.(const (fun name config_file verbosity verbose_http ->\n setup_logging ~verbose_http:verbose_http.value verbosity.value;\n load_config ~fs ~name ~config_file)\n $ name_term default_name\n $ config_file_term\n $ verbosity_term\n $ verbose_http_term default_name)\nWhile this looks complex, all it's doing is to combine various declarations\nof command-line parameters. This combines individual terms for the bot name, config file path, verbosity level,\nand HTTP-level debugging into a single composable unit.
\nThe verbose_http_term controls logging sources in the Requests\nlibrary, letting you toggle detailed HTTP traces without that being the default verbose output.
let verbose_http_term app_name =\n let env_name = String.uppercase_ascii app_name ^ "_VERBOSE_HTTP" in\n let env_info = Cmdliner.Cmd.Env.info env_name in\n Arg.(value & flag & info [ "verbose-http" ] ~env:env_info ~doc)\nEach bot can then define its command with minimal boilerplate:
\nlet bot_cmd eio_env =\n let info = Cmd.info "echo_bot" ~version:"2.0.0" ~doc ~man in\n let config_term = Zulip_bot.Cmd.config_term "echo-bot" eio_env in\n Cmd.v info Term.(const (fun config -> run_echo_bot config eio_env) $ config_term)\nThe source tracking also helps with debugging by showing where each value\noriginated from -- whether from the command line, environment variables,\nXDG config files, or defaults. This makes it much easier to understand\nwhy a bot is behaving a certain way when deployed.
\n![\"%c\"](\"/images/aoah-zulip-regress-3.webp\" "\\"The")
It's nice to get to the Zulip bot framework at last, since this is one of the\nthings I wanted to fix at the start of the month. It uses a number of things\nI've built this month, including the Requests library to handle HTTP, the INI\ncodec for Python configuration, XDG to handle path resolution, and so on. Each\npiece is small and focused, and generatively replicated from other\n(human-written) exemplar libraries from the OCaml ecosystem.
\nThe only "agentic trick" I learnt today was the value of live debugging, as I\nfound with both JMAP email and Karakeep. Building\nservices amenable to this kind of live mocking is something I'll keep in mind\nfor the future. It's also extremely useful to have good terminal manual pages,\nsince those can also be interrogated by coding agents as well as be used by humans.
", "url": "https://anil.recoil.org/notes/aoah-2025-19", "title": "AoAH Day 19: Zulip bot framework to bring Vicuna the friendly camel back", "summary": "Building an OCaml Zulip bot framework with functional handlers, and pivoting from TOML to INI codecs for Python configparser compatibility", "image": "https://anil.recoil.org/images/aoah-zulip-regress-2.1280.webp", "date_published": "2025-12-19T00:00:00.000000Z", "date_modified": "2025-12-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai", "functional" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-18", "content_html": "After getting my email interfaces automated yesterday, I turned my attention to Zulip integration. But first, I took a segway into another format that it required known as TOML. I noticed TOML 1.1.0 was released today and so I built ocaml-tomlt today.
\nWhat I wanted to explore with this library is whether I could use a coding agent to build a complex functional abstraction from scratch. After building yamlrw and yamlt, I settled on the technique Daniel Bünzli developed with jsont in his paper.
\n ![\"%c\"](\"/images/jsont-paper.webp\" "\\"Daniel")
TOML has become a popular configuration format for other language ecosystems like Rust and Python. Unlike Yaml 1.2, TOML is actually a reasonable human-editable format without the terrifying corner cases and denial of service traps hidden in Yaml.
\nSince Toml 1.1 was just released today, there are existing OCaml libraries that fully supported. In addition, I need one that is pure OCaml with no C dependencies (like yamlrw) and that uses Bytesrw for streaming I/O so that it composes well with my other libraries from this month's coding.
\nThe implementation of tomlt was prompted from "An Alphabet for Your Data Soups" which accompanies his jsont library. Working with untyped data formats like TOML in strongly-typed languages like OCaml requires a lot of tedious dynamic marhsalling, and I'd like to switch to conventional OCaml records or other static types as soon as possible.
\nDaniel's solution is to define a generalised algebraic datatype whose values represent\nbidirectional mappings between subsets of the wire format and my chosen OCaml\ntypes. Waaay back in 2010 when Thomas Gazagnaire and I worked on camlp4-based serialisation, we converted into a generic intermediate\nrepresentation for OCaml types and values. More recently Jeremy Yallop has been\nworking on MacoCaml which performs\nthis transformation at compile time via hygenic macros.
\nUnlike any of these approaches, the functional pearl Daniel came up with allows the programmer to\ndefine direct functional transformations that work in both directions. It's a\nbit more work at runtime and so a bit slower, but in return you get excellent\nerror messages for malformed messages. The core Toml type therefore becomes:
\n(* A codec encapsulates both decoding and encoding *)\ntype 'a t = {\n kind : string;\n doc : string;\n dec : Toml.t -> ('a, codec_error) result;\n enc : 'a -> Toml.t;\n}\nThis means you write your schema once and get both directions for free, and\nuser functions can be placed at every coding step to allow the programmer to\ninterpose custom functionality such as transformation or validation.
\nA Toml config file might look something like this:
\n[server] \n host = "localhost" \n port = 8080 \n \n[database] \n connection_max = 5000\nHere's what using tomlt to parse this looks like in practice:
\ntype config = { host : string; port : int; debug : bool }\n\nlet config_codec =\n Tomlt.(Table.(\n obj (fun host port debug -> { host; port; debug })\n |> mem "host" string ~enc:(fun c -> c.host)\n |> mem "port" int ~enc:(fun c -> c.port)\n |> mem "debug" bool ~enc:(fun c -> c.debug) ~dec_absent:false\n |> finish\n ))\n\nlet () =\n match Tomlt.decode_string config_codec {|\n host = "localhost"\n port = 8080\n |} with\n | Ok config -> Printf.printf "Host: %s\\n" config.host\n | Error e -> prerr_endline (Tomlt.Toml.Error.to_string e)\nThe functional pattern is almost identical to the yamlt or jsont codecs I've been building.\nYou don't have to define a codec, as tomlt also provides custom index operators to navigate tables directly:
\nlet config = Toml.of_string {|\n [server]\n host = "localhost"\n port = 8080\n\n [database]\n connection_max = 5000\n|} in\n(* Navigate nested tables with .%{} *)\nlet host = Toml.(config.%{["server"; "host"]} |> to_string) in\nlet port = Toml.(config.%{["server"; "port"]} |> to_int) in\nPrintf.printf "Server: %s:%Ld\\n" host port;\n\n(* Update values *)\nlet config' = Toml.(config.%{["database"; "enabled"]} <- bool true) in\nprint_endline (Toml.to_string config')\nThe syntax is a little verbose due to the module opening, but it's still a\npretty nice way to poke around TOML files interactively!
\nAnother area where TOML differs from other formats is that it four distinct\ndatetime formats: offset datetimes, local datetimes, local dates, and local\ntimes. tomlt tries to unify this a little via a single codec that normalises everything to Ptime.t, but allows the codec to supply sensible defaults (e.g. for a missing timezone, or a missing date).
\n(* All of these decode to Ptime.t with sensible defaults *)\n(* when = 2024-01-15T10:30:00Z -> offset datetime *)\n(* when = 2024-01-15T10:30:00 -> local datetime *)\n(* when = 2024-01-15 -> date at midnight *)\n(* when = 10:30:00 -> time on today's date *)\n\nlet event_codec = Tomlt.(Table.(\n obj (fun name when_ -> { name; when_ })\n |> mem "name" string ~enc:(fun e -> e.name)\n |> mem "when" (ptime ()) ~enc:(fun e -> e.when_)\n |> finish\n))\nFor applications that need to preserve the exact format, there's also a\nptime_full function which returns a polymorphic variant indicating precisely\nwhat was present in the source config file.
The secret to vibing seems to be having a specification oracle to guide the\nagent, and TOML has a toml-test suite\nthat's perfect for this purpose:
\n\n\ntoml-test is a language-agnostic test suite to verify the correctness of TOML parsers and writers.
\nTests are divided into two groups: "invalid" and "valid". Decoders or\nencoders that reject "invalid" tests pass the tests, and decoders that accept\n"valid" tests and output precisely what is expected pass the tests. The\noutput format is JSON, described below.\n-- Toml-test GitHub, 2021
\n
![\"%c\"](\"/images/aoah-toml-ss-1.webp\" "\\"The")
After building yamlrw, yamlt, and now tomlt,\nI'm convinced that the bidirectional codec pattern is a good approach for\nagentic OCaml programming. It's a little verbose to express by hand, which\nleads down the ppx route for most. But with agentic generation and oracle\nspecification testing, the coding agent was particularly helpful with both\nfiguring out the TOML grammar and exposing all the variations of codecs\nrequired for parsing all those datetime variants.
\nHaving the TOML 1.1 specification as context and\nmy earlier Claude OCaml RFC skill helped a lot as well, to\nallow the ocamldoc to be cross referenced. And of course, the key design\ninsights at the heart of the library came from Daniel Bünzli publishing jsont and\nalso uploading his paper. This Tomlt library is a generative clone of his\nideas, but a useful one to my personal workflows this advent!
\nTomorrow in Day 19, I'll continue with my original goal of getting\na Zulip bot working!
After building a JSON Pointer library yesterday, I proceeded\nto complete my OCaml JMAP\nlibrary today so that I could wrestle my overflowing email inbox under control.\nEmail is central to our digital lives and yet we have mostly ceded control to third-party services for something that unlocks access to almost any service we use.
\nLuckily, I've been self-hosting my own email for some time, so I do have full local access to about three decades worth of messages. However, I've been hampered by existing email clients which are mostly geared towards a temporal view and not towards easy programmability. So today's exercise has been to build an ocaml-jmap that lets me write little agentic programs to help me manage my ever overflowing inbox!
\n ![\"%c\"](\"/images/shriram-vibe-1.webp\" "\\"Shriram")
Historically, the protocol of choice for accessing email has been IMAP, and for years I used the Lwt ocaml-imap library that Nicolás Ojeda Bär wrote back when he was here in Cambridge. However, IMAP is a pretty convoluted protocol that hasn't evolved much over time, and requires a number of accreted extensions that are patchily implemented.
\nThe JSON Meta Application Protocol (or JMAP) was developed in response to this. The developers wrote:
\n\n\nJMAP is the result of efforts to address shortcomings [in existing protocols], providing a modern, efficient, easy-to-use API, built on many years of experience and field testing.\n-- JMAP, a modern open email protocol, Gondwana and Jenkins, May 2019
\n
JMAP's exciting because it allows for the reuse of a number of other protocol implementations I've already built these past few weeks. I can use ocaml-requests and conpool for HTTPS connections, and my ever-growing dependence on jsont means that I can get reasonable error messages while debugging the implementation.
\nServers that support JMAP are still thin on the ground, but all my email endpoints do now support it. At work in the Cambridge Computer Lab, the enlightened sysadmins gave us Fastmail accounts as an alternative to having Exchange inflicted on us by the central University. Over at Recoil, my personal mail is switching to Stalwart JMAP. While there's still no good JMAP client that replaces my day-to-day email, all these services also expose IMAP for that purpose. This clears the way for me to vibecode up a JSON library in OCaml for the programmatic fragments that I so crave!
\nInterestingly, I had also tried to build an OCaml JMAP waaay back when Claude Code was first released, and the difference in nine months of model development is massive; browse my first attempt here and see what a mess it was.
\nBut now that I've got a bunch of examples under my belt, the latest round of vibing of ocaml-jmap was extremely fast. The agent had examples of jsont use from jsonfeed and the Karakeep REST client, and ocaml-requests provides direct-style HTTP requests with authentication bearer support. I vibesplained a Javascript tutorial to help me view messages.
\nHowever, I wanted to go a little further than just notebook style tutorials. In addition to compiling OCaml code to JavaScript, there's also really good support for programming browser API directly through libraries like Brr. These expose an OCaml interface that allows us to build up browser-specific logic to better integrate within it.
\n![\"%c\"](\"/images/aoah-jmap-ss-1.webp\" "\\"Connecting")
So I took the notebook idea one step further and prompted up an entire JMAP client that runs in the browser (albeit a very simple one!). This Json_brr library looks just like normal OCaml, except that it uses libraries designed to run in the browser as its dependencies. The interface shows some differences
\ntype connection\nval api_url : connection -> Jstr.t\n(** [api_url conn] returns the API URL for requests. *)\n\n(** {1 Session Establishment} *)\n\nval get_session :\n url:Jstr.t ->\n token:Jstr.t ->\n (connection, Jv.Error.t) result Fut.t\n(** [get_session ~url ~token] establishes a JMAP session.\nInstead of string, this uses Jstr.t for JavaScript strings. Instead of Eio, this also uses Fut.t to encode future promises.\nBut aside from those things, it's just plain OCaml! The implementation makes Websocket requests to connect to a remote JMAP server in the browser.
let fetch_json ~url ~meth ~headers ?body () =\n Console.(log [str ">>> Request:"; str (Jstr.to_string meth); str (Jstr.to_string url)]);\n (match body with\n | Some b -> Console.(log [str ">>> Body:"; b])\n | None -> Console.(log [str ">>> No body"]));\n let init = Brr_io.Fetch.Request.init\n ~method':meth\n ~headers\n ?body\n ()\n in\n let req = Brr_io.Fetch.Request.v ~init url in\n let* response = Brr_io.Fetch.request req in\nTo use jmap.html, I obtained a (read only!!) API key from my live email server, and then managed to connect to my live email and get a protocol debugger, all from my browser.
\n![\"%c\"](\"/images/aoah-jmap-ss-2.webp\" "\\"The")
One important performance point about how this works is that some of the base libraries I'm using take advantage of OCaml's modularity in order to expose browser specific backends. For example, Jsont_brr uses the native browser JSON parser while still working with Jsont codecs. This sort of casual modularity is an extremely nice and undersung feature of OCaml, and was also key in MirageOS being portable to so many embedded devices.
\nSo now we have a working library, I wanted to go beyond the conventional email clients. With agentic coding, I should be able to construct hundreds of domain-specific bits of code that help me manipulate my email the way I want it done. This was the dream of ##selfhosting and open source in the first place, but it was too much work to write all that glue code... until now.
\n![\"%c\"](\"/images/aoah-jmap-ss-3.webp\" "\\"The")
One papercut that bugs both me and Mark Elvers is the flood of email notifications we get that are rapidly out of date. I get thousands a day from GitHub, and ideally they would be marked as read and filed away automatically when I finish with it on the remote service, and not stay in my Inbox. This is now easily solveable using OCaml JMAP, so I built a proof of concept to filter away my Zulip notifications nicely.
\nI built a domain-specific CLI called jmapq which exists solely to implement specialist workflows. I'll break this out into a private repo for my own use later, this is just a prototype!
\nZulip sends a nicely structured email with a subject like "#Blogs > My 2025 Advent of Agentic Humps: a new library daily [Cambridge Energy & Environment Group]" every so often which I instructed the agent to parse into an OCaml data structure:
\n(** Parsed information from a Zulip notification email subject.\n Subject format: "#Channel > topic [Server Name]" *)\nmodule Zulip_message = struct\n type t = {\n id : string;\n date : Ptime.t;\n thread_id : string;\n channel : string;\n topic : string;\n server : string;\n is_read : bool;\n labels : string list;\n }\n\n (** Parse a Zulip subject line of the form "#Channel > topic [Server Name]" *)\n let parse_subject subject =\n (* Pattern: #<channel> > <topic> [<server>] *)\n let channel_re = Re.Pcre.regexp {|^#(.+?)\\s*>\\s*(.+?)\\s*\\[(.+?)\\]$|} in\n match Re.exec_opt channel_re subject with\n | Some groups ->\n let channel = Re.Group.get groups 1 in\n let topic = Re.Group.get groups 2 in\n let server = Re.Group.get groups 3 in\n Some (channel, topic, server)\n | None -> None\nOnce we have this, I then added another CLI command to list all the Zulip notifications in my email, which it outputs using nicely structured JSON output that is parseable by jq.
\n![\"%c\"](\"/images/aoah-jmap-ss-4.webp\" "\\"Given")
Then I instructed the CLI to give me more grouped views, so I can see notifications in my email by topic and by server (I now also get notifications from other Zulips I'm on like the OCaml or Lean).
\n![\"%c\"](\"/images/aoah-jmap-ss-5.webp\" "\\"The")
And then the really sketchy bit. I swapped API keys to have a read/write one (it could delete all my email in theory), and then -- while only mildly sweating -- ran the timeout command. I did add a dry-run mode so I could see the query first before I let it rip.
\n![\"%c\"](\"/images/aoah-jmap-ss-6.webp\" "\\"Running")
And et voila, it added the right keywords and marked things as unread, and my live email view in Fastmail now allows for keyword searches that show me exactly what I want.
\n![\"%c\"](\"/images/aoah-jmap-ss-7.webp\" "\\"I'm")
Today's been great; I managed to find a papercut that's been bugging me for\nyears, and code up something quickly that genuinely makes my personal workflows\na little bit better. I used to do this a lot before the pandemic, but somehow\nthe shift to online services has really damaged my ability to program my own\ndigital life.
\nI feel back in control again; although\nocaml-jmap should be used by\nanyone else with extreme caution, I'm rigging up some ZFS snapshots of my own\nemail so I can code up a few hundred custom agents over things and see how it\ngoes. But mostly, I'm happy to have found a glimmer of the Resonant Computing manifesto\nthrough the medium of strong OCaml types, self hosted services and agentic coding glue:
\n\n\nDedicated: Software should work exclusively for you, ensuring contextual\nintegrity where data use aligns with your expectations. You must be able to\ntrust there are no hidden agendas or conflicting interests.\n-- Resonant Computing Manifesto, 2025
\n
I'm going to work on wrestling GitHub and Netdata under control next. And what about the other side of the Zulip notifications? Well, I'll knock up a Zulip bot in OCaml tomorrow, so stay tuned for that!
After the successful HTML5 translation yesterday, I realised\nthat I know next to nothing about HTML5 parsing and had leant extremely heavily\non agentic coding. This approach has also been useful to help me explore\ndiverse codebases in a combination of languages. So today I\nset my sights on understanding the pedagogical impacts of agentic coding a bit\nmore. Can we use coding agents to help us iteratively explore complex\nprotocols?
\nI decided to build a JMAP email client implementation in\nOCaml that I need for myself[1] but with the added twist of seeing how I\ncould engineer agents to "vibesplain" a protocol to me that I'm unfamiliar\nwith.
\nOCaml has superb tooling to help with this; it can not only compile to efficient native code but also to JavaScript and WASM that runs standalone in the browser. I turned to my colleagues Jon Ludlam, Patrick Ferris and Arthur Wendling for help with the tooling, since they've been leading the way on scientific programming, visualisations and webcomponents in OCaml.
\nToday's work resulted in an ocaml-json-pointer (RFC6901) implementation along with an interactive notebook tutorial that bundles the entire OCaml compiler toolchain alongside it. There's even another one for Yaml just to illustrate how easy this is to replicate once we've built the first one.
\nI first began by informally scanning the JMAP RFCs with a cup of tea, and noticing that it needed support for something called JSON Pointer. I know nothing about this, and so I used my Claude Internet RFC skill to sketch out an OCaml implementation based on the specification.
\nThis is straightforward, but I didn't know how to evaluate the generated interface for fitness since I'm not familiar with the topic. Ideally, I could use the coding agent to explain JMAP pointers to me by using the generated OCaml code illustratively so I could do a code review of its output at the same time as educating myself. For this to work, we would have to machine check the generated documentation and code to make sure it all compiles.\nThis seems like a really good use to put coding agents to!
\nI dub vibesplaining as using an agent to generate executable code and an executable tutorial that can be interacted with by the prompter in order to assist with both understanding the code and iterating on the implementation.
\nWhen writing Real World OCaml about a decade ago, we built it using a tool called mdx that allowed for embedding OCaml sections within a Markdown document. Mdx can compile these markdown documents by executing the embedded OCaml, and promote the outputs directly into the document itself. You can see an example in the Guided Tour chapter where every single OCaml phrase has been machine generated. Yaron Minsky has talked extensively about how successful this "expect test" approach is within Jane Street: they use it for hardware waveforms, exploratory programming and interactive specification.
\n ![\"%c\"](\"/images/aoah-json-ss-2.webp\" "\\"What")
Upon consulting Jon Ludlam, he told me that odoc v3 supports executable code fragments already, so that we don't even need a separate Markdown file anymore. The comments in OCaml code can themselves contain OCaml code that can be compiled into JavaScript and run in a browser!
\nI had never even looked at the tooling for this aspect of odoc, but Patrick Ferris\nhas been building a BibTeX OCaml that is extensively using\nthis workflow. His dune file just\nadds a new stanza:
\n(mdx\n (files index.mld bib.mli)\n (libraries bib bytesrw astring))\nThis stanza handles the build logic for odoc and mdx to interface and extract\nexecutable code out of interfaces\nfiles\nlike:
\n(** ... \n Use {! decode} to read your Bibtex data and {! encode} to write\n it. For example, here is a little program to format a Bibtex file\n from a string.\n\n {@ocaml[\n open Bytesrw\n\n let format s =\n let reader = Bytes.Reader.of_string s in\n let writer = Bytes.Writer.of_out_channel Out_channel.stdout in\n Bib.encode (Bib.decode reader) writer\n ]}\n\n You can then use it with something like:\n\n {@ocaml[\n # format "@string{ hello=world }";;\n @string{\n hello=world\n }\n - : unit = ()\n ]}\nOCaml inception right here! The # at the start of the fragment indicates that it's an OCaml toplevel phrase, otherwise it's library code.
I used this expert knowledge to then vibesplain up a\ntutorial.mld,\nwhich is a standalone ocamldoc fragment that explains JSON Pointers to me.\nHere's an illustrative fragment:
\nFrom RFC 6901, Section 1:\n\n{i JSON Pointer defines a string syntax for identifying a specific value\nwithin a JavaScript Object Notation (JSON) document.}\n\nIn other words, JSON Pointer is an addressing scheme for locating values\ninside a JSON structure. Think of it like a filesystem path, but for JSON\ndocuments instead of files.\n\nFor example, given this JSON document:\n\n{x@ocaml[\n# let users_json = parse_json {|{\n "users": [\n {"name": "Alice", "age": 30},\n {"name": "Bob", "age": 25}\n ]\n }|};;\nval users_json : Jsont.json =\n {"users":[{"name":"Alice","age":30},{"name":"Bob","age":25}]}\n]x}\n\nThe JSON Pointer [/users/0/name] refers to the string ["Alice"]:\n\n{@ocaml[\n# let ptr = of_string_nav "/users/0/name";;\nval ptr : nav t = [Mem "users"; Nth 0; Mem "name"]\n# get ptr users_json;;\n- : Jsont.json = "Alice"\n]}\nThis tutorial is not only available in the code repo, but it's also present in the\ngenerated HTML library documentation for the JSON Pointer library as well. It will,\nfor example, appear on the central ocaml.org if this package\nis ever submitted to the official package repository.
\nHowever, the tutorial would be even useful to me if it was fully interactive so\nI can mess around with it while learning the codebase.\nArthur Wendling recently released a webcomponent for OCaml called\nx-ocaml that allows a full OCaml compiler to be embedded\ninto a webpage. This is how I built the recent\nICFP 2025 OxCaml tutorial, but I figured it would also be an easy\nway of redistributing a fully standalone version of the JSON Pointer library.
\nBeing a webcomponent, using x-ocaml is as simple as including a script header and adding\nan <x-ocaml> tag with the OCaml code. The only extra thing I needed was to create a Dockerfile to install my local library\ninto a separate opam switch and then run the x-ocaml shell script to generate the libraries blob.
We still need to compile our odoc source tutorial into HTML with the right x-ocaml tags. Jon Ludlam explained to me odoc mld can be compiled straight to HTML using a sequence of commands:
$ odoc compile <file.mld> \n$ odoc html-generate page-file.odoc -o . \n$ odoc support-files -o \nI then used my html5rw library from yesterday to write a utility to transform the odoc HTML output by rewriting to introduce the <x-ocaml> tags. I've published this as a odoc-xo binary. This then sit inside some dune rules in my JSON Pointer repository to automate the conversion from mld to interactive Javascript. You can see the promotion rules here; I'll wrap them into a Claude skill later this week.
You can browse the interactive tutorial\nhere for JSON Pointer, which I\nfound very useful to exploring the specification interactively. I did have a\nquick go at compiling my earlier Claude OCaml bindings in order\nto be able to invoke Claude during my explorations, but I got interrupted by a\nvery pleasant mince pie celebration so I'll leave that as an exercise to the\nreader.
\nThere are still a few tooling rough edges to smooth out here, but nothing that can't be automated with a Claude skill and some more binaries like odoc-xo. In Day 17 I'll finish up the JMAP implementation as well.
\nMore generally, what I find fascinating about agentic skills is that once you apply a template somewhere, it can be reused very quickly. It took me a few hours and a lot of input from Jon Ludlam and Patrick Ferris to come up with the first version today, but then it took less than 10 minutes to generate an equivalent one for other repositories. For example, here's the Yaml library explainer tutorial if you really want to know more.
\n ![\"%c\"](\"/images/aoah-yamlrw-doc-1.webp\" "\\"All")
These tutorials aren't high quality writing, especially compared to one where I've really thought it through and spent time on for specific teaching outcomes. However, agent generated "vibesplained" tutorials can also be very personalised even if not redistributed. For example, I want to know about recent advances in TCP, and so I am feeding the agent my mirage-tcpip stack and the latest RFCs, and requesting a tutorial about new concepts that I might have missed in recent years. Having the capability to provide contexts and generate executable tutorial notebooks is a new capability I've discovered today.
\nThis does, of course, depend on having a high quality baseline of documentation in my dependent libraries, and of exemplar human-written rules and documentation. Thank you to Jon Ludlam, Patrick Ferris, Arthur Wendling and Daniel Bünzli for going to the (long) effort of giving me the base material to work from here. I am very appreciative of your contributions to OCaml open source.
\nBeing a professor is not accurately measured by research or teaching outputs, but by how overloaded your INBOX is.
\n ↩After my success with Yaml 1.2 in pure OCaml, I found JustHTML, a new Python library for parsing HTML5 by Emil Stenström (via Simon Willison posting about it).\nEmil wrote JustHTML using coding agents as well, and then Simon ported it to JavaScript in a few hours.
\nMy question, though, is how difficult is to go in the other direction and move towards a strongly typed interface like OCaml's. Could we ultimately distill down the extremely complex set of rules around parsing HTML all the way into a proof assistant like Lean, but hopping via OCaml and Haskell to provide convenient executable pitstops?
\nToday's task was to vibespile the Python into ocaml-html5rw, a pure OCaml HTML5 parser and serialiser that passes the browser test suite 100%.
\nI took a very similar approach to my earlier Yaml 1.2 port,\ndepending purely on Bytesrw as the\nstring decoding/encoding codec. I instructed the agent not to use any other\nexternal libraries, and to build up a test suite that could decode the\nhtml5lib-tests to act as an external oracle. I also used my earlier Claude skills to tidy up OCaml code that was generated.
\nI also used an earlier trick to get my freshly generated OCaml test suite (which is itself parsing the\nthird-party html5lib expect tests) to output a standalone HTML report. This was\nextremely useful to see progress, but also for me to understand what sort of thing HTML5 parsing involves. You can browse a snapshot to judge for yourself.
\n ![\"%c\"](\"/images/aoah-html5-ss-tests.webp\" "\\"The")
I'd never actually realised before doing this that, unlike many parsers, HTML5\nparsing never actually fails. The WHATWG\nspecification is a living standard\nthat defines error recovery rules for almost every possible malformed input,\nensuring all HTML documents produce a valid DOM tree just like browsers do.\nSo, the biggest danger here is that we parse a minor syntax error into a nonsensical\nDOM that is "far away" from the author intention.
\nThe HTML5 port took a few hours, and the resulting library seems to pass the\nHTML5 tests without much drama. One footgun is that the test runner itself was\nquite complex, and hidden in there was some skipping of test cases. So my\nreview of this library actually happens in reverse: I read the test runners\nfirst to figure out what's going on, only working backwards to the library\nitself. A very strange workflow...
\n![\"%c\"](\"/images/aoah-html5-ss-1.webp\" "\\"The")
There's nothing too surprising so far; after all Simon Willison ported it to\nJavaScript in no time at all. So what makes this interesting to do in OCaml\nvs Python or Javascript? The obvious thing is modules and ease of\nrefactoring, so I spent some time critically going through the resulting\nmodule structure of the library itself.
\nA lot of the code inside the library looked suspiciously like it was reinventing the Unicode wheel, with lots of character-encoding specific manipulation. So I cloned some key libraries from the OCaml ecosystem that are both fairly standalone and well engineered: astring, uutf, and uunf by Daniel Bünzli.
\n![\"%c\"](\"/images/aoah-html5-ss-3.webp\" "\\"The")
\n![\"%c\"](\"/images/aoah-html5-ss-4.webp\" "\\"The")
I cloned a few candidate libraries, and the agentic analyse determined that only some of them were relevant to the problem at hand, so I discarded the rest and focussed on the top set.
\nThe resulting diff crunched the size of the library down considerably. Since we had extensive test coverage, the 100% pass rate at the end built up some confidence that semantics weren't changed too badly. The existence of OCaml interface files also meant I could inspect those separately in the diff and be satisfied that only implementations had changed.
\nModules and types are defining feature of OCaml, so I pointed the agent at the WHATWG standard and asked it to introduce explanations directly into the interface files themselves. This is quite different from an informal specification; the guidelines can be browsed directly and navigated around via the odoc HTML output.
\n ![\"%c\"](\"/images/aoah-html5-ss-odoc.webp\" "\\"Learn")
When I was browsing the types, I realised that there were too many strings involved in parsing errors, and so the agent helped synthesise them into an extensible OCaml variant that describes things much more precisely. This has gotten me thinking about 'doing verification in reverse'; just like AI for maths is shifting what it means to do math, this sort of thing is shifting what it means to do formal specification.
\nI'll end by asking the same questions that Simon did:
\n\n\nI’ll end with some open questions:
\n\n
\n- Does this library represent a legal violation of copyright of either the Rust library or the Python one?
\n- Even if this is legal, is it ethical to build a library in this way?
\n- Does this format of development hurt the open source ecosystem?
\n- Can I even assert copyright over this, given how much of the work was produced by the LLM?
\n- Is it responsible to publish software libraries built in this way?
\n- How much better would this library be if an expert team hand crafted it over the course of several months?\n-- I just ported JustHTML from Python to JavaScript, Simon Willison, 2025
\n
I feel the last question is answered most easily: an expert team that has access to these tools and the domain knowledge about HTML5 should be able to do a good job. While I have no experimental evidence in this domain about that fact, we did find earlier this year that expert-level retrieval of conservation evidence could be significantly boosted via access to living evidence databases. I feel agentic search sits alongside the same 'needle-in-a-haystack' productivity boost; scanning through thousands of HTML5 test cases to find the problem is something that agents are better at than humans who do not have domain knowledge (like me, in this case).
\nThe question of copyright and licensing is difficult. I definitely did some\nediting by hand, and a fair bit of prompting that resulted in targeted code\nedits, but the vast amount of architectural logic came from JustHTML. So I\nopted to make the LICENSE a joint one\nwith Emil Stenström. I did not follow the transitive dependency through to the Rust\none, which I probably should.
\nI'm also extremely uncertain about every releasing this library to the central\nopam repository, especially as there are excellent HTML5\nparsers already available. I haven't\nchecked if those pass the HTML5 test suite, because this is wandering into the\nagents vs humans territory that I ruled out in my groundrules.\nWhether or not this agentic code is better or not is a moot point if releasing\nit drives away the human maintainers who are the source of creativity in the code!
\nI note that throughout my entire AoAH adventure so far, most of the code I've\ngenerated has been spectacularly unoriginal. I've received tons of help and\nexamples of how to use tools from colleagues as input, but the aggregate set\nof OCaml code output that I would class as refreshingly interesting from me\nis pretty minimal.\nSo in the long term, I don't think this process is "helping the core" of the\ncommunity by coming up with beautiful functional pearls.
\nHowever, the libraries are satisfying an important need for utility: some\nthings like Yaml and HTML5 are fundamentally such ugly formats that I find it\nhard to argue for elegant solutions therein, and yet we need to manipulate them\nto live in the real world. So I'm ending up with a utilitarian plea, with some\nangst that this might be smothering the functional flame that make's OCaml so\nmuch fun in the first place. I must ask Jenny Gibson for her views on where\nplay fits into programming the next time I see her!
\nTomorrow in Day 16 we'll use this new library to help with\n"vibesplaining" code via executable notebooks.
With the Requests library under my belt, I finally got to what\nI actually need for myself: vibe coding OCaml library interfaces to my\n##selfhosted services that contain most of my data.
\nTo start with, I use\nKarakeep across all my devices to bookmark things, and\nI'd like to be able to programmatically search through tags, for example by\ntaking all outbound links from the blogs that I read and autosynching them with\nmy remote service. Karakeep on the server side does some cool things like\nscreenshot links and create local webarchives.
\nUnfortunately, Karakeep doesn't publish an OCaml interface. Fortunately, my\nnew bestie Claude helped me build\nocaml-karakeep without\nmuch input from me!
\n ![\"%c\"](\"/images/karakeep-ss.webp\" "\\"I")
With a ton of pre-Christmas organising to do this weekend, I didn't have a lot of\nspare time. So I thought I'd use everything I've learnt so far and try to\none-shot this as bravely/foolishly as possible. I setup a repo with:
\nAnd then I prompted the agent to build jsont specifications derived from the\nKarakeep source, and then hook this up to Requests, and then iterate using my\nlive key to debug failures until the API spec was right.
\nThis almost worked first time! While the testcases generated worked, I also got the agent to generate a karakeep CLI command. I'd made sure that, as with xdge, the Requests library also exposes Cmdliner terms that permit easy integration of HTTP requests into any CLI (to configure logging, proxies, that sort of thing).
\n![\"%c\"](\"/images/karakeep-ss-3.webp\" "\\"I")
Since I didn't have time to do a full coverage test of the API, I asked the agent to exercise the functionality and do live debugging. This is where the jsont codec was incredibly useful, since it provided all the right hints to the agent to take action. Good error messages aren't just for humans any more!
\nA typical error, for example was:
\n> main.exe bookmarks summarize wqa1sc1z42xcxfwwrsxeggbl\nmain.exe: [ERROR] Karakeep error: JSON error: Missing members in bookmark object:\n content\n createdAt\n id\nFile "-", line 1, characters 0-615: \nThe agent had enough data from that error message to figure things out and make the jsont codec more permissive with option types.
\n![\"%c\"](\"/images/karakeep-ss-2.webp\" "\\"The")
The library also made good use of the stateful HTTP interface from Requests, for example by modifying all the default requests to use the API key:
\nlet create ~sw ~env ~base_url ~api_key = \n let session = Requests.create ~sw env in \n let session = \n Requests.set_auth session (Requests.Auth.bearer ~token:api_key) \n in \n { session; base_url }\nThe karakeep.proto subpackage has all the dedicated jsont decoders, which take care of conversion to-and-from OCaml types. I found these quite readable:
(** Type of content a bookmark can have *) \ntype bookmark_content_type = \n | Link (** A URL to a webpage *) \n | Text (** Plain text content *) \n | Asset (** An attached asset (image, PDF, etc.) *) \n | Unknown (** Unknown content type *) \n \nval bookmark_content_type_jsont : bookmark_content_type Jsont.t\nThe new agent trick I learnt today is the power of debugging against live services. The jsont descriptions derived from the Karakeep source code provided just enough information in error messages that the agent could fix them by trying to run the command against my live service. Luckily, nothing got deleted this time, but in the future projects like Shelter will be ever more important.
\nMichael Dales asked an interesting question: "Why OCaml over an even more strict language?". My instinct is that OCaml is pretty much the best choice here; I think it's pretty hard to find a usable language with a stronger typing discipline. The more exotic the type systems get, the harder it is to do proof discharge. However, I believe that with some reinforcement learning magic, OxCaml's lifetimes might be gamechanging when combined with agents that can also reason about performance and memory in addition to conventional type safety.
\nAnyway, I now have a karakeep binary and library that's working great for my search needs. Back to bookmarking Christmas presents I needed to have obtained yesterday!
Now I had some prerequisite libraries, I turned my attention to having a batteries-included OCaml HTTP tool with features like request throttling and redirect loop detection. I've hacked on OCaml HTTP protocol libraries since 2011, but these higher level features weren't necessary in things like Docker's VPNKit. The problem with building one now is that there are loads of random quirks needed in real-world HTTP, which would take ages to figure out if I start from scratch.
\nLuckily, there's an entire ecology of HTTP clients built in other languages that could use for inspiration as well! Today, I gathered fifty open-source HTTP clients from a variety of other language ecosystems, and agentically synthesised a specification across all of them into one OCaml client using Eio.
\nI'm not sure what the collective verb is for a group of HTTP clients, so dubbed this whole process a 'heckle' of HTTP coding!
\nThe first thing is to find the heckle of other implementations, which I grabbed from this awesome-http repository. I vibed up a git cloner that fetched the sources to my local dev repo, so the agent could access everything.
\n ![\"%c\"](\"/images/awesome-http.webp\" "\\"To")
Then came the big task of differentially coming up with a spec. I used my earlier Claudeio OCaml library to build a custom agent that iterates over each of the fifty repos and uses the structured JSON output to emit the recommendations using a simple schema. I left this churning for an hour while I went for my morning run, and came back to records like this:
\n"priority_features": [ {\n "priority_rank": 1,\n "category": "Security & Spec Compliance",\n "title": "Strip sensitive headers on cross-origin redirects",\n "description": "When redirecting to a different origin (host/port/scheme),\n automatically strip Authorization, Cookie, Proxy-Authorization, and\n WWW-Authenticate headers to prevent credential leakage to\n unintended domains. Also strip headers on HTTPS->HTTP protocol downgrade.",\n "rfc_references": [ "RFC 9110 Section 15.4 (Redirection)" ],\n "source_libraries": [\n "reqwest", "got", "node-fetch", "okhttp", "axios", "superagent",\n "http-client", "needle" ],\n "affected_files": [ "lib/requests.ml", "lib/one.ml", "lib/http_client.ml" ],\n "implementation_notes": "Implement same_origin check comparing scheme, host,\n and port. Create a list of sensitive headers to strip. Call\n strip_sensitive_headers before following any redirect where origin changes.",\n "cross_language_consensus": 8 },\nThere were hundreds of recommendations, ranked by their class (security, new or missing features, etc), but also how many other language ecosystems viewed\nthis as important along with a list of specific libraries. Each recommendation also included\nimplementation notes specific to OCaml, since the agent had been instructed to differentially\ncompare with the strawman OCaml implementation.
\nAt this point, I did a manual scan and dropped a few language ecosystems that weren't immediately useful.\nFor example, I'd added in a few Lean HTTP clients after\nreading Martin Kleppmann predict that AI will make formal verification go mainstream. However, the Lean clients weren't quite feature complete enough yet, and also don't seem to include any actual verification (it's using Lean as a conventional programming language) so just reverting to Haskell as the reference there seemed fine.
\nThe agent came back with a nice summary when asked to focus on just Python, Rust and Lean and helped confirm my hypothesis:
\n\n\nKey gaps compared to Python requests and Rust reqwest:
\n\n
\n \nGap \nImpact \n\n \nNo proxy support \nBlocks enterprise/corporate adoption \n\n \nNo compression \nLarger downloads, slower performance \n\n \nNo Response.json() \nExtra boilerplate for common use case \n\n \nNo auth stripping on redirects \nPotential credential leakage \n\n \nNo custom CA certs \nCan't use with self-signed/internal PK \n\n \nNo client certs (mTLS) \nMissing enterprise auth scenario \nThe Lean HTTP library is interesting for its strong typing but is\nserver-focused and lacks many client features. It's not a good model for your\n"batteries included" goal.
\n
The full list of projects I ended up using was pretty epic: JavaScript (Axios, node-fetch, Got, superagent, Needle),\nPython (Requests, urllib3, httplib2, GRequests, Uplink), Java (Eclipse Jetty, OkHttp, Heritrix, Apache HttpClient, Google HTTP Client, Http Request), Rust (reqwest, hyper, Isahc, Surf, curl-rust), Swift (Alamofire, SwiftHTTP, Net, Moya, Just, Kingfisher), Haskell (Req, http-client, servant-client, http-streams), Go (Req, Resty, Sling, requests), C++ (Apache Serf, cpr, cpp-netlib, Webcc, Proxygen, cpp-httplib, NFHTTP, EasyHttp), PHP (Guzzle, HTTPlug, HTTP Client, SendGrid HTTP Client, Buzz), Shell/C (HTTPie, curl, aria2, HTTP Prompt, Resty, Ain). Thank you to all the authors for publishing your respective open-source code!
\nThe second phase was to build a summarisation agent that took the hundreds of recommendations and crunched them up into a unified priority list. Rather than do this by hand, it's very convenient to let the agent tool use take care of scanning the JSON files, backed up by some forceful prompting to make sure it covers them all (there is no guarantee it will, but hey, this is AI). I also used my Claude Internet RFC skill to fetch the relevant specifications so they could also be cross linked from the recommendations.
\nThe summariser crunched through them in about a minute using the latest Claude Opus 4.5 model. The resulting recommendations was a very clean list of features in priority order. Here's an example of another security feature I'd never considered before:
\n\n\nImplement Header Injection Prevention (Newline Validation)
\nValidate that user-provided header names and values do not contain newlines\n(CR/LF) which could enable HTTP request smuggling attacks. Reject headers\ncontaining these characters with a clear error message.
\n\n
\n- RFC References:\n
\n\n
\n- RFC 9110 Section 5.5 (Field Syntax)
\n- RFC 9112 Section 2.2 (Message Format)
\n- Cross-Language Consensus: 5 libraries
\n- Source Libraries: haskell/http-client, rust/hyper, php/guzzle, java/okhttp, go/req
\n- Affected Files:
\nlib/headers.mllib/http_client.mllib/error.ml- Implementation Notes: Add validation in Headers.set/Headers.add that\nrejects header names/values containing
\n\\ror\\ncharacters. Add InvalidHeader\nerror variant with the offending header name for debugging.
After this, the rest was 'normal' agentic coding, whereby I put the agent in a loop\nbuilding the recommendations in order, regularly using my ocaml-tidy-code\nto clean up the generated mess (somewhat), and afterwards running it through a module refactoring.
\n![\"%c\"](\"/images/aoah-heckle-ss-1.webp\" "\\"The")
The resulting ocaml-requests library does work, but I feel this is the first time I've lost the thread on the exact architecture of the library, so this will take some careful code review.
\nThe basic library fulfills its original mandate fairly well; there is a simple high level interface that's direct style:
\nEio_main.run @@ fun env ->\nSwitch.run @@ fun sw ->\nlet req = Requests.create ~sw env in\nRequests.set_auth req (Requests.Auth.bearer "your-token");\nlet user, repos = Eio.Fiber.both\n (fun () -> Requests.get req "https://api.github.com/user")\n (fun () -> Requests.get req "https://api.github.com/user/repos") in\nlet user_data = Response.body user |> Eio.Flow.read_all in\nlet repos_data = Response.body repos |> Eio.Flow.read_all in\n...\nThis is straightforward, and there is also a stateless Requests.One module that contains the one-shot equivalents. There's also an ocurl binary that exercises it all via the CLI. For example, here's ocurl downloading a recent paper and following redirect chains:
> dune exec -- bin/ocurl.exe https://doi.org/10.1038/s43016-025-01224-w -Iv\nocurl: Creating new connection pool (max_per_endpoint=10, max_idle=60.0s, max_lifetime=300.0s)\nocurl: Created Requests session with connection pools (max_per_host=10, TLS=true)\nGET https://doi.org/10.1038/s43016-025-01224-w\nocurl: > GET https://doi.org/10.1038/s43016-025-01224-w HTTP/1.1\nocurl: > Request Headers:\nocurl: > Accept-Encoding: gzip, deflate\nocurl: > User-Agent: ocaml-requests/0.1.0 (OCaml 5.4.0)\nocurl: Creating endpoint pool for doi.org:443 (max_connections=10)\nocurl: TLS connection established to doi.org:443\nocurl: < HTTP/1.1 302\nocurl: < Response Headers: <trimmed>\nocurl: < date: Sun, 14 Dec 2025 14:22:18 GMT\nocurl: < expires: Sun, 14 Dec 2025 14:48:37 GMT\nocurl: < location: https://www.nature.com/articles/s43016-025-01224-w\nocurl: \nocurl: Following redirect (10 remaining)\nocurl: \nocurl: Request to https://www.nature.com/articles/s43016-025-01224-w ===\nocurl: > GET https://www.nature.com/articles/s43016-025-01224-w HTTP/1.1\nocurl: > Request Headers:\nocurl: > Accept-Encoding: gzip, deflate\nocurl: > User-Agent: ocaml-requests/0.1.0 (OCaml 5.4.0)\nocurl: > <trimmed>\nocurl: Following redirect (9 remaining)\nocurl: \nocurl: > GET https://idp.nature.com/authorize?response_type=cookie&client_id=grover&redirect_uri=https://www.nature.com/articles/s43016-025-01224-w HTTP/1.1\nocurl: > Request Headers:\nocurl: > Accept-Encoding: gzip, deflate\nocurl: > User-Agent: ocaml-requests/0.1.0 (OCaml 5.4.0)\nocurl: \nocurl: Following redirect (8 remaining)\nocurl: > GET https://idp.nature.com/transit?redirect_uri=https://www.nature.com/articles/s43016-025-01224-w&code=99850362-a71b-426f-a325-887d4ebc2346 HTTP/1.1\nocurl: > Request Headers:\nocurl: > Accept-Encoding: gzip, deflate\nocurl: > Cookie: idp_session=sVERSION_1fdd8b701-8a14-4c8d-b31f-01af7f3653a2; idp_session_http=hVERSION_149b210b2-c08c-40b1-949f-4125872fff82; idp_marker=4792551b-f6f8-4b89-a02b-9032ea177821\nocurl: > User-Agent: ocaml-requests/0.1.0 (OCaml 5.4.0)\nocurl: \nocurl: Following redirect (7 remaining)\nocurl: \nocurl: Request completed in 0.972 seconds\nocurl: Request completed with status 200\nQuite verbose, but you can see there's a lot going on with a seemingly simple HTTP request beyond just one request!
\nWhere it gets a little messier is the internals, which were built as a result\nof iterative agentic work. I don't entirely have confidence (beyond peering at\nthe code) that all those edge cases identified from 50 other libraries are all\ntested in my implementation.
\nTo mitigate this, I put together a httpbin set of ocaml-requests cram tests which use an ocurl binary (which models upstream curl, but using this OCaml code) in order to run\na better of command line tests. These do things like:
\nTest cookie setting endpoint:\n\n $ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?session=abc123" | \\\n > grep -o '"session": "abc123"'\n "session": "abc123"\n\nTest setting multiple cookies:\n\n $ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?session=abc123&user=testuser" | \\\n > grep '"cookies"' -A 4\n "cookies": {\n "session": "abc123",\n "user": "testuser"\n }\n }\nWhile httpbin is extremely convenient to providing a local endpoint that understands HTTP, this still doesn't cover the full battery of tests. Ideally in the future, we should be able to use the 50 test suites from other language's libraries somehow. I haven't quite thought this through, but it would be extremely powerful if we could bridge OCaml over to a wider set of test suites.
\nI've accomplished what I set out to do at the start of the day, but I feel like\nI've gone further off the 'conventional coding' path than ever before. On one\nhand, it's incredible to have churned through fifty other libraries in a\nmatter of hours. On the other hand, I have almost no intuition about the\ndetailed structure of my resulting library without going through it\nline-by-line.
\nThere are many important details like the intermingling of connection handling,\nexceptions and resource cleanups that are likely shaky. A lot of common issues\nwill be taken care of by the fact that Eio Switches\nhandle resource cleanup very well, but it's unlikely to be 100%. This definitely requires\nhuman attention, which I'll do over the next few months.
\nHowever, I've also got very little emotional attachment to the structure of the\nnew Requests library, unlike other libraries I've written by hand -- it's so\neasy to refactor with a bit more agentic coding! Alex Bradbury\nasked me if I'm enjoying this whole process, and today I realised that I am. If I\nviewed this as conventional coding, then I would hate it. But agentic coding is\nextraordinarily different from conventional coding, much more akin to top-down\nspecification and counterexample driven abstract refinement\nusing natural language!
\nI'm doing a lot of machine learning research elsewhere, and this agentic\ncoding adventure reminds me of a wonderful quote from Emma Goldman\nthat I heard while listening to 3rd Reith Lecture today:
\n\n\n"If can't dance, it's not my revolution!"\n-- Emma Goldman, 1931
\n
It's important to find joy in whatever we're working on, and I'm glad I am enjoying myself\nhere, even though I remain highly uncertain about how to integrate this with the open source\npractises I've experienced in the past three decades.\nI'm going to forge ahead with using Requests for a while on some real-world\nAPIs that I need access to, and see how it goes. But this is definitely not\nready for wider use just yet!
\nIn Day 14 we'll take a look at using Requests to build OCaml\ninterfaces to some of the ##selfhosted services I use.
After yesterday's library bonanza for HTTP cookie handling, I implemented a TCP/TLS connection pooling library. This is useful for an HTTP client as it provides the network-level mechanisms for keeping track of outgoing network connections by their DNS name. This allows for more flexible outgoing connection management without worrying about overloading remote endpoints.
\nFor example, github.io has four A records:
> host github.io\ngithub.io has address 185.199.110.153\ngithub.io has address 185.199.109.153\ngithub.io has address 185.199.108.153\ngithub.io has address 185.199.111.153\nWith this new connection pooling library, my application should be able to\nconnect to the github.io name and keep track of all the outgoing connections\non the basis of it being called github.io and load balance the number of\noutgoing connections accordingly.
In the interests of exploring something new, I also decided to add in visualisation\nsupport to figure out what the library is spending its time on.\nI decided to generate self-contained visualisations,\ninspired by Jon Ludlam rediscovering the joy of SVGs yesterday!
\nThe core library itself is pretty straightforward, as it's reasonably similar\nto the ocaml-conduit mechanism of\nproviding a name-based resolver. The interface should be a matter of creating a\nconnection pool to keep track of state and then requesting a connection from it\nto a hostname:
\nmodule Endpoint: sig\n type t\n val make : host:string -> port:int -> t\nend\n\ntype connection_ty = [Eio.Resource.close_ty | Eio.Flow.two_way_ty] \ntype connection = connection_ty Eio.Resource.t\nval connection : sw:Eio.Switch.t -> t -> Endpoint.t -> connection\nThis uses Eio's resource mechanism to allow this connection to be used like any other.
\nOne of the coolest things about Eio's error handling is the ability to stack errors by re-raising exceptions and adding more context to it. I prompted the agent to also create connection-specific errors for Eio, but to integrate them into the Eio.Io extensible type. This allows errors from failures to look like this:
\n[WARNING] Connection attempt 3 to localhost:8088 failed:\n Eio.Io Net Connection_failure Refused Unix_error\n (Connection refused, "connect-in-progress", ""),\n connecting to tcp:127.0.0.1:8088,\n connecting to localhost:8088,\n after 3 retry attempts\nThe conpool library can keep a lightweight reporting stack of what it's been\ndoing while propagating the error, which makes a big different to the quality\nof the end logging.
\nI activated Claude Marketplace's front end module\nand instructed it to generate me a self-contained HTML file with the results of\nthe output of the stress test.
\nThis...just worked. Here's a HTML\nsnapshot of the results\nof a conpool run, with all the various configurations tests pushed together\ninto a visualisation. I've also seen Sadiq Jaffer do the same thing when iterating\non TESSERA CNN visualisations for his intermediate runs.
\n ![\"%c\"](\"/images/aoah-vis-ss-1.webp\" "\\"Standalone")
I figured I'd have a go at integrating at event tracing, but the various\npackaging problems around the tools defeated my quick attempt. There's\nmeio which I really want to use, but\nit requires various upstream PRs merging, and I'll also need to figure out what\nto do if the ring buffer overflows. I'll have to come back to this later; for\nnow the self-contained visualisations are fine.
\nI found myself doing the same prompting repeatedly to tidy up generated code to\nmake it more idiomatic, so I had Claude go back over my history to compact my\ninstructions into a reusable skill, which I uploaded to\nocaml-tidy-code.\nRunning this over all the code is good to do after test cases have been\ngenerated, as then it's fairly easy to verify that things are working well.
\n![\"%c\"](\"/images/aoah-cleanup-ss-1.webp\" "\\"The")
I find the Eio types to be quite complex after we stripped out objects, and so\nhaving an agent code up the idiomatic boilerplate required for registering Eio\nexceptions, pretty printers and resources was quite nice. It's also good to\nhave another Claude skill to further refine my workflow for cleaning up code.
\nThe self-contained visualisations are also extremely useful; in addition to the\nstandalone HTML one, I also vibed up a full event logging system using the\nRuntime_events that\nSadiq Jaffer implemented. I decided not to go with that due to some difficulties\nwith dealing with full ring buffers, but I'll come back to in the future.\nThere's clearly a need for a library that can register library-level events and\ndispatch them to the OCaml custom events buffer, to remote logging endpoints,\nand to clients like terminal UIs or web monitors.
\nTomorrow, we'll pull this all together into a Day 13 HTTP client!
I'm switching focus for a few days to build a complete HTTP(S) client to use in my literature downloader. This requires building a few support libraries before we build the full client, so I figured I'd dive in them in the next few days. First up is RFC6264 HTTP Cookie support. There are some excellent existing cookie libraries already on opam, notably http-cookie and ocaml-cookie, but I wasn't sure what their coverage of the protocol is, and there's no Eio serialisation support.
\nSo I thought I'd have a go at a different approach today using agentic coding: can we synthesise a complete HTTP Cookie implementation purely from the RFC 6265 prose itself, and then differentially compare this OCaml implementation against the others? In theory, running a single test suite across all three libraries might be a good way of discovering how to improve the existing implementations. In the long-term, http-cookie is probably the upstream library I want to use, but I don't want to generate a giant diff against it today due to my groundrules of not disturbing other maintainers.
\nI downloaded RFC 6265 and gave the agent all my previously generate code in the same directory to act as examples. My first run went verrrrry strangely as it generated a plan for a carrier-grade NAT implementation, until I realised that I'd typoed the RFC number and picked 6264 by mistake. Ah, such human frailty...
\nOnce I got the right RFC number, I stuck Claude into planning mode with the RFC text and also instructed it to search through opam to find relevant packages. This opam search went well, as it identified some missing dependent logic that a full implementation of cookies would require. So I went down a sub-rabbithole of implementing those dependent packages first.
\nFirst up is Punycode, as RFC6265 Section 5.1.2 requests that:
\n\n\nConvert each label that is not a Non-Reserved LDH (NR-LDH) label,\nto an A-label (see Section 2.3.2.1 of [RFC5890] for the former\nand latter), or to a "punycode label" (a label resulting from the\n"ToASCII" conversion in Section 4 of [RFC3490]), as appropriate\n(see Section 6.3 of this specification).)
\n
I'd never heard of this Punycode before, but perusing\nRFC3490 introduces it such:
\n\n\nUntil now, there has been no standard method for domain names to use\ncharacters outside the ASCII repertoire. This document defines\ninternationalized domain names (IDNs) and a mechanism called\nInternationalizing Domain Names in Applications (IDNA) for handling them in a\nstandard fashion. IDNs use characters drawn from a large repertoire\n(Unicode), but IDNA allows the non-ASCII characters to be represented using\nonly the ASCII characters already allowed in so- called host names today.\nThis backward-compatible representation is required in existing protocols\nlike DNS, so that IDNs can be introduced with no changes to the existing\ninfrastructure. IDNA is only meant for processing domain names, not free\ntext.\n-- RFC3490, 2003
\n
Ok, ok so this is some DNS thing needed to reliably compare cookie domains that\nmight be in different languages. The RFC is a little unusual in that it embeds\nC code and also test vectors, so this seems ideal for an agentic session.
\nI pushed ocaml-punycode to Tangled. It's fairly straightforward, with one module for the core Unicode (using uufn for Unicode normalisation and the domain-name by Hannes Mehnert for RFC1035 Internet Domain Name handling.\nAlthough we didn't use cram tests for this, the Punycode alcotest seem to encode the test vectors from the RFC faithfully, and also do roundtrip testing.
\nOne trick that helped get more idiomatic OCaml code here was to do the generation in two passes. First I asked it to transcribe the RFC into OCaml along with the test cases, and then a second pass to refactor the code using higher-order functions and Stdlib combinators. You can see the results of the second pass here, which does look like more normal OCaml that I might write.
\nCoding with the RFC as the source spec actually resurrected an issue I've been thinking about since 2015. Back then, Luke Dunstan wanted to add multicast DNS support to ocaml-dns, and as part of that he proposed using extension attributes in the code to link the relevant section of the RFC spec to the associated OCaml function.
\nCoding agents go a long way to automating this. In the punycode ocamldoc, I instructed it to use the online RFC repository as an anchor base, and it successfully links almost every type to where it got the specification constraints from.
\ntype error =\n| Overflow of position\n (** Arithmetic overflow during encode/decode. This can occur with\n very long strings or extreme Unicode code point values.\n See {{:https://datatracker.ietf.org/doc/html/rfc3492#section-6.4}\n RFC 3492 Section 6.4} for overflow handling requirements. *)\n| Invalid_character of position * Uchar.t\n (** A non-basic code point appeared where only basic code points\n (ASCII < 128) are allowed. Per\n {{:https://datatracker.ietf.org/doc/html/rfc3492#section-3.1}\n RFC 3492 Section 3.1}, basic code points must be segregated\n at the beginning of the encoded string. *)\nI've therefore mechanised this insight into a\nclaude-ocaml-internet-rfc\nskill which formalises my future approach to RFCs and allows this part of the\nworkflow to be reused in future OCaml code.
\n![\"%c\"](\"/images/http-cookie-1.webp\" "\\"This")
But then the exploration of the OCaml RFCs told me that for real compliance, we\nneeded to support Public Suffix lists, something else\nI'd never heard about!
\n\n\nA "public suffix" is one under which Internet users can (or historically\ncould) directly register names. Some examples of public suffixes are com,\nco.uk and pvt.k12.ma.us. The Public Suffix List is a list of all known public\nsuffixes.
\nThe Public Suffix List is an initiative of Mozilla, but is\nmaintained as a community resource. It is available for use in any software,\nbut was originally created to meet the needs of browser manufacturers. It\nallows browsers to, for example:
\n\n
\n- Avoid privacy-damaging "supercookies" being set for high-level domain name suffixes
\n- Highlight the most important part of a domain name in the user interface
\n- Accurately sort history entries by site\n-- Public Suffix Lists, 2025
\n
This one's different from the previous library in that there's a large dataset that needs to be embedded inside the library, and we need reasonably efficient datastructures to traverse them and do the domain comparison (using the Punycode logic from earlier for domain name normalisation).
\nSo my approach for this was to download the dataset and clone the public suffix list wiki and prompt the agent to come up with a generation architecture that would first parse the list into an OCaml data structure and then provide a Public_suffix OCaml interface that we could use without having to download the data as a library user.
The agent managed all the dune build rules fine for the generated package, so I pushed a ocaml-publicsuffix Git repository using the opam metadata skill. I also then used the earlier ocaml-internet-rfc skill to add references to the various RFCs we used.
\nThe interface is very simple, since the dataset is embedded as OCaml code:
\nlet psl = Publicsuffix.create () in\n\nPublicsuffix.public_suffix psl "www.example.com"\n(* Returns: Ok "com" *)\n\nPublicsuffix.public_suffix psl "www.example.co.uk"\n(* Returns: Ok "co.uk" *)\n\nPublicsuffix.registrable_domain psl "www.example.com"\n(* Returns: Ok "example.com" *\nAfter these segues, we finally have enough for an Eio-based Cookie library that's derived from the main cookie RFC!
\nThis was pretty plain sailing given the previous libraries. The published ocaml-cookeio library library has subpackages for the Cookie jar interface that provide a pretty simple persistent cookie implementation that we can use in the next few days for our HTTP client.
\nval create : unit -> t\n(** Create an empty cookie jar. *)\n\nval load : clock:_ Eio.Time.clock -> Eio.Fs.dir_ty Eio.Path.t -> t\n(** Load cookies from Mozilla format file.\n\n Loads cookies from a file in Mozilla format, using the provided clock to set\n creation and last access times. Returns an empty jar if the file doesn't\n exist or cannot be loaded. *)\n\nval save : Eio.Fs.dir_ty Eio.Path.t -> t -> unit\n(** Save cookies to Mozilla format file. *)\n\n(** {1 Cookie Jar Management} *)\n\nval add_cookie : t -> Cookeio.t -> unit\nI didn't expect to generate three libraries in one day, but I was massively sped up by creating this Claude skill for Internet RFC handling. The three libraries today, punycode, publicsuffix and cookeio do a lot of fairly ad-hoc checking that's specified throughout a variety of RFCs, but do benefit from public test sets to check their behaviour.
\nNow that these are out of the way, I'm going to continue on with another essential library on Day 12 tomorrow: Eio TCP/TLS connection pooling.
\nUpdate 13th Dec 2025: Romain Calascibetta and Hannes Mehnert both kindly alerted me to the fact that there are existing implementations of punycode and publicsuffix. The first one by cfcs/punycode wasn't found by my agent since it's not in the opam repo, and public-suffix was missed since my checkout was just a week out of date! But to re-emphasise my groundrules, this is not a competition of agents vs humans. I will very happily use these "proper" libraries in the long term, but in the short-term I cannot contribute a giant chunk of AI-generated code onto those upstream projects. Figuring out what to do about this is one of the points of this month-long adventure...
After building a reasonably complete Sortal contacts manager and\ntrying out OxCaml's Bonsai_term, I thought I'd have a second go\nat a terminal UI using a newly announced Mosaic library by Thibaut Mattio.
\nI first noticed this library when Thibaut presented his OCaml coding with AI talk at FunOCaml. It's quite different from Bonsai in that Mosaic uses OCaml's effects to provide a more direct-style API, and so seems worth experimenting with. So today's task is to port Sortal to use Mosaic and see what this terminal UI looks like!
\n ![\"%c\"](\"/images/mosaic-cli-1.webp\" "\\"Thibaut")
The environmental setup for Mosaic was similarly complicated to yesterday's with Bonsai, but for different reasons. Mosaic requires OCaml 5.4.0 or higher, so I had to relax some constraints in dependent packages. But it also requires pinning vendored packages, which I did manually (specifically, the vendored tree-sitter package). I later automated this vendoring with unpac.
Once this manual package messing was done, the rest was straightforward. I left my remote Linux system vibing with access to all the relevant source code (important since this is a bleeding edge package so the parametric memory of the agent will know nothing about it). Luckily, the interaction between Eio and Mosaic is far easier, so we ended up with a single-process OCaml binary for the TUI; a relief after yesterday's JSON-RPC gymnastics.
\nOnce again, the ability to paste in images is key to debugging terminal UIs. I wish I had the time to automate this via a skill, but I'll come back to this later in the month perhaps.
\n![\"%c\"](\"/images/sortal-mosaic-ss-1.webp\" "\\"I")
![\"%c\"](\"/images/sortal-mosaic-ss-2.webp\" "\\"However")
The working terminal application was far simpler than the earlier Bonsai version by virtue of being a single binary. In fact, the code is pretty readable and lives in a single file. Like with Bonsai, what really helped was specifying that the agent should use the Tea example components as inspiration, and it used those to both fix the theming but also introduced Markdown rendering to make the default UI look really smart.
\n![\"%c\"](\"/images/sortal-mosaic-ss-3.webp\" "\\"The")
![\"%c\"](\"/images/sortal-mosaic-ss-4.webp\" "\\"The")
There was also an amusing keybinding bug that both Bonsai and Mosaic suffered from in the first cut of the agent code. When I searched for "/sadiq" it would crash consistently on the last letter of his name. Why?!! About 15 minutes of debugging code ensued, but at one point I realised that "q" was also the "quit" keybinding, and the app was just exiting cleanly. The agent didn't spot this at all, so some human intuition was useful in building the UI logic!
\nThis Mosaic terminal is also very usable, and some things like terminal mouse clicking works too. It's less plug-and-play than Bonsai in the longer term as I'll have to build more components like text editors. However, the easier integration with the wider OCaml ecosystem really helps here since there is only one problem to solve ("build me a TUI against an existing data model") rather than two ("port all your dependencies to OxCaml") as well.
\nBoth of these TUIs will require more focussed attention from me to learn their data models and rendering logic; neither Bonsai nor Mosaic particularly benefited from asking the agent to explain their core architectures to me. Only excellent human-written documentation will achieve that.
\nI am very tempted to build a Claudeio TUI using Mosaic as one of the future advent day projects, though! But tomorrow, I'm going to switch gears in Day 11 and work on HTTP cookies towards a HTTP downloader tool.
", "url": "https://anil.recoil.org/notes/aoah-2025-10", "title": "AoAH Day 10: Building a TUI for Sortal using Mosaic", "summary": "Building a simpler single-process terminal UI for Sortal using Mosaic's effects-based direct-style API, with Eio integration and discovering multimodal image debugging for terminal layouts.", "date_published": "2025-12-10T00:00:00.000000Z", "date_modified": "2025-12-10T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-9", "content_html": "After building a reasonably complete Sortal contacts manager,\nI decided to try to do a proper job of a terminal user interface. The first\noption for a modern UI is something that Yaron Minsky announced last week:\nbonsai_term, which also gives me\na chance to dip into the OxCaml ecosystem with my agentic hacking!
\n ![\"%rc\"](\"/images/x-bonsai.webp\" "\\"I")
Switching to the OxCaml compiler is a relatively straightforward process by following the installation instructions. There is a custom oxcaml/opam-repository remote that contains all the bleeding edge packages required.
\nHowever, stepping outside of the Jane Street OxCaml package universe and interfacing with bleeding edge upstream OCaml packages does require a minor act of god to succeed. Luckily, David Allsopp is one such OCaml god that's been helping me work through a bunch of constraints to get a consistent package universe. This is a topic for a future note, but I just wanted to warn the casual reader that this aspect of OxCaml isn't quite ready for casual use yet: if you want to play with it, then stick with the Jane Street package set for now and you'll make fine progress.
\nI however, thought I'd go in the deep end and make Bonsai work with Sortal, including the Romeo-and-Juliet-esque blending of Bonsai's use of Async with Sortal's Eio. This is complex because they each have their own run queues that don't compose cleanly -- one of the motivating problems we wrote about in early OCaml effects work back in 2017.
\nThere have been attempts since then; Thomas Leonard built an async_eio bridge and even a lwt/async/eio chimera that could run all three in one process!
\nI gave this a quick shot, but there has been a lot of bitrot and code movement in the intervening years since these prototypes, especially as OxCaml has zoomed off with all of its extensions. So after my morning espresso I decided to instruct the coding agent to instead use JSON-RPC to communicate between a Sortal server (that it spawns) and a Bonsai terminal client. After all, we've got really good jsont codecs so JSON-RPC shouldn't be too difficult.
\nThe only environment that really worked here was giving the agent a monorepo with all the relevant Jane Street packages checked out with the OxCaml switch configured, and my eio OxCaml patched packages. I later automated this monorepo assembly with unpac.
\nI then had to build the application in two phases:
\n![\"%c\"](\"/images/sortal-claude-ss-planning.webp\" "\\"The")
There was some drama, but I did end up with a working terminal UI here, although perhaps one that requires a lot more iteration before I consider it sound. The most obvious result is that I have no idea how to code review this, as I need to spend more time learnings the basics of Bonsai and then get to terminal coding and then apply it my specific Sortal usecase. But, as the only user of this thing, the artefact that came out after an hour's agentic coding isn't completely bad.
\nI was only confident enough to push this to a branch and not main until I do more experimentation, but I did learn a few fun things.
\nOne big drawback with coding agents is that they don't have a spatial sense of how UI elements can be laid out. There are workaround for browsers such as using headless Chrome, but often the results of a bad web layout just need repeated iteration.
\nHowever, Sadiq Jaffer pointed out to me that Claude Code also supports multimodal images as part of its prompts, which could be fed into it to debug the UI! It's as simple as dragging and dropping images into the Claude Code prompt.
\nFor example, look at the two screenshots below that represent before and after triggering a bug:
\n![\"%c\"](\"/images/sortal-claude-ss-before.webp\" "\\"Before")
![\"%c\"](\"/images/sortal-claude-ss-after.webp\" "\\"After")
Simply dragging these two images allowed the agent to figure out (visually) that there was a layout problem, and then apply it to a reasonable looking code fix that refactored the component layout logic.
\n![\"%c\"](\"/images/sortal-claude-ss-result.webp\" "\\"The")
It seems really worthwhile to build a proper Claude Skill that wraps this process up. I could imagine taking your favourite terminal recording library and hooking it up to an interactive Pty to automate this prompting process of UI layout.
\nThe other fun thing was experimenting with Bonsai terminal components, since there are some incredibly nice ones present. These range from TUI bar charts to full blown text editor components (with vi bindings!) to tree views.
\nCheck out some of the example gifs in the main repo like the weighted tree one below.
\n![](\"https://www.cl.cam.ac.uk/~avsm2/weighted-tree.gif\")
So I think working through the constraint problems are well worth it in order to use so many cool components. The text editor especially is extremely usable out of the box as a replace for ed or nvi, which is incredible! There are also pager replacement components which remove the need for less/more, but with the same bindings.
\nI call today a "negative but worthwhile result". The Bonsai-term architecture is too complex with the client-server split, although agentic coding made it possible to try it out really quickly. I think in the future it's worth investing in Eio and Async interoperability properly so that everything can run in one process, which dramatically simplifies a TUI. Thomas Leonard has already done the groundwork here, so it's a matter of putting time in the integration and keeping it working.
\nBonsai itself shows how powerful TUIs can be, and in the Day 10 I'll look at yet another new TUI library just announced a few days ago!
I was not expecting to find a bunch of activist librarians at the lovely spires of King's College Chapel last week, but I was very glad that I did! I gave a talk to the Confederation of Open Access Repositories group that was having a meeting about "Turning scholarly publishing on its head". Luckily, I had my budding Four Ps for Collective Intelligence fresh on my brain, so I discussed it with the assembled librarians. The crowd was a really interesting mix of the open research team at Cambridge, their French equivalents in CNRS, academic researchers like myself and Albert Cardona interested in non-traditional outputs, and of course digital librarians from all over the world.
\nThe conference was held in the lovely King's College Cambridge, and was described as such:
\n\n\nTired of long delays, expensive fees, and a lack of transparency about editorial decisions when publishing your articles? Join us at King’s College, Cambridge on December 3rd for an in-depth discussion about a better future for scholarly publishing using the Publish, Review, Curate model.\n-- Confederation of Open Access Repositories, 2025
\n
![\"%c\"](\"/images/coar-25-3.webp\" "\\"Open")
The first thing I learnt was the Cambridge Open Research team is distinct from the University library. The OR team is responsible for Apollo that is a permanent repository of scholarly publications here (aka the place where I constantly forget to deposit my papers to be REF-eligible).
\nThis is also distinct from Cambridge Open Engage (where I often drop preprints that aren't suitable for arXiv), which is run by Cambridge University Press (another independent entity that's a big contributor to the solvency of the University).
\nI'm still a bit muddled up about the exact incentives of all these diverse groups, but they all seem to be working together well, so I don't think I need to know too much more than this!
\n![\"%c\"](\"/images/coar-25-4.webp\" "\\"Discussions")
Peter-Sutton Long from the OR team gave an excellent overview of the activities of the OR team, and one thing that caught my eye was that there has been an initiative called "Diamond Open Access Journals at Cambridge" which hosts various journals like the Cambridge Journal for Climate Research or the Cambridge Journal for Visual Culture. These are "hyper-local" journals, but obviously very important to nourish to ensure that any local knowledge is kept alive, that might not otherwise make it into hallowed halls of major publishing houses like Springer.
\n![\"%c\"](\"/images/coar-25-5.webp\" "\\"Apollo")
It looks like Diamond/Cam is a relatively new initiative within the University, so I asked the wisest[1] person I know -- Bill Sutherland -- about the relevance of this to his own work on conservation evidence. Bill has run a diamond open access publication called the "Conservation Evidence Journal" since 2004, and has been managing this ever since!
\nBack when they set it up, the CEJ was one of very few open journals, but nowadays initiatives like the Cambridge OR one should make it significantly easier to manage. Bill also mentioned that it's getting quite expensive to manage the thousands of articles in CEJ, so I doubly hope to connect the Cambridge OR team with the Conservation Evidence team to see if there could be some helpful sharing of digital platforms there.
\nThe CEJ has been a critical source of causal ground truth for our collaboration with CE on evidence pipelines, so it's really really important that the knowledge about what actions on the ground make a difference is widely available and as open as possible.
\nMy talk was preceded by the brilliant Helena Gellersen from the Memory Laboratory talking about her own experiences with preprints and open publishing. She had gone through quite an adventure with conventional publishing to get some of her research out, and noted that the length of time it took for an early career research was about the same time a PhD takes here in Cambridge!
\nSomething clearly has to give here, as we also discussed at the Royal Society earlier this year. Stefanie Haustein at that meeting subsequently published this brilliant critique of the dominance of commercial publishers in academia:
\n\n\nThe domination of scientific publishing in the Global North by major commercial publishers is harmful to science. We need the most powerful members of the research community, funders, governments and Universities, to lead the drive to re-communalise publishing to serve science not the market.\n-- The Drain of Scientific Publishing, Nov 2025
\n
The arguments this paper really vibed with what the crowd at the COAR conference was saying: "The drain is four-fold, depriving the research system of Money, Time, Trust and Control". And for early career researchers who have a limited runway in which to get their insights out into the world, there is a critical time gap introduced by editorial decisionmaking gatekeeping peer review.
\nThere are some interesting alternative pathways forming. Just this week, the IJCAI conference is offering a Primary Paper Initiative whereby they charge $100 to submit a paper, but waive the fee for papers in which no author appears on any other paper. All proceeds raised from this will go towards compensating peer reviewers! This is a model that might actually work, especially given the waiver for submitting just one paper.
\n![\"%c\"](\"/images/coar-25-6.webp\" "\\"Helena")
I followed up with my own talk about the 4 principles for collective knowledge networks, which is the first outing of these since I only came up with them a couple of weeks ago. These were some of my takeaways from the discussions that followed:
\nFirstly, it was amazing to be in a room full of people that care about metadata. Several people commented on how interesting the Rogue Scholar initiative is, so I hope to find a way to get Martin Fenner discussing some his own experiences with topics like subject classification in these COAR forums.
\nThere were some questions about my interest in ATProto instead of ActivityPub. My answer to this is that they will almost certainly converge or be bridged, but the difference is where they start from. ATProto bootstrapped itself centrally via BlueSky and has around 40m users quickly as a result. It is now increasingly supporting alternative implementations and services. This makes it (in my mind) much easier to experiment with protocol evolution without having to update every single endpoint, which is often the case with ActivityPub. That's not to say one is better than the other!
\nHowever, I increasingly think that the green fields for open access infrastructure should not be solely focussed on the crowded area of paper publishing, but move to non-traditional scholarly outputs such as datasets, hyperlocal grey literature and code management. These are all areas where my group is spending vast amounts of effort, and is underserved:
\nSo my message to the COAR community after attending their first (and most energising meeting) is to open up to new forms of data mediums, and to craft infrastructure that unlocks learnings from them. I feel that solely going up against the conventional scholarly publishing incumbency is an unnecessary waste of precious energy, as it looks increasingly brittle from a commercial perspective.
\n![\"%c\"](\"/images/coar-25-7.webp\" "\\"In")
Minutes after publishing this post, Martin Fenner wrote me with some interesting experiments they've been doing with Rogue Scholar and peer review:
\n\n\n[...] we ran an experiment trying PRC with a blog post together with the MetaROR platform, the result is summarized by Chris Marcum in https://doi.org/10.54900/bymaz-4fw37. And just week we had a demo from CottageLabs regarding their upcoming COAR Notify integration coming to Zenodo and InvenioRDM, and thus Rogue Scholar.
\n
So MetaROR is another way of doing peer review across non-traditional open publishing. So much to look at in 2026!
\nUsefully, also the first person I saw at Espresso Lane this morning.
\n ↩I've been accumulating a lot of contacts that I use to write cross references\non my website. This works by using\nCmarkit to parse my custom Markdown,\nand spot entries like [@sadiqj] and convert those into a full reference like\nSadiq Jaffer.
Today, I want to build a full CLI application that stores all my\ncontacts as Yaml files in my home directory using XDG conventions, and give me\na simple search interface so I can quickly autocomplete these posts from my\neditor. I call this little application "Sortal".
\nThe data model I want behind sortal is that I can have a flat set of Yaml files somewhere\nin my XDG path, and that the CLI and library will build those up into data structures for\nquerying and printing. I have around 2500 contacts or so, so this is very manageable without\na fancy database.
\nSortal uses many of the previous libraries I've been building up so far. I prompted\nthe agent to generate a standalone OCaml project, and to analyse my existing (extremely hacked together)\nwebsite code to determine a reasonable semantic for Sortal's schema, and then to design\na CLI that uses yamlt and xdge along with jsont/cmdliner\nto plan a user interface with subcommands. I also blended in Fmt\nand Logs to get nice colours in my terminal. I'm using a lot of libraries from Daniel Bünzli, which is no coincidence as the model needs to have far less context if it only has to use a few, well-designed and modular dependencies.
\nArchitecturally, I used Claude's planning mode for this\nwith the best Opus 4.5 model, along with instructions to maintain a separation between the core\njsont schemas, the library logic, and the cmdliner terms. A useful tip is to ensure that you prompt\nClaude to "ask for any clarifications" after working, and it'll drop you into a custom terminal user\ninterface that structures followup questions. This is a very convenient way of batching answers to the\ncode model.
\n![\"%c\"](\"/images/aoah-ss-1.webp\")
I did have to do some prompting to refine how the agent designed the xdge and cmdliner integration, specifically by selecting only the XDG dirs that Sortal actually uses. It does work beautifully with the CLI once fixed, as the man page shows:
\n![\"%c\"](\"/images/sortal-ss-1.webp\")
I also prompted the model to use the man pages to guide a better CLI design, and it came up with a reasonable set of subcommands:
\n![\"%c\"](\"/images/sortal-ss-2.webp\")
I'm actually opting not to do any fancy testing for this library, since it's basically a bunch of data munging at this stage and I'm going to using it myself for my own data.
\nThe one thing I did do was to prompt the agent to have an automated import script from my existing Yaml formats, so I could run both in parallel. But aside from that, just keeping an eye the model's inferred success criteria was helpful:
\n- dune build @check succeeds\n- All tests pass (dune runtest)\n- Documentation builds without warnings (dune build @doc)\n- No __ references in generated HTML docs\n- CLI executable works correctly\n- sortal.schema builds without eio/xdge dependencies\n- Existing API unchanged (backward compatible)\nThis is all the sort of thing I would have done myself by hand, but the model has now enough previous libraries and memory and the ocaml-metadata skill to figure out my preferred style of libraries.
\nThe proof is in the pudding, so after knocking up a quick import script for my existing contacts, here's it showing Sadiq Jaffer.
\n![\"%c\"](\"/images/sortal-ss-3.webp\")
The associated Yaml is very straightforward, out of my ~/.local/share/sortal directory:
version: 1\nkind: person\nhandle: sadiqj\nnames:\n - Sadiq Jaffer\nemails:\n - address: sj514@cam.ac.uk\n type: personal\n - address: sadiq@toao.com\norganizations:\nurls:\n - url: https://toao.com\nservices:\n - url: https://github.com/sadiqj\n kind: github\n handle: sadiqj\n primary: false\norcid: 0009-0006-4120-3244\nfeeds:\n - type: atom\n url: https://toao.com/feeds/posts.atom.xml\nAs a giant quality of life improvement, I also coded up a Git_store module that automated the commit and push of the XDG directory to a private repo on every CLI change. This gives me a super quick way of synching my contacts.
I also left the version field in there to allow for future schema evolution. I guess this would need a jsont codec that only reads the version field and preserves unknown object members and passes them to a versioning module. Something for the future!
\nWell, it works! I'm using this now for this website, and I'm going to add more quality-of-life things as I need it, such as support for thumbnails.
\nIt's very encouraging to see an emerging workflow in OCaml: this language is brilliant at sketching out a big complicated service, and then refactoring to break it up into composable libraries. Having said that, I don't think the agentic coding exhibits particularly good taste in library design, and is nowhere near capable of building libraries of the quality of Daniel Bünzli, but they are very good at using those libraries.
\nAlso of note is that I have zero PPX in this CLI, which I would have used if I were building by hand. Instead, building up boilerplate combinator functions (like jsont codecs) is done pretty well by the coding agent, and results in semantically richer code. I very much appreciate Patrick Ferris maintaining ppxlib, but I do not miss having a dependency on it!
\nTomorrow in Day 9 we'll try Bonsai_term for a terminal UI, and then Mosaic to see how alternative approaches might work.
", "url": "https://anil.recoil.org/notes/aoah-2025-8", "title": "AoAH Day 8: Building a contacts CLI manager with Sortal", "summary": "Creating Sortal, a CLI contacts management application using Yaml storage, XDG directories, Git-based synchronization, and integrating all previously built libraries into a cohesive CLI tool.", "date_published": "2025-12-08T00:00:00.000000Z", "date_modified": "2025-12-08T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-7", "content_html": "After the excitement of building an entire Yaml 1.2 parser yesterday, I began to put it to use. Since I've been steadily converting all my JSON parsers to use jsont codecs, it would be convenient if a single JSONt codec definition could also convert that schema to Yaml. In theory, Yaml is a superset of JSON, except it isn't actually. But it's close enough that we should be able to build a yamlt library that can accept a jsont codec and spit out Yaml (or the reverse).
\n ![\"%c\"](\"/images/yamlt-tangled.webp\")
I used all the previous tricks learnt so far, with all the dependent libraries available in the source tree for the agent to browse. The source code to jsont was absolutely essential, since the agent needed to learn how to traverse the jsont codecs to build a different (yaml) translator.
\nOne thing that was essential was guidance on how to do certain conversions where the results are ambiguous. For example, null handling in Yaml is much more permissive than in JSON, so I opted for any dictionary or array types to resolve nulls in the Yaml to the empty dictionary. It would also be ok to just raise an error in that case, but it seems safer to try to recover from what a human might do.
\nAs before, cram tests are a very effective way of exercising codepaths. I got the agent to generate a comprehensive suite of cram tests that do things like:
\nEncode arrays to JSON and YAML formats\n\n $ test_arrays encode\n JSON: {"numbers":[1,2,3,4,5],"strings":["hello","world"]}\n YAML Block:\n numbers:\n - 1\n - 2\n - 3\n - 4\n - 5\n strings:\n - hello\n - world\n YAML Flow: {numbers: [1, 2, 3, 4, 5], strings: [hello, world]}\nAnd also negative results:
\nAttempting to decode an object file with an array codec should fail\n\n $ test_arrays int ../data/objects/simple.yml\n JSON: int_array: ERROR: Missing member values in Numbers object\n File "-", line 1, characters 0-28:\n YAML: int_array: ERROR: Missing member values in Numbers object\n File "-":\nWhich is a convenient way of checking that location tracking is working.
\nThis all worked fairly straightforwardly. The Yamlt interface even includes support for multidocument Yaml files:
\nval decode_all :\n ?layout:bool ->\n ?locs:bool ->\n ?file:Jsont.Textloc.fpath ->\n ?max_depth:int ->\n ?max_nodes:int ->\n 'a Jsont.t ->\n Bytes.Reader.t ->\n ('a, string) result Seq.t\n(** [decode_all t r] decodes all documents from a multi-document YAML stream.\n Returns a sequence where each element is a result of decoding one document.\n Parameters are as in {!val-decode}. Use this for YAML streams containing\n multiple documents separated by [---]. *)\nThere's a lot going on this function, but it's easy to use. The optional parameters control details like Yaml layout and location tracking and defence against billion laughs attacks, but ultimately accept the jsont codec and a Yaml bytesrw source and give the result back as a lazy Seq.t.
\nUsing this library is very easy. The example code illustrates this:
\nmodule Config = struct\n type t = { name : string; port : int }\n\n let make name port = { name; port }\n\n let jsont =\n Jsont.Object.map ~kind:"Config" make\n |> Jsont.Object.mem "name" Jsont.string ~enc:(fun c -> c.name)\n |> Jsont.Object.mem "port" Jsont.int ~enc:(fun c -> c.port)\n |> Jsont.Object.finish\nend\n\n(* Use the same codec for both JSON and YAML *)\nlet from_json = Jsont_bytesrw.decode_string Config.jsont json_str\nlet from_yaml = Yamlt.decode_string Config.jsont yaml_str\nI've started using yamlt in my website code without issue. It's very convenient\nbeing able to describe a format as a jsont codec (which take care of conversion\nto OCaml types and a concrete wire format), but then also exposing this as Yaml\nfor easier human editing. The line number tracking means that it's much easier\nfor me to trace back errors, thanks to jsont being so careful about this in its\nimplementation and interface.
\nI also notice that Patrick Ferris has been hacking on a BibTeX codec which is based on the same principles as jsont and also uses Bytesrw. I hope to also integrate this into my website code soon too! But first, in Day 8 we will build a complete CLI application that uses some of these libraries we've built so far.
", "url": "https://anil.recoil.org/notes/aoah-2025-7", "title": "AoAH Day 7: Converting between JSON and Yaml with yamlt", "summary": "Building yamlt to enable jsont codec definitions to work with both JSON and Yaml, providing data manipulation with location tracking and good error messages for both formats.", "date_published": "2025-12-07T00:00:00.000000Z", "date_modified": "2025-12-07T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-6", "content_html": "I did the palate cleanser of Bytesrw-eio yesterday for a good reason. Back in 2017, I wrote the OCaml Yaml bindings that a lot of projects use in the OCaml ecosystem, and I'm having trouble maintaining it.
\nSince Yaml is an monstrously convoluted spec, I opted back then to bind to the C libyaml using ocaml-ctypes. This was a good decision a decade ago, but maintaining this has been a nightmare due to the complexity of vendoring the C library, dealing with security issues there, and exposing a reasonable OCaml interface. The ocaml-yaml implementation also doesn't pass the full Yaml test suite.
\nAnd the worst thing is, I cannot find the motivation to figure out how Yaml really works. It's the world's worst serialisation format, with lots of corner cases and memory blowups inherent in how it works. So I decided to dive in and see if I could build a pure OCaml Yaml 1.2 implementation using bytesrw and the source spec.
\nTL;DR: it worked. It actually seems to have come up with a reasonable, pure OCaml implementation that I'm now using! It needs more validation and external code review, but this has been on my TODO list for years now.
\nAs with previous projects, I carefully set up the source directory using all the previous libraries to act as style guides, along with the source code to the key dependency of bytesrw and the associated bytesrw-eio libraries.\nSince the yaml spec is so complex, I also added in the source code to my original ocaml-yaml library, which includes the vendored version of the C libyaml.
\nIn a slight twist, I also instructed the agent to look at the Git history for ocaml-yaml, since there have been a decade of bug reports about bad ways of interpreting yaml reported, including one from Martin Jambon that I haven't gotten around to looking at yet for the main library. There have also been frequent requests for a pure OCaml version to make cross compilation to iOS easier, as well as compilation on OpenBSD (failing due to the vendoring of the C library), and of course C memory leaks and spec violations. All of these would disappear with a spec-compliant implementation, so I fed the agent these to use as regression examples from user bug reports.
\n ![\"%c\"](\"/images/aoah-yamlrw-tests.webp\" "\\"It's")
The key to the development loop succeeding was adding integration to an external test suite. Yaml has the yaml-test-suite which has thousands of little examples of good and bad specs, along with the expected outputs in both JSON and Yaml. For example, test 36F6 takes this input yaml:
\nplain: a\n b\n\n c\nand expects this error "Multiline plain scalar with empty line" and the following parser events in a custom DSL exposed by the test suite:
\n+MAP\n=VAL :plain\n=VAL :a b\\nc\n-MAP\n-DOC\n-STR\nSo I instructed the agent to also build up a test suite DSL that could output in a format compatible with the test suite. A simple custom loader and JSON converter then output in the exact format required by the checked in test suite files.
\nI also added in some of the pathological tests from my original OCaml yaml, including an implementation of the Yaml bomb:
\na: &a ["lol","lol","lol","lol","lol","lol","lol","lol","lol"]\nb: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]\nc: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]\nd: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c]\ne: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d]\nf: &f [*e,*e,*e,*e,*e,*e,*e,*e,*e]\ng: &g [*f,*f,*f,*f,*f,*f,*f,*f,*f]\nh: &h [*g,*g,*g,*g,*g,*g,*g,*g,*g]\ni: &i [*h,*h,*h,*h,*h,*h,*h,*h,*h]\nThis simple Yaml file exponentially allocates lols into a billion\nlaughs, so I prompted the\nagent to also add in depth tracking to terminate parsing after a configurable\nnumber of nodes or depths have been crossed.
\nWith the test structure setup, it was plain sailing. I prompted the agent to maintain a strict separation between a pure Yaml parser (with a single dependency on bytesrw), and then have Unix and Eio converters using the Bytesrw unix one, or the bytesrw-eio one I coded up yesterday.
\nAs a useful aid to debugging, I also prompted the test suite to output nice HTML, which you can browse here. It's convenient to have a rendered version of the entire test suite!
\nI also coded up a quick core-bench\nlibrary to differentially test both the original Yaml library and this new one\non the full yaml test suite, and the pure OCaml one seems around 20% faster.\nI'm going to do a bit more memory benchmarking in addition to performance\nbefore I'm confident in these results, but the vibe coded smoke test was\nreassuring to see that I wasn't dramatically slower. Memory usage remains a\nrisk of being high, though; something to look at for a future day.
\nThis was the first day of the advent adventure where I really felt like I'd hit\na breakthrough! Yamlrw is a dropin replacement for all of my uses of\nocaml-yaml now, and I can't find any regressions. After a bit more code review,\nI'm going to post on the OCaml forums to request existing users to test this\none and see if they can find any regressions.
\nIt's also nice having a streaming Eio Yaml parser, which will be convenient\nfor some projects in the remaining days, like my contacts manager. But first,\nin Day 7 we'll build a nice yamlt codec library...
After the Claude exertions of yesterday, I needed something easier to cool my laptop down. I wanted to learn how to use another new library from Daniel Bünzli called Bytesrw, which provides composable byte stream readers and writers. It supplies ways to serialise Bytesrw to Unix file descriptors, so I figured I'd add in an Eio library for this. Along the way though, I was generating a growing number of opam packages, so I also learnt how to use Claude Skills to automate my opam metadata on Tangled as well.
\nThe Bytesrw-eio library is exceedingly simple and exposes just two functions, so it was pretty easy for the agent to code up given the source repositories already had Unix equivalents.
\nval bytes_reader_of_flow :\n ?slice_length:int -> _ Eio.Flow.source->\n Bytesrw.Bytes.Reader.t\n\nval bytes_writer_of_flow :\n ?slice_length:int -> _ Eio.Flow.sink ->\n Bytesrw.Bytes.Writer.t\nWhile coding this simple library, I realised my bottleneck lay in managing the\ngrowing number of opam packages that I have lying around! Can I also use agents\nto manage these libraries?
\n
I'm maintaining all the libraries at a custom aoah-opam-repository overlay on Tangled. Each individual repository I publish also uses a Spindle CI action to build that library on every push. This overlay later became the input to monopam and unpac for assembling monorepos.
\nFor every package I publish, I need to:
\nThe problem with scripting these up is that there's subtle parameterisation which changes slightly across each library. Some might have an extra external dependency, another might have a custom build step, and others might just have tests that require specific actions. This seems ideal for a coding agent that can take a template of actions and then propose metadata changes automatically.
\nThe idea behind Claude skills is incredibly simple: you just create a ~/.claude/skills/my-skill that contains a SKILL.md and the associated scripts that need to run. Nothing else; no state, no MCP server, just a simple file! The header looks as follows:
—\nname: ocaml-metadata\ndescription: Standards for OCaml project metadata files. Use when initialising a new OCaml library/module, preparing for opam release, setting up testing infrastructure, or searching the OCaml ecosystem for dependencies. Not for normal code edits.\nlicense: ISC\n—\n\n# OCaml Project Metadata Standards\n\n## When to Use This Skill\n\nInvoke this skill when:\n\n1. **Initializing a new OCaml project** - Setting up dune-project, LICENSE, README, CI, etc.\n2. **Preparing for opam release** - Ensuring all metadata is correct for publication\n3. **Setting up testing infrastructure** - Especially for Eio-based libraries that need mock testing\n4. **Searching the OCaml ecosystem** - Finding and fetching dependency sources for reference\n5. **Adding third-party source references** - Using `opam source` to study library implementations\n\n**Do not use for:**\n- Regular code edits or bug fixes\n- Simple function additions\n- Refactoring existing code\nThe Yaml frontmatter is the only thing loaded by Claude at startup (thus saving context space). Then it uses that information to load the rest when needed on demand. For example, for the claudeio repository it spins it up on demand.
\n![\"%c\"](\"/images/aoah-ss-4.webp\" "\\"The")
If you browse the full skill repository you will also find even more detailed (and personalised) instructions about how to structure tests and retrieve sources to my laptop. My workflow is that if Claude gets something wrong workflow-wise, I now prompt it to also fix the skill after its done. The agent then revises the workflow information in this repository and (hopefully) doesn't make the same mistake twice.
\n![\"%c\"](\"/images/aoah-ss-3.webp\" "\\"The")
Getting back to the problem at hand, testing the bytesrw-eio adapter was a simple matter of prompting the agent to\ncome up with some read and write tests,\nand then I deliberately introduced an error to sanity check they were failing\nwhen expected and then refined the tests to include those failures as well.
\nOne frustration here I had is a long-standing problem in OCaml where we don't\nhave a single agreed IO representation. Eio uses\nCstruct which is backed by Bigarray\n(which lives out-of-heap), and then others use bytes/string (which live\nin-heap). The reason is usually because out-of-heap values can be non-moving\n(allowing for zero-copy IO), but in return it results in GC pacing issues that are hard to fix. This is relevant to\nthis Bytesrw-eio library because we introduce a copy between Bigarray (from\nEio) to Bytesrw (which obviously uses bytes). At some point in the future, I\nreckon we need to get a non-moving bytes mechanism into OCaml 5.x so that we\ncan all just use one IO type and avoid these dratted copies.
The bytesrw-eio package itself was simple enough, and it added to my long-term\ntodo list to revisit OCaml IO abstractions again in the\nfuture. The big breakthrough today though, was figuring out Claude skills. I'm\nnot using MCP at all any more, and will be creating more domain-specific skills\nfor OCaml actions as I proceed through the month.
\nNext, onto Day 6 where we'll build a pure Yaml codec using this library!
By this point, I've got three useful libraries and my use of Claude is getting better. So naturally I want to automate my invocations of the claude CLI, but I hit a roadblock: there are no OCaml SDK bindings! However, there appear to be SDKs in Python, Go and many others. So today will involve having a stab at generating Claude OCaml bindings using Eio, so I can use Claude to write more OCaml!
I prodded around the Python and noted that the communications protocol between the SDK and the CLI is JSON-RPC. I'd noticed when hacking with jsont that it includes a json-rpc codec, so adopting the same approach as I did with ocaml-jsonfeed seems reasonable: code up the core protocol using JSONt codecs, and then handle serialisation and process coordination using Eio.
\nFor context to the agent, I gave it the Python and Go Claude SDKs to digest what the actual Claude protocol involves, and then all my previous OCaml libraries and the sources to jsont and Eio (i.e. the lessons learnt from the previous couple of days with xdge and jsonfeed).
\nOne important prompt was to instruct it to first generate a claude.proto subpackage that only has jsont codecs and OCaml types, and then to use that package in the coordination layer with Eio. This avoids mixing up concerns in one giant module, as an unprompted Claude is prone to do.
Using jsont at the codec layer made all the difference, since I could get the model to debug the wire-level messages independently of the transport layer. In fact, I left the agent running in a loop where it looked at the error outputs from its own regression tests (against a live Claude instance) and then proceeded to fix them. This was only possible because of the excellent error instructions from jsont. For example, with the structured output test I got:
\nstructured_output_demo.exe: [ERROR] Failed to decode incoming message: Missing member tool_name in Rule object\nFile "-", line 1, characters 451-515:\nFile "-", line 1, characters 451-515: at index 0 of\nFile "-", line 1, characters 450-515: array<Rule object>\nFile "-": in member rules of\nFile "-", line 1, characters 423-515: Update object\nFile "-", line 1, characters 423-515: at index 0 of\nFile "-", line 1, characters 422-515: array<Update object>\nFile "-": in member permission_suggestions of\nFile "-", line 1, characters 88-515: Permission object\nFile "-": in member request of\nFile "-", line 1, characters 0-515: ControlRequest object\nLine: {"type":"control_request","request_id":"055cb59c-2f9f-457d-8c98-0a2c5a48c577","request":{"subtype":"can_use_tool","tool_name":"Bash","input":{"command":"find /Users/avsm/src/git/knot -type f -name \\"*.ml\\" -o -name \\"*.mli\\" -o -name \\"*.md\\" -o -name \\"*.html\\" -o -name \\"*.go\\" -o -name \\"dune\\" -o -name \\"dune-project\\" | head -100","description":"List all relevant files in the repository"},"permission_suggestions":[{"type":"addRules","rules":[{"toolName":"Read","ruleContent":"//Users/avsm/src/git/knot/**"}],"behavior":"allow","destination":"session"}],"tool_use_id":"toolu_011w6XYAbALBytxMaLxtnBGd","agent_id":"05bf9384-6c4b-4edd-898f-962d945ff724"}}\nThis was enough information for Claude to pick up the problem and address it in the codec:
\nClaude: I found the issue! The Rule decoder in proto/permissions.ml is expecting\nsnake_case field names (tool_name, rule_content) but the Claude CLI is sending\ncamelCase field names (toolName, ruleContent). The rest of the permission\nsystem already uses camelCase consistently.\nI did find a lot of breakage when using different versions of the upstream\nClaude SDKs. For example, permission handling is just...broken... in some\nversions, but they seem to quite quickly push changes. I suspect they might be\nusing a bit too much bleeding edge Claude in developing Claude!
\n![\"%c\"](\"/images/claude-ss-perm-1.webp\" "\\"I")
However, their breakage did exercise my agent quite nicely into switching\nbetween the Python and Go SDKs to come up with a good answer, and also\nhighlighted why agentic coding is so different from one-shot coding LLMs. It's\npretty crazy seeing an agent dynamically introspect itself to come up with the\narchitecture I specified, against a live service!
\nIt's now quite convenient to have a Claude OCaml wrapper, but I stopped short of making a really nice Eio interface as the upstream project is moving so quickly.
\nI'd like to eventually use this as a basis for a distributed Claude to unify my local and remote Docker development, and also integrate with our local initiatives like Ryan Gibb and his work on package management and Patrick Ferris's cool new Shelter. For now, I'm holding the fort on just doing simple OCaml invocations of Claude and not trying anything too exotic until the CLI itself settles down and stabilises.
\nOnto Day 5 next, where we use Claude skills for the first time!
As part of the ARIA Engineering Ecosystem Resilience\nprogram, we've been convening a series of workshops here at the Cambridge\nConservation Initiative to explore the\npotential of combining two very radically different approaches to modeling. Joe Millard wrote this to frame the discussion:
\n\n\nEcology and ecosystems are inherently agent-based. In other words, patterns\nin biodiversity in both space and time emerge as a function of the local\ninteraction of many types of individual organisms, both with each other and\nwith their abiotic environment.
\nGenerative agent-based models, such as Concordia enable the simulation of\nmultiple interacting large language models. Given LLMs now possess\nsignificant ecological knowledge,\nit is possible that models such as Concordia will enable the meaningful simulation of ecological interactions.
\nThe biotic and abiotic environment in which ecological agents interact in a given\necosystem is likely measurable via remotely monitored earth-observation data. Raw EO\ndata, however, is unwieldy, containing large quantities of information that can be\ndifficult to interpret. Earth-system models, such as TESSERA or AlphaEarth\nare foundational AI models which compress large quantities of EO data into "embeddings", unambiguous\nand consistent digital representations of the structure of the Earth’s surface.\n-- Foundational AI to forecast ecosystem resilience, J. Millard, A. Pili, K. Berthon, R. Fletcher, L. Dicks
\n
We held two separate workshops to explore this; one for a\ndeep-dive into the technical details, and another to invite conservation\npractitioners to drive our modeling direction in a realistic and positive\ndirection. This was all lead by Lynn Dicks and the stellar organisation of\nJoe Millard, Katherine Berthon, Arman Pili and Rob Fletcher, with input from me, Srinivasan Keshav and David Coomes.\nI'll go into each talk next, or you can watch the playlist yourself.
\nFor the technical day, we held it in the lovely Ferguson Nazareth[1] room at Pembroke College, contrasting ancient rows of books with group brainstorms about machine learning for improbably complex simulations!
\nWe began with Srinivasan Keshav and David Coomes going through TESSERA in some detail, with a particular focus on its use for ecological downstream tasks. Julia P.G. Jones observed that this was the clearest explanation of the utility of TESSERA to a non-machine-learning person that she had heard, and I agree! Do give this a watch if you'd like a gentle intro to geospatial foundation models.
\n\nThen we switched tacks to something entirely different, with Sasha Vezhnevets and Logan Cross from DeepMind telling us about their approach to generative social simulation:
\n\n\nConcordia is a library to facilitate construction and use of generative agent-based models to simulate interactions of agents in grounded physical, social, or digital space. It makes it easy and flexible to define environments using an interaction pattern borrowed from tabletop role-playing games in which a special agent called the Game Master (GM) is responsible for simulating the environment where player agents interact (like a narrator in an interactive story).\n-- Concordia on GitHub, 2025
\n
What was fascinating about this is how LLMs just change the game for defining simulations. Instead of having to specify every last thing, the LLMs understand a certain amount of human culture and context by virtue of pre-training, and so can take more nuanced decisions than a discrete simulator. So the "defaults" are intriguingly much more useful for ecological simulations, which are deeply connected and social in nature and extremely difficult to model computationally via brute force methods.
\n\nThe Concordia source code is all open, so I've started messing around with some local LLM simulations. It's really as easy as specifying some prompts and parallelisation; quite different from conventional agent-based modeling! There was a v2 release this summer and a video tutorial as well.
\nThen Julia P.G. Jones, the host of one of my favourite podcasts "Tuesdays with Team Counterfactual" gave us a dive into why causal modeling is so important for real nature simulations. I've been steadily learning from Julia and Thomas Swinfield over the past five years in 4C about the importance of causal inference to try and find those pesky hidden confounders over lots of observational data.
\n\nJulia highlighted just how much more complex the observational and causal variables are in biodiversity vs something like climate, and observed that climate modeling is not analogous to modeling biodiversity and ecosystem services. This is because biodiversiy is highly multi-scale, and local changes will have an impact nearby (unlike the global climate). But it's also possible to construct (quasi-)experimental and natural experiments to test various aspects, which is difficult to do with climate. The discussion of after her talk went on for quite some time, so the second half of the video is also useful I hope.
\nRebecca Spake then connected the dots with ecology and the industry that has driven billions of dollars of investment into machine learning: advertising. This began via her amazing pet turkey (watch the talk!) and then dove deep into how to use causal information to gain predictive power over the outcomes of interventions. It's worth noting that almost all natural experiments in ecology seem to be centered around counterfactuals due to the difficulty of exposing any single organism outside of a habitat, so this puts a lot of importance on robust causal modeling.
\n\nBecks went into detail about how conventional individual treatment effects are difficult to test with real-world data, and so a "virtual ecologist" simulation approach is needed to simulate ecosystem responses under both treatment and control. If we could use some of the techniques discussed above like TESSERA and Concordia to make these simulations as high fidelity as possible, then we can dramatically improve the evidence available to motivate expensive on-the-ground interventions to protect some aspect of biodiversity.
\nThere were fewer talks in the second practitioners workshop, but no discussion of AI and nature would be complete without Sam Reynolds giving an entertaining[2] talk covering our recent horizon scan and also our work on large scale literature analysis and using conservation evidence to design policymaking frameworks that can rapidly replicate positive interventions usefully around the world.
\n\nAn interesting term I hadn't heard before is human bycatch resulting from using automated nature monitoring. Issues of privacy and human rights (often for indigenous populations who might not have agency over such monitoring technology) were discussed at length.
\nThe workshop was also attended by the ARIA programme manager Yannick Wurm, who took the time to explain ARIA's structure and how his program is evolving rapidly as they get feedback from workshops like ours. I am finding interacting with ARIA a breath of fresh air: the workshops I've been to have been well organised and useful beyond just looking for funding by sparking new thoughts. I was initially not going to be taking part in this program at all due to concerns about licensing and IP ownership, but a quick post on social media was picked up and my concern was addressed within days. ARIA has an exceptionally free IPR policy that makes it so much easier for us to maintain our open-source code while being funded from a variety of sources, and I'm appreciative of this.
\nOn the other hand, it also means that we need to keep up with all the changes happening to their opportunity spaces in order to keep abreast of all the changes in thinking that are resulting! I am hosting one last workshop for this opportunity space on the 12th December with Oisin Mac Aodha and Mike Harfoot to go into range mapping. Both workhops will have a proper writeup about our actual discussions and recommendations (which I haven't covered here!), so stay tuned for those ahead of the Christmas break.
\n![\"%c\"](\"/images/aria-er-4.webp\" "\\"Yannick")
![\"%c\"](\"/images/aria-er-1.webp\" "\\"Shane")
![\"%c\"](\"/images/aria-er-3.webp\" "\\"The")
Random fun facts from the workshop:
\nBy Day 3 of the Advent of Agentic Humps, I now have the\nconfidence to build a slightly more complex library that uses\nEio to implement the XDG Base\nDirectory Specification with a twist: let's use Eio capabilities to sandbox XDG paths by default.
\n![\"%c\"](\"/images/cross-xdg.webp\" "\\"Lots")
The XDG Spec is\ncomprehensive, but full of lots of informal rules about directories. Some of\nthe rules are pretty easy to follow:
\n\n\n\n
$XDG_DATA_DIRSdefines the preference-ordered set of base directories to\nsearch for data files in addition to the$XDG_DATA_HOMEbase directory. The\ndirectories in$XDG_DATA_DIRSshould be separated with the separator used\nfor$PATHon the platform (typically this is a colon:).
While others are harder to enforce in code:
\n\n\nThe directory MUST be on a local file system and not shared with any other\nsystem. The directory MUST be fully-featured by the standards of the\noperating system. More specifically, on Unix-like operating systems
\nAF_UNIX\nsockets, symbolic links, hard links, proper permissions, file locking, sparse\nfiles, memory mapping, file change notifications, a reliable hard link count\nmust be supported, and no restrictions on the file name character set should\nbe imposed. Files in this directory MAY be subjected to periodic clean-up. To\nensure that your files are not removed, they should have their access time\ntimestamp modified at least once every 6 hours of monotonic time or the\n'sticky' bit should be set on the file.
We can do bits of that, but POSIX doesn't really expose everything we need to\nmechanically verify some aspects of filesystem support. Still, we have a big\nlong spec, so let's see what happens if we throw an agent at it!
\nMy general approach with Claude was to download a copy of the XDG spec, instruct\nthe agent to digest it. But then, I also supplied the previous\ntwo libraries as examples of "my OCaml style" that it could draw\nfrom. Having more code available to act as an oracle to guide the model towards\nsomething I find acceptable is a useful way to avoid lots of prompting.
\nThe first attempts with using completely crashed and burnt as the agent failed\nto grok the admittedly very complicated Eio types for capabilities (which do involve\nsubtyping and phantom polymorphic variants). I then cloned the Eio source code\nand specifically instructed the agent to read the Eio README,\nas it has extensive information about best practises to use the library. This is\nsimilar to my earlier trick with jsont to instruct it to read\nthe cookbooks.
\nThings got much better after this; the agent had to iterate quite a few times,\nbut did eventually converge on the right types.
\nval config_dir : t -> Eio.Fs.dir_ty Eio.Path.t\n(** [config_dir t] returns the path to user-specific configuration files.\n\n {b Purpose:} Store user preferences, settings, and configuration files.\n Configuration files should be human-readable when possible.\n\n {b Environment Variables:}\n - [${APP_NAME}_CONFIG_DIR]: Application-specific override (highest priority)\n - [XDG_CONFIG_HOME]: XDG standard variable\n - Default: [$HOME/.config/{app_name}]\n\n @see <https://specifications.freedesktop.org/basedir-spec/latest/#variables>\n XDG_CONFIG_HOME specification *)\nYou can see one nice feature here that would have taken a while to code by\nhand. The type t for the XDGe library is typically constructed by exposing Cmdliner terms to allow\nother applications to "plug in" XDG support by just specifying a single term.
This term takes care of adding of the ordering of environment variable, command-line arguments,\nand default values in the right order. The manual page for an example binary shows how this works from the CLI.
\n![\"%c\"](\"/images/xdge-man-page.webp\" "\\"In")
You can see the Cmdliner support in action with the test cases, where I adopted\nthe same cram-based testing strategy as earlier with jsont.
\nThis allows for a nice repository structure where I can simply add in the XDG\nCmdliner term to the test case binaries, and have all the gory details of config setup\nhandled by the xdge library. For example, in the cram tests you can\nsee how the source of a XDG path is tracked (i.e. did it come from the CLI, from an environment variable or the defaults?):
\n $ export HOME=./test_home\n $ unset XDG_CONFIG_DIRS XDG_DATA_DIRS\n $ XDG_CONFIG_HOME=/tmp/xdge/xdg-config \\\n > XDG_EXAMPLE_CONFIG_DIR=./app-config \\\n > ./xdg_example.exe --config-dir ./cli-config\n === Cmdliner Config ===\n XDG config:\n config_dir: ./cli-config [cmdline]\n \n === XDG Directories ===\n XDG directories for 'xdg_example':\n User directories:\n config: <fs:./test_home/./cli-config> [cmdline]\n data: <fs:./test_home/./test_home/.local/share/xdg_example> [default]\n cache: <fs:./test_home/./test_home/.cache/xdg_example> [default]\n state: <fs:./test_home/./test_home/.local/state/xdg_example> [default]\n runtime: <none> [default]\n System directories:\n config_dirs: [<fs:/etc/xdg/xdg_example>]\n data_dirs: [<fs:/usr/local/share/xdg_example>; <fs:/usr/share/xdg_example>]\n\nCommand-line argument overrides both types of environment variables. Even\nthough both XDG_CONFIG_HOME and XDG_EXAMPLE_CONFIG_DIR are set, the\n--config-dir flag takes precedence and shows [cmdline] source. Other\ndirectories fall back to defaults since no other command-line args are\nprovided.\nI ran the cram tests quite a few times and read through them to make sure\nthe shell sessions and explanations made sense, and then read through\nthe xdge source code itself which was pretty simple. There are some\nfeatures which Eio doesn't expose functionality for yet that are OS-specific\n(like checking the mount type), but they can come in a future iteration.
\nCmdliner is one of the gems in the open-source OCaml community due to how\neasy it makes it to build "Unix-style" applications with sensible manual\npages and consistent argument parsing. However, even after using it for\nyears I can never remember its term language without referring to the manual,\nand I always find myself cut-and-pasting from previous code and editing it.
\nUsing the agent definitely helped me out here. A lot of the XDG logic\nis fairly boilerplate, but extremely useful to express in a typed way.\nI anticipate now using this library in almost every CLI tool I build\nin OCaml, as it has enough information exposed in the interface to let\ndownstream CLI-coding agents pick the right base directories to use by\ndefault.
\nOnto Day 4 then, where we'll go recursive by wrapping Claude in OCaml using Claude!
", "url": "https://anil.recoil.org/notes/aoah-2025-3", "title": "AoAH Day 3: XDG filesystem paths using Eio capabilities", "summary": "Building an XDG Base Directory Specification library with Eio capabilities and Cmdliner integration, providing sandboxed filesystem access patterns with full environment variable and CLI override support.", "date_published": "2025-12-03T00:00:00.000000Z", "date_modified": "2025-12-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "aoah", "ocaml", "agents", "llms", "ai" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/aoah-2025-2", "content_html": "Day 2 of the Advent of Agentic Humps dawns with building a slightly more complex library than before, via the JSONFeed specification that is a more modern version of Atom.
\nJSONfeed is a successor to Atom for website feeds, that has a nice informal specification about how to parse it. However, it also has a growing number of extensions which also need to be implemented somehow, as well as some informal rules to map RSS/Atom to JSONFeed.
\nThere is no existing OCaml implementation that I could find, and I need it to integrate my website with Rogue Scholar more easily for permanent DOIs.
\n ![\"%c\"](\"/images/jsonfeed.webp\")
Unlike the Crockford implementation, parsing JSON involves selecting a third-party library dependency cone. By default the agent chose Yojson (presumably because its the most popular in its training set). I would conventionally use my own ezjsonm library that builds over the lower level jsonm, but I noticed a deprecation notice on jsonm towards a newer library by Daniel Bünzli called jsont.
\nAfter seeing the announcement of jsont about a year ago, I gave it a quick try:
\n\n\nJsont is an OCaml library for declarative JSON data manipulation. It provides:
\n\n
\n- Combinators for describing JSON data using the OCaml values of your choice. The descriptions can be used by generic functions to decode, encode, query and update JSON data without having to construct a generic JSON representation.
\n- A JSON codec with optional text location tracking and layout preservation. The codec is compatible with effect-based concurrency.
\nThe descriptions are independent from the codec and can be used by third-party processors or codecs.\n-- jsont, Daniel Bünzli, 2025
\n
The codec for a JSON type is expressed using combinators, and then separately serialised or deserialised. I found that it had fantastic error messages since it could use the codec to come up with the reason why some input has been rejected. But on the flipside, writing the codecs involved a lot of boilerplate, so I found it quite time consuming.
\n
But now, with Claude, I could use it to scan a spec and automate codec construction using jsont! So I decided to try a complex coding case where I fed the prose JSONFeed spec to Claude, and instructed it to build jsont codecs. As a separate phase, I then built test cases and serialisers.
\nThe very first run of Claude crashed and burned with jsont as it didn't have enough examples to figure out the interface from scratch, resulting in a lot of type errors and no working code.
\nSo I took a different tack: I ran opam source jsont to get the source code into the current working directory, and then prompted a fresh Claude session to "ultrathink about the interface of jsont.0.2.0, paying particular attention to the cookbook". The cookbook is a section of the (excellent) documentation in Jsont that describes real-world usage, and the agent also picked up on the jsonrpc testcase in the jsont repository.
Once it had this bit of example-driven context, the agent proceeded to build a very credible set of codecs:
\nThe user exposed API is idiomatic OCaml:
\ntype t\nval jsont : t Jsont.t\nval create :\n title:string -> ?home_page_url:string ->\n ?feed_url:string -> ?description:string ->\n ?user_comment:string -> ?next_url:string ->\n ?icon:string -> ?favicon:string ->\n ?authors:Author.t list -> ?language:string ->\n ?expired:bool -> ?hubs:Hub.t list ->\n items:Item.t list -> ?unknown:Unknown.t ->\n unit -> t\nThe full set of constructors and validators can be read in jsonfeed.mli.
\nFor test cases, I first got Claude to synthesise a variety of JSONFeed corpuses that exercised the spec, and manually inspected it to check it all seemed ok. I then made use of Dune Cram tests to write CLI-based validators that run.
\nAll of the boilerplate around writing the test cases just worked out of the box, with a little bit of manual prompting from me required to guide the agent towards some known edge cases (like extensions handling).
\nHere's an excerpt from the cram tests:
\nMissing Required Fields\n------------------------\n\nTest missing title field:\n $ ./test_location_errors.exe data/missing_title.json title\n {"status":"error","message":"Missing member title in JSON Feed object",\n "location":{"file":"data/missing_title.json","line":1,"column":1,\n "byte_start":0,"byte_end":65},"context":"$"}\n [1]\n\nTest missing version field:\n $ ./test_location_errors.exe data/missing_version.json title\n {"status":"error","message":"Missing member version in JSON Feed object",\n "location":{"file":"data/missing_version.json","line":1,"column":1,\n "byte_start":0,"byte_end":51},"context":"$"}\n [1]\nThe code is available on anil.recoil.org/ocaml-jsonfeed with online docs. My first user for this might be Michael Dales in his own website.
\nThe use of Claude has made using jsont the default choice for me now. As a first phase, I can ask the LLM to synthesise down a spec into a combinator-based codec, and then validate that separately. Crucially, the good error messages from jsont help the agent to root-cause why some tests fail, and give me the option of either clarifying the spec or fixing the test case if that was the error.
\nI did, however, still have to make the judgement call of which libraries to use at the start. The agent also happily spat out Yojson and Ezjsonm based implementations for me, but I simply prefer the jsont approach. If you had other priorities like pure performance, you might go for Yojson instead.
\nOnto Day 3, where we'll then build our first Eio based library!
Our neighbours France and the UK announced a Franco-British AI collaboration a few months ago dubbed the Entente CordIAle. Last week we held a couple of days of workshops with our Oxford and French buddies deep diving into details of what a partnership might actually involve; a particular pleasure with France given my group's long history of working with Inria on OCaml and other open source projects.
\nI sprinted back from Birmingham to speak about our research on connecting the dots on terrestrial life via TESSERA, LIFE, food and the scholarly literature. The other talks were vastly ambitious, best summarised in the Scriberia visual:
\n![\"%c\"](\"/images/aicam-french-6.webp\" "\\"The")
There's a great blog post from AI@CAM (who hosted the workshops on our Cambridge end) on how AI is changing our practise of science:
\n\n\nOn 20 November, as part of an emerging UK-France collaboration in AI, ai@cam brought together researchers working at the intersection of AI and science. The conversation moved across different problems: how do you forecast weather without traditional numerical weather prediction? How do you model how a drug affects tissue structure? How do you predict molecular structures from mass spectrometry data? How do you quantify the extinction impact of agricultural land use? How do you extract reliable insights from the neuroscience literature? How do you diagnose dementia earlier? These aren't separate questions. They're connected by something deeper: a rethinking of how the scientific pipeline works, from data to actionable insight.\n-- How AI is Changing the Practice of Science, AI@CAM, Nov 2025
\n
![\"%rc\"](\"/images/aicam-french-2.webp\")
One of my favourite talks was related to open-source; I hadn't realised that the scikit-learn project that's used widely in machine learning actually began as a Google Summer of Code project and then incubated by Inria Saclay subsequently. Gaël Varoquaux gave a fantastic overview of his interest in machine learning for tabular data, which struck me as a similar argument to Shriram Krishnamurthi making the case for data-centricity in teaching.
\n![\"%rc\"](\"/images/aicam-french-3.webp\")
After this, we had a formal reception at the French Embassy in London hosted by Her Excellency Hélène Tréheux-Duchêne, Ambassador to the United Kingdom. It was as posh as you may expect, and Neil Lawrence represented Cambridge with a splendid speech about the importance of soverignity and openness in this age of frontier AI.
\nI also enjoyed meeting my counterparts from Oxford to learn more about their own work. In particular, Sandra Kiefer gave me some interesting reading on her recent work in complexity theory and algorithms, which I need to get up to speed on!
\n![\"%c\"](\"/images/aicam-french-7.webp\" "\\"I")
Subsequently, I'm also grateful to AI@CAM for giving our Conservation Evidence Copilots project followon funding to help develop the "Conservation Co-pilot: Making Environmental Evidence Accessible", lead by Sam Reynolds:
\n\n\nWith over 1 million users already accessing the Conservation Evidence database, this project will develop the first ‘Conservation Co-pilot’ – an AI-powered chat interface that retrieves, summarises, and presents conservation evidence to answer user questions. The challenge is ensuring that AI faithfully represents scientific evidence without misrepresentation. Building on rigorous evaluation research comparing frontier AI models with human experts, the team will create an agentic system that draws together evidence while maintaining faithfulness to source material. The tool will be game-changing for conservation decision-makers seeking evidence to guide action.\n-- Seven New AI-deas Projects Advance AI for Science, Citizens, and Society, 2025
\n
![\"%rc\"](\"/images/aicam-french-4.webp\")
Read more about this from the French contingent and the LinkedIn post from Alec Christie, who also evangelised our Conservation Evidence Copilots project extremely well across the event!. While I've enjoyed the recent diplomatic visits from India as well, I am also glad to be able to sit down and do some hacking this week when things are a bit quieter.
\nCarl Henrik Ek and I are also very grateful to Neil Lawrence for taking the time to sketch out the logic in his recent paper on The Inaccessible Game to our undergraduates at Pembroke after this event as well. It was a heady mix of thermodynamics, information theory and GENERIC in the best possible way!
\n![\"%c\"](\"/images/aicam-french-5.webp\" "\\"Back")
Let's start day 1 of the Advent of Agentic Humps with a gentle introduction to agentic coding. Firstly, I've chosen to exclusively use Claude Code for this since it's CLI driven. I tried some of the other Copilot and Cursor IDEs, but I just couldn't adjust to how busy the displays were.
\nWith Claude, my setup first involved a custom devcontainer using Docker on a Linux host, and my local Mac laptop. I coordinate both of these via Git repositories hosted up at Tangled with a self-hosted knot.
\nI setup a remote Docker container on a Linux host where I can leave the agent running unattended in a sandbox. I coded up a custom Claude devcontainer for this which takes care of firewalling the agent at the DNS level and installing relevant OCaml packages.
\nWith this setup, I can run claude --dangerously-skip-permissions and have it only access my repositories on Tangled and GitHub.
As a bit of fun, the wrapper slopper script to start the session passes the session shell output back to Claude afterwards, to give it an opportunity to rewrite the Dockerfile with additional libraries I installed. It's shaky, but it occasionally does something useful.
\n![\"%c\"](\"/images/aoah-ss-5.webp\" "\\"The")
On my Mac and Linux desktops, I just use Claude code directly with my usual vi-based OCaml coding setup. This is just a straightforward addition to my usual workflow with Claude running in a terminal.
\nThe only major change is that I assume that all code generated is slop by default, and is so until "promoted" via review and manual editing to the next level. I therefore stage all my work-in-progress slop code in a dedicate slop repository which I'll reset and delete regularly. This is a convenient way of doing coordination, nothing more.
\nProblem. I've integrated my blog with Rogue Scholar but the mechanism for doing so required generating Base32 encoded random DOIs for the Rogue Scholar DOI assigner to pick up from my blog feed. There is no existing Base32 library in OCaml, so let's make one.
\nApproach.\nI looked over at the spec, and also (with permission from Martin Fenner) at his reference Go implementation on GitHub. I then instructed Claude to synthesise a spec from the online spec and the Go implementation, and used Sonnet 4.5 to come up with a plan. I then instructed it to write the OCaml implementation as a single module with no dependencies other than the stdlib. I then separately had a session to take just the OCaml interface file and synthesise alcotest-based checks from the spec, to act as test cases.
\nResults. The crockford code is up with online docs. I assigned co-copyright to Front Matter due to cross-generating the code with their implemenation as a reference. Unsure if this is the right thing to do, but it seems better to be generous about copyright assignment in this case.
\nReflections. This is a pretty simple day 1 task, as it's a single module of OCaml and relatively easy to test. Still, I had to be careful to not just generate the tests in the same session as the code to ensure that the agent didn't just specialise the tests to the code it had written.
\nOnto Day 2 then, where we'll build a more substantive JSONfeed library!
I've been building some big collective knowledge systems recently, both for scholarly literature or to power large-scale observational foundation models. While the modalities of\nknowledge in these systems are very different, they share a common set of design principles I've noticed while building\nindividual pieces. A good computer architecture is one that can be re-used, and I've been mulling over what this exactly is for some time.
\nI found the perfect place to codify this at the ARIA Workshop on Collective Flourishing that Sadiq Jaffer and I attended in Birmingham last week. I posit there are "4 P's" needed for any collective knowledge system to be robust and accurate: permanence, provenance, permission and placement. If these properties exist throughout our knowledge graph, we can make robust networks for rapid evidence-based decision making. They also form a dam against the wave of agentic AI that is going to dominate the Internet next year in a big way.
\nWill building these collective knowledge systems be a transformative capability for human society? Hot on the heels of COP30 concluding indecisively, I've been getting excited by decision making towards biodiversity going down a more positive path in IPBES. We could empower decisionmakers at all scales (local, country, international) to be able to move five times faster on actions about global species extinctions, unsustainable wildlife trade and food security, while rapidly assimilating extraordinarily complex evidence chains. I'll talk about this more while explaining the principles...
\nThis post is split up into a few parts. First, let's introduce ARIA's convening role in this. Then I'll introduce the principles of permanence, provenance, permission, and placement. The post does assume some knowledge of Internet protocols; I'll write a more accessible one for a general audience at a future date when the ideas are more baked!
\nThe ARIA workshop was held in lovely Birmingham, hosted by programme manager Nicole Wheeler. It explored four core beliefs that ARIA had published about this opportunity space in collective flourishing:
\n\n\n\n
\n- Navigating towards a better future requires clarity on direction and path > we need the capability to make systemic complexity legible so we can envision and deliberate over radically different futures.
\n- Simply defining our intent for the future is not enough → we need a means of negotiating our fragmented values into shared, actionable plans for collective progress.
\n- Our current cognitive, emotional, and social characteristics are not immutable constants → human capacity can and will change over time, and we need tools to figure out together how we navigate this change.
\n- Capabilities that augment our vision, action, and capacity are powerful and can have unintended consequences → we must balance the pressing need for these tools with the immense responsibility they entail.\n-- ARIA Collective Flourishing Opportunity Space, 2025
\n
I agree with these values, and translating these into concrete\nsystems concepts seems a useful exercise. The workshop was under Chatham\nHouse rules, which\nhamstrings my ability to credit individuals, but the gathering was a\nuseful and eclectic mix of social scientists and technologists. There was also\na real sense of collective purpose: a desire to reignite UK growth and decrease inequality.
\n![\"%c\"](\"/images/aria-ci-1.webp\" "\\"While")
First, why come up with these system design principles at all? I believe strongly both in building systems from the groundup, and also in\neating my own dogfood\nand using whatever I build. I also define knowledge broadly: not just academic\npapers, but also geospatial datasets, blogs and other more conventionally\n"informal" knowledge sources that are increasingly complementing scholarly\npublishing as a source of timely knowledge.[1]
\nTowards this, several of my colleagues such as Jon Sterling have been building systems like Forester, and I've got my own homebrew Bushel, and Patrick Ferris has Graft, and Michael Dales has decades of Atom/RSS on his sites. These sites are very loosely coupled -- they're built by different people over many years -- but there is already the beginnings of a rich mesh of hyperlinks across them.
\nTo get to the next level of collective meshing (not just for these personal sites but for biodiversity data as well), I posit we need to explicitly engineer in support for permanence, provenance, permission and placement right into the way we access data across the Internet. If we build these mechanisms via Internet protocols, collective knowledge can be meshed without any one entity requiring central control. I view this as being vital for the Internet as a whole to evolve and adapt into the coming decades and combat enshittification.
\nI'll now dive into the four principles: (i) permanence; (ii) provenance; (iii) permission; and (iv) placement.
\nFirstly, knowledge that is spread around the world needs a way to be retrieved\nreliably. Scholarly publications, especially open-access ones, are distributed\nboth digitally and physically and often replicated. While papers are "big" enough\npieces of work to warrant this effort, what about all the other outputs we have\nsuch as (micro)blogs, social media posts, and datasets? A reliable addressing\nsystem is essential to be retrieve these too, and we can do this via standard\nInternet protocols such as HTTP and DNS.
\nThe eagle-eyed among you might notice that my site now has a unique DOI for most post.\nA Digital Object Identifier is something you more conventionally\nsee associated with academic papers, but thanks to the hard work of Martin Fenner\nwe can now have them for other forms of content! Martin set up the Rogue Scholar\nwhich permits any standards-compliant site with an Atom or JSONFeed to be assigned a DOI automatically.
\nThis post, for example, has the\n10.59350/418q4-gng78 DOI assigned to\nit, which forms its unique DOI identifier. This can be resolved into the real\nlocation by retrieving the DOI URL, which issues a HTTP redirect to this post:
\n$ curl -I https://doi.org/10.59350/418q4-gng78\nHTTP/2 302\ndate: Wed, 26 Nov 2025 12:48:43 GMT\nlocation: https://anil.recoil.org/notes/fourps-for-collective-knowledge\nCrucially, this DOI URL is not the only identifier for this post, as you can still\nalso use my original homepage URL. However, it's an identifier that can\nbe redirected to a new location if the content moves, and also has extra metadata associated\nwith it that help with keeping track of networks of knowledge.
\n ![\"%c\"](\"/images/aria-ci-4.webp\" "\\"You")
Let's look at some more details of the extra useful metadata, by peering at one of my recent posts to see how Rogue Scholar augments the metadata for it. They (i) track author identities; (ii) build a reference mesh across items; and (iii) archive clean versions to replicate content with an open license.
\nFirstly, the authorship information helps to identify me concretely across name variations. My own ORCID forms a unique identifier for my own scholarly publishing, and this is now tied to my blog post.\nYou can then search for my ORCID and find my posts, but also find it in other indexing systems such as CrossRef which index scholarly metadata. OpenAlex has just rewritten their codebase and released it a few weeks ago, with 10s of millions of new types of works indexed.
\nCurating databases that are this large across decades clearly leads to some inconsistencies as people move around jobs and change their circumstances. Identifying "who" has done something is therefore a surprisingly tricky metadata problem. This is one of the areas where ATProto, Bluesky and Tangled have a lot to offer, by allowing the social graph to be shared among multiple differentiated services (e.g. microblogging or code hosting).
\n ![\"%rc\"](\"/images/aria-ci-5.webp\" "\\"Bluesky")
Secondly, references from links within this post are extracted out and linked\nto other DOIs. I do this by generating a structured JSONFeed\nwhich breaks out metadata for each post by scanning the links within my source\nMarkdown. For example, here is an excerpt for one of the "references" fields\nin my post:
{ "url": "https://doi.org/10.33774/coe-2025-rmsqf",\n "doi": "10.33774/coe-2025-rmsqf",\n "cito": [ "citesAsSourceDocument" ] }, {\n "url": "https://doi.org/10.59350/hasmq-vj807",\n "doi": "10.59350/hasmq-vj807",\n "cito": [ "citesAsRelated" ] },\nThis structured list of references also includes CITO conventions to also list how the citation should be interpreted, which may be useful input to LLMs that are interpreting a document. I've published an OCaml-JSONFeed library that conveniently lists all the citation structures possible.\nThis reference metadata is hoovered up by databases such as CrossRef which use them to maintain their giant graph databases that associate posts, papers and anything else with a DOI with each other.
\nTo make this as easy as possible to do with any blog content online, Rogue Scholar has augmented how it scans posts so that just adding a "References" header to your content is enough to make this just work. We now have an interconnected mesh of links between diverse blogs and papers and datasets, all using simple URLs!
\nThirdly, the metadata and Atom feeds are used to archive the contents of the post via the Internet Archive Archive-It service. This is also not as straightforward as you might expect; the problem with archiving HTML straight from the source is that the web pages you read are usually quite a mess of JavaScript and display logic, whereas the essence of the page is hidden.
\nFor example, look at the archive.org version of one of my posts vs the Rogue Scholar version of the same post. The latter is significantly cleaner, since the "archival" version actually uses my blog feed instead of the original HTML.\nThe feed reader version strips out all the unnecessary display gunk so that it can be read by clients like NetNewsWire or Thunderbird. There is some work that needs to happen on the Atom feed generation side to really make this clean; for example, I learnt about how to lay out footnotes to be feed-reader friendly.
\n ![\"%c\"](\"/images/aria-ci-2.webp\" "\\"The")
![\"%c\"](\"/images/aria-ci-3.webp\" "\\"The")
To wrap up the first P of Permanence, we've seen that it's a bit more involved than "simply archiving it". Some metadata curation and formatting flexibility really helps to clean up the connections. If you have your own blog, you should sign up to Rogue Scholar. Martin has just incorporated it as a German non-profit organisation, showing he's thinking about the long-term sustainability of such ventures as well.
\nThe enormous problem we're facing with collective intelligence right now is that the Internet is getting flooded by AI generated slop. While there are obvious dangers to our collective sanity and attention spans, there's also the pragmatic problem that recursive training causes model collapse. If we just feed our models the output of other language models, we greatly dilute the quality of the resulting LLMs and the overall quality of collective knowledge.
\nWe observed the societal implications in our recent Nature comment:
\n\n\nThe publication of ever-larger numbers of problematic papers, including fake ones generated by artificial intelligence, represents an existential crisis for the established way of doing evidence synthesis. But with a new approach, AI might also save the day.\n-- Will AI speed up literature reviews or derail them entirely?, 2025
\n
We urgently need to build accurate provenance information into our collective knowledge networks to distinguish where some piece of knowledge came from. Efforts like Rogue Scholar and Kagi Small Web do this by human judgement: a community keeps an eye on the feeds and filters out the obviously bad actors. Shane Weisz also pointed out to me that crowdsourcing communities also often self-organise like this. For example, iNaturalist volunteers painstakingly critique AI output vs human experts for species detection. Provenance in these systems would help them scale their efforts without burning out.
\nLuckily though, we do have some partial solutions already for keeping track of\nprovenance:
\nWhat's missing is a provenance protocol by which each of these "islands of provenance" can interoperate across each other's boundaries. Almost every project runs its own CI systems that never share the details of how they got their data and code. Security organisations are now recommending Software Bill of Materials be generated for all software, and Docker Hardened Images are acting as an anchor for wider efforts in this space. The IETF is moving to advance standards of provenance but perhaps too slowly and conservatively given the rapid rise of AI crawlers.
\nAn area I'm going to investigate in the future is how HTTP-based provenance headers\nmight help glue these together, so that a collective knowledge crawler doesn't\nneed to build a global provenance graph (which would be overwhelmingly massive)\nto filter out non-trusted primary content.
\n
The Internet is pretty good about building giant public databases, and it's\nalso pretty good at supporting storing secret data. However, it's terrible\nat supporting semi-private access to remote sites.
\nConsider a really obvious collective knowledge case: I want to expose my draft\npapers that I'm working on with a diverse group of people. I collaborate with\ndozens of people all over the world, and so want to selectively grant them\naccess to my works-in-progress. Why is this so difficult to do?
\nIt's currently easy to use individual services to grant access; for example,\nI might share my Overleaf or my Google Drive for a\nproject, but propagating those access rights across services is near impossible\nas soon as you cross a project or API boundary. There are a few directions we\ncould go to break this problem down into easier to solve chunks:
\nWhen we look at the current state of the publishing industry, it becomes clearer why a few publishers are hoovering up smaller journals. Maintaining the infrastructure for open/closed access is just a lot easier in a centralised setup than it is when distributed. However, it's crucial for expanding the ceiling on our collective knowledge that we support such federated access to semi-private data. Let's consider another sort of data to see why...
\nZooming out to a global usecase, biodiversity data is a prime example of where\neverything can't be open. Economically motivated rational actors (i.e.\npoachers) are highly incentivised to use all available data to figure out where\nto snarf a rare species, and so some of this presence data is vital to keep\ntight control over. But the pendulum swings both ways, and without robust\npermissions mechanisms to share the data with well intentioned actors, we\ncannot make evidence-driven planning decisions for global topics such as food consumption.
\nI asked Neil Burgess, the Chief Scientist of UNEP-WCMC, and Violeta Muñoz-Fuentes about their views on how biodiversity data might make an impact if connected together. They gave me a remarkable list of databases they maintain (an excerpt reproduced with permission here):
\n\n\n\n
\n- Protected Planet, 10s of thousands of users. The world's most trusted, up-to-date, and complete source of information on protected areas and other effective area-based conservation measures (OECMs). Includes effectiveness of protected and conserved area management; updated monthly with submissions from governments, NGOs, landowners, and communities.
\n- UN Biodiversity Lab, 1000s of governments, NGO users. A geospatial platform with 400+ of the world’s best data layers on nature, climate change, and sustainable development. Supports country-led efforts for planning, monitoring, and reporting; linked to the Convention on Biological Diversity's global nature agreements.
\n- CITES Wildlife Trade View, 1000s of government and NGO users. Visualizes legal wildlife trade globally, by species and by country.
\n- CITES Wildlife Trade Database, 10000s of government, NGO, research users. Contains records of all legal wildlife trade under CITES (>40,900 species globally).
\n- IBAT, 1000s of businesses. Spatial tool for businesses to calculate potential impacts on nature. IBAT is an alliance of BirdLife International, Conservation International, IUCN, and UNEP-WCMC.\n-- An excerpt of UNEP-WCMC tools and systems (N. Burgess, personal communication, 2025)
\n
This is just a short excerpt from the list, and many of these involve illegal activities (tracking them, not doing them!). The value in connecting them together and making them safely accessible by both humans and AI agents would be transformative to the global effort to save species from extinction, for example by carefully picking and choosing what trade agreements are signed between countries. A real-time version could change the course of human history for pivotal global biodiversity conferences where negotiations decide the future of many.
\nSo I make a case that we must engineer robust permission protocols into the heart of how we share data, and not just for copyright and legal reasons. Some data must stay private for security, economic or geopolitical reasons, but that act of hiding knowledge currently makes it very difficult to take part in a collective knowledge network with our current training architectures. Perhaps federated learning will be one breakthrough, but I'm betting on agentic permissions being where this goes instead.
\n ![\"%c\"](\"/images/aria-ci-6.webp\" "\\"The")
The final P is one that we thought we wouldn't need to worry about thanks to\nthe cloud back in the day: placement. A lot of the digital data\ninvolved in our lives is spatial in nature (e.g. our movement data), but also\nmust be accessed only from some locations. If we don't engineer in location as\na first-class element of how we treat collective knowledge, it'll never be a truely\nuseful knowledge companion to humans.
\nWe explained some spatial ideas in our recent Bifrost paper:
\n\n\nPhysical containment creates a natural network hierarchy, yet we do not currently take advantage of this. Even local interactions between devices often require traversal over a wide-area network (WAN), with consequences for privacy, robustness, and latency.
\nInstead, devices in the same room should communicate directly, while physical barriers should require explicit networking gateways. We call this spatial networking: instead of overlaying virtual addresses over physical network connections, we use physical spaces to constrain virtual network addresses.
\nThis lets users point at two devices and address them by their physical relationship; devices are named by their location, policies are scoped\nby physical boundaries, and spaces naturally compose while maintaining local autonomy.\n-- An Architecture for Spatial Networking, Millar et al, 2025
\n
Think about all the times in your life that you've wanted to pull up some specific knowledge about the region you're in, and how bad our digital systems currently are at dealing with fine-grained location. I go to the gym every Sunday morning like clockwork with Dave Scott, and yet my workout app treats it like a whole new experience every single time I open it.
\nSimilarly, if you have a group of people in a meeting room, they should be able to use their physical proximity to take advantage of that inherent trust! For example, photos weren't allowed the Collective Intelligence workshop due to the Chatham House rules, but it would have been really useful to be able to get a copy of other people's photos for me to have a personal record of all the amazing whiteboarding brainstorming that was going on.
\nProtocol support for placement, combined with permissioning above, would allow us to build a personal knowledge network that actually fits into our lives based on where we physically are.
\nWhen I started working on planetary computing, the most obvious\nchange was just how vast the datasets involved are. We've just installed a\nmulti petabyte cluster in the\nComputer Lab just to deal with the embeddings from TESSERA,\nand syncing those takes weeks even on a gigabit link. This data can't just\ncasually move, which means that in turn we can't use many cloud-based services\nlike GitHub for our hosting. And this, in turn, disconnects us from all the\ncollective knowledge training happening there for their code foundation models.
\nAn alternative is to decouple the names of the code and data from where it's hosted.\nThis is a feature explicitly supported by the AT Protocol that underpins Bluesky.\nThere are dozen of alternative services that are springing up\nthat can reuse the authentication infrastructure, but allow users to choose where their data is hosted.
\nThe one we are using most here in my group is Tangled, which is a code hosting service\nthat I've described before and use regularly.\nWhat makes this service different from other code hosting is that while I can remain social and share, the\nactual code is stored on a "knot" that I host. I run several: one on my personal recoil.org domain,\nand another for my colleagues in the Cambridge Computer Lab. Code that sits there can also be run using local CI\nwhich can access private data stored in our local network, by virtue of the fact that we run our own infrastructure.
\nTo wrap up the principle of placement, I've made a case for why explicit control over locations (of people, of code, of data, of predictions) matter a lot for collective intelligence, and should be factored into any system architecture for this. If you don't believe me, try asking your nearest LLM for what the best pub is near you, and watch it hallucinate and burn!
\n![\"%c\"](\"/images/okavango-2.webp\" "\\"Offline")
I jotted down these four principles to help organise my thoughts, and they're by no means set in stone. I am reasonably convinced that the momentum building around ATProto usage worldwide makes it a compelling place to focus prototyping and research efforts on, and they are working on plugging gaps such as permission support already. If you'd like to work on this or have pointers for me, please do let me know! I'll update this post as they come in.
\n(I'd like to thank many people for giving me input and ideas into this post, many of whom are cited above. In particular, Sadiq Jaffer, Shane Weisz, Michael Dales, Patrick Ferris, Cyrus Omar, Aadi Seth, Michael Coblenz, Jon Sterling, Nate Foster, Aurojit Panda, Ian Brown, Srinivasan Keshav, Jon Crowcroft, Ryan Gibb, Josh Millar, Hamed Haddadi, Sam Reynolds, Alec Christie, Bill Sutherland, Violeta Muñoz-Fuentes and Neil Burgess have all poured in relevant ideas, along with the wider ATProto and ActivityPub communities)
\nAs I write this, the UK's budget announcement was released an hour early by the OBR, throwing markets into real-time uncertainty.
\n ↩I've just released geotessera 0.7 to pypi for our TESSERA\ngeospatial foundation model, following on from the first release earlier this year. To recap:
\n\n\nTESSERA is a foundation model for Earth observation that processes Sentinel-1\nand Sentinel-2 satellite data to generate representation (embedding) maps. It\ncompresses a full year of Sentinel-1 and Sentinel-2 data and learns useful\ntemporal-spectral features.\n-- Temporal Embeddings of Surface Spectra for Earth Representation and Analysis
\n
With this new release, there's convenient documentation to show how you can freely access 150TB+\nof CC-BY-licensed embeddings of the earth's surface. We've been getting a growing influx of requests for diverse regions of the world, and so our focus for the next few\nmonths is attaining complete coverage of our v1 model on the whole planet.
\nA new coverage map visualiser tracks our progress towards complete global embeddings;\nto complete the set, we have a giant inferencing task ahead of us. AMD kindly provided us access to an MI300X cluster for us to train the TESSERA models on, and now we are using our in-house Dawn cluster along with machines donated by Tarides and Jane Street to run every tile on earth through our pipeline. The\noutput of this is a very large set of 128 dimensional vectors in numpy format,\nwhich we then spatially sort and make available for download. You can browse the tiles on our HTTP server, where you can see one such tile and associated[1] checksums.
\n ![\"%c\"](\"/images/geotessera-coverage-ui.webp\" "\\"The")
All pretty simple stuff -- it's just some floating point numbers! -- except for the sheer number of them. Each tile of vectors is around 160MB in size, and so we need around 250TB of storage per year, and we want them from 2017 through to 2024. And by the time we're done inferring all these, 2026 will be upon us and we'll need to generate another 250TBs in short order for the 2025 seasons. Ultimately, to keep up with this we need to build a scalable, open and federated pipeline to handle this, which the 0.7 geotessera release lays the groundwork for.
\nEarlier releases of geotessera used the Pooch library as a registry for all the embeddings. Pooch is a very convenient Python library that handles the details of fetching, checksumming and caching, but is really designed for grabbing a few large datasets, for example from Zenodo. The Tessera manifests repository for Pooch now has over a million files and takes minutes to initialise the data structures, which in turn makes using GeoTessera interactively slow.
\nOne lesson Sadiq Jaffer and I learnt from building our giant literature database is how good the Parquet file format is. So in this 0.7 release, we switch from a text format to track all the tiles to a couple of GeoParquet databases. To help visualize this, I've created an avsm/geotessera HuggingFace dataset where you can browse the format, as well as an interactive coverage map.
\nNow when you initialise GeoTessera 0.7+, it will download the two parquet\nfiles instead of using the old git registry. The landmasks parquet rarely changes (it just provides the mapping\nbetween ocean/land tiles and the relevant UTM projection). The registry\ncontains the tiles for all years, and weighs in around 100MB so far compressed.\nIn future releases, I'll probably split out them out to be per-year, which will\nmake them smaller again, but they're all in one place for convenience right now\nas the size of the registry download is about the same as a single embedding\ntile, so it's not hugely significant to optimise at this stage.
\nHot on the heels of the 0.7.0 release, Janne Mäyrä submitted a PR to add support for a\nfile format called Zarr which is a really optimised way to store tensors.
\n\n\nZarr is a community project to develop specifications and software for\nstorage of large N-dimensional typed arrays, also commonly known as tensors.\nA particular focus of Zarr is to provide support for storage using\ndistributed systems like cloud object stores, and to enable efficient I/O for\nparallel computing applications.
\nZarr is motivated by the need for a simple, transparent, open, and\ncommunity-driven format that supports high-throughput distributed I/O on\ndifferent storage systems. Zarr data can be stored in any storage system that\ncan be represented as a key-value store, including most commonly POSIX file\nsystems and cloud object storage but also zip files as well as relational and\ndocument databases.
\n-- Zarr Homepage, 2025
\n
This is exactly what we need given the giant storage requirements above, so\nI've fixed up and merged this feature and included it into a 0.7.1 point\nrelease. I've not had much chance to actually play with the format yet, but\ngetting it into a release is the best place to start. I got one positive\nmessage from Andres Zuñiga-Gonzalez that he's been using Zarr already in his Local Climate Zone\nexperiments.
\nThe other feature that's gone into the new library are higher level APIs to use\nthe embeddings. A very common usecase when building downstream tasks is to have\nto sample embeddings for a set of labels, which are then used to train\nclassifiers. It's quite cumbersome to select these out by hand, particularly with\nlarger regions of interest.
\nThere is now a new sample_embeddings_at_points library call\nthat extracts embedding values at arbitrary lon/lat coordinates and\ngroups points by tile for efficient batch processing. You can see this in action\nin the new geotessera-examples\nrepository where we're starting to put sample code for downstream tasks that\nuse geotessera.
The first one here is the code to detect solar panels worldwide, as demoed by\nSadiq Jaffer in his recent PROPL 25 talk. Browse though the\nsource code and\ngive it a spin. The results can visualised using QGIS, and I'm working on a notebook\ninterface for this later.
\n ![\"%c\"](\"/images/tessera-f1.webp\" "\\"Parametric")
The second one is the parametric UMAP script to do false colour visualizations of\nany ROI. You can see a high-res "arty version" that has been contour traced using a pomap algorithm which renders as a high-res SVG as well.
\nThat's it for now with GeoTessera updates. Enjoy the new releases, and if you have any requests for embeddings please get them in our queue.
\nMark Elvers is currently syncing our embeddings to Scaleway via a Ceph cluster we setup for this purpose, and I'll be announcing even more federation options for TESSERA over the coming weeks. We're really grateful to the outpouring of offers of help with our computational needs from our users and cloudy friends! And we also know that TESSERA doesn't have a proper homepage yet; we'll work on this right after the immediate embeddings bottleneck is handled for our current users (contributions from interested web designers are very welcome here).
\nThere are two numpy files in there because we store the quantized embeddings to save space. To dequantize them, the scales are multiplied to each of the bands in the main file. Geotessera takes care of this for you.
\n ↩There's a buzz forming around the upcoming AI Impact\nSummit next year in India, following up the\nAI Safety Summit here and the France Action\nSummit earlier this year. I\nheaded down to a couple of events in London this week to help set the agenda,\nparticularly around the importance of FAIR and ethical AI for sustainability being on the political agenda.
\nThe ATI CETAS, the FCDO, and the High Commission of India hosted a day long session for the\nIndian and UK contingents to get to know each other and to find opportunities for\ncollaboration. While the details of the day are under Chatham House rules, I learnt a\nlot from the discussions that is already public.\nThe opening came from Abhishek Singh, the CEO of the India AI Mission that got over a billion dollars in funding last year to drive equitable AI deployment there:
\n\n\nThe IndiaAI Mission aims to build a comprehensive ecosystem that fosters AI innovation by democratizing computing access, enhancing data quality, developing indigenous AI capabilities, attracting top AI talent, enabling industry collaboration, providing startup risk capital, ensuring socially impactful AI projects, and promoting ethical AI. This mission drives responsible and inclusive growth of India's AI ecosystem [...]\n-- India AI Mission, 2024
\n
Abhishek made a compelling case about why India is so differentiated in this AI race.\nFirstly, it has a huge diversity of population, languages and climates. It's quite common to have voice-only interfaces, such as those by Gram Vaani that I learnt about at COMPASS 2024 in multiple Indian languages. India has been investing heavily in making sure that local languages are supported via the Bhashini initiative, which is an example of the central government funding efforts for the public good. Aadi Seth pointed me to the "Field Guide for Inclusive Language" in India, which has a profile on how Gram Vaani and Gram Sabhas is being used in rural India to bridge the digital divides.
\n![\"%c\"](\"/images/ukindia-dialogue-2.webp\" "\\"The")
Their recent AI Governance Guidelines also seemed reasonable to me. It identified risks in malicious use, bias and discrimination, transparency failures, systemic risks, loss of control, and national security challenges, but didn't overspecify solutions to these. The report was chaired by IIT-Madras, where I've been collaborating with KC Sivaramakrishnan ever since he left Cambridge, so there's a nice connection there too!
\nAfter the round table, we broke out into detailed planning sessions. I joined the climate one, where Eviatar Bach gave me a splendid ad-hoc tutorial on the difficulty of subseasonal weather forecasting and pointed me to their notes on inverse problems. I delivered an explanation of TESSERA and how terrestial forecasting is important towards areas such as farming, and we also learnt from Po Yang and others present about the latest in agricultural robotics.
\nOf particular note is the large amount of work being done by Wadhwani AI towards "real world" AI deployment in India; for example, doing uncertainty aware inferencing to build trust with thousands of cotton farmers across India. I'm very much looking forward to building up more concrete collaborations here, in addition to the work we're starting with the CoRE Stack following PROPL25.
\nWithout pause, the next day I went back to London to have a roundtable dinner\norganised by the wonderful Amanda Brock from OpenUK for\nfurther discussions. This time, the focus was on the critical role of open\nsource as a medium for exchange between the UK and India.\nThere were some wonderful discussions here again, ranging from open agentic infrastructure, to building open communities in the vibe coding world, to new communities around crowdsourcing nature data!
\n![\"%c\"](\"/images/ukindia-dialogue-8.webp\" "\\"The")
I saw Luke Marsden again after a decade following DockerCon 2016,\nand learnt he is now working on a cool local agentic inference stack\ncalled helix.ml, which sounded like exactly what we\nneed with all the complex agentic coding happening for building our collective knowledge database.
\n![\"%rc\"](\"/images/ukindia-dialogue-5.webp\")
Ironically, Luke used to work on a brilliant ZFS appliance for Docker with his old\nClusterHQ startup when I saw him last. Nowadays, I find my\nown group hacking on the same sort of thing. Patrick Ferris\nhas been using ZFS in his Shelter shell,\nand Mark Elvers has been building a giant petascale storage cluster\nfor our Tessera embeddings. I love how open source goes full circle all the time!
\nDon Syme not only leapt to the defence of higher order languages[1], but more importantly\ngave a thoughtful note on the difficulty of building community when\neveryone is AI coding.\nCommunity in open-source isn't just code; it's about\n"soaking in" the whole lifecycle of crafting something together. We're in\nreal danger of losing the incentive to communicate and socialise if we're all vibe coding at\nhigh speed, and that in turn will signal a real problem in how we mentor and build distributed\ncommunities across geographies.
\nCoincidentally, there's been a lot of online discussion about this same topic in the past few\ndays. KC Sivaramakrishnan reflected on foundations for hacking on OCaml, and\nPatrick Ferris on his experiences picking up systems research.\nMeanwhile, a keen first-time contributor to OCaml decided that artisinal coding is the future and caused a ruckus among the maintainers. It's so easy to contribute PRs now -- which is great! -- but there's also no empathy for the amount of work it'll cause maintainers who have to deal with this flood of long-term technical debt.\nMy gut feeling is that more in-person events (such as attending the AI summit and running hackathons) is the only way forward in the short-term, to recruit contributors who care about the long-term as well.
\nI spoke to the assembled group about our work on LIFE and TESSERA, and\nwas bowled over the by the desire to engage on the topic from many of those\npresent. I particularly enjoyed sitting beside BBC tech journalist Zoe Kleinman\nwho I ended up explaining my obsession with tracking hedgehogs to.
\nIt does feel to me like this hedgehog tracking project has really resonated with\npeople. Satellite mapping can be a little abstract as a concept, but spotting hedgehogs from space is an\nimmediate draw to a topic people are interested in immediate usecases where AI can help figure out the state of nature in their own backyards.
\n![\"%c\"](\"/images/ukindia-dialogue-9.webp\" "\\"I")
Aside from this, some random enjoyable tidbits of the evening:
\n![\"%c\"](\"/images/ukindia-dialogue-6.webp\" "\\"The")
![\"%c\"](\"/images/ukindia-dialogue-3.webp\" "\\"There")
Follow the discussion about this on LinkedIn as well. I look forward to more followups with the ATI, OpenUK and our colleagues in India!
\nA story to tell another day, since this meeting was also under Chatham House rules.
\n ↩Jacqueline Wilson wrote a brilliant article about our recent paper on the biodiversity impacts of food consumption up on the main Cambridge website, complete with interviews with Thomas Ball and Alison Eyres.
\n\n\nHow does your dinner affect the risk of 30,875 species of land-dwelling animal going extinct?
\nDr Thomas Ball can tell you. Depending on what you’re eating he can calculate\nthe likelihood of the global demise of every mammal, bird, amphibian and\nreptile over the next 100 years. He’ll tell you that not all dinners are\nequal.\n-- cam.ac.uk, 2025
\n
If you'd like to explore this dataset in more detail after reading the article, check out the interactive explorer below:
\n ![\"%c\"](\"/images/food-life-globe-1.webp\" "\\"Explore")
I got to shake Jensen Huang's hand as he received the 2025 Hawking Fellowship this evening at the Cambridge Union! He's a fitting winner for this award; he's not only tech's longest running CEO (33 years!), but also a founding engineer who deeply understands the technology stack. He also bucks the trend among bigtech and famously doesn't believe in firing people, preferring to "torture them into greatness"[1] instead.
\nAt the awards event this evening, Jensen made some remarks that stuck with me. He noted that he doesn't really like competing: "Nvidia is a company that succeeds when other companies succeed". By building platforms, Nvidia can focus on creating a better future that others aren't, and not on beating someone else. This mindset has had real adverse costs, but Nvidia has stoically taken hardware bets[2] that it believes in across six waves of computing generations.
\nIt was quite delightful to hear this attitude of focussing on the future, and\nretaining a basic optimism and curious mindset even as his company becomes larger than most economies.
\nMy question to him was simple. My 816 year old institution has had exams as\nits staple assessment mechanism for centuries. Is AI finally the breaking point\nby which competitive stack ranking of brilliant students no longer makes sense? What would his advice to Cambridge University be now?
\n![\"%c\"](\"/images/jensen-hawking-4.webp\" "\\"Jensen")
His answer was comprehensive and nuanced. At Nvidia, he's discounted 360 peer reviews\nor firing the bottom bracket of employees as ineffective. The\nreason for this is that those employees\nmight be underperforming temporarily because they had the courage to take\nrisks, and in the next cycle are a source of creativity to come up with fresh\nideas and avoid ruts. If you weed out all the risktakers regularly, you've just\nweeded out a bunch of original thinkers! So instead, they focus on a culture\nof helping peers to improve, and trust individuals to avoid resting and vesting.
\nSo how does this philosophy apply to Cambridge, a residential institution with a huge emphasis on high student contact time and full of mavericks? Jensen noted that the premise of testing students via exams is based around hiding some information (the exam) and\nmaking students find the answers (from their memory). When Google rose in popularity\nin the early 2000s, there was a lot of soul searching then about whether libraries were now obsolete.\nToday, AI makes finding expert information about specialist topics easier than ever, and makes it proportionately harder to use "information hiding" as the mechanism of assessment.[3]
\n![\"%c\"](\"/images/jensen-hawking-2.webp\" "\\"Every")
So his view was that information hiding will become an ineffective form of\nscarcity, much like how digitising books made learning library skills far less\nof a useful discriminator. And instead, we could focus our pedagogical efforts on the\nqualities that will remain useful to our students throughout their careers and lives: the courage to be creative and take risks,\nthe resilince to fail in a public setting, and the intellectual honesty to\nrecognise when a direction needs to be altered.
\nI've been having long conversations with Carl Henrik Ek and Amanda Prorok about the\nvalues we really believe in for Computer Science at Pembroke, which are aligned\nwith Jensen's remarks above. I don't think we'll be abolishing exams any time\nsoon, but I wouldn't be suprised if their format dramatically changed by the\nend of this decade.
\nNvidia is a remarkably agile company despite its huge size thanks to the culture\nthat Jensen Huang established. Other large organisations with similar\nagility that I've worked with include Jane Street[4] and their famous lack of\nhierarchy. Cambridge also has a unique\nstructure with the independent Colleges and University coalition that mean we\nresist centralisation, but exam culture is so deeply entrenched in our Tripos that\nseeing a quick path away from it might take some time -- but is possible!
\n![\"%c\"](\"/images/jensen-hawking-3.webp\")
Another very encouraging aspect in Jensen's talk was his deep belief in the\nscientific method, and the important of intellectual honesty in our collective\nactions. He talked about his excitement about AI for drug discovery,\nbut also how weird it is that we have to "discover" drugs in the first place.\nThere are clearly physical laws governing the interactions betweeen biological\nentities, so why isn't biology more of an "engineering practise"?
\nHis approach, of course, is the application of machine learning to build\nbiological models. And with the advent of multimodal models, he'd like to be able\nto talk to proteins and interrogate them Socratically ("how would you bind to this other molecule?").\nThis chimes very well with the research we've been doing in my group for the past few years...
\nI've just been involved in two bold ARIA discovery grants that encourage us to ask exactly these sorts of big questions around understanding complex ecological systems using machine learning. The first is with Joe Millard on exploring the potential of combining distinct forms of foundational AI to simulate ecosystem resilience in digital space. The second is with Oisin Mac Aodha up in Edinburgh to think about species distribution prediction using a combination of machine learning, causal literature and remote observations.
\nKey to both of these succeedings are the foundational machine learning models we've been developing. The TESSERA geospatial model unlocks the modality of ground observation alongside conventional LLMs. The Cambridge TAP literature pipeline gives us deep causal connections from millions of experiments.
\nWe haven't even begun to plumb the potential of combining some of these models in the future since we're deeply limited by our computational resources in the University at the moment. Luckily, both Nvidia and AMD[5] have been enthusiastic about supporting open research through GPU access, and we also have Dawn and soon Isembard. I'm really enjoying research life right now, and particularly so after seeing that someone as successful as Jensen Huang still retains a childlike curiosity about the world. Congratulations to him on a well-deserved Hawking Fellowship!
\n![\"%c\"](\"/images/jensen-hawking-1.webp\")
Some entertaining misc factoids from his talk:
\nA big thank you to Andy Grant for the invitation to tonight's event! It was a lot fun.
\nRemarkably similar to a Cambridge University degree...
\n ↩The Geforce I bought back in 2001 came with expensive shader hardware that nothing supported. That feature lead to GPGPU programming that led to modern machine learning becoming computationally feasible.
\n ↩This is distinct from the problem of LLMs having a negative effect on learning in schools and university. I'm focussing on exams as the primary means of assessment.
\n ↩The Signals and Threads episode describing how Jane Street regularly swaps employee desks is a wild ride.
\n ↩AMD donated us access to enough MI300x machines to make the training of the first version of TESSERA possible earlier this summer.
\n ↩Possibly the only good marketing book I've read.
\n ↩I had an amazingly fun week at\nICFP/SPLASH in Singapore;\nit was the first time that these two major programming languages conferences\nwere held simultaneously. My submissions turned into a bit of a success\ndisaster;\nI ended up chairing a workshop, giving several talks and a keynote, and\norganising a tutorial, and helping out a bunch of colleague and students. And\nif this wasn't enough to fill up the week, collaborators I hadn't seen in a few\nyears were also presenting a tonne of interesting work, so there was no time to\nbreathe!
\nSo much went on that I've split up the post into a five parter:
\n![\"%c\"](\"/images/icfp-20.webp\" "\\"The")
Other colleagues who wrote up their experiences at ICFP 2025 include:
\nThis is part 5 of a series of posts[1] about ICFP 2025.
\nIn addition to giving a bunch of talks about\nDocker, post-POSIX and\nplanetary computing, the greatest fun at a huge conference\nlike ICFP and SPLASH is seeing talks given by my students (they grow up so\nfast!) and collaborators, and generally floating around random talks trying to\ndeceipher ancient Greek lambdas floating on a projector.
\nI've been wanting to try to do something with Hazel ever\nsince Cyrus Omar showed it to me at last year's PROPL.\nMax Carroll picked up the idea of doing gradual type-level debugging for his Part II undergraduate project\nat the Computer Lab. He not only aced his project, but wrote up a\npaper\nfor the HATRA\nworkshop:
\n\n\nWe explore how to provide programmers with an interactive interface for\nexplaining the process by which static types and dynamic casts are derived,\nwith the goal of improving the debugging of static and dynamic type errors.
\nTo this end, we define mathematical foundations for a decomposable\nhighlighting system within a bidirectional system, and show how these can be\npropagated through dynamic types in a cast system. Our prototype\nimplementation in the gradually typed Hazel language includes a web-based\nuser interface, through which we highlight the importance of type level\ndebugging.\n-- Decomposable Type Highlighting for Bidirectional Type and Cast Systems, Carroll 2025
\n
![\"%c\"](\"/images/icfp-18.webp\" "\\"Max")
Max Carroll delivered a fantastic first conference talk, complete with a live\ndemo demonstrating type-level debugging in action; give it a watch if you're\ninterested in live programming!
\nOne issue we had during this project was finding a decent corpus of functional\ncode with errors to use to test out Max's debugger. Hazel's a pretty young\nlanguage, and finding a large codebase is difficult, let alone a bunch of code\nwith errors. Patrick Ferris decided to accelerate this process by building a\nhazel_of_ocaml and\npresenting this work at the TyDE workshop.
\n ![\"%c\"](\"/images/icfp-pf341-tyde.webp\" "\\"Patrick")
With Patrick's transpiler, we grabbed Eric Seidel's\ncorpus of ill-typed OCaml that he built\nfor his research on dynamic witnesses for static type\nerrors. Max successfully used\nthis translated corpus to build his type-level debugger, and is planning to\ncontinue to work on this in his Part III project this year.
\nI've been spending a lot of time with my friend Claude recently, and so have Sadiq Jaffer and\nJon Ludlam. We wrote up our experiences with interfacing\nOCaml with coding agents, and Sadiq presented it to an interactive crowd at the OCaml Workshop.
\n ![\"%c\"](\"/images/icfp-26.webp\" "\\"Sadiq")
Aside from the very sensible\nguidance on MCP and\ntools, I discovered a couple of things from this work:
\nSheera Shamsu gave a fantastic talk on building a formally\nspecified garbage collector for OCaml to a very crowded room! This was rather\ntopical given our musings on multiple runtimes in\nthe shift from OCaml 4 to 5.
\n ![\"%c\"](\"/images/icfp-21.webp\" "\\"Sheera")
\n\n[...] we propose a strategy for crafting a correct, proof-oriented GC from\nscratch, designed to evolve over time with additional language features. Our\napproach neatly separates abstract GC correctness from OCaml-specific GC\ncorrectness, offering the ability to integrate further GC optimizations,\nwhile preserving core abstract GC correctness. As an initial step to\ndemonstrate the viability of our approach, we have developed a verified\nstop-the-world mark-and- sweep GC for OCaml. The approach is mechanized in Fstar\nand its low-level subset Lowstar.\n-- A Mechanically Verified Garbage Collector for OCaml, Shamsu et al 2025
\n
Chatting to KC Sivaramakrishnan afterwards, it seems that there's interest in shifting to\nLean from Fstar to investigate if the ergonomics of the proofs are better. But\nas baselines go, the mechanically verified collector always beat the\nconservative Boehmm-GC, which means it's no worse than the current more\nconservative choice. That's good work!
\n![\"%c\"](\"/images/icfp-22.webp\" "\\"A")
Most "wonderful ICFP experiences" usually include crossing the lines\nto go hang out with other language communities.
\nBack in 2014, I stayed up all night before my keynote to the Haskell Symposium\ntrying to encode OCaml functors as Haskell typeclasses and even got help on\nstage from friendly Haskellers. This year, Richard Eisenberg was my absolute\nhighlight with a superb session on what\nhe's learnt from being the rare breed of someone steeped deeply both in\nHaskell and OCaml. The room was so packed for this talk that they had to\ncreate an overflow room streaming it in the corridors!
\n ![\"%c\"](\"/images/icfp-23.webp\" "\\"Richard")
Richard talked about his experiences with being both an OCaml and Haskeller,\nand went through a series of examples illustrating the differences between the\ntwo. He didn't get very far before the audience got involved, with both\nHaskellers and OCamlers putting their 2c in! For that reason, the stream\nrecording might not work so well.
\n ![\"%c\"](\"/images/icfp-25.webp\")
It's worth watching the talk rather than me going through each of his examples,\nbut I did have a long morning coffee with Simon Peyton Jones when I got back to Cambridge about what\nthe essential difference is between OCaml and Haskell. Laziness seems like a\ndetail, but purity is absolutely key; it percolates through every other design\ndecision (like ordering of variables, or module generativity, and so on) since\nside-effects lurk everywhere in OCaml.
\n![\"%c\"](\"/images/icfp-19.webp\" "\\"Jane")
I think it's really important to have these cross-community in-person moments. One call to action in Richard's talk was for us to consider having a unified "Haskell/ML Symposium" where long-form research papers could be shared, with shorter language-specific workshops. One audience member asked why this couldn't just be the ML Workshop, and Richard promptly pointed out that it has "Higher-order, Typed, Inferred, Strict" in the title! Just excising one word might unify two communities long split for decades...
\nAside from language matters, I think it would be a good idea to bring more of the functional programming community together more often outside of the "main ICFP track" (which is high pressure and quite squeezed for time with little discussion outside the corridor tracks). I really miss CUFP, since for a decade this was where the functional hackers would all meet up towards the tail end of the main conference. This year however, the workshops were run in parallel with the main ICFP and OOPSLA, which I think sadly diluted the community bonding a bit.
\n![\"%c\"](\"/images/icfp-24.webp\" "\\"KC")
Speaking of teaching, noone in the world can school me better than Shriram Krishnamurthi when\nit comes to matters of computer science pedagogy. I grabbed him at the lunch\nbreak and asked him for advice on our upcoming reform of the Cambridge Computer\nScience Tripos (I teach the first course). His opinions were legion,\nand he kindly gave me a quick spin around what they are working on at Brown.
\nThe SMoL (Standard Model of Languages) has a nice web interface and quiz, just like the one he helped on for our OxCaml tutorial. SMoL is deliberately language agnostic:
\n\n\n\n
\n- If students master SMoL, they have a good handle on the core of several of these languages.
\n- Students may find it easier to port their knowledge between languages: instead of being lost in a sea of different syntax, they can find familiar signposts in the common semantic features. This may also make it easier to learn new languages.
\n- The differences between the languages are thrown into sharper contrast.
\n- Students can see that, by going beyond syntax, there are several big semantic ideas that underlie all these languages, many of which we consider “best practices” in programming language design.\n-- Fixing Standard Misconceptions about Program Behaviour, 2024
\n
Much like Richard Eisenberg's talk on Haskell/OCaml, the SMoL tutor shows multiple\nlanguages for the same problem, rotating across Python, Scala, JavaScript and\nso on. I like this idea a lot for our Foundations of CS course, as I've been\nconsidering rotating in Hazel into the mix to ease the\nsyntactic shock of using OCaml. SMoL takes this concept much further, and is\nbacked by serious user studies on students.
\nI also really liked the Pyret approach of starting to\nteach using tables as a core datastructure, and not lists or arrays. However,\nI'll need to think hard about how this teaching model would work under Cambridge's\nquirky supervision model.
\nThis is on my queue to work on over the winter, while Jon Ludlam kindly covers my undergraduate lectures for this year while I'm on sabbatical!\nOn my reading list from chatting to him:
\n![\"%c\"](\"/images/icfp-27.webp\" "\\"Not")
Webassembly has also gone a long way since I last looked into it. I had a long chat with Phillipa Gardner on the\nnature hike\nto learn about her work on SpecTec,\nwhich is a single source-of-truth DSL that describes both the\nWasm specification and the artefacts like the\ninterpreter.
\nAfter that, Ben Titzer told me about WALI which is an alternative approach to WASI that simply exposes Linux kernel interfaces straight to the wasm runtime. I'm rather amenable to this given my case for shared memory IO earlier in the week at VMIL, so this is now on my list of things to investigate! KC Sivaramakrishnan, Chris Casinghino and I discussed what an OxCaml wasm unikernel might look like (a lot of buzzwords, I know), and we are pretty close to OxCaml making it possible to write runtimes using itself -- it just needs support for "external memory", which is a topic the Jane Street interns worked on over their summer projects.
\nOverall, I had a brilliant -- if exhausting! -- week in ICFP in Singapore. I\nloved the city, I loved the vibes around the conference, and it was totally\nworth the trip. Huge thanks to Ilya Sergey and the organising team for making\nthis happen!
\n
![\"%c\"](\"/images/icfp-5.webp\" "\\"The")
![\"%c\"](\"/images/icfp-17.webp\" "\\"The")
10th Oct 2025: Typo fixes spotted by Shriram.
\nSee also in the ICFP25 series: chairing PROPL25, the OxCaml tutorial, multicore at Jane Street and Docker, post-POSIX IO and what I learnt.
\n ↩This is part 4 of 5 of a series of posts[1] about ICFP 2025.
\nAfter the excitement of presenting my Docker experience report, I went straight into giving a keynote talk at VMIL\n2025. This talk\nbubbled up intrusive thoughts I've had resulting in the past 25 years: every\nsystem I've worked on, ranging from Xen to\nDocker on all seem to boil down to "make shared memory go\nfast".
\nI'd started to believe it was time for change in the way we approach IO about 12 years ago when I talked about wierd IO\nbehaviour to a packed audience at\nFOSDEM, and now I believe it's even more true in 2025.
\nSo I made one key argument to the audience: it's time to accept that standards\nsuch as POSIX are now holding back the development of good language runtimes,\nand we need to embrace the diversity of highly concurrent, shared-memory\ninterfaces. And unfortunately, there's no portable subset amongst these, and so\nthis may require a rethink of our frontend language interfaces as well.
\n ![\"%c\"](\"/images/icfp-38.webp\" "\\"The")
After explaining io_uring on\nLinux, and then the Windows and macOS variants, I showed how we're trying to\nsupport these in our OCaml 5 Eio library. While Eio has plenty of\nreally cool features,\nthe defining one for me is that it abstracts away IO operations (like\nFlow.copy)\nsufficiently that the backend can do highly parallel dispatch to the kernel\nover shared memory interfaces like uring.
\nCrucially, we treat the shared memory interface in Eio as the first-class citizen,\nwith the POSIX(ish) backends relegated to a compatibility role. This way, future\ninterfaces to the programmer can be "parallel first".
\nThe discussion with the audience after the talk was just fantastic. Stephen Kell,\nwho has\nthought\ndeeply\nabout the interaction between the kernel and userspace asked whether I was\nbeing too harsh on POSIX, which has served us faithfully for many years. And\nindeed, I agree with Stephen! POSIX gives us a fine boot layer, and a fine\ninteraction layer (for terminals anyway), and a great single threaded\ninterface. Where it falls over is the highly concurrent and parallel world of\nhigh performance computing, where must align our data paths and not have any\ninterference from third party code.
\nSo perhaps we need to restructure runtimes to explicitly have a "boot phase"\n(POSIX) where they are establishing their resources, and then switch into a\n"steady phase" (uring and friends) where they are blasting data at high speeds.\nThese are really quite distinct modes of operation, and both are useful. Note\nthat "high speed" here applies to embedded systems as well; if I do less work\non those systems in the CPU, then I'll get better energy usage, so low-overhead\nmechanisms like uring are useful there too.
\nOne other thought I had from Stephen's excellent question was the important\nrole of POSIX in portability for many years. Moving forward into the next\ndecade, what standards body will help developers write portable software for\nall these different IO interfaces? It seems inevitable that this will fragment\ninto per-language specifications instead of the operating system (or C-level)\ninterfaces we've had for the past 50 years.
\nI was also slightly surprised by how few people had used io_uring, in a room\nfull of language VM developers. One of the most useful things I did when\ndeveloping the OCaml uring\nbindings was to build a\nparallel file\ncp,\nbut I never got around to any real networking code.
So I've started to build a "raw" OCaml 5 + uring HTTP server that is completely\nnon-portable, but serves to show how it works at the lowest level. This should\nalso give us a nice benchmark against which to test higher level interfaces. My plan\nis to make this work with OCaml 5 first, and subsequently add OxCaml support. Sadiq Jaffer pointed\nme to the magic caml_alloc_local FFI function\nadded in OxCaml that allows allocation directly into the OCaml stack from C, which should be all I need to\nmake the shared memory interface never allocate into the heap.
\nPatrick Ferris also spent some ICFP time hacking on integrating OxCaml into the OCaml uring bindings:
\n\n\nThe idea being that a completion queue entry (a notification that some operation has completed) could be fully represented using 64 bits (two 32-bit, unboxed values).\n-- Patrick at ICFP 2025
\n
Having had my regular morning coffee with Simon Peyton Jones back in Cambridge, he then pointed out\nthat Haskell also has a uring backend\nPR lingering for years, and\nso perhaps we should do the same exercise there too, to understnad it all! I\ncan't say no to Simon, but if a Haskell expert is interested please do get in\ntouch so I don't have to inflict my OCaml-style Haskell on the world...
\n3rd Nov 2025: Add link to Patrick's uring bindings.
\nSee also in the ICFP25 series: chairing PROPL25, the OxCaml tutorial, multicore at Jane Street and Docker, post-POSIX IO and what I learnt.
\n ↩This is part 3 of 5 of a series of posts[1] about ICFP 2025.
\nIt's been about six years since we wrote the papers on parallelism and effects,\nand four years since we helped to release\nupstream OCaml 5.0 with multicore support, a mammoth\neffort\nthat took up years of work for my OCaml Labs and\nTarides crew. After the release came out, I focussed on\nbuilding applications using OCaml 5 for my own work on planetary computing, for example on using the new features with the\nfledgling Eio library to get some experience with\ndirect-style OCaml programming.
\nMeanwhile, big OCaml users have also been adapting their codebases to shift\nfrom OCaml 4 to 5. Jane Street have expanded their tools and compiler team and\ndriven through their production switch to the multicore runtime, and Docker for\nDesktop is progressing with their switch to direct-style code via Eio for\nhundreds of millions of users! Read on to learn more...
\nAlthough it was the last talk of the entire conference week, I'm discussing\nthis first as it was the most exciting thing I learnt at ICFP! At the\nREBASE [2] workshop,\nYaron Minsky announced that Jane Street's production servers are now running\non the OCaml 5 runtime!
\n ![\"%c\"](\"/images/icfp-29.webp\" "\\"Yaron")
That was the good news (our runtime is trading trillions of dollars, wow). The\nbad news is that there was a bumpy road internally for Jane Street to go from\nthe version that was first released (OCaml 5.0 on Dec\n2022) to their current tree. Ron gave a really\ngood roundup of some of the effort that went into release engineering their\ninternal rollout. Since the entire OCaml runtime was\nrewritten as part of the multicore\nruntime upgrade, Jane Street encountered GC pacing\nissues\nand other unexpected changes in resource usage as a result of the new runtime\nbehaviours. This took some significant design and engineering effort and to fix, which Ron\ncovers in detail in his talk.
\nSo what was the root cause of this bumpiness? I think a lot of it was just\nnormal release engineering; we clearly\nsignalled\nwhen releasing OCaml 5.0 that it was not yet feature complete, and that the\n4.x runtime would continue to be supported for some years.
\n\n\nThe developer team released OCaml 5.0.0 in December 2022. OCaml 5.x features\na full rewrite of its runtime system for shared-memory parallel programming\nusing domains and native support for concurrent programming using effect\nhandlers.
\nOwing to the large number of changes, especially to the garbage collector,\nOCaml 4.14 (the final release in the OCaml 4.x series, originally released in\nMarch 2022) remains supported for the time being. Maintainers of existing\ncodebases are strongly encouraged to evaluate OCaml 5.x and to report any\nperformance degradations on our issue tracker.\n-- ocaml/ocaml README
\n
The latest OCaml 5.4.0 release that came out just a couple of\nweeks ago is the first\nrelease with full feature parity with the OCaml 4.x LTS branch. Features such\nas statmemprof,\nthe MSVC port, GC compaction, thread sanitizer, RISC-V and\nS390X architecture support all had to be engineered back in. OCaml development\nin recent years has been very developer intensive because of the need to not only\nreintroduce these features, and also keep up with the torrent of new features being introduced to OCaml 5.x.\nAll in all, I think it's been a very successful few years for the language to have kept steadily improving!
\nHowever, Ron's talk was an excellent chance for us to reflect on what we might\nhave done differently with the benefit of hindsight. David Allsopp posits that we should\nhave maintained the OCaml 4 and 5 runtimes simultaneously:
\n\n\nOne of the early ideas was to merge just the runtime changes as a separate\nruntime, leaving all the language changes to a subsequent update. The main\nthing here would have been to upstream the immense changes to the allocator\nand garbage collector along with the domains and fibers machinery, while not\nyet exposing it.
\nI remember the concern being that having essentially a runtime variant (not\nunlike the debug runtime) might lead to very slow uptake at actually testing\nit and possibly a maintenance burden. i.e. we were concerned at maintaining\ntwo runtimes. This would probably have resulted in something like OCaml\n4.15.0, with an experimental official multicore-aware runtime.\n-- Reflections on ICFP 2025, David Allsopp
\n
I agree with this. Although we weren't sure in 2021 that it would be possible\nto have two simultaneous runtimes[3] it was clear by the time we were\nengineering the 5.0 PR that this would be possible. Still, a few years to stabilise\nmulticore performance vs the decade-old 4.x runtime isn't bad going at all, so I don't have\ndeep regrets about the approach we did take!
\nThe other area where all of us agreed more effort is necessary is on continuous performance engineering.\nIn the runup to 2022, Tom Kelly, KC Sivaramakrishnan and I set up Sandmark, which\nwas a body of microbenchmarks and some macrobenchmarks. Tarides setup a continuous benchmarking service around this, and I hosted a bunch of carefully tuned machines with specific BIOS settings in the Cambrige Computer Lab.
\nRoll on six years, and the maintainence cost of this service becomes clear.\nIt's quite a bit of effort to maintain the old machines (one is running Ubuntu\n16.04! I'm not telling you which one) to keep consistency in previous results.\nNew machines all come with a proportional configuration effort, and their results\nhave to be interpreted. Operating systems have to be upgraded, and tuned afresh.\nContinuous benchmarking is itself a continuous process of engineering, and should be treated as such!
\nOur discussions at ICFP centred around the idea that we should not only maintain this\nbenchmarking infrastructure, but add an incentive to macro projects to submit\nrepresentative tests of their performance. Rocq, Why3,\nSemgrep,\nor Frama-C, for example, should all have test cases run within this\ninfrastructure that move beyond microbenchmarks to realistic performance\npatterns that will show up issues with GC pacing in a way that microbenchmarks\ndo not.
\nThe challenge with doing this so far has been the difficulty of getting many of these\nbig projects to compile on a random OCaml trunk snapshot (essential to test them against\na pre-release OCaml compiler). Solving this will take some thought (particularly around\nppx usage), but the effort seems worthwhile as we move into a new phase of OCaml 5\nengineering now that feature parity has been achieved. Stay tuned for more on this from\nTarides!
\nI also had the opportunity to share both a retrospective on the work Dave Scott\nand I have been doing on Docker Desktop for some years, and\nalso the efforts from new OCamlers like Patrick Ferris and Ryan Gibb on helping us\nto port some aging OCaml code over to OCaml 5.
\nWe got a paper accepted to ICFP on the topic, and so I had a lot of fun\npresenting\n"Functional Networking for Millions of Docker Desktops" to the mainline ICFP audience! I first discussed the\npast; how we joined Docker and came up with HyperKit and\nVPNKit to solve\nscaling problems that Docker was facing early in its growth.
\n ![\"%c\"](\"/images/icfp-37.webp\" "\\"Me")
Our experience report makes the broad case for library operating sytems and\nfunctional programming being a good fit, especially with strict languages like\nOCaml which offer thin interfaces to the OS interfaces.
\n\n\nOur use of library-oriented programming to deliver Docker for Desktop is [...] a very\nuseful way to build the "invisible systems glue" code that is pervasively needed in many systems\nprogramming tasks.
\nThere are an ever-growing number of hardware and software interfaces to\naccess the outside world, most obviously with GPUs for machine learning workloads but also\nFPGAs and new storage and persistent memory devices. These usually require significant\nretrofitting to work with existing codebases, and so building translation adapters like VPNkit and\nusing library VMMs like Hyperkit will become more common in the future.\n-- Functional Networking for Millions of Docker Desktops, 2025
\n
And looking into recent specifics, Patrick Ferris's contributions to VPNKit also allow\nthis codebase to move to OCaml 5, and take advantage of direct-style IO! In a nutshell, it lets\nold code like this:
\n![coverage map visualiser](\"https://github.com/ucam-eo/tessera-coverage-map/blob/main/tessera_coverage.png\"){kind=link}
![renders as a high-res SVG](\"http://www.cl.cam.ac.uk/~avsm2/cb2-pumap.svg\"){kind=link}
module Make_packet_proxy\n(I: Mirage_flow.S) (O: Mirage_flow.S) = struct\n let run incoming outgoing =\n let rec loop () =\n I.read incoming >>= function\n | Error err -> fail "%a" I.pp_error err\n | Ok `Eof -> Lwt.return_unit\n | Ok (`Data buf) -> begin\n O.write outgoing buf >>= function\n | Ok () -> loop ()\n | Error err -> fail "%a" O.pp_error err\n end\n in loop ()\n...migrate to direct-style code Eio like this:
\nmodule Proxy = struct\n let run incoming outgoing =\n try\n while true do\n Eio.Flow.copy incoming outgoing\n done\n with\n | End_of_file -> ()\n | Write_error err ->\n fail "%a" pp_write_error err\n | Read_error err ->\n fail "%a" pp_read_error err\nend\nThe old code[4] had monadic concurrency, functors for parameterising the IO\ndrivers, and error handling duplicated across OCaml exceptions and the\nconcurrency monad. The new code uses direct control flow constructs like\nwhile, and also one form of error handling. This is all still a\nwork-in-progress, but looking a solid approach with no blockers except hacking\ntime to get it merged.
Read the ICFP paper to learn more about\nthis, or browse my slides! It's exciting to see\nproduction code here get simpler and faster as we move to OCaml 5. The reasons\nwhy this happens are explored further in my VMIL keynote talk\nthat I gave the next day, where I make a case for runtimes focussing on\npost-POSIX IO! And beyond that, we have OxCaml waiting in the wings\nfor even more performance gains. OCaml is living in exciting times.
\nSee also in the ICFP25 series: chairing PROPL25, the OxCaml tutorial, multicore at Jane Street and Docker, post-POSIX IO and what I learnt.
\n ↩As far as I can tell, REBASE is SPLASH's equivalent of the\nvenerable CUFP series that I helped run earlier in the\ncentury.
\n ↩Our 2020 parallelism paper proposed two minor GC\nstrategies, one of which broke the C FFI and so didn't make the cut in the end\ndue to the amount of ecosystem churn it would cause.
\n ↩A fun historical note is that I gave one of the first talks about VPNKit in Jane Street London about a decade ago! Back then we had a deep discussion about whether to use Lwt or Async, and it looks like we'll now meet again via OxCaml.
\n ↩This is part 2 of 5 of a series of posts[1] about ICFP 2025.
\nSeveral extensions to "oxidize" OCaml (Rust performancew with ML ergonomics!) have been developing rapidly\nin a fork called\nOxCaml. I helped an intrepid crew from Jane Street,\nIIT-M, Tarides, Brown and Cambridge pull together a really fun tutorial in ICFP\n2025 that you can try out too! TL;DR: Work through the online\nslides, try the\nactivities, and take the\nquiz to give us feedback.
\n ![\"%c\"](\"/images/oxcaml-codespace.webp\" "\\"Just")
If you watch Yaron Minsky's talk about moving to OCaml 5, towards the end he discusses the language\nextensions Jane Street needs to eke out every ounce of performance while\nwriting code in a safe, functional OCaml style. These extensions currently materialise in\na forked version of the mainline OCaml compiler. Earlier in the year, a bunch\nof us from Cambridge and Tarides helped\nJane Street get the first public release out the door, and to setup opam\nrepositories and tutorials for this fork to be developed in the open.
\n ![\"%c\"](\"/images/oxcaml-release-summer25.webp\" "\\"An")
Yaron Minsky summarised the OxCaml release thusly:
\n\n\nOxCaml's extensions make OCaml a better language for performance engineering.\nIt also supports data race free parallel programming, and a bunch of other\ngoodies.
\nOxCaml is in an interesting spot. It's an experimental language, whose\nextensions will change quickly and mercilessly.
\nBut it's also a production quality compiler. Indeed it's the compiler we use\nin production everyday.
\nSo why are we doing this? Our primary hope is to use this to build awareness\nabout our work, and to help pave the way for getting these extensions\nupstreamed to mainline OCaml.
\nBut we're also interested in building adoption among enthusiasts and\nresearchers who don't mind working with a language that is changing quickly\nunder their feet. We think there's a ton to learn from the collaboration.\n-- Yaron Minsky on BlueSky, June 2025
\n
So the status in the summer of 2025 was that there existed an open source code\ndrop of OxCaml, but it had a lot of rough edges and it would take some effort\nto make it usable outside of the walls of Jane Street.
\nThe language extensions in OxCaml fall into three broad categories of features that:
\nRight now, all of these are all developed in the OxCaml monorepo by a\ngrowing number of developers within Jane Street. Without access to the\ninternal Jane Street developer resources, KC Sivaramakrishnan and I started to work with\nRichard Eisenberg and Chris Casinghino to submit a tutorial proposal to ICFP; there's no\nbetter way to learn something than a deadline looming over our heads!
\nWriting the tutorial proved much more difficult than I'd expected, as OxCaml\nhas a very large number of extensions\n(such as modes and kinds) that are not only evolving fast, but also sometimes\ninteract differently in combination. It not only requires learning the compiler extensions,\nbut also keeping up with the OxBase libraries that\ncompose them into usable interfaces. KC Sivaramakrishnan and I struggled to pack in all the\ndifferent concepts; I spent a whole weekend just attempting to get a parallel quicksort\ncompiling!
\nLuckily, Richard Eisenberg had the idea to bring in experts in programming language\npedagogy in the form of Shriram Krishnamurthi, Will Crichton and Gavin Gray to save the day.\nGavin completely redesigned our slides to radically simplify the examples\n(beginning with a gensym rather than a sorting algortithm), and also\ndesigned a quiz to test user knowledge before and after. Max Slater, Megan Del Vecchio and\nNadia Razek from Jane Street also leapt in to give us the inside line on new\ndevelopments.
This collaboration made for a nice split in efforts; KC Sivaramakrishnan, Patrick Ferris and I\nfocussed on the longer handson activity examples, while Gavin finished the Slipshow-based slides to deliver the presentation itself. I got a OxCaml GitHub DevContainer working that allowed participants to spin up a full OxCaml environment in just a few minutes, to ensure that as many people could participate during the conference as possible.
\nSome excitement ensued when we realised that there hadn't been a public release\nof the OxCaml packages since our first public release earlier in the year! Meanwhile,\nhundreds of improvements had accumulated upstream, including a number of significant\ninterface and type system changes. It seemed a little regressive\nto present an out-of-date version of the tutorial to a demanding ICFP audience.
\nSo, in a late night call after we arrived in Singapore, with Diana Kalichenko working\ntirelessly on compilation fixes from New York, we refreshed all the tutorial\nexamples and released the latest minus19\ncompiler version of the\ncompiler with four months of developments! We fixed the compilation\nproblems that resulted in our slides, and the new Devcontainer images finally rebuilt\naround 10 seconds before the tutorial started. No sweat.
\n![\"%c\"](\"/images/icfp-4.webp\" "\\"Gavin,")
The tutorial itself held at NUS went fantastically! Both sessions were completely full,\nwith participants online as well (Edwin Torok in particular gave all of it a thorough\nspin on Discord, thanks!).
\nThe broad sweep of audience feedback centred around stuff like this:
\nlocal and stack_ inferred? The compiler does this analysis by default and will locally allocate when possible, but it wasn't clear to the tutorial attendees that this is the case.![\"%c\"](\"/images/icfp-1.webp\" "\\"Full")
The answer depends on if you want to get into compiler and language design or\nnot! The number of mode axes in the OxCaml compiler are growing rapidly as\nmore usecases are covered, so we'll clearly need to develop this tutorial\nfurther, and the "final form" of OxCaml is by no means stable yet. Jane Street\niterates quickly on language changes since they control all the code in their monorepo\nthat uses it, and have extensive production engineering infrastructure. From the outside,\nit'll be hard to keep a large codebase in sync... for now.
\nMy high-level takeaway from discussions with the developers is that there's\nanother 12-24 months of active language evolution left, so it'll continue to be\na moving target for a while. But some features like locals have been around for\nlonger than others features and are more stable. Knowing this\nreally helps to plan out how I'm going to use OxCaml "from the outside".
\nIt's also reassuring to see that Jane Street is serious about investing in\neducation resources for the language as well. Yaron Minsky posts:
\n\n\nWe've had an exciting couple of weeks full of opportunities to teach people about the exciting (and mildly bewildering) features of OxCaml.\nAnd...we're looking to hire an experienced educator to help us in this work. Please share this with anyone you think might be a good fit!\n-- Yaron Minsky on Bluesky, Oct 2025
\n
![\"%c\"](\"/images/icfp-3.webp\" "\\"There")
If you want to have a go at the tutorial and quiz yourself, then it's all still\nopen for participation! Follow the\nslides and then take the\nquiz. And most importantly,\nshare your improbable stunts online so we can see what's going on. I'm hacking on a\nio_uring oxhttpserver myself, and I heard rumours that KC Sivaramakrishnan\nhas been peering into the eBPF sources...
\nSee also in the ICFP25 series: chairing PROPL25, the OxCaml tutorial, multicore at Jane Street and Docker, post-POSIX IO and what I learnt.
\n ↩This is part 1 of 5 of a series of posts[1] about ICFP 2025.
\nThe first outing of PROPL was\nlast year in London, and this time around Dominic Orchard and I invited KC Sivaramakrishnan to be\nthe PC chair and held it at ICFP/SPLASH. The uptake was\nencouraging, and we got enough submissions to have a proper published\nproceedings in the ACM\nDigital Library for the first time! Our proceedings summary\nis a quick read to give you an idea of the breadth of the papers and talks this year.
\n ![\"%c\"](\"/images/propl25-header.webp\" "\\"A")
The workshop itself had slightly less in-person attendance than last year, but\nthis was due to the heavily multi-track structure of ICFP. We had less total\ntime than\nlast year as the morning slot was taken up by the ICFP keynote (the\nawesome Satnam Singh), and with six parallel sessions people were filtering in and\nout of all the events. The online attendance made up for it, with quite a few\npeople tracking the SIGPLAN live\nstream.
\nThe papers were exactly\nwhat I'd dreamed would happen -- a variety of practitioners describing their\ncomputational challenges mixed together with solutions. I'll summarise the day's\nproceedings next!
\n ![\"%c\"](\"/images/icfp-6.webp\" "\\"Dominic")
The first batch of talks I'll cover were about how to specify some core\ncomputational models related to climate and biodiversity science.
\nFirstly, climate modelling was up with Chinmayi Prabhu\npresenting a paper on\nclimate model coupler verification that discussed\nthe difficulty of folding multiple global climate models into combined ones,\nsomething that is normally done via (underspecified and somewhat black magic)\ncoupler\ncomponents. Chinmayi had found some bugs in production coupler code, and\ndescribed a hybrid verification strategy that used both static and runtime\ntechniques to improve the state of affairs.
\n\n\nThe continuous exchange of data through couplers creates the risk of subtle\nerrors propagating across components, potentially distorting scientific\nconclusions. In this paper, we argue for lightweight formal verification\ntechniques applied at the coupler interface to improve both coupler and model\ncorrectness.\n-- Towards Modelling and Verification of Coupler Behaviour in Climate Models
\n
![\"%c\"](\"/images/icfp-11.webp\" "\\"Chinmayi")
Then we heard about hydrology modeling using GPUs over in India, where\nalgorithms to trace the path of surface water flows (e.g. flow accumulation,\nwatershed delineation or runoff simulation) are hard to execute for large areas\nat reasonably fine spatial and temporal resolutions.
\n\n\nLibraries like GDAL that use multi-threaded CPU-based implementations running\non a single host may be slow, and distributed infrastructures like Google\nEarth Engine may not support the kind of computational primitives required by\nthese algorithms.
\nWe have developed a GPU-accelerated framework that\nre-engineers these four algorithms and is able to process areas as large as\nriver basins of 250,000 km2 on commodity GPU workstations.\n-- GPU-Accelerated Hydrology Algorithms for On-Prem Computation
\n
Aadi Seth explained not only the basics of flow algorithms, but why it's\nessential to GPU accelerate them to get a reasonable spatial scale and\nresolution. They ended up with a set of small parallelizable primitives that are either\npixel independent, or a sequence of 'long pixel' operations that need more\ncoordination.
\n ![\"%c\"](\"/images/icfp-13.webp\" "\\"Aadi")
Continuing on in the previous theme of novel computation models, Michael Dales presented "Yirgacheffe: A Declarative Approach to Geospatial Data",\na new Python library that\nallows spatial algorithms to be implemented concisely and supports parallel\nexecution and resource management. (read more...)
\nMichael used our work on the LIFE biodiversity metric as the\nmotivating usecase for his demo, as we have petabytes\nof rasters processed using Yirgacheffe. He also set up a nice new\nhomepage for the project and used PROPL to launch it,\nincluding adopting a shiny new\nexecutable notebook called Marimo which I liked the look\nof!
\n ![\"%c\"](\"/images/icfp-36.webp\" "\\"Michael")
Switching tack from computation to managing large-scale datasets, we had a\nnumber of papers discussing how to orchestrate these both for full execution\nbut also in a developer friendly way for local use.
\nThe first extremely ambitious talk was from Jean-Michel Lord who\npresented "Programming Opportunities for the Global Biodiversity Observation Network". GEOBON is a global network of researchers dedicated to improving the acquisition, coordination and delivery of biodiversity information at a global scale. This PROPL collaboration came up up after I met Andrew Gonzalez at the\nNAS earlier in the year and learnt about BON-in-a-Box, which uses Docker under the hood!
\n ![\"%c\"](\"/images/icfp-8.webp\" "\\"Jean-Michel")
Jean-Michel described how off-the-shelf software could (almost) be enough to integrate the world's biodiversity dataset pipelines, but needed some help from their maintainers. Most notably, BON-in-a-box facilitates peer review of computation pipelines (as opposed to the science underpinning them), which is the first time I've seen peer review applied to scientific code. This "connecting the dots" across diverse biodiversity datasets is vital towards building a comprehensive model of life on this planet, and computer science is a crucial piece of the puzzle to make sense of all the data. Andrew Gonzalez also just announced that all the major players are getting together later this month at the Living Data 2025 conference in Colombia, further underlying how timely getting involved with BON-in-a-Box is.
\nAadi Seth had a second paper on the challenge of building spatio-temporal catalogue dataflow\ngraphs. These\ncatalogues are really important in environmental\nscience to connect the dots in the often "gappy" field\ndatasets. The STAC-D extension proposes dataflow pipelines to help compose\nthese together via YAML specifications that describe dependencies between the\n(large) datasets involved.
\n\n\nWe propose STACD (STAC extension with DAGs), an extension to STAC\nspecifications that incorporates Directed Acyclic Graph (DAG) representations\nalong with defining algorithms and version changes in the workflows. We also\nprovide a reference implementation on Apache Airflow to demonstrate STACD\ncapabilities such as selective recomputation when some datasets or algorithms\nin a DAG are updated, complete lineage construction for a dataset, and\nopportunities for improved collaboration and distributed processing that\narise with this standard.\n-- STAC Extension with DAGs
\n
This one really reminded me of the work I did ages ago with Derek G Murray and Malte Schwarzkopf\non the CIEL execution engine. Dynamic dataflow graphs seem\nto come up again and again as the beating heart of coordination languages, and\nwe've made surprisingly little progress in making them more ergonomic. Some\ncombination of Docker and Build Systems à la Carte would\ngo a long way here!
\nThe third talk was from the the Catalysts Foundation\nfrom India, and highlighted the importance of data science for promoting health\nand wellbeing in some of the most vulnerable rural communities, who are at\nserious risks of increasing intensity due to climate change.
\n\n\nClimate change presents multifaceted public health challenges, from\nheat-related mortality and vector-borne disease expansion to water\ncontamination and respiratory ailments. The 2022 Lancet Countdown Report\ndemonstrates a host of health effects of climate change ranging from\nheat-related illness and mortality to the spread of vector-borne and\nwater-borne pathogens, to rising food insecurity as cropping patterns change.\nCurrent public health systems lack integrated, real-time data capabilities to\nidentify vulnerable populations and coordinate timely responses to these\nclimate-induced health threats, particularly in resource-constrained\nsettings.\n-- Precision Action Towards Climate and Health (PATCH)
\n
![\"%c\"](\"/images/icfp-9.webp\" "\\"Prerak")
Prerak talked about the difficulty of combining geospatial data with machine\nlearning inference, and keeping track of the resulting outputs in a systematic\nway. What I found particularly interesting about their "PATCH" system is that it not\nonly has core computing facilities (a health reporting platform, a spatial counterfactual\nmap for interventions and a communications channel for different stakeholders), but they\nalso extensively partner with local state governments in India (like the India Meteorological Department and the All India Institutes of Medical Sciences.
\nPatrick Ferris formalised many of the above problems with his thoughtful seminar on "what we talk about when we talk about scientific programming". Patrick came up with three axes along which the scientific method of hypothesis falsification needs to operate when dealing with computational data: dynamism, scale and access controls. If you'd like to read more about Patrick's thoughts on this topic, his paper last year on "Uncertainty at scale: how CS hinders climate research" is also well worth a read.
\n ![\"%c\"](\"/images/icfp-31.webp\" "\\"Patrick")
All these talks highlighted the difficulty of managing large and often very messy datasets in practise. So how do we move towards more principled platforms to fix the situation? That's what the next group of talks covered!
\nBy far my favourite aspect of PROPL was he sheer ambition on display when it\ncomes to leveraging the network effects around computer technology to\naccelerate the pace of environmental action. The next batch of papers is all\nabout evolving notebooks to be global scale!
\nCyrus Omar led the charge with his call to action for "A FAIR Case for a Live Computational Commons", which\nhe described as a live planetary wiki! What is needed to turn something like\nWikipedia into a live, updated computation graph with incremental, real-time\nvisualisation? This would effectively be one giant program running and being\nupdated by tens of thousands of contributors in real time.
\n\n\nThis paper proposes Fairground, a computational commons designed as a\ncollaborative notebook system where thousands of scientific artifacts are\nauthored, collected, and maintained together in executable form in a manner\nthat is FAIR, reproducible, and\nlive by default. Unlike existing platforms, Fairground notebooks can\nreference each other as libraries, forming a single planetary-scale live\nprogram executed by a distributed scheduler.\n-- A FAIR Case for a Live Computational Commons, Omar et al 2025
\n
Many of the answers to how to do this lay in the talks at this week's\nICFP/SPLASH: programming languages with clean semantics for incremental\ncompilation, purely functional with effect tracking, and mergeable semantics\nfor managing large scale data structures. Cyrus' own Hazel\nlanguage is a perfect example of an ergonomic interactive language that has\nclean, functional semantics while retaining usability.
\n ![\"%c\"](\"/images/icfp-32.webp\" "\\"Cyrus")
Roly Perera then switched tack and\ndiscussed how we might achieve more\ntransparent climate reporting, via a new class of "transparent programming\nlanguages" like his own Fluid project.
\n\n\nWith traditional print media, the figures, text and other content are\ndisconnected from the underlying data, making them hard to understand,\nevaluate and trust. Digital media, such as online papers and articles,\npresent an opportunity to make visual artifacts which are connected to data\nand able to reveal those fine-grained relationships to an interested user.\nThis would enable research outputs, news articles and other data-driven\nartifacts to be more transparent, self-explanatory and explorable .\n -- fluid, explorable, self-explanatory research outputs
\n
Roly showed in his talk how the latest advances in Fluid helped to automate\nproviding "drill down" explanations for topics in the energy transition and\ndecarbonisation, adaptation to climate change, or risk mitigation strategies\nthat required policy changes justified by data.
\n ![\"%c\"](\"/images/icfp-30.webp\" "\\"Roly")
The Fluid website has loads of interactive examples\nfor you to explore, so continue on there if interested in this topic.
\nCristian Urlea then discussed the crucial problem of Bridging Disciplinary Gaps in\nClimate Research, since the\npoint of all these planetary computing systems is to make them accessible to\nnon-computer-science-expert users (the "vernacular programmers").
\n\n\nCurrent scientific computing practices pose major barriers to entry,\nparticularly for interdisciplinary researchers and those in low and\nmiddle-income countries (LMICs). Challenges include steep learning curves,\nlimited access to expert support, and difficulties with legacy or\nunder-documented software. Drawing on real-world experiences, we identify\nrecurring obstacles in the usability, accessibility, and sustainability of\nscientific software.\n-- Bridging Disciplinary Gaps in Climate Research, 2025
\n
I greatly enjoyed the framing of this paper of "reimagining scientific\nprogramming as a shared public good", a point also made by Roberto Di Cosmo\nrecently in his Nature comment that we must stop\ntreating code like an afterthought, and instead record, share and value it.
\nOnce we have all these planetary scale notebooks, what sorts of new programs\nmight we run on them? The last group of talks covered some radically different\nideas here, and I was involved with all of them!
\nFirst, Sadiq Jaffer and Robin Young presented our own TESSERA project, which is a new geospatial foundation model for programming with observations of the planet from space. This is pretty scifi stuff!
\n\n\nRemote sensing observations from satellites are critical for scientists to\nunderstand how our world is changing in the face of climate change,\nbiodiversity loss, and desertification. However, working directly with this\ndata is difficult. For any given satellite constellation, there are a\nmultitude of processed products, data volume is considerable, and for optical\nimagery, users must contend with data sparsity due to cloud cover. This\ncomplexity creates a significant barrier for domain experts who are not\nspecialists.
\nPre-trained, self-supervised foundation models such as TESSERA aim to solve this by offering pre-computed\nglobal embeddings. These rich embeddings can be used in-place of raw remote\nsensing data in a powerful “embedding-as-data” approach. For example, a\nsingle 128-dimensional TESSERA embedding for a 10-meter point on Earth can\nsubstitute for an entire year of optical and radar imagery, representing its\ntemporal and spectral characteristics. While this could democratise access to\nadvanced remote sensing-derived analytics, it also creates a new programming\nchallenge: a lack of tools designed for this new approach.
\n-- Building a Usable Library for Planetary-Scale Embeddings, Jaffer 2025
\n
![\"%c\"](\"/images/icfp-12.webp\" "\\"Sadiq")
Sadiq did a live demo by demonstrating the\nGeotessera Python library I've been\nhacking on. He went one step further and coded up a\nsimple classifier that found most of the solar farms on the planet, with a demo\nthat ran in minutes on his laptop!
\nAfter this, Andres Zuñiga-Gonzalez showed how he's been using airborne data to uncover socioeconomic stratification of urban nature in England, all via a pipeline that combines all sorts of geospatial algorithms to distill metrics about modern urban life.
\n\n\nUsing high-resolution LiDAR (Vegetation Object Model), Sentinel 2 imagery,\nand open geospatial datasets for over 28 million buildings across England, we\nintegrate raster, vector, and socioeconomic data within a scalable\ncomputational framework. Tree segmentation was performed using adaptive\nlocal-maximum filtering, canopy cover estimated at 1 m resolution, and park\naccessibility derived from network-based walking distances.
\nInequality in access to nature was quantified via Gini coefficients and\nmodelled with spatial error regressions against socioeconomic deprivation.\nOur results reveal that while most urban areas meet the 3-tree proximity\nrule, fewer than 3% achieve 30% canopy cover, and only a minority satisfy all\nthree components simultaneously.\n-- Airborne assessment uncovers socioeconomic stratification of urban nature in England
\n
![\"%c\"](\"/images/icfp-14.webp\" "\\"Andres")
You can read more about this in his preprint that\nexplains in detail the programming pipeline (including route finding across\ntree canopies across the whole of England), and the bigger picture behind\n3-30-300 mapping.
\nAnd last but definitely not least, Ryan Gibb presented possibly the most topical\ntalk for a programming language venue: repurposing the classic Robin Milner\nbigraphs for environmental goodness!\nRyan (and Josh Millar) observe that existing distributed computing models poorly\ncapture spatial structure, hindering dynamic collaboration and access control.\nThey argue that that space must be treated as a first-class concept in\nprogramming models, and use this to better coordinate the complex sensor\nsystems we need for environmental science.
\n ![\"%c\"](\"/images/icfp-35.webp\" "\\"Ryan")
It was also wonderful to see Roy Ang attend the workshop, as it was his Part II project which imported\nOpenStreetMap into BigraphER that sparked off the idea in the first place.\nYou may also be interested in our preprint "An Architecture for Spatial Networking" that explores this concept of spatial networking in more detail.
\nAs always, the corridor track of discussions after the conference was the most valuable part of attending PROPL. We had the opportunity to put up some posters during the main banquet session, and it was busy!
\n\nOne thing that leapt out at me from the discussions was the need for a hosted service with the ergonomics of Docker, the interactive flexibility of Jupyter, the peer community of Wikipedia, and the semantic cleanliness of Hazel. This is overwhelmingly difficult to do in a topdown manner, but we do have now a growing community of practitioners and computer scientists who share a vision of making this happen. I had long conversations with most of the attendees of PROPL about how we might make this a reality, and my plan is spend a good chunk of my sabbatical time this year hacking on this.
\nThe other really energizing thing was seeing all the side hacking going on. I\nspotted Patrick Ferris porting all the Python Geotessera code over to OCaml in a\nskunkworks effect of which I heartily approve. I looked over the Hazel UI with\nCyrus Omar to figure out what a planetary view might look like. Aadi Seth figured\nout spatial datastructures for the CoRE stack with Ryan Gibb and Andres Zuñiga-Gonzalez while we\nstood around in the blazing Singapore heat. Michael Coblenz lead a session at\nHATRA\non user centric approaches to types and reasoning assistants.
\nAll of these strands could weave together powerfully into groundup systems that\nsolve the problems we've been defining at PROPL! I'm feeling\nenergized and tired, and look forward\nto continuing the discussions started in London (2024) and Singapore (2025)\ninto real systems in 2026!
\n![\"%c\"](\"/images/icfp-33.webp\" "\\"Ilya")
![\"%c\"](\"/images/icfp-34.webp\" "\\"As")
See also in the ICFP25 series: chairing PROPL25, the OxCaml tutorial, multicore at Jane Street and Docker, post-POSIX IO and what I learnt.
\n ↩A world without nature feels rather impermanent, doesn't it? It's difficult to\nimagine a healthy future without clean air, fresh water and diverse wildlife.\nYet important policy is being decided at\nthe moment that will sideline "nature-based solutions" for net-zero carbon\ntargets. While it is true that anything involving nature is fundamentally less\npredictable than human edifice, it is not true that it can't be\nquantified through science-based methods! Advances in\nremote sensing mean we have better resolution views into nature than ever\nbefore in human history, and we can leverage those towards protecting what's\nleft. The wrong economic incentives are pushing us into a dangerous crossroads where several policy paths effectively abandon\nnature.
\nBack in January 2024, I hosted a workshop on permanence and\ndurability\nat Pembroke College attended by sixty experts in this topic, with their\nrecommendation reflected in the continuous improvement report on\npermanence\nfrom the ICVCM in May this year. I now join 40 other colleagues today in\nsigning an open letter to the UN Article 6.4 supervisory body strongly calling\nout the scientific imperative to incentivise natural climate solutions on the path\nto net zero.
\nA number of proponents of climate action policy are advocating for only\nconsidering 'entirely durable' carbon sequestration as the ones we should\naccount for. That usually means carbon sequestration technologies, and rules\nout nature based solutions due to arbitrary thresholds. However, temporary\nremovals can deliver immediate climate benefits which are urgently needed,\nand we've proposed several schemes of\nequivalent permanence accounting along with\nothers that propose\ncredible paths forward.
\n![\"%c\"](\"/images/iiasa-2.webp\" "\\"Sir")
I argue instead for a principle of parallel action: we need to invest in both\nfully durable carbon sequestration technologies as well as nature based\nsolutions. Humanity can no long afford to take single bets; warming and\nbiodiversity loss has advanced to the point where concerted parallel action is required\nto overshoot sufficiently to bring balance back. Setting aside nature at this\ncritical juncture is unbelievably short-sighted, and the incentives we\nestablish for nations this decade will echo through the remainder of the\ncentury. The future of food security, arresting\npandemic spread and soil\nhealth all depend on\nprotecting nature.
\nIf you agree, please read the open letter and sign up. See also this great explainer about nature-based durability.
\n![\"%c\"](\"/images/iiasa-1.webp\" "\\"John")
Choosing where we source the food that we eat makes a difference to the\nenvironment, but by how much? After churning through around 100 petabytes of\ndata, beginning with our LIFE metric and moving onto food\nprovenance maps and import/export data for the world, we now know the answer\ncan vary by three orders of magnitude for species\nextinction risks.
\nOur paper in Nature Food came out today with\nall the tasty details and implications for food policies worldwide. In order to make the data easier\nto explore, I knocked up an interactive global explorer\nusing the data that the team (lead by Thomas Ball) generated.
\n
The methodology is described in the paper:
\n\n\nThe marginal impact of current food consumption on biodiversity can be viewed\nas the forgone opportunity to restore biodiversity arising through ongoing\nagricultural land use. By linking LIFE with spatial crop and pasture\ndistributions, consumption, production and trade data from the Food and\nAgriculture Organisation of the United Nations, [...]\nwe quantify the > opportunity cost to biodiversity of producing or consuming 1kg of each\nFAO-aligned food commodity in 174 countries, taking the feed and grazing\nrequirements for animal products into account.\n-- Food impacts on species extinction risks can vary by three orders of magnitude
\n
The results themselves are absolutely fascinating. Firstly, ruminant meat has\nmedian consequences 100 times higher than that of legumes and pulses, even when\naccounting for the high protein content. That burger, while delicious, has an\nenormous biodiversity cost somewhere in the world! Bear in mind that more than\n3/4 of human-appropriated land surface on the planet is dedicated to the\nproduction of animal products while\nproviding only 17% of global calories.
\nBut if you really want a burger, then choosing where you source the meat\nfrom makes a huge difference as well. There is a great deal of spatial variance\nabout the selection of each commodity.
\n ![\"%c\"](\"/images/food-life-globe-2.webp\")
The vertical length show the spatial variance. It gets more interesting when you break it down by country and examine the imports and exports of each one:
\n ![\"%c\"](\"/images/food-life-globe-3.webp\")
The biodiversity leak becomes clear now! Some countries do consume enormous amounts of meat, but do so from domestic production (i.e. a local political decision). Others, like the UK or Japan, import vast amounts of damaging food from other countries which are highly biodiverse, resulting in the demand that is one of the primary drivers of deforestation.
\nThe data is out there now, under a CC-BY-SA license, and the interactive FOOD explorer allows you to break down impacts per country and look through 150 different foodstuffs.
\nI also mentioned the vast amount of data processing involved in coming up with these numbers; Michael Dales has uploaded the calculators over to our quantifyearth/ code organisation, and will be presenting the magic library Yirgacheffe that drives much of this processing at PROPL25 in October. Tune in if you'd like to hear more about the computer science side of this sort of analysis!
We've been having great fun at the EEG\nrecently releasing embeddings of\nour new TESSERA geospatial foundation\nmodel.
\n\n\nTESSERA is a foundation model for Earth observation that processes Sentinel-1\nand Sentinel-2 satellite data to generate representation (embedding) maps. It\ncompresses a full year of Sentinel-1 and Sentinel-2 data and learns useful\ntemporal-spectral features.\n-- Temporal Embeddings of Surface Spectra for Earth Representation and Analysis
\n
A foundation model is\ndesigned to be used for downstream tasks without having to retrain a full model\nfor every individual task. Our preprint paper\ndescribes what sorts of geospatial tasks you can solve more quickly, ranging from crop\ntype classification, forest canopy height estimation, above-ground biomass\ncalculations, wildfire detection, forest stocks, and many more.
\n
TESSERA is an open model that is trained only on public satellite data (thanks\nESA!),\nand we make tiled embeddings available for download that have 128-dimensional\nvectors precomputed for every 10m2 surface of the planet. This makes using\nTESSERA really simple in existing GIS workflows.
\nTo make it even simpler, I've just published a new Python library called\ngeotessera which provides a\nprogrammatic and CLI interface to accessing these.
\n![](\"https://raw.githubusercontent.com/ucam-eo/tessera-coverage-map/refs/heads/main/geotessera.gif\")
The full set of TESSERA embeddings are petabytes when generated[1], so it's important that you download just the ones\nyou need for a given region of interest. We chunked up the embeddings into image tiles where each pixel represents a 10m2 are, and are hosting these at the Computer Lab on dl.geotessera.org.[2] Geotessera uses Pooch to build up a registry of manifests in Git, and provides helper functions to calculate which tiles you need.\nautomate.
\nYou can take this for a spin very quickly if you have uv installed. First, let's check global coverage of what's available:
\nuvx geotessera coverage\nThis will drop a figure like the below into tessera_coverage.png. You can also refine the map to an area of interest by passing in a GeoJSON, shapefile or manual bounding box to the command-line arguments.
![](\"https://raw.githubusercontent.com/ucam-eo/tessera-coverage-map/refs/heads/main/map.png\")
We're still churning through the inference (and prioritising areas of interest for our early adopters), so the green spots represent full coverage from 2017-2024, the blue represents just 2024, and orange for in-between.\nNow, we want to download the embeddings themselves. Let's do Cambridgeshire:
\nuvx geotessera download \\\n --output cb \\\n --region-file https://raw.githubusercontent.com/ucam-eo/geotessera/refs/heads/main/example/CB.geojson\nThis will drop a bunch of GeoTIFFs into the cb/ directory which you can inspect using GDAL in the normal way. Note that the local UTM coordinates are preserved (this varies by latitude) and that there are 128 bands per TIFF.
% gdalinfo -stats cb/tessera_2024_lat51.95_lon0.05.tif\n<...>\nPixel Size = (10.000000000000000,-10.000000000000000)\nMetadata:\n GEOTESSERA_VERSION=0.5.1\n TESSERA_DATASET_VERSION=v1\n TESSERA_DESCRIPTION=GeoTessera satellite embedding tile\n TESSERA_TILE_LAT=51.95\n TESSERA_TILE_LON=0.05\n TESSERA_YEAR=2024\n AREA_OR_POINT=Area\nImage Structure Metadata:\n COMPRESSION=LZW\n INTERLEAVE=PIXEL\nCorner Coordinates:\nUpper Left ( 293612.221, 5765288.255) ( 0d 0'24.03"W, 51d59'59.39"N)\nLower Left ( 293612.221, 5753888.255) ( 0d 0' 0.61"E, 51d53'50.90"N)\nUpper Right ( 300932.221, 5765288.255) ( 0d 5'59.33"E, 52d 0' 9.01"N)\nLower Right ( 300932.221, 5753888.255) ( 0d 6'23.09"E, 51d54' 0.49"N)\nCenter ( 297272.221, 5759588.255) ( 0d 2'59.76"E, 51d56'59.99"N)\nBand 1 Block=256x256 Type=Float32, ColorInterp=Gray\n Description = Tessera_Band_0\n Minimum=-4.017, Maximum=11.446, Mean=3.649, StdDev=2.089\n Metadata:\n STATISTICS_MINIMUM=-4.0171508789062\n STATISTICS_MAXIMUM=11.445509910583\n STATISTICS_MEAN=3.6492421642927\n STATISTICS_STDDEV=2.088705775134\n STATISTICS_VALID_PERCENT=100\nOnce you have the GeoTIFFs locally, you can drop into your normal GIS workflows. But\nyou can also continue to use the CLI to do false colour visualisations, for\nexample using PCA, to help visualise what's going on.
\nuvx geotessera visualize cb cb.tiff\nuvx geotessera serve cb.tiff --open\nThese two commands will first output an RGB mosaic of the tiles (false colour, like the one at the start of this post), and then tile them using LeafletJS so you explore them with OpenStreetMap in the background.
\nAt this point you are probably itching to do some actual machine learning. You can try out the Tessera interactive Jupyter notebook next!
\ngit clone https://github.com/ucam-eo/tessera-interactive-map\ncd tessera-interactive-map\nuv venv\nsource .venv/bin/activate\nuv pip install -r requirements.txt\ncode app.ipynb\nThis will spin up the environment in VS Code as a notebook, where if you run the cells you get an interactive bounding box that you can use to do manual classification by simply marking labels. Here's a video that demonstrates this, courtesy of Robin Young:
\n\nHave fun with this! All of this is really bleeding edge stuff, so if you run into issues (likely) then please do file an issue and let us know. In fact, let us know if you build something where you didn't find any bugs, too!
\nCoding in Python again after a few years has been a fun experience for me, but\nI'm yearning to return to OCaml again. Accordingly, I've been building out an\nimplementation of GeoTessera in native OCaml, using\neio. This is also a perfect usecase\nfor oxcaml extensions to speed up floating point\nprocessing, and Thibaut Mattio has just published\nRaven for handling numpy format arrays in\nOCaml. Stay tuned for more on that...
\nThere's also plenty to be done on improvements to GeoTessera; I'll be adding in\nmodules to help with machine learning workflows as soon as the external uses of them\nhave stabilised a bit more. I'm also enjoying getting re-familiar with modern\nPython tooling. uv is a remarkable piece of work, but I'm still figuring out how to\n(e.g.) run notebooks directly using it without running into package issues.
And finally, storage management remains a real headache as we are striping and syncing hundreds of terabytes of storage and keeping it\nperformant. As we go back to generate\nembeddings for earlier years, we'll be hitting petabytes easily. While the normal\nanswer is to store this on a cloud, the problem is the egress bandwidth is hugely\nexpensive, and it's imporant we have a local storage cluster for this. Any tips\non how to build out a cheap such cluster are welcome!
\nWe ran the inference on a combination of AMD MIX300 and the Dawn cluster you may have seen me talking about on the BBC a while back.
\n ↩The humungous number of SSDs are jury-rigged onto machines kindly donated to the Lab by Jane Street.
\n ↩That's a wrap for the next decade with Aarhus 2025,\nwhere I presented our paper on "Steps towards an Ecology for the Internet".\nI was a little unsure about how to approach the presentation, largely because\nthe ideas seem a little crazy if they'd been proposed even a year ago! Luckily\nmy co-authors strengthened my spine with encouragement and gin,\nand the event was tremendous fun packed with useful insights.
\nOur key observation is that the Internet is dangerously ossifying into monocultures\nat multiple levels. Inspired by wild ecosystems, we're proposing mixing in more natural\nselection\ninto edge deployments by using AI code models to mutate end-hosts and tailor\nthem to their environment. Generative AI is notoriously\nunpredictable,\nwhich turns out to be a useful property if you actually want more local\nsoftware diversity! For example, this lets us cook up "antibotty" networks that\nfight back against global viruses via locally adapted vigilantes (antibodies).
\n ![\"%c\"](\"/images/aarhus-1.webp\")
Beyond just making thing more resilient, injecting more software diversity gives\nus the hooks to make computers in our environment do what we actually want\nthem to.\nTo quote Mark Weiser, "the world is not a desktop", and computers\nshould be so ubiquitously blended into our day-to-day lives that they are\ninvisible to the user. We're\ngetting further away from that dream every day with the monotony of the FAANG\nsoftware monocultures, and a wild west of botnets\nsweeping through billions of devices.
\nSince this conference only happens once a decade, I put myself in the right mindframe by reading through my old ideas and seeing how they had aged.\nI was struck by how much Databox (2015), Droplets (2011) and\nDigital Yurts (2009) all stood up surprisingly well[1] in 2025.\nThis time around, we have a fresh edge with the rise of coding models and a relative glut of edge computation. The question is how to harness these new technologies for the health of the Internet and not yet more central lockin.
\n![\"%c\"](\"/images/aarhus-4.webp\" "\\"The")
The talks were arranged in a panel with three other great speakers, who discussed crises through the lenses urban gardening, computing supply chains and two-loops models of change. There were some thought-provoking questions from the audience!
\nFirstly, is framing these questions as a "crisis" just saturating us with a constant bombardment of problems we need to react to? Should we be building more (emotionally and systemically) sustainable platforms for engendering change? I certainly agreed with this, but I don't yet have a clear sense of what this means, beyond finding Our World In Data an inspiration of how fun data exploration can be.
\nA couple of things I need to follow up on reading:
\n![\"%c\"](\"/images/aarhus-5.webp\" "\\"Waving")
The stage discussion then veered into the role of lifecycles in these cultural\nsystems. An audience member asked about the role of biological mutualism and\ncooperation in any future digital framework, and I pointed out\nsome\nexamples from our paper\nabout how cooperative ensembles in nature can be very stable, but also do not\nhave to last forever. When applying this "nihil aeternum est" principle to\nhuman systems, how sacrosanct are concepts like "democracy" as we move forward?\nIf our views on these institutions remain unchanging, then they will also\nbecome brittle and collapse as the context in which they operate changes into a hot\ncrowded world mired in polycrises.
\nThis let me bring out an idea I've been ruminating on for a while. While the principles of equal suffrage are vital, one dimension where we could relax it is for intergenerational representation. One issue with taking political decisions for the long-term is that wealth pools with the old, who have far less to gain from long-term thinking than the young[2] voters. So why are the young so underrepresented? Imagine a completely made up voting system that looked like this:
\n\n\nA voting system assigns each individual a number of votes inversely\nproportional to their age. An 18-year old will have the maximum number of\nvotes, and they gradually degrade until anyone age 70 or higher gets just one\nvote. The number of votes decay rapidly with age, so that every year the new\n18-year old cohort will control (say) 40% of the total vote. This sort of\nsystem ensures that the older generations (where wealth pools) must educate\nthe newly minted voters every year, or risk losing control of their agendas.\n-- Anil's entirely made up voting system
\n
I've found some variations of the theme, like Demeny voting that gives parents a proxy\nvote for their children or even cases for disenfranchisement of the elderly.\nBut neither of these quite capture what I have in mind, which is to build\nintergenerational education firmly into how our society operates. If 40% of\nthe entire voting block appears newly every year, then education on civic\nmatters need to happen like clockwork and be incorporated into our curriculums.
\nBut of course, there are huge barriers to trying out these experiments in civic\nsociety. But there are no such barriers to running these experiments in\nmicrocosms of common pool\nresources\nor even on digital systems. So after these fascinating conversations\nat the conference, I'm going to think about the Cambridge Green Blue and apply it there.
\n![\"%c\"](\"/images/aarhus-3.webp\" "\\"The")
I was also struck by how far behind self-hosting is, even among an audience\nthat should be heavily in favour of it. I think my talk was one of only a few\nthat mentioned BlueSky and the Fediverse, and alternative communication\nmechanisms. I also demoed Claude in a corner to show how it could help manage\ninfrastructure that would\nordinarily take a sysadmin, but could now be reasonably handled by a non-expert\n(with care!).
\nOne of the attendees commented to me afterwards that they remembered the\nDatabox talk from a decade ago, and wondered why it\nhadn't taken off. Maybe now is the time for Ryan Gibb's work on digital\nislands to hit the mainstream!\nThey've certainly never been needed more than now; I am deeply glad to\nsee my colleagues like Jon Sterling also working on solutions\nin this space.
\nIn the spirit of self-hosting, I also used the great new\nSlipshow tool (that uses\njs_of_ocaml) to write the\npresentation. Slipshow lets the presentation be written in\nMarkdown,\nand I used Claude Code to handle all the styling for me. The whole presentation\ntook about an hour to put together, and can be viewed\nstandalone as a single web page as it\ninlines all the assets.
Using HTML/JS/CSS for talks is really convenient, so I'm sold on using Slipshow\nfor my upcoming presentations this year! It's also excellent to be using\njs_of_ocaml. I think the only thing on my "wishlist" is to be able to run a\nheadless browser and output PDF snapshots of each of the slips. I'm also not\nyet sure how my 100MB videos will encode, but I'll figure that out ahead of my\nnext talk (in the Royal Society at the start of September to the Austrian\ngovernment). The author of the software, Paul-Elliot also\nkindly reached out to get feedback, and I was really pleased to see his work\nwas supported by the NGI Commons Fund!
![\"%c\"](\"/images/aarhus-8.webp\" "\\"Slipshow")
Aarhus is also a spiritual twin city to Cambridge. It was a gorgeously sunny\nweek, with bicycles available everywhere and a lovely Latin quarter to hang out\nin.
\n![\"%c\"](\"/images/aarhus-2.webp\" "\\"Hanging")
The venue itself at Aarhus University was really nice to explore and see what\nthe students are up to. Lots of music and creative arts in the same area. When\nI was here last year to present at\nMatchpoints\nwe were at the Modern Art museum, and the amount of new building work in the\ncity was remarkable to see. There was also a lovely forest park I went for an\nearly morning jog in to get some nature!
\n![\"%c\"](\"/images/aarhus-7.webp\" "\\"Excellent")
Not coincidentally, most of these ideas were cooked up with Jon Crowcroft somewhere in the picture at a nearby pub.
\n ↩We used the ideas of discount factors to adjust for impermanence in our forest carbon work in Nature Climate Change a few years ago too.
\n ↩Since I wrote about the new ATProto-powered Tangled Git forge a few months ago, it's come along by leaps and bounds!
\nFirst, and most excitingly, they've added continuous integration via Spindles which are built in a nice ATProto style:
\n\n\nWhen you push code or open a pull request, the knot hosting your repository\nemits a pipeline event (sh.tangled.pipeline). Running as a dedicated service,\nspindle subscribes to these events via websocket connections to your knot.
\n
The pipelines are Nix-only right now, so I braved using it[1] for a new GPS Exchange Format library in OCaml that I wrote. The pipelines should look familiar, and the description format very straightforward.
\nSecondly, the service has added support for JJ stacked pull requests, which are the closest I've seen to the Jane Street Iron diff workflow which I've been wanting to try in open source for ages. You can see the interdiff review process on a recent PR by Winter who add support for engine-agnostic Spindle workflows, which should pave the path for a Docker or BuildKit engine alongside the existing Nixery-based one.
\nAnd thirdly, the general quality-of-life of the web frontend has improved dramatically, with a nice timeline, repo list, and profile pages. I'm running two knots right now (one on Recoil, and one in the Cambridge Computer Lab), and both have been very painfree. I wrote one of the earliest Dockerfiles for it, but there's now a community-maintained Knot Docker setup which I've switched to. Doesn't take very long at all; give it a try!
\nBecause I've been using Tangled so much, I added support for Tangled metadata to Dune to make OCaml package maintainence easier. This will appear in Dune 3.21 in a few months, but in the meanwhile enjoy the first Tangled.sh package on opam. It's a simple GPX library I used in my recent trip to Botswana. All you need in your dune-project will be:
(lang dune 3.21)\n(name mlgpx)\n(generate_opam_files true)\n(source (tangled @anil.recoil.org/ocaml-gpx))\nThe only major thing I'm missing from Tangled is support for private repositories now, but I'm very content using it for public content today. Beware as usual that it's still in alpha, so don't trust super-ultra-mega-important stuff to it unless you've git mirrored elsewhere.
\nI'm emerging reenergised from an epic trip to the\nOkavango Delta in Botswana, where we spent\nweeks\nin the wilderness gathering ground truth for TESSERA\n(and enjoying the wildlife!). Piecing together our locations was quite\nimportant, and so I took a cue from Ryan Gibb and deployed\nOwnTracks and HomeAssistant Device\nTracker before I\nheaded out there. There were four interesting tech pieces that resulted from\nthis: a local iPhone app to determine my GPS accuracy while entirely remote;\nthen merging my Home Assistant location database hile in the field,\nthen reverse engineering 8 years worth of location data out of an old iOS app to backfill data,\nand finally deploying my own self-hosted OwnTracks on Recoil for the longer term.
\n![\"%c\"](\"/images/okavango-1.webp\")
Device power is at a premium while in the remote wilderness, and a lot of the\nusual methods that iOS uses to figure out precise locations (such as Wifi SSIDs)\nsimply don't exist there. So I need some way to figure out if our devices actually\nhad a location lock, and if so what the error was (so I could wiggle my phone in\nthe air until it got a lock).
\nSince we had utterly minimal connectivity, I managed to text Nick Ludlam back\nat London home base to seek his help. Nick\nvibed an iPhone\nPWA that uses heic.js to dump\nout all the Exif tags for a given photograph locally on my device. This solved\nthe immediate problem while working entirely offline! You can grab its\nsource for yourself or try it\ndirectly.
\nI've used Home Assistant for many years to manage\nmy household devices. They supply a handy iOS\napp which uses Location\nServices to keep track of where you are, in order to trigger events ("Turn on\nthe heating when I arrive home"). When I was out in Botswana, we had a bunch of\ncameras with us such as Canon R5 MKII and an Olympus E-M1 MKII. These cameras\ndon't track GPS reliably, which makes them a pain to realign with iPhone and\nother footage after the fact.
\n
Luckily, I realised that my Home Assistant iOS app is actually recording its\nlocation locally to my device, and very occasionally syncing it back home on\nthe rare occasions where we had signal (we discovered that by standing on top\nof a old termite mound and holding your phone high in the air you could\noccasionally get one bar of connectivity for important texts).
\nI retrieved my Home Assistant database from Cambridge (via Tailscale), and\nused hass2geo to dump out GPX format\ntraces of our locations. Then, I vibe coded a Python script to convert the\nwaypoint format into "trackfiles", so that they could be loaded into the\nLightroom map view.\nAfter this it was relatively easy going, as Lightroom matches up the GPS\ntraces to the camera picture times, and stamps them accordingly. It's not\nperfect as it doesn't interpolate between track traces, but good enough for\nthe sort of accuracy I wanted ("in the NG32 reserve in Botswana").
\nIn the long term though, this isn't a great solution: Home Assistant only\nkeeps a few days of location history, and getting fine-grained GPS traces\nvia the API requires manually dumping out the internal database.
\n![\"%c\"](\"/images/okavango-3.webp\" "\\"It")
I then looked more closely at which apps were using location on my phone, and\ndiscovered the Life Cycle app that had been\nthere since 2018! This was an app I installed way back to categorise where I\nwas spending my time, and then promptly forgot about it when the pandemic\nstarted. The app developers are generally privacy friendly, and it also had an\noption to output a CSV file of all the location history.
\nUnfortunately, this CSV file is incomplete, as it includes locations as\nnames, but not their exact GPS location (which the app has internally).\nI opened a support request with the app developer, but they responded\nthat they don't plan to add any new features to the app. That's fair\nenough, but I really wanted to get the data out!
\n ![\"%c\"](\"/images/okavango-4.webp\" "\\"This")
The obvious solution is to extract the app database from my iPhone and\nreverse engineer it. While this used to be easy, the\napp sandboxes mean\nthat modern iPhones are very locked down. I had to manually initiate a tethered\nand unencrypted backup to my Mac (as opposed to an iCloud backup), and then\nextract the Life Cycle app from my phone using iMazing.
\n$ ls \n 11440 Aug 11 15:43 .lock\n 192 Aug 11 15:43 Container/\n 1555 Aug 11 15:43 iTunesMetadata.plist\n 6661627 Aug 11 15:43 Life Cycle.imazingapp\n 96 Aug 11 15:43 Payload/\n$ cd Container/Library/ && file life.db\nlife.db: SQLite 3.x database, user version 20, last written using SQLite\nversion 3043002, file counter 1, database pages 3443, cookie 0x65, schema 4,\nUTF-8, version-valid-for 1\nWe've found the life.db, and its now in an unencrypted sqlite3 format!\nBut what format is it actually in? I didn't have time to figure it out,\nbut I pointed Claude Code at the database in an attempt to reverse engineer\nthe format into something I could export. After consulting Ryan Gibb, he\nsuggested using OwnTracks as a self-hosted location tracker.
Rather than get Claude to directly interpret the data, I directed it\nto examine the sqlite3 database and to emit a script that would convert\nit to a format that OwnTracks could use. It did this pretty accurately:
\n# Discovered schema with 24 tables tracking location, motion, activities\nsqlite3 life.db ".tables"\nsqlite3 life.db ".schema"\n\n# Created JSON export script for primary observational data\n- LocationEvent: Raw GPS coordinates (31,743 records)\n- Motion: Movement classification (77,385 records)\n- Activity: Behavior categorization (20,273 records)\n- Visit: Stationary location periods (9,903 records)\n\n2. Initial Upload Implementation\n\n# Core transformation mapping\nLifeCycle → OwnTracks:\n- timestamp → tst (Unix epoch)\n- latitude/longitude → lat/lon\n- hAccuracy → acc\n- altitude → alt (with validation)\n- speed → vel (m/s to km/h conversion)\n- WiFi context → SSID/BSSID\n\nFeatures implemented:\n- Duplicate prevention via state tracking\n- Resume capability for interrupted uploads\n- Quality filtering (>500m accuracy rejected)\n- Progress monitoring\n\n3. Enhanced Data Integration\n\n# Added motion activities from 77K motion records\n"motionactivities": [\n {"type": "automotive", "confidence": 2},\n {"type": "walking", "confidence": 0}\n ]\nThis implementation went well beyond what I had in mind: in addition to just\ntranslating the GPS, it also looked at the contextual data (such as the iOS\nmotion tracking information), and also wrote a comprehensive script (available\nhere) that collected it all together and uploaded it to my server.
\nOverall, the fact that I could reverse engineer a decade old app, autodetect\nits data structures and convert with high fidelity into a self-hosted infrastructure\nin about 15 minutes of agent-driven coding is a capability I find absolutely incredible.\nThis sort of coding isn't suitable for everything, of course, but it's great for\nthe tedious glue code which makes self-hosting often so painful.
\n ![\"%c\"](\"/images/okavango-5.webp\" "\\"On")
I now had my past eight years of locations in one place, so it seemed a\ngood time to switch away completely to a self-hosted location system\ndedicated to that purpose.
\nDeploying the aforementioned OwnTracks requires two things: a secure MQTT\nendpoint to receive events from the iOS app, and then a recorder that\nsubscribes to it and compacts the results into location records. The recorder\nexposes an HTTP API that frontends can use to render map like interfaces,\nor it works headlessly as well for other API uses of your location.
\nI used this docker-compose.yml, as I couldn't find an off-the-shelf\nsetup:
\nservices:\n frontend:\n image: caddy:2-alpine\n depends_on:\n - mqtt\n ports:\n - "443"\n - "443/udp"\n - "80"\n volumes:\n - ./caddy/data:/data\n - ./caddy/htdocs:/htdocs\n - ./caddy/conf:/etc/caddy\n restart: always\n owntracks-frontend:\n image: owntracks/frontend\n volumes:\n - ./frontend/config.js:/usr/share/nginx/html/config/config.js\n environment:\n - SERVER_HOST=otrecorder\n - SERVER_PORT=8083\n restart: unless-stopped\n otrecorder:\n image: owntracks/recorder\n restart: unless-stopped\n environment:\n - VIRTUAL_HOST=<host>\n - TZ=Europe/London\n - OTR_USER=<user>\n - OTR_PASS=<pass>\n - OTR_HTTPHOST=0.0.0.0\n - OTR_HTTPPORT=8083\n - OTR_HOST=mqtt\n - OTR_PORT=1883\n volumes:\n - ./owntracks/config:/config\n - ./owntracks/store:/store\n mqtt:\n container_name: mqtt\n build: .\n environment:\n - TZ=Europe/London\n ports:\n - "185.33.27.72:8883:8883"\n volumes:\n - ./mosquitto/data:/mosquitto/data\n - ./mosquitto/logs:/mosquitto/logs\n - ./mosquitto/conf:/mosquitto/config\n - ./mosquitto/conf/passwd:/etc/mosquitto/passwd:ro\n - ./caddy/data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/track.recoil.org:/etc/mosquitto/crt:ro\n restart: unless-stopped\nYou can replace the direct environment variables with an env_file\n(recommended!). This compose file uses Caddy to set up LetsEncrypt certificates\nfor the MQTT server automatically, and then you just need to drop in a\ncaddy/conf/Caddyfile with the HTTP auth:
domainname.org {\n encode gzip\n reverse_proxy http://owntracks-frontend\n basicauth * {\n user hashed-password\n }\n}\nThis uses normal browser auth to protect your location, and you can also run\nthe whole thing behind Tailscale or similar to avoid having an Internet-visible\nport exposed too.
\n ![\"%c\"](\"/images/okavango-6.webp\" "\\"This")
I've been interested in "spatial programming" for ages, and while I was away\nRyan Gibb and Josh Millar uploaded a preprint we've been working on a system for\nprogramming with physical locations called "Bifrost". We're\nparticularly interested in the intersection between physical devices, locations\nand how to manage their interactions. This is a continuation of the work on\nOsmose and spatial naming. Stay tuned!
I've been hacking with Sadiq Jaffer (^),\nJon Ludlam (^) and\nRyan Gibb (^) on various approaches to\nimproving the agentic coding experience for OCaml.
\nWe jotted down our notes in a draft paper to keep track of everything going on, including summarising previous experiments with Qwen3 for FoCS. Since then, there's been a flurry of extra activity from others which we need to integrate!
\nThis is all fairly straightforward MCP work that improves the short-term experience. We'll get to the RL-VR ideas later...\nIf anyone else is hacking on something agent related do post on OCaml Discuss and let us know! I'm hoping to update the paper later in August to roundup the efforts above.
I've just taken Kyutai's speech-to-text model for a spin on my Mac laptop, and it's stunningly good. As background, this is what the prolific Laurent Mazare has been hacking on; he has made a ton of contributions to the OCaml community as well, such as ocaml-torch and starred in a very fun Signals to Threads episode on machine learning at Jane Street back in 2020.
\nYou can get the microphone-to-speech running on your Mac in a few commands, assuming you have uv installed (which you should!).
\ngit clone https://github.com/kyutai-labs/delayed-streams-modeling\ncd delayed-streams-modeling\nuvx --with moshi-mlx python scripts/stt_from_mic_mlx.py\nIt understands my accent near perfectly; if that isn't a machine learning miracle, I don't know what is! I'm looking forward to trying this out more with our Low power audio transcription with Whisper project over the summer with Josh Millar and Dan Kvit.
", "url": "https://anil.recoil.org/notes/kyutai-streaming-voice-mlx", "title": "Using Kyutai's low latency audio models on macOS in one command", "summary": "Quick setup guide for running Kyutai's high-quality speech-to-text model locally on Mac using their MLX implementation.", "date_published": "2025-07-16T00:00:00.000000Z", "date_modified": "2025-07-16T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "llm", "ai", "audio" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/nmcab-py710", "content_html": "I was a bit sleepy getting into the Royal Society Future of Scientific\nPublishing\nconference early this morning, but was quickly woken up by the dramatic passion\non show as publishers, librarians, academics and funders all got together for a\n"frank exchange of views" at a meeting that didn't pull any punches!
\nThese are my hot-off-the-press livenotes and only lightly edited; a more cleaned up version will be available\nfrom the RS in due course.
\n![\"%c\"](\"/images/rspub-1.webp\" "\\"Sir")
Sir Mark Walport was a delightful emcee for the proceedings of the day, and\nopened how important the moment is for the future of how we conduct science.\nAcademic publishing faces a perfect storm: peer review is buckling under\nenormous volume, funding models are broken and replete with perverse\nincentives, and the entire system groans with inefficiency.
\nThe Royal Society is the publisher of the world's oldest continuously published\nscientific journal Philosophical Transactions\n(since 1665) and has convened this conference for academies worldwide. The\noverall question is: what is a scientific journal in 2025 and beyond?\nWalport traced the economic evolution of publishing: for centuries, readers\npaid through subscriptions (I hadn't realised that the early editions of the RS\nused to be sent for free to libraries worldwide until the current commercial\nmodel arrived about 80 years ago).. Now, the pendulum has swung to open access\nthat creates perverse incentives that prioritize volume over quality. He called\nit a "smoke and mirrors" era where diamond open access models obscure who\nactually pays for the infrastructure of knowledge dissemination: is it the\npublishers, the governments, the academics, the libraries, or some combination\nof the above? The profit margins of the commercial publishers answers that\nquestion for me...
\nHe then identified the transformative forces that are a forcing function:
\nThe challenges ahead while dealing with these are maintaining metadata\nintegrity, preserving the scholarly archive into the long term, and ensuring\nsystematic access for meta-analyses that advance human knowledge.
\nThe opening pair of speakers were unexpected: they brought a historical and\nlinguistic perspective to the problem. I found both of these talks the\nhighlights of the day! Firstly Professor Aileen\nFyfe drew upon her research\nfrom 350 years of the Royal Society archives. Back in the day, there was no\nreal fixed entity called a "scientific journal". Over the centuries, everything\nfrom editorial practices to publication methods over to dissemination means\nhave transformed repeatedly, so we shouldn't view the status quo as set in stone.
\n![\"%c\"](\"/images/rspub-2.webp\" "\\"Professor")
While the early days of science were essentially people writing letters to each\nother, the post-WWII era of journals marked the shift to "scale". The tools for\ndistance communication (i.e. publishing collected issues) and universities\nswitching from being teaching focused over to today's research-centric\npublishing ecosystem were both key factors. University scientists used to\nproduce 30% of published articles in 1900; by 2020, that figure exceeded 80%.\nThis parallels the globalization of science itself in the past century;\nresearch has expanded well beyond its European origins to encompass almost all\ninstitutions and countries worldwide.
\nAmusingly, Prof Fyfe pointed out that a 1960 Nature editorial asked "How many more new\njournals?" even back then! The 1950s\ndid bring some standardization efforts (nomenclature, units, symbols) also\nthough citation formats robustly seem to resist uniformity. English was also\nexplicitly selected as the "default language for\nscience, and peer review\nwas also formalised via papers like "Uniform requirements for manuscripts submitted to biomedical journals" (in 1979). US Congressional hearings\nwith the NSF began distinguishing peer review from other evaluation methods.
\n![\"%c\"](\"/images/rspub-3.webp\" "\\"Professor")
All of this scale was then "solved" by financialisation after WWII. At the turn of the\n20th century, almost no journals generated any profit (the Royal Society\ndistributed its publications freely). By 1955, financial pressures and growing scale of submissions forced a\nreckoning, leading\nto more self-supporting models by the 1960s. An era of mergers and acquisitions\namong journals followed, reshaping the scientific information system.
\nProfessor Vincent Larivière then took the stage to dispel some myths of English monolingualism in scientific publishing. While English offers some practical benefits, the reality at non-Anglophone institutions (like his own Université de Montréal) reveals that researchers spend significantly more time reading, writing, and processing papers as non-native language speakers, and often face higher rejection rates as a result of this.\nThis wasn't always the case though; Einstein published primarily in German, not English!
\nHe went on to note that today's landscape for paper language choices is more\ndiverse than is commonly assumed. English represents only 67% of publications,\na figure whic itself has been inflated by non-English papers that are commonly\npublished with English abstracts. Initiatives like the Public Knowledge\nProject has enabled\ngrowth in Indonesian and Latin America for example. Chinese journals now\npublish twice the volume of English-language publishers, but are difficult to\nindex which makes Lariviere's numbers even more interesting: a growing majority\nof the world is no longer publishing in English! I also heard this in my trip\nin 2023 to China with the Royal Society; the scholars we met had a sequence of\nChinese language journals they submitted too, often before "translating" the\noutputs to English journals.
\n![\"%c\"](\"/images/rspub-4.webp\" "\\"Professor")
All this leads us to believe that the major publisher's market share is smaller than commonly believed, which gives us reason for hope to change! Open access adoption worldwide currently varies fairly dramatically by per-capita wealth and geography, but reveals substantive greenspace for publishing beyond the major commercial publishers. Crucially, Larivière argued that research "prestige" is a socially constructed phenomenon, and not intrinsic to quality.
\nIn the Q&A, Magdalena Skipper (Nature's Editor-in-Chief) noted that the private sector is reentering academic publishing (especially in AI topics). Fyfe noted the challenge of tracking private sector activities; e.g. varying corporate policies on patenting and disclosure mean they are hard to infdex. A plug from Coherent Digital noted they have catalogued 20 million reports from non-academic research; this is an exciting direction (we've got 30TB of grey literature on our servers, still waiting to be categorisd).
\n![\"%c\"](\"/images/rspub-5.webp\" "\\"Professor")
Our very own Bill Sutherland opened with a sobering demonstration of "AI\npoisoning" in the literature, referencing our recent Nature\ncomment. He did the risky-but-catchy\ngeneration of a plausible-sounding but entirely fabricated conservation study\nusing an LLM and noted how economically motivated rational actors might quite\nreasonably use these tools to advance their agendas via the scientific record.\nAnd recovering from this will be very difficult indeed once it mixes up with\nreal science.
\n![\"%c\"](\"/images/rspub-6.webp\" "\\"Bill")
Bill then outlined our emerging approach to subject-wide synthesis via:
\nGoing back to what ournals are for in 2025, Bill then discussed how they were\noriginally vehicles for exchanging information through letters, but now serve\nprimarily as stamps of authority and quality assurance. In an "AI slop world,"\nthis quality assurance function becomes existentially important, but shouldn't\nnecessarily be implemented in the current system of incentives. So then, how do\nwe maintain trust when the vast majority of submissions may soon be\nAI-generated? (Bill and I scribbled down a plan on the back of a napkin for\nthis; more on that soon!)
\n![\"%c\"](\"/images/rspub-7.webp\" "\\"Bill")
Dr. Sophie Meekings then took the stage to discuss the many barriers facing early career researchers (ECRs). They're on short-term contracts, are dependent on others people's grant funding, and yet are the ones conducting the frontline research that drives scientific progress. And this is after years spent on poorly paid PhD stipends!
\nECRs require:
\nDr. Meekings then noted how the precarious nature of most ECR positions creates cascading complications for individuals. When job-hopping between short-term contracts, who funds the publication of work from previous positions? How do ECRs balance completing past research with new employers' priorities? Eleanor Toye Scott also had this issue when joining my group a few years ago, as it took a significant portion of her time in the first year to finish up her previous publication from her last research contract.
\nIf we're going to fix the system itself, then ECRs need better incentives for PIs to publish null results and exploratory work, the councils need to improve support for interdisciplinary research that doesn't fit traditional journal boundaries (as these as frontiers between "conventional" science where many ECRs will work), and recognition that ECRs often lack the networks for navigating journal politics where editors rule supreme.
\nDr. Meekings summarized ECR needs with an excellent new acronym (SCARF) that drew a round of applause!
\n![\"%c\"](\"/images/rspub-8.webp\" "\\"Dr")
The audience Q&A was quite robust at this point. The first question was about how might we extend the evidence synthesis approach widely?\nBill Sutherland noted that we are currently extending this to education working with Jenny Gibson. Interconnected datasets across subjects are an obvious future path for evidence datasets, with common technology for handling (e.g.) retracted datasets that can be applied consistently. Sadiq Jaffer and Alec Christie are supervising projects on evidence synthesis this summer on just this topic here in Cambridge.
\nAnother question was why ECRs feel that double blind review is important. Dr. Meekings noted that reviewers may not take ECR peer reviews as seriously, but this coul dbe fixed by opening up peer review and assigning credit after the process is completed and not during. Interestingly, the panel all like double-blind, which is the norm in computer science but not in other science journals. Some from the BMJ noted there exists a lot of research into blinding; they summarised it that blinding doesn't work on the whole (people know who it is anyway) and open review doesn't cause any of the problems that people think it causes.
\nA really interesting comment from Mark Walport was that a grand scale community project could work for the future of evidence collation, but this critically depends on breaking down the current silos since it doesn't work unless everyone makes their literature available. There was much nodding from the audience in support of this line of thinkin.g
\nThe next panel brought together folks from across the scientific\npublishing ecosystem, moderated by Clive Cookson of the Financial Times. This\nwas a particularly frank and pointed panel, with lots of quite direct messages\nbeing sent between the representatives of libraries, publishers and funders!
\n![\"%c\"](\"/images/rspub-9.webp\" "\\"Amy")
Amy Brand (MIT Press) started by delivered a warning about conflating "open to\nread" with "open to train on". She pointed out that when MIT Press did a survey\nacross their authors, many of them raised concerns about the reinforcement of\nbias through AI training on scientific literature. While many of the authors\nacknowledged a moral imperative to make science available for LLM training,\nthey also wanted the choice of making their own work used for this. She urged\nthe community to pause and ask fundamental questions like "AI training, at what\ncost?" and "to whose benefit?". I did think she made a good point by drawing\nparallels with the early internet, where Brand pointed out that lack of\nregulation accelerated the decline of non-advertising-driven models. Her\nclosing question asked if search engines merely lead to AI-generated summaries,\nwhy serve the original content at all? This is something we discuss in our\nupcoming Aarhus paper on an Internet ecology.
\nDanny Kingsley from Deakin University Library then delivered a biting perspective as a representative of libraries. She said that libraries are "the ones that sign the cheques that keeps the system running", which the rest of the panel all disagreed with in the subsequent discussion (they all claimed to be responsible, from the government to the foundations). Her survey of librarians was interesting; they all asked for:
\nKingsley had a great line that "publishers re monetizing the funding mandate",\nwhich Charlotte Deane later also said was the most succinct way she had heard\nto describe the annoyance we all have with the vast profit margins of\ncommercial publishers. Kingsley highlighted this via the troubling practices\nin the IEEE and the American Chemical Society by charging to place repositories\nunder green open access. Her blunt assessment was that publishers are not\nnegotiating in good faith. Her talk drew the biggest applause of the day by\nfar.
\nAfter this, John-Arne\nRøttingen\n(CEO of the Wellcome Trust) emphasised that funders depend on scientific\ndiscourse as a continuous process of refutations and discussions. He expressed\nconcern about overly depending on brand value as a proxy for quality, calling\nit eventually misleading even if it works sometimes in the short term. Key\npriorities the WT have is ensuring that reviewers have easy access to all\nliterature, to supporting evidence synthesis initiatives to translate research\ninto impact, and controlling the open body of research outputs through digital\ninfrastructure to manage the new scale. However, his challenge lies in\nmaintaining sustainable financing models for all this research data; he noted\nexplicitly that the Wellcome would not cover open access costs for commercial\npublishers.
\nRøttingen further highlighted the Global Biodata Coalition (which he was a\nmember of) concerns about US data resilience and framed research infrastructure\nas "a global public good" requiring collective investment and fair financing\nacross nations. Interestingly, he explicitly called out UNESCO as a weak force\nin global governance for this from the UN; I hadn't even realised that UNESCO\nwas responsible for this stuff!
\nFinally, Prof Charlotte Deane from the EPSRC also discussed what a scientific\njournal is for these days. It's not for proofreading or typesetting anymore and\n(as Bill Sutherland also noted earlier), the stamp of quality is key. Deane\nargued that "research completion" doesn't happen until someone else can read it\nand reasonably verify the methods are sound; not something that can happen\nwithout more open access. Deane also warned of the existential threat of AI poisoning since "AI can make fake papers at a rate humans can't\nimagine. It won't be long before mose of the content on the Internet will be AI\ngenerated".
\nThe audience Q&A was very blunt here. Stefanie Haustein pointed out that we\nare pumping of billions of dollars into the publishing industry, many of which\nare shareholder companies, and so we are losing a significant percentage of\neach dollar spent. There is enough money in the system, but it's very\ninefficiently deployed right now!
\nRichard Sever from openRxiv asked\nhow we pay for this when major funders like the NIH have issued a series of\nunfunded open data mandates over recent years. John-Arne Rottingen noted that\nUNESCO is a very weak global body and not influential here, but that we need\ncoalitions of the willing to build such open data approaches from the bottom\nup. Challenging the publisher hegemony can only be done as a pack, which lead\nnicely onto the next session after lunch where the founder of\nOpenAlex would be present!
\nAfter lunch (where sadly, the vegetarian options were terrible but\nluckily I had my trustly Huel bar!), we reconvened with a panel debating\nwho the stewards of the scientific record should be. This brought together\nperspectives from commercial publishers (Elsevier), open infrastructure advocates (OpenAlex),\nfunders (MRC), and university leadership (pro-VC of Birmingham).
\nVictoria Eva (SVP from\nElsevier)\nopened by describing the "perfect storm" facing their academic publishing\nbusiness as they had 600k more submissions this year than the previous year.\nThere was a high level view on how their digital pipeline "aims to insert\nsafeguards" throughout the publication process to maintain integrity. She\nargued in general terms to view GenAI through separate lenses of trust and\ndiscoverability and argud that Elsevier's substantial technological investments\nposition them to manage both challenges well. I was\npredisposed\nto dislike excuses from staggeringly profitable commercial publishers, but I\ndid find her answers to providing bulk access to their corpus unsatisfying.\nWhile she highlighted their growing open access base of papers, she also noted\nthat the transitionon to open access cannot happen overnight (my personal\ntranslation is that this means slow-walking). She mentioned special cases in\nplace for\nTDM\nin the Global South and healthcare access (presumably at the commercial\ndiscretion of Elsevier).
\nJason Priem from OpenAlex\n(part of OurResearch) then offered a radically\ndifferent perspective. I'm a huge fan of OpenAlex, as we use it extensively in\nthe CE infrastructure. He disagreed with the conference framing of\npublishers as "custodians" or "stewards," noting that these evoke someone\nmaintaining a static, old lovely house. Science isn't a static edifice but a\ngrowing ecosystem, with more scientists alive today than at any point in\nhistory. He instead proposed a "gardener" as a better metaphor; the science\necosystem needs to nourish growth rather than merely preserving what exists.\nExtending the metaphor, Priem contrasted French and English garden styles:\nFrench gardens constrain nature into platonic geometric forms, while English\ngardens embrace a more rambling style that better represents nature's inherent\ndiversity. He argued that science needs to adopt the "English garden" approach\nand that we don't have an information overload problem but rather "bad\nfilters"\n(to quote Clay Shirky).
\n![\"%c\"](\"/images/rspub-11.webp\" "\\"Jason")
Priem advocated strongly for open infrastructures since communities don't just produce papers: also software, datasets, abstracts, and things we don't envision yet. If we provide them with the "digital soil" (open infrastructure) then they will prosper. OpenAlex and Zenodo are great examples of how such open infrastructure hold up here. I use both all the time; I'm a huge fan of Jason's work and talk.
\nPatrick Chinnery from the Medical Research Council brought the funder perspective with some numbers: publishing consumes 1 to 2% of total research turnover funds (roughly £24 million for UKRI) . He noted that during the pandemic, decision-makers were reviewing preprint data in real-time to determine which treatments should proceed to clinical trials and decisions had to be reversed after peer review revealed flaws. He emphasised the the need for more real time quality assurance in rapid decision-making contexts.
\nAdam Tickell from the University of Birmingham declared the current model "broken", and not that each attempt at reform fails to solve the basic problem of literature access (something I've faced myself). He noted that David Willetts (former UK Minister for Science) couldn't access paywalled material while minister of science in government (!) which significantly influenced subsequent government policy towards open access.\nTickell was scathing about the oligopolies of Elsevier and Springer, arguing their profit margins are out of proportion with the public funding for science. He noted that early open access attempts from the Finch Report were well-intentioned but ultimately insufficient to break the hegemony. Perhaps an opportunity for a future UK National Data Library...\nTickell closed his talk with an observation about the current crisis of confidence in science. This did make me think of a recent report on British confidence in science, which shows the British public still retains belief in scientific institutions. So at least we're doing better than the US in this regard for now!
\n ![\"%c\"](\"/images/rspub-ss-1.webp\" "\\"Stefanie")
The Q&A session opened with Mark Walport asked how Elsevier manages to publish so many articles. Victoria Eva from Elsevier responded that they receive 3.5m articles annually with ~750k published. Eva mentioned something about "digital screening throughout the publication process" but acknowledged that this was a challenge due to the surge from paper mills. A suggestion of paying peer reviewers was raised from the audience but not substantively addressed. Stefanie Haustein once again made a great point from the audience about how Elsevier could let through AI generated rats with giant penises with all this protection in place; clearly, some papers have been published by them with no humans ever reading it. This generated a laugh from the audience, and an acknowlegment from the Elsevier rep that they needed to invest more and improve.
\nMy laptop power ran out at this point, but the next panel was an absolute treat as it had both Kaitlin Thaney and Jimmy Wales of Wikipedia fame on it!
\n![\"%c\"](\"/images/rspub-12.webp\" "\\"Hylke")
Jimmy Wales pointed out an interesting point from his "seven rules of trust" is that a key one is to be personal with human-to-human contact and not run too quickly to technological solutions. Rather than, for example, asking what percentage of academic papers showed evidence of language from ChatGPT, it's more fruitful to ask whether the science contained within the paper is good instead of how it's written. There are many reasons why someone might have used ChatGPT (non-native speakers etc) but also many reasons unrelated why the science might be bad.
\nKaitlin Thaney pointed out the importance of openness given the US assault on\nscience\nmeans that the open data repositories can be replicated reasonably as well.
\nIan Mulvaney pointed out that Nature claims to have invested $240m in research\ninfrastructure, and this is a struggle for a medium sized publisher (like his\nown BMJ). Open infrastructure allows sharing and\ncreation of value to make it possible to let these smaller organisations\nsurvive.
\nWhen it comes to policy recommendations, what did the panel have to say about a more trustworthy literature?
\nI'm updating this as I go along but running out of laptop battery too!
For the past few years, Sadiq Jaffer and I been working with our colleagues in\nConservation Evidence to do analysis at scale on the\nacademic literature. Getting local access to millions of fulltext papers has not\nbeen without drama, but made possible thanks to huge amounts of help from our\nUniversity Library who helped us navigate our\nrelationships with scientific publishers. We have just published a comment\nin Nature about the next phase\nof our research, where are looking into the impact of AI advances on evidence synthesis.
\n
Our work on literature reviews led us into assessing methods for evidence\nsynthesis\n(which is crucial to rational policymaking!) and specifically about how recent advances in AI may\nimpact it. The current methods for rigorous systematic literature review are expensive and slow, and authors are already struggling to keep up with the rapidly expanding\nnumber of legitimate papers. Adding to this, paper retractions are increasing near\nexponentially and already\nsystematic reviews unknowingly cite\nretracted papers, with most remaining uncorrected even a year (after notification!)
\nThis is all made much more complex as LLMs are flooding the landscape with\nconvincing, fake manuscripts and doctored data, potentially overwhelming our\ncurrent ability to distinguish fact from fiction. Just this March, the AI\nScientist formulated hypotheses, designed and\nran experiments, analysed the results, generated the figures and produced a\nmanuscript that passed human peer\nreview for an ICLR\nworkshop! Distinguishing genuine papers from those produced by LLMs isn't just\na problem for review authors; it's a threat to the very foundation of\nscientific knowledge. And meanwhile, Google is taking a different tack with a\ncollaborative AI co-scientist who acts as a multi-agent assistant.
\nSo the landscape is moving really quickly! Our proposal for the future of\nliterature reviews builds on our desire to move towards a more regional,\nfederated network approach. Instead of having giant repositories of knowledge\nthat may be erased unilaterally,\nwe're aiming for a more bilateral network of "living evidence databases".\nEvery government, especially those in the Global South, should have the ability to build their\nown "national data libraries" which represent the body\nof digital data that affects their own regional needs.
\nThis system of living evidence databases can be incremental and dynamically\nupdated, and AI assistance can be used as long as humans remain in-the-loop.\nSuch a system can continuously gather, screen, and index literature,\nautomatically remove compromised studies and recalculating results. We're\nworking on this on multiple fronts this year; ranging from the computer science\nto figure out the distributed-nitty-gritty [1], over to working with the\nGEOBON folk on global biodiversity data\nmanagement, and continuing\nto drive the core LED design at Conservation Evidence. It feels like a
\nRead our Nature Comment piece (comment on LI) to learn more about how we think we can safeguard evidence synthesis against the rising tide of "AI-poisoned literature" and ensure the continued integrity of scientific discovery. As a random bit of trivia, the incredibly cool artwork in the piece was drawn by the legendary David Parkins, who also drew Beano and Dennis the Menace!
\nMy instinct is that we'll end up with something ATProto based as it's so convenient for distributed system authentication.
\n ↩The exam marking is over, and a glorious Cambridge summer awaits! This year, we\nhave a sizeable cohort of undergraduate and graduate interns joining us from\nnext week.
\nThis note serves as a point of coordination to keep track of what's\ngoing on, and I'll update it as we get ourselves organised.\nIf you're an intern, then I highly recommend you take the time to carefully\nread through all of this, starting with who we are,\nsome ground rules, where we will work,\nhow we chat, how to get paid, and of course social activities to make sure we have some fun!
\nWe're working on quite the diversity of projects this summer, ranging from classic\ncomputer systems and programming problems all the way through to environmental\nscience. Here's a recap of what's going on.
\nFirst we're working against the evidence database we've been building for the past couple of years:
\nWe're then heading into remote sensing and working on some mapping projects:
\nDropping down towards embedded systems and fun "real-world" projects, we have:
\nGoing back to classic computer science, we have a few programming language and systems projects:
\nSince there are so many of us this summer, it's imperative that you're all\nproactive about communicating any problems or clarifications you need. If something\nhere doesn't make sense, or you have a better idea, then just reach out to any\nof the supervisors or me directly!
\nDo also take time to learn from each other. Read up on not just your own project in the\nlist above, but take some to read the remainder so that you have a sense of what everyone\nis working on. When you see each other, it'll be much easier to chat about what's going\non and find opportunities for commonality.
\nThe projects above have been carefully selected to not be on the critical path for any\ndeadlines. If it's not going well from your perspective, then it's ok to take a step back\nand figure out why! We're hear to learn and discover things, so take the time to do so.
\nThis will be different for everyone, since it depends on which home department will house the project.\nSome of us will be in the David Attenborough Building, in the third floor where the CRI is:
\nThose working on the Zoology Museum itself (Arissa-Elena Rotunjanu, Beatrice Spence and Anna Yiu) will have an health and safety induction on Monday with Tiffany Ki and find offices there.
\nThe rest of us will be in the Computer Lab over in West Cambridge:
\nIt'll probably take a week to let this all shake out, so please do shout if you find yourself stuck in your room and without an office! You should of course arrange to meet your immediate supervisors regularly according to whatever schedule and location works for you.
\nThe way you get paid weekly is via the Cambridge Casual Worker system. This has a few important steps that you must pay attention to, or you will not get paid!
\nSince we're all going to spread around Cambridge physically, it's important to have a chat channel. Hannah McLoone is setting up a WhatsApp group for social things (see below), but we also use Matrix as our "hackers choice" for day-to-day messaging.
\nWe host a Computer Lab Matrix server on which anyone with a valid Raven account can create an account. Since Matrix is a decentralised chat system, it is also possible to use other accounts from third-party servers, and also to join channels elsewhere.
\nTo create an account:
\neeg.cl.cam.ac.uk as your homeserver.Once you create your account, you will be in the "EEG" Matrix space. A Matrix space is a collection of channels, and you should join "EEGeneral" as the overall channel for the group. We'll create a separate room just for intern chats. We also have a bot in the room that posts our blogs to the channel, so you can keep up with what the group members are all chattering about. Ryan Gibb runs the CL matrix server, and there are occasional quirks, so just let us know if you run into any problems. I am @avsm:recoil.org on there, not avsm2 as I use my personal Matrix for a bunch of stuff.
It's important to get some downtime this summer and recharge. Hannah McLoone has been setting up a social group for the interns to hang out together, and we'll organise a punting excursion at some point to get us out to the river. Of course, many of us will be travelling this summer (I'm heading off to Botswana in late July for instance), so please do also make suggestions.
", "url": "https://anil.recoil.org/notes/eeg-interns-2025", "title": "EEG internships for the summer of 2025", "summary": "Coordination note for summer 2025 undergraduate and graduate internships covering projects from evidence databases to remote sensing and embedded systems.", "date_published": "2025-06-28T00:00:00.000000Z", "date_modified": "2025-06-28T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "urop" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/53zjq-ft509", "content_html": "The BIOMASS forest mission satellite was successfully boosted into space a couple of days ago, after decades of development from just down the road in Stevenage. I'm excited by this because it's the first global-scale P-band SAR instrument that can penetrate forest canopys to look underneath. This, when combined with hyperspectral mapping will give us a lot more insight into global tree health.
\nWeirdly, the whole thing almost never happened because permission to use the P-band was blocked because it might interfere with US nuclear missile warning radars back in 2013.
\n\n\nMeeting in Graz, Austria, to select the the 7th Earth Explorer mission to be flown by the 20-nation European Space Agency (ESA), backers of the Biomass mission were pelted with questions about how badly the U.S. network of missile warning and space-tracking radars in North America, Greenland and Europe would undermine Biomass’ global carbon-monitoring objectives.
\nEurope's Earth observation satellite system may be the world's most dynamic, but as it pushes its operating envelope into new areas, it is learning a lesson long ago taught to satellite telecommunications operators: Radio frequency is scarce, and once users have a piece of it they hold fast.\n-- Spacenews (2013)
\n
Luckily, all this got sorted by international frequency negotiators, and after\nbeing built by Airbus in Stevenage\n(and Germany and France, as it's a complex instrument!) it took off without a hitch. Looking forward to getting my hands on the first results later in the year over at the Centre for Earth Observation.
\nCheck out this cool ESA video about the instrument to learn more, and congratulations to the team at ESA. Looking forward to the next BIOSPACE where there will no doubt be initial buzz about this.
\n\nUpdate 28th June 2025: See also this beautiful BBC article about the satellite, via David Coomes.
Every ten years, the city of Aarhus throws a giant conference to discuss new agendas for critical action and theory in computing. Back in 2016, Hamed Haddadi, Jon Crowcroft and I posited the idea of personal data stores, a topic that is just now becoming hot due to agentic AI. Well, time flies, and I'm pleased to report that our second dicennial thought experiment on "Steps towards an Ecology for the Internet" will appear at the 2025 edition of Aarhus this August!
\nThis time around, we projected our imaginations forward a decade to imagine an optimistic future for the Internet, when it has exceeded a trillion nodes. After deciding in the pub that this many nodes was too many for us to handle, we turned to our newfound buddies in ##conservation to get inspiration from nature. We asked Sam Reynolds, Alec Christie, David Coomes and Bill Sutherland first year undergraduate questions about how natural ecosystems operate across all levels of scale: from DNA through to cells through to whole populations.\nWe spent hours discussing the strange correspondences between the seeming chaos in the low-level interactions between cells through to the extraordinary emergent discipline through which biological development typically takes place.
\nThen, going back to the computer scientists in our group and more widely (like Cyrus Omar who I ran into at Bellairs), it turns out that this fosters some really wild ideas for how the Internet itself could evolve into the future. We could adopti biological process models within the heart of the end-to-end principle that has driven the Internet architecture for decades!
\n ![\"%c\"](\"/images/ecology-ss-1.webp\" "\\"Correspondences")
The ideas we sketch out in the paper were indeed quite crazy a year ago. We float the idea of "antibotty" networks that adopt the same command-and-control software as malicious botnets, except to patch our local community's networks and proactively protect them from roving viruses. We discuss how we might automatically modify the source code of our open source software stacks to introduce diversity into operating system kernels that have become homogenous due to the runaway success of Linux. We wonder if mutualism is the only eventual steady state of Internet communities, with the existing extractive surveillance economy just a blip that is collapsing under the weight of its own unfitness.
\nWhile these were fantastical ideas, we are already seeing the techniques to make them real (and more). AI-driven Agentic programming is now a regular part of my toolbox, and sophisticated code sandboxing like CHERI, FIDES and Deluminator can enforce fine-grained containment throughout the software stack, with techniques like unikernels driving whole-stack changes. And when it comes to imagining a future of Internet community, others like Maria Farrel and Robin Berjon have written wonderfully inspiring pieces on [the need to rewild the Internet] that are starting a chorus about the need for change and ##selfhosting.
\nNature is unstoppable in its ferocity and desire to eat the unfit, and the combination of artificial and natural life may be just what we need to bring the Internet back to mutualistic stability into the coming decade. Your own thoughts and reactions on our paper are extremely welcome ahead of Aarhus in August! I am sweating at the thought of presenting this work in a coherent way as the paper reads as wildly as you might imagine. I am, however, confident in my excitement and gratitude for having the chance to think about the problem in a more interdisciplinary and longer-term way, and that this is just the start of a line of thinking that may mutate into something special.
\n![\"%c\"](\"/images/ecology-2016.webp\" "\\"Two")
\n![\"%c\"](\"/images/ecology-2026.webp\" "\\"Two")
Apple made a notable announcement in WWDC 2025 that they've got a new containerisation framework in the new Tahoe beta. This took me right back to the early Docker for Mac days in 2016 when we announced the first mainstream use of the hypervisor framework, so I couldn't resist taking a quick peek under the hood.
\nThere were two separate things announced: a Containerization framework and also a container CLI tool that aims to be an OCI compliant tool to manipulate and execute container images. The former is a general-purpose framework that could be used by Docker, but it wasn't clear to me where the new CLI tool fits in among the existing layers of runc, containerd and of course Docker itself. The only way to find out is to take the new release for a spin, since Apple open-sourced everything (well done!).
\nTo get the full experience, I chose to install the macOS Tahoe beta, as there have been improvements to the networking frameworks[1] that are only present in the new beta. It's essential you only use the Xcode 26 beta as otherwise you'll get Swift link errors against vmnet. I had to force my installation to use the right toolchain via:
\nsudo xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer\nOnce that was done, it was simple to clone and install the container\nrepo with a make install. The first\nthing I noticed is that everything is written in Swift with no Go in sight.\nThey still use Protobuf for communication among the daemons, as most of the\nwider Docker ecosystem does.
![\"%c\"](\"/images/macos-ss-1.webp\" "\\"I")
Let's start our daemon up and take the container CLI for a spin.
$ container system start\nVerifying apiserver is running...\nInstalling base container filesystem...\nNo default kernel configured.\nInstall the recommended default kernel from [https://github.com/kata-containers/kata-containers/releases/download/3.17.0/kata-static-3.17.0-arm64.tar.xz]? [Y/n]: y\nInstalling kernel... \n⠙ [1/2] Downloading kernel 33% (93.4/277.1 MB, 14.2 MB/s) [5s]\nThe first thing we notice is it downloading a full Linux kernel from the Kata Containers project. This system spins up a VM per container in order to provide more isolation. Although I haven't tracked Kata closely since its launch in 2017, I did notice it being used to containerise confidential computing enclaves while Zahra Tarkhani and I were working on TEE programming models a few years ago.
\nThe use of Kata tells us that container spins up a new kernel using the\nmacOS Virtualization framework every time a new container is started. This\nis ok for production use (where extra isolation may be appropriate in a\nmultitenant cloud environment) but very memory inefficient for development\n(where it's usual to spin up 4-5 VMs for a development environment with a\ndatabase etc). In contrast, Docker for Mac uses a single Linux kernel and runs\nthe containers within that instead.
It's not quite clear to me why Apple chose the extra overheads of a\nVM-per-container, but I suspect this might be something to do with running code securely\ninside the many hardware enclaves\npresent in modern Apple hardware, a usecase that is on the rise with Apple\nIntelligence.
\nOnce the container daemon is running, we can spin up our first container using Alpine, which uses the familiar Docker-style run:
$ time container run alpine uname -a \nLinux 3c555c19-b235-4956-bed8-27bcede642a6 6.12.28 #1 SMP\nTue May 20 15:19:05 UTC 2025 aarch64 Linux\n0.04s user 0.01s system 6% cpu 0.733 total\nThe container spinup time is noticable, but still less than a second and pretty acceptable for day to day use. This is possible thanks to a custom userspace they implement via a Swift init process that's run by the Linux kernel as the sole binary in the filesystem, and that provides an RPC interface to manage other services. The vminitd is built using the Swift static Linux SDK, which links musl libc under the hood (the same one used by Alpine Linux).
\nWe can see the processes running by using pstree:
\n|- 29203 avsm /System/Library/Frameworks/Virtualization.framework/\n Versions/A/XPCServices/com.apple.Virtualization.VirtualMachine.xpc/\n Contents/MacOS/com.apple.Virtualization.VirtualMachine\n|- 29202 avsm <..>/plugins/container-runtime-linux/\n bin/container-runtime-linux\n --root <..>/f82d3a52-c89b-4ff0-9e71-c7127cb5eee1\n --uuid f82d3a52-c89b-4ff0-9e71-c7127cb5eee1 --debug\n|- 28896 avsm <..>/bin/container-network-vmnet\n start --id default\n --service-identifier <..>network.container-network-vmnet.default\n|- 28899 avsm <..>/bin/container-core-images start\n|- 29202 avsm <..>/bin/container-runtime-linux\n --root <..>/f82d3a52-c89b-4ff0-9e71-c7127cb5eee1\n --uuid f82d3a52-c89b-4ff0-9e71-c7127cb5eee1 --debug\n|- 28896 avsm <..>/container-network-vmnet start --id default\n --service-identifier <..>network.container-network-vmnet.default\nYou can start to see the overheads of a VM-per-container now, as each container\nneeds the host process infrastructure to not only run the computation, but also to\nfeed it with networking and storage IO (which have to be translated from the\nhost). Still, its a drop in the ocean for macOS these days, as I'm running 850\nprocesses in the background on my Macbook Air from an otherwise fresh\ninstallation! This isn't the lean, fast MacOS X Cheetah I used on my G4 Powerbook anymore,\nsadly.
\nI then tried to run a more interesting container for my local dev environment:\nthe ocaml/opam Docker images that we use\nin OCaml development. This showed up an interesting new twist in the Apple\nrewrite: they have an entire ext4 filesystem implementation written in\nSwift!\nThis is used to extract the OCI images from the Docker registry and then\nconstruct a new filesystem.
\n$ container run ocaml/opam opam list\n⠦ [2/6] Unpacking image for platform linux/arm64 (112,924 entries, 415.9 MB, Zero KB/s) [9m 22s] \n⠹ [2/6] Unpacking image for platform linux/arm64 (112,972 entries, 415.9 MB, Zero KB/s) [9m 23s] \n⠇ [2/6] Unpacking image for platform linux/arm64 (113,012 entries, 415.9 MB, Zero KB/s) [9m 23s] \n⠼ [2/6] Unpacking image for platform linux/arm64 (113,059 entries, 415.9 MB, Zero KB/s) [9m 23s] \n⠋ [2/6] Unpacking image for platform linux/arm64 (113,104 entries, 415.9 MB, Zero KB/s) [9m 24s] \n# Packages matching: installed \n# Name # Installed # Synopsis\nbase-bigarray base\nbase-domains base\nbase-effects base\nbase-threads base\nbase-unix base\nocaml 5.3.0 The OCaml compiler (virtual package)\nocaml-base-compiler 5.3.0 pinned to version 5.3.0\nocaml-compiler 5.3.0 Official release of OCaml 5.3.0\nocaml-config 3 OCaml Switch Configuration\nopam-depext 1.2.3 Install OS distribution packages\nThe only hitch here is how slow this process is. The OCaml images do have a lot of individual\nfiles within the layers (not unusual for a package manager), but I was surprised that this took\n10 minutes on my modern M4 Macbook Air, versus a few seconds on Docker for Mac. I filed a bug upstream to investigate further since (as with any new implementation) there are many edge cases when handling filesystems in userspace, and the Apple code seems to have other limitations as well. I'm sure this will all shake out as the framework gets more users, but it's worth bearing in mind if you're thinking of using it in the near term in a product.
\nI was super excited when this announcement first happened, since I thought it might be the beginning of a few features I've needed for years and years. But they're missing...
\nIn OCaml-land, we have gone to ridiculous lengths to be able to run macOS CI on our own infrastructure. Patrick Ferris first wrote a custom snapshotting builder using undocumented interfaces like userlevel sandboxing, subsequently taken over and maintained by Mark Elvers. This is a tremendous amount of work to maintain, but the alternative is to depend on very expensive hosted services to spin up individual macOS VMs which are slow and energy hungry.
\nWhat we really need are macOS containers! We have dozens of mechanisms to run Linux ones already, and only a few heavyweight alternatives to run macOS itself within macOS. However, the VM-per-container mechanism chosen by Apple might be the gateway to supporting macOS itself in the future. I will be first in line to test this if it happens!
\nWaaaay back when we were first writing Docker for Mac, there were no mainstream users of the Apple Hypervisor framework at all (that's why we built and released Hyperkit. The main benefit we hoped to derive from using Apple-blessed frameworks is that they would make our app App-Store friendly for distribution via those channels.
\nBut while there do exist entitlements to support virtualisation on macOS, there is no support for iOS or iPadOS to this day! All of the trouble to sign binaries and deal with entitlements and opaque Apple tooling only gets it onto the Mac App store, which is a little bit of a graveyard compared to the iOS ecosystem.\nThis thus remains on my wishlist for Apple: the hardware on modern iPad adevices easily supports virtualisation, but Apple is choosing to cripple these devices from having a decent development experience by not unlocking the software capability by allowing the hypervisor, virtualisation and container frameworks to run on there.
\nOne reason to run Linux containers on macOS is to handle machine learning workloads. Actually getting this to be performant is tricky, since macOS has its own custom MLX-based approach to handling tensor computations. Meanwhile, the rest of the world mostly uses nVidia or AMD interfaces for those GPUs, which is reflected in container images that are distributed.
\nThere is some chatter on the apple/container GitHub about getting GPU passthrough working, but I'm still unclear on how to get a more portable GPU ABI. The reason Linux containers work so well is that the Linux kernel provides a very stable ABI, but this breaks down with GPUs badly.
\nI have mixed feelings about the Containerization framework release. On one hand, it's always fun to see more systems code in a new language like Swift, and this is an elegant and clean reimplementation of classic containerisation techniques in macOS. But the release fails to unlock any real new end-user capabilities, such as running a decent development environment on my iPad without using cloud services. Come on Apple, you can make that happen; you're getting ever closer every release!
\nI don't believe that Docker or Orbstack are too threatened by this release at this stage either, despite some reports that they're being Sherlocked. The Apple container CLI is quite low-level, and there's a ton of quality-of-life features in the full Docker for Mac app that'll keep me using it, and there seems to be no real blocker from Docker adopting the Containerization framework as one of its optional backends. I prefer having a single VM for my devcontainers to keep my laptop battery life going, so I think Docker's current approach is better for that usecase.
\nApple has been a very good egg here by open sourcing all their code, so I believe this will overall help the Linux container ecosystem by adding choice to how we deploy software containers. Well done Michael Crosby, Madhu Venugopal and many of my other former colleagues who are all merrily hackily away on this for doing so! As an aside, I'm also just revising a couple of papers about the history of using OCaml in several Docker components, and a retrospective look back at the hypervisor architecture backing Docker for Desktop, which will appear in print in the next couple of months (I'll update this post when they appear). But for now, back to my day job of marking undergraduate exam scripts...
\nvmnet is a networking framework for VMs/containers that I had to reverse engineer back in 2014 to use with OCaml/MirageOS.
\n ↩I stayed on for a few days extra in Washington DC after the biodiversity extravaganza to attend a workshop at legendary National Geographic Basecamp. While I've been to several NatGeo Explorers meetups in California, I've never had the chance to visit their HQ. The purpose of this was to attend a workshop organised by Christian Rutz from St Andrews about the "Urban Exploration Project":
\n\n\n[The UEP is a...] global-scale, community-driven initiative will collaboratively track animals across gradients of urbanization worldwide, to produce a holistic understanding of animal behaviour in human-modified landscapes that can, in turn, be used to develop evidence-based approaches to achieving sustainable human-wildlife coexistence.\n-- Christian Rutz's homepage
\n
This immediately grabbed my interest, since it's a very different angle of biodiversity measurements to my usual. I've so far been mainly involved in efforts that use remote sensing or expert range maps, but the UEP program is more concerned with the dynamic movements of species. Wildlife movements are extremely relevant to conservation efforts since there is a large tension between human/wildlife coexistence in areas where both communities are under spatial pressure. Tom Ratsakatika for example did his AI4ER project on the tensions in the Romanian Carpathian mountains, and elephant/human conflicts and tiger/human conflicts are also well known.
\nThe core challenge posed at the workshop was how to build momentum for the UEP's vision of fostering human–wildlife coexistence in the world's unprotected areas (often, this is areas near urban expansion zones like cities). The UEP idea sprang from Christian's earlier efforts after the pandemic on the COVID-19 Bio-Logging that built up a database of over 1 billion satellite fixes for ~13,000 tagged animals across ~200 species. The lead student on that work, Diego Ellis Soto has since graduated and was also at the UEP workshop sitting beside me!
\n![\"%c\"](\"/images/ngs-2.webp\" "\\"NatGeo")
The workshop itself wasn't fully public (not because it's secret, but just because the details are still being iterated on), so here are some high-level takeaways from my conversations there...
\nI've used iNaturalist and OpenStreetMap extensively for wildlife occurrence and urban data, but I'm less familiar with how animal movement data is recorded. Martin Wikelski was at the workshop and explained the ICARUS project to me, which collected data fitted to animals via GPS transmitters. This is then fed into the MoveBank service that is custom-designed for movement data.
\nUnlike most other biodiversity data services though, MoveBank data is not immediately made public (due to the sensitivity of animal movements), but is licensed to the user that made it. For that reason, it's less of a "social" service than iNaturalist, but still has a staggering 11 million records added every day. This data is then fed into GBIF, although it is downsampled to a single record per day. Martin also indicated to me that they're considering federating Movebank to other countries, which is important as biodiversity data resilience was a hot topic in our meeting a few days before.
\n![\"%c\"](\"/images/ngs-3.webp\" "\\"The")
I was really struck by how deeply the National Geographic staff were thinking about and co-designing solutions for along with the academics involved. I got chatting to Ian Miller, the chief scientist at NatGeo about his scientific background (he's worked on all seven continents!) and how our conservation evidence database might be of use to help the Society figure out the long-term impacts of their projects. I also met the person with the coolest job title there: Alex Tait, who is The Geographer at the NGS. Alex, along with Lindsay Anderson and other NGS staff who participated, all had infectious enthusiasm about exploration combined with an encyclopedic knowledge of specific projects that they support involving explorers across the world.
\nThese projects ranged from the Amazon River Dolphins (to understand aquatic health) over to cephalopod empathy) and many more. These gave me a new perspective on the importance of storytelling as a key mechanism to help connect the dots from conservation actions to people; something that I've been learning from Bill Sutherland's video series as well!
\n ![\"%c\"](\"/images/ngs-5.webp\" "\\"I")
It's also worth noting that the NGS support goes beyond "just" filmmaking. Our own Charles Emogor is also an Explorer, and recently received support from their Exploration Technology Lab to get a bunch of biologgers to support his research on mapping hunting pressures. Rather than placing a few big bets, the Society seems to focus on investing widely in a diverse range of people and geographies.
\nA lot of the discussion at the workshop naturally focussed on charismatic mammals such as the amazing work done by the Zambian Carnivore programme. However, I also had in mind the importance of addressing issues closer to home in the UK as well so that we didn't ignore Europe.
\nLuckily, before the workshop, I had grabbed a coffee with Silviu Petrovan from the CCI, who has been bringing me up to speed on the National Hedgehog Monitoring programme (did you know that British hedgehogs are now vulnerable to extinction?). This particular effort seems to tick a lot of boxes; it's a local and beloved species in the UK, it requires evidence-based interventions to avoid making the problems worse, and also requires combining data sources (from camera traps to species distribution models to urban planning to the GPS Movebank data) to build up a really accurate high res picture of what's going on.
\nI brought up UK hedgehog conservation at the NatGeo workshop, and then while down at Earthfest at Google a few days later I learnt from Drew Purves that they've developed an extremely high-res map of woodland and hedgerows in the UK. I've therefore created a new student project on hedgehog mapping and hope to recruit a summer internship for this. It would be extremely cool to put the pieces together with a very concrete project such as this as a first small step for the UEP.
\n![\"%c\"](\"/images/ngs-1.webp\" "\\"NatGeo")
I found the whole experience of visiting National Geographic inspirational, and not just because of the projects discussed. The walls of their HQ are full of incredible photographs of explorers all over the world, and a seemingly unbounded enthusiasm for exploring the unknown. I kind of thought I'd aged out on applying to become an explorer, but Kathy Ho has been encouraging me to apply, and the same was echoed by the lovely conversations with NatGeo staffers.
\nI'm therefore putting on my thinking hat on for what my Explorers project proposal should be, as I am on academic sabbatical next year and have more freedom to travel; suggestions are welcome if you see me at the pub!
\n![\"%c\"](\"/images/ngs-4.webp\" "\\"I")
I spent a couple of days at the National Academy of Sciences in the USA at the invitation of the Royal Society, who held a forum on "Measuring Biodiversity for Addressing the Global Crisis". It was a packed program for those working in evidence-driven conservation:
\n\n\nAssessing biodiversity is fundamental to understanding the distribution of biodiversity, the changes that are occurring and, crucially, the effectiveness of actions to address the ongoing biodiversity crisis. Such assessments face multiple challenges, not least the great complexity of natural systems, but also a lack of standardized approaches to measurement, a plethora of measurement technologies with their own strengths and weaknesses, and different data needs depending on the purpose\nfor which the information is being gathered.
\nOther sectors have faced similar challenges, and the forum will look to learn from these precedents with a view to building momentum toward standardized methods for using environmental monitoring technologies, including new technologies, for particular purposes.\n-- NAS/Royal Society US-UK Scientific Forum on Measuring Biodiversity
\n
I was honoured to talk about our work on using AI to "connect the dots" between disparate data like the academic literature and remote observations at scale. But before that, here's some of the bigger picture stuff I learnt...
\n ![\"%c\"](\"/images/nas-rs-cover.webp\" "\\"Identifying")
The need for urgent, additional action came across loud and clear from all the top actors in biodiversity. On the bright side, we have made stellar progress in measuring more dimensions of biodiversity accurately than ever before in human history. But, the field of biodiversity does not have a single "simple question" that needs answering, unlike many other science challenges in physics or chemistry. The ecosystem of nature measurements need to span scales ranging from the micro (from fungi and soil health) to the macro (species richness and diversity), with geographical coverage across the planet but also hyperlocal accuracy for ecosystem services.
\nOne key question asked at the forum was how we can get to interoperable, pragmatic tools that enable all the actors involved in conservation actions (from the governments that set policy, to the private sector that controls the supply chains, to the people who have to live in and depend on natural services) to work together more effectively on gathering all the data needed.
\nThis interoperability has to emerge during a rapid shift towards digital methods, which are vulnerable to being deleted and edited at scale with decades of painstaking observations at risk at the moment. And in the middle of all this, machine learning is swooping in to perform data interpolation at scale, but also risks dividing and polluting observations with inaccurate projections.
\n
This is all quite the challenge even for a gung-ho computer scientist like me, and I was struggling with the enormity of it all! But things really clicked into place after the inspirational Julia P.G. Jones pointed me at a fantastic big-picture paper:
\n\n\nDrawing reasonable inferences from current patterns, we can predict that 100 years from now, the Earth could be inhabited by between 6-8 billion people, with very few remaining in extreme poverty, most living in towns and cities, and nearly all participating in a technologically driven, interconnected market economy.
\n[...] we articulate a theory of social–environmental change that describes the simultaneous and interacting effects of urban lifestyles on fertility, poverty alleviation, and ideation.
\nFrom Bottleneck to Breakthrough: Urbanization and the Future of Biodiversity Conservation
\n
They observe that the field of conservation has often "succumbed to jeremiad, bickering, and despair". Much of this angst springs from the (failed) bets made by Paul Ehlrich, who thinks humans are going to be wiped out because of unbounded expansion. In response, conservation has become "the art of slowing declines" rather than achieving long term wins. But instead of being moribund, the paper paints an optimistic, practical endgame for conservation:
\n\n\nWe suggest that lasting conservation success can best be realized when:
\n\n
\n- the human population stabilizes and begins to decrease
\n- extreme poverty is alleviated
\n- the majority of the world's people and institutions act on a shared belief that it is in their best interest to care for rather than destroy the natural bases of life on Earth.
\n
It turns out that most of these conditions can be reasonably projected to happen in the next fifty years or so. Population is projected to peak by the turn of the century, extreme poverty might reasonably be eradicated by 2050, and urban landuse will stabilise at 6% of terrestrial land by 2030-ish.
\n ![\"%c\"](\"https://academic.oup.com/view-large/figure/118140827/biy039fig4.jpeg\" "\\"Connecting")
Given this projection, the paper then points out that conservation doesn't need to save nature "forever". Instead, we have to save enough nature now to "breakthrough" from the great acceleration of WWII until we stabilise landuse.
\n\n\nThe profound danger is that by the time the foundations of recovery are in place, little of wildlife and wild places will be left. If society focuses only on economic development and technological innovation as a mechanism to pass through the bottleneck as fast as possible, then what remains of nature could well be sacrificed.\nIf society were to focus only on limiting economic growth to protect nature, then terrible poverty and population growth could overwhelm what remains.
\nEither extreme risks narrowing the bottleneck to such an extent that our world passes through without its tigers, elephants, rainforests, coral reefs, or a life-sustaining climate. Therefore, the only sensible path for conservation is to continue its efforts to protect biodiversity while engaging in cities to build the foundations for a lasting recovery of nature.\n-- From Bottleneck to Breakthrough
\n
This puts what we need to achieve today in a far, far more pragmatic light:
\n\n\n[...] it means that conservation faces another 30–50 years of extreme difficulty, when more losses can be expected. However, if we can sustain enough nature through the bottleneck—despite climate change, growth in the population and economy, and urban expansion—then we can see the future of nature in a dramatically more positive light.
\n
Conservation is all about solving difficult opportunity-cost decisions in society.\nScience can help calculate credible counterfactuals that allow policymakers to balance\nlimited resources to minimise nature harm while maximising benefit to humans. We can also figure out new economic methods to figure out the value of future actions. When combined, this can help conservation break through the bottleneck of the next fifty years of nature loss... and computer science can make a serious accelerative impact here (yay!).
\n![\"%c\"](\"/images/nas-rs-5.webp\" "\\"What")
Having got my existential big-picture crisis under control, here are some more concrete thoughts about some of the joint ideas that emerged from the NAS meeting.
\nWe've been doing a lot of work on mechanisms to process and ingest remote sensing data. All of our techniques also apply to biodiversity, except that the pipelines are even more complex due to the multi-modal nature of the data being stored. This can be clearly seen in this review on the decline of insect biodiversity that speaker Nick Isaac and my colleague Lynn Dicks published last month.
\n ![\"%c\"](\"/images/nas-rs-1.webp\" "\\"(source:")
The data itself isn't just from one source; instead, we need a pipeline of spatial (at different resolution) measurements, of different types (visual, acoustic, occurrence), of different provenance (experts, crowdsourced, museum), and from different hypotheses tests (evidence bases).
\nOnce the ingestion pipeline is in place, there's a full range of validation and combination and extrapolation involved, often involving AI methods these days. The output from all of this is then tested to determine which conservation actions to take.
\n![\"%c\"](\"/images/nas-rs-3.webp\" "\\"Nick")
Andrew Gonzalez also talked about the ambitious global biodiversity observing system that he's been assembling a coalition for in recent years. They are using Docker as part of this via their Bon in a Box product but hitting scaling issues (a common problem due to the size of geospatial tiles).
\n ![\"%c\"](\"/images/nas-rs-7.webp\" "\\"Andrew")
There's a good tie in for collaboration with us here via the next-generation time-travelling shell that Patrick Ferris is developing that can handle this via ZFS snapshots. Michael Dales has been applying this to scaling the LIFE and FOOD pipelines recently with Alison Eyres and Thomas Ball. And meanwhile Josh Millar and Hamed Haddadi have been researching embedded biodiversity sensors. The overall theme is that we need to make the hardware and software stack involved far easier to use for non-expert programmers.
\n![\"%c\"](\"/images/nas-rs-8.webp\" "\\"A")
Another problem that several speakers discussed was how complex biodiversity observations are to manage since they span multiple scales. In my talk, I described the new TESSERA geospatial foundation model that Frank Feng, Srinivasan Keshav and Sadiq Jaffer have been leading in Cambridge. As this is a pre-trained foundation model, it needs to be finetuned to specific downstream tasks. A number of people came up after my talk with suggestions for collaborations here!
\nFirstly, Kat Bruce (fresh from spraying pondwater with Prince William) explained how NatureMetrics are gathering eDNA from many diverse sources. The data is of varying licenses depending on which customer paid for the acquisition, but overall there is a lot of information about species presence that's very orthogonal to the kind of data gathered from satellite observations.
\n![\"%c\"](\"/images/nas-rs-4.webp\" "\\"Kat")
Secondly, Barnabas Daru from Stanford described his efforts to map plant traits to species distribution models. This complements some work David Coomes has been leading recently in our group with Ian Ondo and Neil Burgess on mapping rare plants globally. The basic problem here is that plant occurrence data is extremely data deficient and spatially biased for 100k+ species, and so we'll need cunning interpolation techniques to fill in the data gaps.
\n![\"%c\"](\"/images/nas-rs-12.webp\" "\\"Barnabas")
When back in Cambridge, I'm going to arrange for all of us to chat to see if we can somehow combine eDNA, fungal biodiversity, plant traits and satellite foundation models into a comprehensive global plant species map!
\nThere was also huge enthusiasm for another of our projects on analysing the academic literature at scale. While we've been using it initially to accelerate the efficiacy and accuracy of systematic reviews for Conservation Evidence, there are a huge number of followup benefits for having a comprehensive data corpus.
\nFirstly, Chris Elphick pointed out a metasynthesis where they manually integrate recent hypotheses about insect stressors and responses into a network (3385 edges / 108 nodes). It found that the network is highly interconnected, with agricultural intensification often identified as a root cause for insect decline. Much like the CE manually labeled dataset, it should be possible to do hypothesis searches in our LLM pipeline to expand this search and make it more dynamic.
\nSecondly, Oisin Mac Aodha, fresh from a recent talk in Cambridge, discussed his recent work on few-shot species range estimation and also WildSAT text/image encoding. His example showed how you could not only spot a species from images, but also use text prompts to refine the search. An obvious extension for us to have a go at here is to combine our large corpus of academic papers with these models to see how good the search/range estimation could get with a much larger corpus of data.
\n![\"%c\"](\"/images/nas-rs-13.webp\" "\\"I")
And thirdly, I finally met my coauthor David Williams in the flesh for the first time! We've worked together recently on the biodiversity impact of food, and we had a long discussion over dinner about whether we could glean more behavioural data about how people react from the wider literature. This would require us expanding our literature corpus into grey literature and policy documents, but this is something that Sadiq Jaffer and I want to do soon anyway.
\nThe connective tissue across these seemingly disparate projects is that there is a strong connection between what you can observe from space (the canopies of trees) to the traits expressed via knowledge of plant physiology and their DNA. If we could figure out how to connect the dots between the observed species to the physiological traits to the bioclimatic range variables, we could figure out where the (many) data-deficient plant species in the world are! I'll be hosting a meeting in Cambridge soon on this since we're already working on it.
\nThe most unexpectedly cool talk was Ron Milo showing us visualisations of the mass distribution of all life on earth. His work really puts our overall challenge into context, as it shows just how utterly dominated wildlife is by domesticated animals.
\n![\"%c\"](\"/images/nas-rs-11.webp\" "\\"The")
It struck me just how important these sort of high-level visualisations are in putting detailed numbers into context. For example, he also broke down global biomass that showed that plants are by far the "heaviest" living thing on earth, and that the ocean organisms do still dominate animal biomass.
\n![\"%c\"](\"/images/nas-rs-9.webp\")
![\"%c\"](\"/images/nas-rs-10.webp\")
My favourite new animation library on the block is AnimeJS, and so once I plan to try to do some nice animations for LIFE and FOOD along these lines after the academic term finishes.
\nAnd that's a wrap on my notes for now! I'm still hanging out in the US for a bunch more meetings (including one at National Geographic HQ), so I'll update this note when the official RS/NAS videos and writeup comes out.
\n(Update 5th June: the full talk videos series is now online at the National Academy of Sciences channel. Enjoy!)
What might a Dame of the Realm, a Fellow of the Royal Society, the latest member of the UK Joint Nature Conservation Committee, and me all covet? That's right: a Junior Ranger badge from Shenandoah National Park! After an intense few days, Bill Sutherland, Julia P.G. Jones, EJ Milner-Gulland and I headed into nature to experience the spectacular landscapes of the Blue Ridge Mountains in Virginia and do some birding.
\nThe National Park Service in the US runs a wonderful program for anyone aged 8+ (which we just about qualified for) to introduce people to nature, and Shenandoah is no exception. We visited the local ranger lodge in the park, and picked up a program booklet. They're full of activities for kids to do, but of course adults also pick up a lot of random knowledge (such as the endemic salamander species in the region).
\n![\"%c\"](\"/images/shen-1.webp\" "\\"EJ")
The activities are rigorous: in addition to crosswords and drawings about the park, we had to compose poems and haiku, and also experience nature. First, we had to dance like butterflies... (I was a Monarch, so I occasionally had to respawn while migrating).
\n\n...and then hug a tree, of which there were many around, and they told us their stories...
\n![\"%c\"](\"/images/shen-2.webp\")
...and try not to fall off sweeping overlooks as we clambered around...
\n![\"%c\"](\"/images/shen-3.webp\")
And once that was done, we returned to the ranger lodge to swear our ranger oaths, with a delightful Ranger who knew lots about the local area, the forest/meadow management, and where we might spot more of the salamanders (which are apparently under some threat from an invasive look-a-like that is not endemic but outcompeting the local salamandar).
\n\nDuring our hikes to the various lovely sites, we also took some time to record the 101st video for Bill Sutherland's wonderful Conservation Concepts video series, about the work Sadiq Jaffer and I have been helping out the Conservation Evidence Copilots team on.
\n\nI highly recommend watching Bill's whole channel series as it's full of endlessly fascinating facts, and congratulations to Bill on reaching the century mark on the number he's done!
\n![\"%c\"](\"/images/shen-4.webp\" "\\"I'm")
Josh Millar has just released the latest survey paper he lead on energy-aware approaches to optimise deep-learning training and inference on embedded devices, such as those benchmarked in "Benchmarking Ultra-Low-Power μNPUs" recently. This comprehensive survey is particularly relevant for IoT and mobile devices that face substantial energy constraints to prolong battery life or operate intermittently via energy harvesting. Josh synthesizes the evolving landscape of energy-aware deep learning approaches, examining their methodologies, implications for energy consumption and system-level efficiency, and their limitations across different network types, hardware platforms, and application scenarios. The work is especially timely given the push to run more AI workloads on edge devices for privacy, latency, and cost reasons.
\n\n\nWe present an overview of such approaches, outlining their methodologies, implications for energy consumption and system-level efficiency, and their limitations in terms of supported network types, hardware platforms, and application scenarios. We hope our review offers a clear synthesis of the evolving energy-aware DL landscape and serves as a foundation for future research in energy-constrained computing.
\n
Any comments, please do let any of us know!
Our paper on how the careful design of LLMs is crucial for expert-level evidence retrieval has been published today in PLOS One and is available fully open access!
\n\n\nOur findings suggest that, with careful domain-specific design, LLMs could potentially be powerful tools for enabling expert-level use of evidence syntheses and databases. However, general LLMs used "out-of-the-box" are likely to perform poorly and misinform decision-makers. By establishing that LLMs exhibit comparable performance with human synthesis experts on providing restricted responses to queries of evidence syntheses and databases, future work can build on our approach to quantify LLM performance in providing open-ended responses.
\n
In a nutshell, we tested 10 LLMs with six different retrieval strategies on their ability to answer questions related to conservation, benchmarked against the Conservation Evidence database that has been hand-assembled by experts over the last two decades. In some of the retrieval scenarios, models were only allowed to use their pretrained knowledge, whereas in others they had access to the relevant parts of the hand-curated database.
\nWe found that language models had very varying results when relying only on their pretrained data, and were particularly bad at answering questions about reptile conservation.\nHowever, given some extra training with the CE database, their performance improved dramatically.\nWhen we put these models head to head with human experts (from the conservation evidence team), with a set of questions and with RAG access to the database, we found that the models were just as good as our experts, but answered the questions much much much faster (near instant).
\nEssentially, LLMs without extra training are likely to perform poorly and misinform decision-makers. This is crucial when considering how to build AI infrastructure for public policymaking.
\nParticular props to Radhika Iyer who did much this work in her summer break last year as part of the UROP program.\nSee also the fantastic EEG seminar video below of the talk that the student group who worked on this over the summer gave towards the end of 2024! And as an opportunistic advert, we are recruiting (with Alec Christie, Sam Reynolds, Sadiq Jaffer, Bill Sutherland and the rest of the CE team) for more undergrads for the coming summer.
\n\nThis summary is adapted from a social media summary by Sam Reynolds. Join the conversation there!
I joined Cambridge's loftily named Environment Sustainability Strategy Committee this academic year, and have attended a couple of meetings with the latest one being held today. While a lot of what goes on is intricately tied into the University's rather special governance structure and the complexity of the College system, there has been significant progress on making all of this more visible more widely.
\nSally Pidgeon, our wonderful head of Enviromental Sustainaibility, has been redeveloping the public website and has put a lot of interesting data online.\nThere is now a new Environmental Sustainability website that tracks the University committment structure more closely, with the areas broken up into Carbon & Energy, Travel & Transport, Waste & Circular Economy, Biodiversity, and Water usage.
\nThese pages makes it far clearer what our University aims are for operational environmental sustainability, and how we're getting there. There's also a dedicated area to track our actual progress along with a bunch of case studies such as our own David Attenborough Building at the CCI!
\nSome highlights from the progress as I read through them:
\n![\"%c\"](\"/images/essc-1.webp\")
![\"%c\"](\"/images/essc-2.webp\")
![\"%r\"](\"/images/bap-1.webp\")
Biodiversity progress is closest to my heart, but also the hardest to assess despite the comprehensive Biodiversity Action Plan from last year (not because anyone's doing a bad job, but biodiversity is just a really complicated metric!). There's a University-wide biodiversity manager now and a really well described set of action points here.\nSo, lots to digest in here, and something I'm still piecing together in the context of the Cambridge Green Blue idea! The overall message seems clear that we need to continue to push harder for progress towards our net-zero goals to be far higher up the University's strategic plan than it currently is. That doesn't necessarily just involve spending more money, but bringing the juggernaut of research innovation around here to bear, as well as shifting landuse for renewable energy while preserving biodiversity and water according to the biodiversity action plan.
\nPatrick Ferris has developed a carbon-intensity based on this reporting style which Ryan Gibb then used in a carbon-aware DNS server recently. This is an example of location-based emissions data being used.
\n ↩In my earlier note about how AI should unite conservation, I talked about the robust debate\nongoing within Cambridge about whether or not we're too "AI obsessed" and are losing track of our goals in the rush to adopt learning algorithms. Jacqueline Garget has written a brilliant roundup about how colleages like Sam Reynolds, Chris Sandbrook and Sadiq Jaffer in the\nCCI are leading conversations to make sure we advance with eyes wide open.
\n ![\"%c\"](\"/images/camacuk-ainature.webp\")
The article covers many areas of concern to us right now: the takeover by big tech companies of data, our own conservation copilot project, and ultimately how people and equity must remain at the centre of this process if we are to avoid causing harm to humans.
\n\n\nHave you ever persisted in following your SatNav even when you knew you were\ngoing in the wrong direction?
\nIf so, you’ll know that placing all your trust in a machine powered by AI, without also engaging your own intelligence, does not always get you where you want to go.
\nThis is the message that a group of conservation scientists at Cambridge is pushing hard.\nEfforts to protect the natural world need all the help they can get - but before embracing AI as the solution, we need discussions about its risks and wider implications.\n-- To save nature, AI needs our help - cam.ac.uk (2025)
\n
Last week, we held a brilliant half-day AI for Climate and Nature Day[1] with AI@Cam that had many of the CCI community present, and this topic was at the forefront of the group discussions at the end.
\n ![\"%c\"](\"/images/aicamday-1.webp\" "\\"An")
I thought Chris Sandbrook's point about societal change was key:
\n\n\nIf we give all our attention to inventing new AI tools to fix specific conservation problems - important as these are - we’re missing a trick."
\nAI’s biggest impact on biodiversity is probably going to be through the ways it changes wider society.\n-- Chris Sandbrook
\n
I've been thinking recently that this principle applies at a local level as well, and not just with respect to AI. We generally to figure out how to change incentives towards more positive collective action, with lightweight ways of keeping score that do not give perverse incentives to cheat.
\nOne really interesting path (pun intended) in this direction is Gabriel Mahler's project on generating urban walkability maps that I've been supervising this year for the CompSci MPhil. Gabriel combines Andres Zuñiga-Gonzalez's urban tree maps with OSM labels in order to help people to really enjoy walking around cities. Imagine you want to bias your experience of walking to work along different dimensions such as the chance of seeing a particular bird you like, or need to go shopping at a local coop, or need to find a safe running route late at night. AI should be a tool that helps you to do all of this, and improve the general experience a human wanting to get the most out of nature, and generally help humans value their wild neighbours.
\nI only had time to do a Bluesky post storm and Jon Ludlam did a roundup as well.
\n ↩Carl Edward Rasmussen recently gave a great talk in our group about his thoughts on mechanisms against climate change. He persuasively argued that the Paris Agreement was doing more harm than good by giving the illusion of being a concrete agreement, but is in reality a huge distraction. Our actual emissions have increased since the Paris agreement was signed!
\nCarl argues that a climate system ultimately only responds to collective actions, and without a global cooperative incentive each nation will spring back to their own isolated short-term incentives that lead to an increase in fossil fuel burning. He has just published the "Themis Mechanism" as a simple alternative for equitable global emission reduction (long form). (6th May 2025: See a new article on Themis as well)
\nThis got me brainstorming with Carl about how to test his theories out and we came up with an idea that is either terrible or awesome; please read on and judge appropriately. I think we should take advantage of Cambridge's unique structure to trial the Themis mechanism via a new competitive decarbonisation sporting league among Colleges that I dub the "Cambridge Green Blue". Given the Chancellor's recent unveiling of an innovation corridor between Oxford and Cambridge, the timing could not be better for an initiative like this. (TL;DR sign up at the bottom of this post if you'd like to participate)
\nFirst, let's understand what Carl is proposing, which is built on three foundations:
\n\n\n\n
\n- Our atmosphere is a shared resource, a commons. Fossil fuel users benefit fully from fuel\nconsumption, while the CO2 cost is spread globally. This dilution effect makes continued\nuse rational for individuals but collectively disastrous. [...] To prevent this,\nwe must cooperate to guarantee positive climate results.
\n- The root cause of climate change is the failure to account for the true cost of emissions.\nBy treating the atmosphere as a free resource, we encourage overexploitation. Themis\ncorrects this unpriced externality by pricing greenhouse gas emissions.
\n- Effective cooperation requires a fair guiding principle. Themis upholds equity: that our\natmospheric resources should be shared equally between all humans.\n -- Carl Edward Rasmussen, The Themis Mechanism
\n
As I noted last week, most tech companies regularly break future carbon pledges due to competitive pressure. So it's good to see that Themis requires only immediate commitments rather than long-term pledges which are impossible to police. Instead of forcing unwilling participants to join, Themis is a coalition in which partners check on each other, learn by doing, and build up mutual trust.
\n ![\"%c\"](\"/images/themis-ss-1.webp\")
The core scheme itself is based on a value Py which is the price of emitting a single ton of CO2 into the atmosphere in year y. Here's how it works:
\nThis mechanism only depends on per-capita emissions for one year, and not on\nany future pledges or historic emissions. If a country has above average per capita emissions, then\nthey pay into the common pool. If they are below average per capita, then the country\nbenefits from payments from the pool. The system permits co-existence with any other\ncarbon reduction efforts, and works with a non-exhaustive pool of nations participating.
\nThe main reason Themis might fail is that participating in the league disadvantages the participants vs those just continuing with business-as-usual. The economics theory behind Themis is similar to a Pigouvian tax\nwhich dates back to a century ago, when the Cambridge economist Arthur Pigou suggested in 1920 that a tax equal to the external cost of pollution could align private costs with social costs. This idea also works for discounting future actions, and is the basis for some of our own work on pricing impermanent but delayed emissions.
\nFrom an economic theory perspective, Pigou and the other prominent Cambridge economist at the time JM Keynes[1] had deep disagreements. Keynes argued for higher interest rates to boost aggregate growth, while Pigou wanted to give people an increase in real wealth relative to prices. Both of their approaches ultimately lost out by the 1980s as free market economics ruled supreme instead, leading to the current "grow, emit and die" competitive spiral of doom we find ourselves in. However, Pigou's theories are clearly ones we should revisit today in light of Themis: by raising the cost of emitting via taxes (or Themis contributions) we can incentivise countries to reduce pollution or decarbonise instead of treating the atmosphere as a free sink to dump into.
\nA modern counterpoint to the "lack of competitiveness" argument from participating in a emissions reduction competition is the increasing evidence of runaway tipping points that might suddenly need everyone to decarbonise really quickly. Katherine Hayhoe, the chief scientist at TNC observes that we can't adapt our way out of this climate crisis due to the sheer magnitude of change that will occur if we continue to emit.
\n\n\nOur infrastructure, worth trillions of dollars, built over decades, was built for a planet that no longer exists [...]\n - Katherine Hayhoe, The Guardian 2022
\n
This is a pragmatic point in favour of countries joining Themis, since participation strengthens their economic infrastructure towards decarbonisation. By joining, countries can trade off some short term losses in their economy with being well hedged for either a "sudden" black swan climate tipping point that requires rapid change in their societal infrastructure, and it also gives them a long-term advantage heading into the inevitable electric future. So perhaps the fact that things are now much worse since Paris could force the emergence of cooperative groups who wish to prepare for sudden change.
\nAs Carl Edward Rasmussen also notes in his Themis proposal, there is consensus among climate scientists that we must cooperate in the planetary commons if we are to succeed.\nBut his proposal seems overwhelmingly difficult to evaluate in a political climate that is moving away from global cooperation. There must be a way to try some of these ideas out at a smaller scale, and especially locally in our sleepy University town!
\nOne area where nations have remained cooperative through no clear immediate financial gain is that of competitive sport. We just had the Paris Olympics with almost every nation in the world competing for no good reason other than a desire to win. And they're not seeking to win money as in most other areas of competition; instead it's just virtual credit in the form of medal tables that are celebrated from the largest to the smallest countries!
\nSporting events such as the Olympics are highly structured events with clear rules dictating almost every aspect. An interesting consequence of decoupling the rules of the games from direct financial incentives is that many sports are not zero-sum games. In rugby union or football for example, the winner gains more than the loser loses. While this structure can encourage match-fixing due to the asymmetry, participants also build trust amongst themselves over the years, for example via promotion through divisions.\nGame theorists often note how stable cooperation emerges in infinitely repeated games. Sports seasons are simply repeated competitions; over time, codes of conduct evolve and become self-policing agreements for mutual benefit (avoiding injuries, preserving dignity in loss, etc). There are clear lessons for the Themis mechanism here, as it also needs to establish long-term cooperation deep into the next century until total CO2 levels decline.
\n![\"%c\"](\"/images/board-game-pd-1.webp\" "\\"If")
Away from physical sports, we also see similar scoring dynamics in boardgames! There is a whole genre of semi-competitive boardgames such as Archipelago which are "competitive games that everyone can lose". This sounds a lot like Themis; we want to be able to stave off emissions disaster, but otherwise be the top dog in our league for every other aspect of our societies! The game rules must be structured so that even selfish players find it in their interest to cooperate to avoid losing. In Archipelago, the rule is simple: if instability within the game hits a certain point, all players lose, which forces even the leader to sometimes help the laggard to save themselves.[2]
\nSo how is this relevant to evaluating the global Themis mechanism from earlier? Everything global must start locally, so I propose a new semi-competitive league here in Cambridge, with willing Colleges as participants, and with virtual points instead of using real currency. And just like the two century old tradition, we should make this sufficiently competitive to gain a coveted sporting blue! To give you some context, being really good at tiddlywinks can gain you a quarter blue.
\nIn the rest of this post, I've written up the structure of this league with Ostrom's principles in mind, by treating the CO2 management problem as a common pool resource.\nCambridge Colleges have been around for centuries and so naturally appreciate the long perspective required; Pembroke was founded in 1347. Our collective collegiate goal is to urgently reduce CO2e that accumulate in the atmosphere and contribute to climate change for hundreds of years. This requires cooperation and learning from each other, but also a certain drive to do better than each other to get to the goal as quickly as we can.
\nThe three key sources of carbon emissions this league would track would initially come from food, heating and travel, noting again that we are only measuring this year's reductions and emissions, not historic or future pledges. We need to design specific mechanisms for each of these, but I'll just sketch out what makes measuring each of these "interesting".
\nStudents, Fellows, visitors and staff all eat a lot of food in the Colleges from all over the world. Communal dining is so central to the Cambridge College experience that it is mentioned in many College statutes as part of our charitable purpose.
\n\n\nIn furtherance of the College’s purposes, Fellows shall be entitled to dine daily free of charge at common table.\n -- Pembroke College Statutes presented to Her Majesty in 2009
\n
Since thousands of meals go through a typical College every day, identifying pragmatic sources of emissions reductions is very important. In a recent committee meeting at Pembroke College, I was incredibly pleased to hear that we've reduced food waste from the kitchens down to just one or two meals a day (which, considering the vast number of meals served is hugely impressive).\nAnd similarly, Darwin College reported on the recent plant based May Ball which was a rather fine party, and the world did not end due to black tie attendees being unable to find a sausage roll.\nHow can we communicate the lessons learnt from the catering teams here to other Colleges? The CGB allows us to rank and categorise these initiatives!
\nResearch, with much of it conducted here in Cambridge, shows us that key gains in food impacts come from reducing ruminant meat consumption and the corresponding damage to tropical forests full of biodiversity.\nImportantly, we're not trying to force every College member to suddenly become vegan, but instead provide sustainable and healthy options.\nAndrew Balmford and Theresa Marteau have both shown that nudging consumers such as Cambridge students and staff towards less damaging choices by default is entirely practical, without alienating those that insist on their meat'n'twoveg:
\n\n\nA study of over 94000 cafeteria meal choices has found that doubling the vegetarian options from 1-in-4 to 2-in-4 increased the proportion of plant-based purchases by between 40-80% without affecting overall food sales.\n-- Veg nudge. Impact of increasing vegetarian availability on meals (paper / followup)
\n
The league does need some way to turn these initiatives into a points based system. This is where my colleague Thomas Ball's recent research is instructive. He's been working on quantifying the biodiversity cost of food imports, broken up by the food type. The CGB food points game could correlate consumption choices with where the food comes from and how much it is wasted, and so we could steadily work across Colleges on reducing our impact year-on-year.
\n ![\"%c\"](\"/images/tball-food-paper-ss-1.webp\" "\\"An")
Turning off the natural gas flows in Colleges is a major challenge. We have some of\nthe oldest buildings in the world around here, and much of the infrastructure is\ncorrespondingly aged. Pembroke has just spent a ton of cash on a communal heat pump for our new development in Mill Lane, which got me thinking about how this aspect of the CGB league could be based around this. The rules and regulations for heat pump installation in the UK are incredibly baroque, as Robert McQueen pointed out recently:
\n\n\nI have a neighbour who embarked on a planning application for a heat pump for his terraced house. There is a difference in ridiculous paperwork necessary simply to install <1m from the boundary compared to the presumed consent in permitted development. Of course now they are waiving that requirement but he's stuck half way through the process. I can't even imagine adding listed requirements into that
\n[...] due to be waived for permitted development - whether that tracks through to the full regulations is anyone's guess. They are already bafflingly inconsistent.\n-- Robert McQueen, Bluesky, Feb 2025
\n
However, the Cambridge City Council isn't sitting still and has been working with the University on this. Ian Leslie pointed me to city-wide explorations into district heating networks for Cambridge that includes a phase 1 report\nthat plots out what it might look like by using different Colleges as sinks and sources!
\n ![\"%c\"](\"/images/cambridge-district-heat-ss-1.webp\")
Darwin College also reports in their 2023 sustainability report the progress they've made on establishing heat pumps in the River Cam.
\n\n\nIn 2022, in a collaboration with six other riverside Colleges, Mott MacDonald were commissioned to monitor\nwater flow, depth and temperature at four locations on the river and to produce a detailed hydrology study.\nThe report, delivered in 2023, confirms the considerable potential of the river to supply heat for space\nand hot water heating for the adjacent Colleges.\n -- Darwin sustainability report, 2023
\n
And famously most recently, Kings College installed 400 solar panels on their world-famous chapel, despite opposition from Historic England. This sets a huge precedent for the rest of Cambridge to take similar action, and they deserve recognition for this from the CGB!
\n![\"%c\"](\"/images/kings-solar-panels.webp\" "\\"The")
So this aspect of the CGB league could focus on building spatial connections across Colleges. Perhaps the College that brings the most benefit to its neighbours by contributing the most towards a district heating mechanism could win this round.
\nFinally, lots of the Colleges do facilitate international travel, for a variety of reasons ranging from pedagogical to developmental. The most obvious one is when conducting in-person interviews, when candidates fly in from all over the world. Since the pandemic, there has been split opinion among Colleges about returning to in-person interviews or not, with Pembroke opting for in-person this year. While there are lots of good reasons to encourage in-person interactions, the carbon cost has been so low down in the discussion points in the meetings I've attended that it might as well not even be a factor. A CGB league might encourage us to tally up the scores across Colleges more systematically to factor in these costs into the overall decisionmaking.
\nThe other opposite end of the spectrum is international air travel for conferences, which are thankfully quite rare as most of our business is conducted locally. We do host events here such as the SCCS student conservation conference that flies in young scholars from all over the world, but this is quite rightly justified as being essential as it brings together underrepresented young students from all over the world who find tremendous value from meeting each other. I've made more extensive notes on the topic of travel mitigation elsewhere in my note on carbon contributions.
\nI've hopefully convinced you that there quite a few interesting dimensions around which we could design our semi-competitive Cambridge Green Blue (CGB) league. I've avoided over-specifying the rules at this early juncture, since I want to bring in more people's thoughts and ideas first. However, here's a strawman attempt.
\n\n\nWe treat the emission of CO2 into the atmosphere as a shared common pool resource (CPR); i.e. we can collectively only emit a limited amount if we are to avoid the worst effects of climate change. Cooperation on a global CPR should ideally happen on a global basis, however that current approach is inadequate. Therefore, we must locally initiate mechanisms which will build up into a global framework from the ground up. Cambridge Colleges are institutions for young people who will be greatly affected by climate change, and Colleges make decisions with long time horizons, and a body of scholars should represent intellectual leadership in a time of crises. Therefore Cambridge Colleges should be an ideal proving ground for exploring cooperative frameworks in practise!
\n
The CGB would select its initial College membership and define baseline rules about how to measure emissions collectively, based around the first interest areas of travel, food and heating described above. Members will then write a rule book that follows the Themis mechanism to establish a virtual price for each tonne of emissions, and we will self-report progress monthly with points assigned to those who are beating their baselines of emissions reduction interventions. The league is used to collectively learn from those who are winning, and equalise the playing field in future seasons for the others to catch up.
\nFollowing Ostrom's principles, the league looks like this:
\nOne very important aspect of this is to adopt a respectful "sportsmanship" rule to the relative ranking of Colleges, and not engage in shaming wars. There is a wide wealth disparity among the Cambridge Colleges, and we could adjust for this using the per-capita rules from the Themis mechanism. Ultimately, it's also about celebrating and learning from every participant and using the competition to spur us on, build each other up, and have fun doing so. We're all in this together.
\nYeah, I think this is worth a try! I have recently joined the University's Environmental Sustainability Strategy committee, and I've found it extremely difficult to educate myself about the local initiatives going on (not because of any individual's fault, but because there are 31 separate constituent Colleges and University and townspeople sharing a fairly small area). If nothing else, this initiative will let us collectively bring together a wiki of all the positive actions happening across Cambridge. If it succeeds though, I'd like to spread the next iteration of the league to other Universities to run their own (I'm looking at you, Oxford), and see if we can turn this into a distributed game.
\nI was reading Andrew Balmford's book Wild Hope over the weekend, and his conclusion at the end was that we must not lose hope in our quest for a biodiverse, equitable world. And given the chaotic start to 2025, I can't think of a better place to start something new than within Cambridge, with our collegiate structure already providing a ready-made framework.
\nSo what next? If you're interested in helping Carl Edward Rasmussen and me organise this, get in touch with either of us! I'm on academic sabbatical for a year from the summer, so I'll have loads of time. I'll edit this post with a list of first Colleges that have been in touch. We'll likely organise a pub get-together in early March (exact date to follow) to brainstorm about this without anyone interested.
\nThis post is the result of many conversations around Cambridgeshire over the past year, ranging from a balmy summer dinner in Ely with Andrew Balmford and Theresa Marteau, chilly autumn cups of tea in my Pembroke office with Carl Edward Rasmussen and Carl Henrik Ek, to misty morning coffees at Pages with Melissa Leach and Michael Dales or at Espresso Lane with Eleanor Toye Scott, to cosy pubs with Ian Leslie, Jon Crowcroft, David Coomes and Patrick Ferris, to College dinners with Sadiq Jaffer and Srinivasan Keshav, and EEG/CSG discussions with Thomas Ball, Alison Eyres, Orlando Timmerman, Thomas Swinfield, Ryan Gibb, Alastair Beresford, Neil Lawrence and Richard Mortier. Many thanks to them for corrections and feedback, and any remaining errors are my own. Changelog: 12th Feb added note on sportsmanship and Carl's NeurIPS@Cam talk. 6th May 2025: added Carl Edward Rasmussen's published article on Themis.
\nI promise I'm not a JMK shill, despite being a JMK Fellow.
\n ↩The keen boardgame player will probably observe that there's always one player who decides to cause trouble just for fun, making everyone lose. This can be dealt with by social means.
\n ↩With the sunsetting of Equinix Metal\nI've also been migrating the Recoil machines over to new hosts in Mythic\nBeasts. This time around, rather than manually\nsetting up services, I've turned to a nice new tool called\nKomodo which helps with deploying Docker\ncontainers across multiple servers. Unlike many other\ncontainer management solutions, Komodo is refreshingly simple. It has a mode\nwhere it can take existing Docker compose files on a\ngiven host, and run them, and provide a web-based monitor to keep an eye on a\nfew machines.
\nThere's an online demo of Komodo available (user/pass\nis demo/demo). The basic idea is that you first register servers (see below for\n"Periphery"), and then add in "Stacks" which represent a service each.
\n![\"%c\"](\"/images/komodo-1.webp\" "\\"The")
Every stack is configured to run a docker-compose.yml service that is already\npresent on the host, and the web UI has a convenient way of pulling, deploying\nand polling the Docker Hub to check for updates.
![\"%c\"](\"/images/komodo-2.webp\" "\\"The")
The autoupdate functionality is quite cool (if a touch risky), as it polls for the\nimages on the Docker Hub and updates to those automagically. While I've activated\nthis for services I'm happy autoupdating, it's also accompanied by a healthy\ndose of ZFS snapshotting so I can roll back if anything\nuntoward happens.
\n![\"%c\"](\"/images/komodo-3.webp\" "\\"The")
Most importantly to me is that I can always switch away from Komodo at any time\nand directly interact with the services on the host using the normal docker CLI.\nKomodo is just coordinating the compose invocations in the lightest way possible,\nand not wrapping them in such a way that I lose access.
Komodo operates across multiple hosts by using something called a periphery agent\nwhich the main host issues RPCs to in order to do something. This is obviously quite a privileged operation, and so rather than\nexpose it to the Internet I setup a Wireguard tunnel mesh across the Recoil hosts for these operations to go over.
\nThe easiest way to do this was via dsnet, which generates the configurations and keys\nsuitable for a wg-quick service to run on each host and connect\nto their peers. Following the instructions let me setup this mesh in minutes; this is a much simpler solution than\nTailscale due to the lack of flexibility, but all I want here is few hosts connected by static interfaces\nand with no need for complex NAT punching. Once the dsnet configuration is\nsetup, all that's needed is to activate the wg-quick service on each of the hosts, and they spin up a virtual interface.
After this, the Periphery setup was straightforward but with one twist. I configured the agent to bind to the wireguard IP, e.g.:
\n/etc/komodo/periphery.config.toml\n################################\n# 🦎 KOMODO PERIPHERY CONFIG 🦎 #\n################################\n\nport = 8120\nbind_ip = "10.100.0.2"\nBut then on reboot the periphery agent would fail to startup due to the wireguard service being too low a priority in the boot order. This was fixed by a systemd tweak (which took me longer to figure out than the rest of the entire setup altogether, since I find systemd utterly inscrutable).
\n/etc/systemd/system/periphery.service\n[Unit]\nDescription=Agent to connect with Komodo Core\nAfter=wg-quick@wg0.service\nThis little tweak to the script, followed by umpteen daemon-reload prods and\nreboots to get systemd happy, did the trick.
I'm pretty happy with Komodo, thank you to the devs! It's a system that's simple enough that I can try\nit out progressively, and can bypass easily if required, and provides a very\nuseful part of the ##selfhosting jigsaw puzzle.
I had a tremendous time participating in last year's horizon scan of AI and Conservation, which laid out the opportunities that technological progress from AI (a catchall phrase here) could bring to hard-working conservation practitioners. Since then, there's been a lot of corridor conversations about future projects (and even dinner with the Wildlife Trusts). However, there has also been discussion about the potential harms of our work, most notably in a response letter to our paper written by Katie Murray and colleagues.
\nMurray et al make two really important points:
\n\n\n\n
\n- [...] importance of ecological expertise must be recognised as much more than just the expert annotation of training data
\n- [...] effort should be made to build capacity for AI development in the Global South, so that the rewards of successful research can be shared\n-- The potential for AI to divide conservation
\n
Myself and the co-authors of the original horizon scan could not agree more with this statement, and Sam Reynolds lead us to publish a response-to-the-response [1] dubbed "Conservation changed but not divided".
\n ![\"%c\"](\"/images/cam-nature-3.webp\")
In our response, we note that:
\n\n\nWe agree wholeheartedly with these points and recognise that the task of equitable integration of AI into conservation is beyond the scope of any single group and requires collective action.
\n[...] Developers of AI tools have a foundational role to play in delivering an equitable AI landscape. Technologies disconnected from pragmatic ecological, cultural, and socioeconomic factors are unlikely to advance the field [...]
\n[...] Developers should adopt participatory design and development principles, identifying conservation actors to guide the process, designing [...] protocols that respect cultural sensitivities and Indigenous and local knowledge [...]
\n[...] All tools should be open source and thoroughly documented, so that they can be easily adapted for local contexts.\n-- Conservation changed but not divided
\n
Many thanks to Katie Murray and colleagues for taking the trouble to call out the issues in our original paper! Both of the letters will be side-by-side in the next issue of Trends in Ecology and Evolution and, of course, we welcome any more perspectives about either of these.
\nThis whole discourse also happened at the same time as Cambridge dove in with a major piece about turbocharging the race to protect nature and climate with AI. The piece itself (lead by the brilliant Jacqueline Garget and Louise Walsh) covers a number of the projects in our AICN project we started last year.
\n ![\"%c\"](\"/images/cam-nature-1.webp\")
The online story itself is a rather gorgeous layout, with pieces on:
\nWhile each of these projects are fascinating research projects, the bit that made me stop and really think was the last interview with me about how AI could heal in the planet. In it, I talk about conservation from a technological lens:
\n\n\nWe need to act fast to mitigate the impacts of climate change, and to protect and restore biodiversity. There’s incredible potential in using AI to augment our work. It enables us to do things much more quickly – it’s like giving humans global knowledge superpowers!
\n
But, after more corridor conversations with colleagues in the CCI[2] more important angles to this story emerged. It's really easy for us to lose sight of the fact that AI is just a piece in the puzzle; a means to an end. We must keep the focus on the giant crisis in biodiversity unfolding in front of our eyes like a slow motion steamroller. Other pieces on the Cambridge website that cover this include the pollinator risk index (with Lynn Dicks), lead poisoning of grouse, carbon emissions from peatland fires, or the risks of biodiversity leakage causing extinctions (by Andrew Balmford).\nIt's not all bad news of course! Cambridge has also covered thriving Antarctic ecosystems and success stories in species restoration.
\nMy overall concerns with current central University (and world's) focus on AI stem from:
\n ![\"%c\"](\"/images/cam-nature-2.webp\" "\\"Would")
I only really started thinking about this properly after talking to Jacqueline and Sam, so I'm grateful to them for sparking the chain of thoughts. I've started reading how other organisations (such as MacArthur's Technology for the Public Good) discuss the role of technology in societal domains, and would be grateful for any pointers to similar initiatives in conservation.
\nI would also dearly love to see a roundup of all the Cambridge environmental coverage in one place, perhaps on the Conservation Research Institute pages, told as a cohesive story from the perspective of the nature research and not the technology that enables just a part of it. If you're an undergraduate looking for something to do this summer, especially from the social sciences or journalism, do get in touch and I'd be delighted to work with you on this for an internship! Or maybe this is something for the first edition of the Cambridge Green Blue competition to assemble next year...
\nThanks to Sam Reynolds, William Morgan, Sadiq Jaffer, Ash Simkins and Alec Christie for corrections and suggestions to this post! *7th May 2025: See also a followup article on this by Jacqueline Garget.
\nAs an aside, I love this long-form, carefully considered mechanism for scholarly discussion, as espoused by the letter back-and-forth in a journal. I wish we had more of this in computer science rather than social media arguments that disappear like tears in the rain just a few scrolls later.
\n ↩Amusingly, they were triggered by an accidental reply-all from me to the whole building rather than a private reply. I hold that this is the best way to start a real conversation!
\n ↩Like many others, my website is under a constant barrage of crawling from bots. I need to figure out which one is hosing me, but I am also resisting having third-party trackers of any form. I took a look at hosting a Plausible instance as OCaml does, but it's yet another service to run and maintain. Then Nick Ludlam pointed me to an old-fashioned server-side log analyser with builtin privacy called Goaccess he's using on his site, which is also perfect for my needs!
\nSetting up Goaccess is extremely simple. It's a single binary with no dependencies outside of ncurses, and just needs some server side logs. I currently use Caddy to front the HTTP2/3 for my custom OCaml webserver, so I just had to configure it to output JSON-format logs.
\nanil.recoil.org {\n reverse_proxy http://localhost:8080\n encode zstd gzip\n log {\n format json\n output file /var/log/caddy/anil.recoil.org.log {\n roll_size 1gb\n roll_keep 100\n }\n }\n}\nThe above causes Caddy to log lines in a JSON format like this:
\n{ "level": "info", "ts": 1745414562.426229,\n "logger": "http.log.access.log0",\n "msg": "handled request",\n "request": {\n "remote_ip": "<snip>", "remote_port": "56839",\n "client_ip": "<snip>", "proto": "HTTP/3.0",\n "method": "GET", "host": "anil.recoil.org",\n "uri": "/assets/home.svg",\n "headers": {\n "Sec-Fetch-Dest": [ "image" ],\n "Sec-Fetch-Site": [ "same-origin" ],\n "Sec-Fetch-Mode": [ "no-cors" ],\n "Priority": [ "u=5, i" ],\n "Accept-Encoding": [ "gzip, deflate, br" ],\n "Accept": [\n "image/webp,image/avif,image/jxl,image/heic,image/heic-sequence,video/*;q=0.8,image/png,image/svg+xml,image/*;q=0.8,*/*;q=0.5" ],\n "User-Agent": [\n "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3.1 Safari/605.1.15"\n ],\n "Referer": [ "https://anil.recoil.org/" ],\n "Accept-Language": [ "en-GB,en;q=0.9" ]\n },\n "tls": {\n "resumed": false, "version": 772, "cipher_suite": 4865,\n "proto": "h3", "server_name": "anil.recoil.org"\n }\n }, <...etc>\n}\nWhile this is a verbose logging format, it compresses very well and has lots of information that can be analysed without the need for any JavaScript. Once the logging is setup, just running goaccess <logfile> spins up a curses configuration from which I can select the Caddy log format.
![\"%c\"](\"/images/goaccess-ss-1.webp\")
After that, there is a simple interactive terminal dashboard that not only shows the usual analytics, but also fun things like operating system and time-of-access frequency patterns.
\n![\"%c\"](\"/images/goaccess-ss-2.webp\")
The tool can also blank out IP addresses in order to preserve privacy in the output analytics, and also spit out an HTML report, although I'm not using this particular functionality. While Plausible looks like the answer for bigger sites, this simple tool is good enough for me. The very first iteration of this site in 1998 used to use Analog (written by my former Xen/Docker colleague Stephen Turner), so it's nice to go back full circle to this sort of tool again!
The sister conference to PROPL was held late last year in Scotland with a bumper attendance from Cambridge. All of the talks from it are now available online at YouTube, or on our ad-free EEG video site.\nThe keynote from Anne Currie was fantastic and wide-ranging (she is the author of the eerily predictive Panopticon series):
\n\nI was also involved with four fun talks up there given by collaborators:
\n\n\n\n\nMore broadly, I'm quite pleased with the ##selfhosting of videos. I'm now operating three sites (with Mark Elvers and Ryan Gibb):
\nMany of the talks in the instances above have been sunset from their respective video hosting sites, so there's a strong element of robustness over time to hosting things ourselves. Each of the instances above also follow each other to provide p2p redundancy if a video hits the socials and goes viral.
It's about the time of the academic year to come up with project ideas! KC Sivaramakrishnan, Andy Ray and I have been looking into FPGA/OCaml matters recently so I thought I'd review the latest in the land of Webassembly for non-traditional hardware targets. It turns out that there are very fun systems projects going on to turn wasm into a "real" target architecture on several fronts: a native port of Linux to run in wasm, a port of wasm to run in kernel space, a POSIX mapping of wasm, and fledgling wasm-CPUs-on-FPGAs.
\nThe first one is a native port of the Linux kernel to run in webassembly (try it in your browser). This isn't an emulation; instead, the various kernel subsystems have been ported to have wasm interfaces, so the C kernel code runs directly as webassembly, with virtual device layers.
\nThe inspiration for this seems to have come from a famous comment eight years ago on the LKML:
\n\n\nOne more general comment: I think this may well be the last new CPU architecture we ever add to the kernel. Both nds32 and c-sky are made by companies that also work on risc-v, and generally speaking risc-v seems to be killing off any of the minor licensable instruction set projects, just like ARM has mostly killed off the custom vendor-specific instruction sets already. If we add another architecture in the future, it may instead be something like the LLVM bitcode or WebAssembly, who knows?\n-- Arnd Bergmann, LKML, 2018
\n
And this port brings us much closer to that! I need to spelunk more into the diffs to the mainline kernel to see how it all works, but some quick notes:
\nOn the opposite end of the architecture spectrum, we have a Linux in-kernel WASM runtime. This one allows running userspace code within the kernel space, as motivated by:
\n\n\nSince WASM is a virtual ISA protected by a virtual machine, we don't need to rely on external hardware and software checks to ensure safety. Running WASM in the kernel avoids most of the overhead introduced by those checks, e.g. system call (context switching) and
\ncopy_{from,to}_user, therefore improving performance.\nAlso, having low-level control means that we can implement a lot of features that were heavy or impossible in userspace, like virtual memory tricks and handling of intensive kernel events (like network packet filtering).\n-- Why run Wasm in the kernel
There are some interesting example applications available that they accelerate. They report on the speedup for an echo and http server that can run in kernel space:
\n\n\nWhen compiled with the singlepass backend (unoptimized direct x86-64 code generation) and benchmarked using tcpkali/wrk, echo-server is ~10% faster (25210 Mbps / 22820 Mbps) than its native equivalent, and http-server is ~6% faster (53293 Rps / 50083 Rps). Even higher performance is expected when the other two Wasmer backends with optimizations (Cranelift/LLVM) are updated to support generating code for the kernel.\n-- kernel wasm benchmarks
\n
The kernel-wasm port lead me to look more closely at the wasmer runtime, which in turn also extends the wasi server-side interface of WASM to support full POSIX compatibility. You can also view this in the browser as a shell, where a variety of applications can be compiled to wasm and run as if you had a shell in the browser!
\nThere is impressive support for POSIX here, as well as an wasmer/wasix SDK to port existing applications like ffmpeg to run in the browser or on in a server JS runtime.
\nSo what's stopping OCaml --via the new wasm-of-ocaml compiler -- from running in the browser? Just the fact that our target runtime depends on the wasm stack switching extension, and wasmer doesnt yet support that extension. Since there, wasmer 2.3 has improved stack switching performance but the extension isn't quite there yet. So if anyone's looking for some experience with language runtime hacking, this might be a good project. I couldn't find any information on whether wasmer is planning on adding support for this extension yet though.
\nAnd last but not least, given all of the above, what would it take to run wasm on an FPGA directly? The existence of the Linux native wasm port is encouraging, since it implies that if you were to get wasm instructions to run directly on an FPGA (just like you might wiht a MIPS FPGA CPU or a RISC-V one), then you could hook up the rest of the OS ecosystem to this as custom drivers.
\nI found a few projects around this space that I need to look into more:
\nAfter first posting this, here are incoming updates. Jonas Kruckenberg tells me that he's got an experimental OS called k23. This is a microkernel that is built around the idea of using Wasm as the primary execution environment:
\n\nThis allows for a number of benefits:
\n\n
\n- Security: WebAssembly is designed to run in a sandboxed environment, making it much harder to exploit.
\n- Modularity: WebAssembly modules can depend on each other, importing and exporting functionality and data, forming a modular system where dependency management is a first class citizen.
\n- Portability: WebAssembly is designed to be very portable. Forget questions like "is this binary compiled for amd64 or arm?". k23 programs just run wherever.
\n- Static Analysis: WebAssembly is famous for being very easy to analyze. This means we can check for bad programs without even running them.\n-- The k23 manual
\n
The European Space Agency organised the first conference on Biodiversity Insights from Space (BioSpace) in February this year, and it seems like it was a huge success. The conference itself sold out within days, and the program was so packed that the organisers had to split it into multiple chunks during the week to cope with everyone. I've only just gotten around to fully browsing the schedule, and it's incredible to see so much variety of work happening in biodiversity and remote sensing. Here's hoping that ESA makes this an annual event in Italy!
\nDavid Coomes, who was on the scientific selection committee, told us about it so we hastily submitted a few abstracts which got selected for presentation! David himself talked about forest disturbance.
\n ![\"%c\"](\"/images/biospace-ss-1.webp\")
Andres Zuñiga-Gonzalez presented the work we've been exploring at Cambridge and Imperial around using ultra low power sensors for biodiversity monitoring and urban health:
\n\n\nUrban trees are essential for supporting biodiversity, as they provide\nhabitats for various species and help regulate water storage and temperature,\nand sequester CO₂ in urban ecosystems.Urban forests have been proposed as a\nnature-based solution to fight climate change and provide ecosystem services\nto citizens. Mapping and monitoring urban trees is vital as it facilitates\nconservation strategies for both flora and fauna, early diagnosis of plant\npathogens, and zoning and urban development.
\nHowever, mapping trees has\nproved difficult for urban planners since they rely on in situ surveys or\ncommunity-led projects that may not cover all areas; one such case is London,\nwhere the official survey only accounts for ~10% of the estimated 8 million\ntrees in the city. Moreover, the geographic coordinates of trees are\nsurprisingly unreliable due to a lack of precision of measuring devices (e.g.\nphones or commercial GPS).
\nWe propose a method for calibrating urban tree\nlocations using physical ground sensors as "anchors". These sensors help\nreconcile spatial mismatches across various spatial datasets, including\nhigh-resolution satellite and aerial imagery and tree surveys collected by\ncity councils or in open-data projects like OSM. These low-power sensors can\nalso collect microclimate and other biodiversity-related data, such as\npassive acoustic animal activity monitoring, providing a richer picture of\ntree and urban ecosystem health and enabling high resolution maps not\npreviously possible. Our ultimate goal is to combine remote sensing\ninformation with ground-based measurements to support reliable data that can\nbe used in geographic-based foundation models to help better urban planning\nstrategies around trees that maximise their benefit to humans and nature.
\n
![\"%c\"](\"/images/biospace-ss-2.webp\" "\\"The")
You can read Andres Zuñiga-Gonzalez's own writeup on his blog and watch the recording! Josh Millar would have made it to a poster presentation, but forgot to register in time and missed out due to how packed the conference was!
\nOnkar Gulati also prepared a poster for his PhD work on the topic of causality measurement. His notes from the conference about the use of SDGs are great:
\n\n\nMy big takeaway from the opening speeches was that this is the first year that the ESA is spending more on building out its data science capabilities than it is on putting satellites into space. To me, this is indicative of the fact that the marginal benefit from putting effort into effectively wrangling huge amounts of data is now greater than that from collecting huge amounts of data at a faster pace.
\n
Given the growing amount of space junk out there, getting more leverage over already gathered data seems very sensible indeed.
\nAnother important point Onkar makes that I've been noticing in my own thoughts about national data libraries is:
\n\n\nA key point multiple speakers made note of (there were a dozen or so speakers\ntalking for perhaps ~10 minutes each) was that introducing frameworks and\nmethodologies to give countries national ownership of their data and the\nability to independently generate compatible statistics was the priority, not\nintroducing new data products. If we can move towards all countries using the\nsame standards, we can enable the aggregation of statistics up in a reliable\nmanner.
\n
Since the February date of this BIOSPACE conference there has, of course, been a huge amount of\ngeopolitical flux in the world. Countries gaining national ownership of their\nown data seems more important than ever.\nOnkar's full writeup is full of\ninsights derived from the conference, so I encourage you to have a direct read!
I was gobsmacked to get a note from the SIGARCH ASPLOS steering committee that our 2013 paper "Unikernels: library operating systems for the cloud" won the most influential paper award at the conference last week! I couldn't make it to Rotterdam myself due to the travel time, but Richard Mortier was already there and so accepted the award on the whole team's behalf!
\nMy officemate Simon Peyton Jones pointed out to me that these 'test of time' awards are his favourite, as they indicate that a piece of research was actually useful over a number of years to other people in the field:
\n\n\nThe ASPLOS Influential Paper Award recognizes historical ASPLOS papers that have had major influence on the field. The Program Committee nominates highly influential papers from any ASPLOS conference that occurred ten or more conferences ago, with the final selections being made by the ASPLOS Steering Committee.\n-- SIGARCH awards
\n
![\"%r\"](\"/images/asplos25-award-1.webp\" "\\"Mort")
My long-time colleague Dave Scott wrote up a great overview of why he likes unikernels, especially in areas like Docker where he is a senior maintainer these days. Dave uses a nice jigsaw puzzle analogy to show the value of a library operating system approach when building complex systems glue; they're good for high assurance applications, for rapid experimentation and iteration, and for deep systems customisation.
\nRichard Mortier's note brought up some memories about how this particular work made me almost rage-quit academia entirely. Back in 2009 after I returned to academia with Horizon fellowship, I spent a year of my life working on lots of libraries for the very first iteration of MirageOS, and published a USENIX workshop paper on it.
\nAfter that, it was time to do the full conference paper, so I spent another year (joined by more colleagues like Thomas Gazagnaire and Dave Scott in the early days) bringing yet more OCaml libraries to life to make the thing actually useful. Charalampos Rotsos and Richard Mortier spent forever on an OpenFlow implementation in pure OCaml; Balraj Singh sorted out the mess I made of the TCP stack congestion algorithms; Steven Smith hacked on Xen with me to add immutable pfn support. There was a lot of intense hacking going on.
\nWe then trimphantly wrote up our work as a submission to OSDI 2012 after staying up all night for several days in a row to get the evaluation done, and... it got rejected. But the paper didn't just get rejected, it got rejected so hard that I couldn't bear to look at another OSDI proceedings for years. So hard that I can still the weight of the punch of that email as I opened it eagerly, a decade on. Some of our colleagues also had rejected OSDI papers that year; the Naiad paper got six reviews but bounced (later to win best paper at SOSP 2013), and Andrew Warfield had another that got nine (!) reviews before being shown the door. But ours got...three reviews indicating it bounced in the very first round, and one review scored us at '1/5', the lowest possible value. It made all that intense work seem like a total waste of our lives.
\nBut then... one of the reviews shone out like a beacon. It was the longest review, and was full of directly actionable feedback. It began very constructively:
\n\n\nThe approach described in this paper is a very reasonable design point,\na natural intersection of the exokernel and libOS and the type-safe OS.\nIt would help to refocus the abstract and intro on the precise benefits that\nUnikernel can provide, and to attribute each benefit to its origin. Let me\ntake a stab at this, based on my read of the paper:
\n
The reviewer then reinterpreted our submission to add more clarity:
\n\n\n\n
\n- "eliminate several classes of security threats via type-safety" -- clearly due to the top-to-bottom use of type-safe OCaml.
\n- "eliminate several classes of security threats via ... an address-space which can be made immutable" -- is there any reason an analogous technique could not apply to a libOS version of libc?
\n- "progressive specialization" -- I didn't find this contribution very exciting. It's nice that I can execute the same OCaml app in a Linux context to debug it; perhaps this is especially important since we don't yet have good symbolic debuggers for OCaml apps standing alone in a VM. But it really doesn't seem like a central benefit.
\n- "developers no longer need to become sysadmins" -- This claim is specious. If a third party packages an app together with a Linux guest-OS stack to become an appliance, there's no reason that appliance would require any sysadmin-ish behavior more than a Unikernel appliance.
\n- "The resulting unikernels are also highly compact" -- Could this property not also be readily achieved with a tuned libc-based (that is, not type-safe) libOS? How precious is this property? Is the working set actually much smaller? And finally, how much of the reduction is because the rewritten application is much less functional than the industry standard it replaces?\n[...the review continues on for several pages]
\n
Now, I didn't agree with all the points in the review, but they were restructured in such a way that made it clear that the reviewer had really thought about it, and had tried to pull out their own insights from the system construction. We ate up this feedback, and resubmitted it in a matter of weeks to ASPLOS adopting much of the structure suggested by this OSDI reviewer, and the paper got in with accepts across the board.
\nThe best bit of all this? The OSDI reviewer voluntarily unblinded their review:
\n\n\nReview by Jon Howell howell@microsoft.com, intentionally unblinded.
\n
Jon's obviously an expert in the field (his own 2011 paper on Drawbridge won the ASPLOS influential paper award last year), but it's how much time he took in helping out a sibling system that stuck with me. His kind, constructive and direct review kept me in academia, and although I still haven't met him in person (life got really busy right afterwards with Unikernel Systems), I definitely still owe him a pint!
\nMort also made me think about what we learnt from all this work that current students might learn from.
\nYou never know which papers will sink or swim in the fullness of time and most never get that popular outside a small niche, so don't worry about that right now when doing the work! Focus instead of honing your systems intuition for why you're building something, and bring out the delta of your contributions clearly in the paper. When you're building a complex system, there's a lot of boilerplate and scaffolding necessary, but the core of the "thing that hasn't been done before" is what you know best, and it can take some time and conversations to figure out what that is.
\n![\"%c\"](\"/images/jc-flyingpig-1.webp\" "\\"In")
Back in the day, most of our discussions about systems research happened after a day of deep hacking down at the pub.\nSince the pandemic, we seem to have lost a big chunk of that social discussion around our work collectively. While I still see people regularly for a swift half, it somehow seems more difficult to gather people in general. Part of that is that I don't go into the department much anymore due to the noise and cold (something Jon Sterling also observed recently), vs my cosy Pembroke office.
\nI'm not sure if it's just me or everyone else also feeling this, but I'm so zoned out after even a few Zoom calls that I'm just not very social afterwards. So one thing I'm aiming to do more consciously after Easter is to try to gather people down at The Mill more for a swift half (where they serve excellent Guinness Zero), and really cut down on remote interactions that aren't necessary.
\n![\"%c\"](\"/images/jc-kingston-1.webp\" "\\"In")
And the last tip is from Barry Schwartz, who noted that the secret to happiness is low expectations. No matter how much works goes into a system, don't bank too much on the big papers making a splash. Instead, enjoy every step of the journey -- from building things, scrapping them, debugging odd failures, throwing ideas around, releasing the software, seeing adoption, scrapping it all and starting again, the whole journey! There will always be a reviewer 2 waiting to ruin your day if you let them, so don't let them in.
\nI also wonder how long paper publishing in its current form will survive; with the sheer number of publications coming out these days and the amount of AI generated output, it's difficult to see something published today having the same ramp as the work we did in the past few decades. Instead, adoption and rapid iteration seem to be the way to go. Thankfully, our University intellectual property rights remain liberal (patents aside, but who cares about those these days for software), so there's nothing stopping us!
\n![\"%c\"](\"/images/jc-mill-1.webp\" "\\"In")
Over in my EEG group, we have a lot of primary and secondary datasets lying around: 100s of terabytes of satellite imagery, biodiversity data, academic literature, and the intermediate computations that go along with them. Our trusty central shared storage server running TrueNAS stores data in ZFS and serves it over NFSv4 to a bunch of hosts. This is rapidly becoming a bottleneck as our group and datasets grow, and Mark Elvers has been steadily adding lots more raw capacity. The question now is how to configure this raw SSD capacity into a more nimble storage setup. If anyone's seen any systems similar to the one sketched out below, I'd love to hear from you.
\nThe first design constraint is to get rid of centralised network storage. This is both slow when compared to a modern NVMEs, and also hard to extend beyond the POSIX-ish API to take advantage of filesystem-specific features like snapshots or reflink clones. We also don't take much advantage of the simultaneous use of the network storage. Instead, we'd like to just make every host materialise the portion of storage it needs locally by cloning it from the remote server.
\nThe alternative I'm considering here is to use ZFS filesystems on the nodes themselves rather than NFS. This has the upside of having the cloned data be directly available on the local disk of the host that's using it, meaning that there's no performance impact as with networked storage. ZFS also scales fairly enormous sizes, and so it seems likely that we won't run into an upper bound due to this choice of filesystem in the medium term.
\nZFS operates through the creation of a zpool across a block of disks, over which datasets can be created in a tree. One of our typical research servers looks like this:
\n$ zfs list\nNAME USED AVAIL REFER MOUNTPOINT\neeg 20.4T 7.37T 7.84T /eeg\neeg/gbif 8.55T 7.37T 8.55T /eeg/gbif\neeg/logs/fetcher 11.3G 7.37T 8.41G /eeg/logs/fetcher\neeg/logs/zotero 12.2G 7.37T 8.29G /eeg/logs/zotero\neeg/papers/doi 3.11T 7.37T 3.11T /eeg/papers/doi\neeg/papers/pmc 843G 7.37T 843G /eeg/papers/pmc\neeg/papers/tei 87.4G 7.37T 85.8G /eeg/papers/tei\neeg/repology 5.92G 7.37T 5.92G /eeg/repolo\nInside the eeg zpool, each of the sub-datasets can themselves be arranged in a hierarchy. Each of them can also have key-value labels and separate properties attached to them, and inherit their parent datasets properties. There are a vast number of ZFS properties that can be tuned.
Once a single host has some important data in a local ZFS dataset, I've started using Sanoid for the snapshot management:
\n\n\nSanoid is a policy-driven snapshot management tool for ZFS filesystems [...] you can use it to make your systems functionally immortal via automated snapshot management and over-the-air replication.
\n
The first use of Sanoid is to regularly take ZFS snapshots of important filesystems. These snapshots will be rotated regularly at different intervals, and subsequently replicated off-host. Here's an example /etc/sanoid/sanoid.conf from the machine above.
[eeg/papers/doi]\n use_template = production\n[eeg/papers/pmc]\n use_template = production\n[eeg/papers/tei]\n use_template = production\n\n[template_production]\n frequently = 0\n hourly = 24\n daily = 30\n monthly = 3\n yearly = 0\n autosnap = yes\n autoprune = yes\nThis sanoid.conf establishes a "production" template that keeps 24 hourly snapshots, 30 daily ones and 3 monthly ones. After some time passes, I can verify this by checking the local filesystem snapshots.
$ zfs list -t snapshot\nNAME USED AVAIL REFER MOUNTPOINT\neeg/p/doi@2024122101 134M - 1.19T -\neeg/p/doi@2024122102 45.3M - 1.19T -\neeg/p/doi@autosnap_2025-02-01_00:00:02_monthly 224M - 2.84T -\neeg/p/doi@autosnap_2025-03-01_00:00:02_monthly 173M - 3.11T -\neeg/p/doi@autosnap_2025-03-08_00:00:03_daily 173M - 3.11T -\neeg/p/doi@autosnap_2025-03-09_00:00:01_daily 0B - 3.11T -\neeg/p/doi@autosnap_2025-03-10_00:00:02_daily 0B - 3.11T -\neeg/p/doi@autosnap_2025-03-11_00:00:02_daily 0B - 3.11T -\n<...etc>\nThese snapshots are incremental, and each subsequent one uses only the differential space taken up by the earlier ones. See this handy guide for the meaning of the different space accounting terms above.
\nOnce happy with the production template, I then automate it within cron:
\n* * * * * TZ=Europe/London /usr/bin/sanoid --cron\nThis currently runs every minute (a little wasteful), in order to quickly check if any snapshots are required. Once happy with the hourly cadence working as expected, I'll drop this back to an @hourly job.
Once Sanoid is merrily making snapshots on the active host, it's also necessary to replicate this off the host for robustness. Since we're not using a networked store, I'd like to replicate the snapshots onto two other hosts, one of which is offsite.\nCrucially, these backup hosts can have their own sanoid configuration with a longer-term horizon of backups (e.g. keeping some yearly snapshots). To make this work, we first use a sister tool syncoid that is included in the Sanoid distribution.
I run syncoid in 'pull mode', which means that the backup server is configured to be able to SSH into the production server(s) in order to fetch the datasets. Under the hood, syncoid uses a combination of ZFS bookmarks and send/recv to incrementally and efficiently transmit the snapshots over the network and reconstruct the filesystems locally.
Once the SSH host keys are configured in the usual way, a series of crontab entries like this is sufficient to fetch all the remote snapshots to the local host. The backup host that's doing the pulling just needs to run syncoid regularly:
@daily /usr/sbin/syncoid backup@marpe:eeg/papers/doi eeg/papers/doi\n@daily /usr/sbin/syncoid backup@marpe:eeg/papers/tei eeg/papers/tei\nAt this point, the backup host now has all the snapshots from the live host (including hourly ones), and can then run Sanoid again in order to decide which ones it wants to keep locally. I haven't put too much effort into optimising these yet, but you can see they're different from the ones above.
\n[eeg/papers]\n use_template = backup\n recursive = yes\n\n[template_backup]\n autoprune = yes\n frequently = 0\n hourly = 30\n daily = 90\n monthly = 12\n yearly = 0\n autosnap = no\nThese will keep a few more hourly snapshots, and three times the number of daily snapshots available on the backup servers, in case a rollback is needed. Since the backup server typically has a lot more raw capacity than the live server, it's practical to do this there rather than on the production hosts.
\nFinally, we can also hook this up to our monitoring scripts with a handy Nagios-compatible interface.
\n# sanoid --monitor-health\nOK ZPOOL eeg : ONLINE {Size:30.6T Free:2.44T Cap:92%}\nIt's a bit trickier to figure out if the root volume of the hosts should be ZFS. This requires that the boot initrd always has a working ZFS kernel module, which sometimes goes wrong on updates if the DKMS shim falters for some reason. In terms of specific distributions:
\nFor my own personal servers, I've been using a normal ext4 root volume, and creating a ZFS for the remainder of the disk, without using LVM underneath it. It's a bit less flexible, but strikes a balance between performance and flexibility.
\nThis basic setup is sufficient to have pull-based ZFS snapshot and replication across multiple hosts, and making it easy to quickly materialise a ZFS dataset onto a given host for use in data processing. It still needs a bunch of development to turn this into a properly robust system though:
\nzfs create into an existing subdirectory of a dataset and have it copy the files within there semi-automatically into the new sub-dataset.syncoid:sync label can be used to tell syncoid to include a particular recursive dataset in its replication. There are some scalability limits in just how many labels you can add before slowing a machine down a crawl (though not as bad as how many live mounts). Patrick Ferris started some WIP ocaml-zfs bindings to libzfs to help explore this question.So lots of work left to do here, but quite good fun as systems hacking goes! When Mark Elvers is done installing our new drives, we'll have a few petabytes of raw capacity to implement this system over...
Dominic Orchard and I had a blast running the first PROPL workshop a couple of years ago, with a full room and engaged audience in POPL in London. Last year, our sister conference LOCO took over, and it's our turn again this year! PROPL will return for a second outing in October, co-located with ICFP/SPLASH in Singapore in October. Read the call for papers here (deadline 3rd July 2025).
\n![\"%r\"](\"/images/propl-1.webp\" "\\"Dominic")
We'd love to get wider participation in computer science interacting with matters of climate and biodiversity:
\n\n\nThere are simultaneous interlinked crises across the planet due to human actions: climate change, biodiversity loss, and desertification. Addressing these challenges requires, amongst other things, a global understanding of the present state of affairs and the effectiveness of our adaptations and mitigations, leveraging both data and computation.
\nHowever, programming the computer systems required to effectively ingest, clean, collate, process, explore, archive, and derive policy decisions from the planetary data we are collecting is difficult and leads to artefacts presently not usable by non-CS-experts, not reliable enough for scientific and political decision making, and not widely and openly available to all interested parties. Concurrently, domains where computational techniques are already central (e.g., climate modelling) are facing diminishing returns from current hardware trends and software techniques.
\nPROPL explores how to close the gap between state-of-the-art programming methods being developed in academia and the use of programming in climate analysis, modelling, forecasting, policy, and diplomacy. The aim is to build bridges to the current practices used in the scientific community.\n -- About PROPL
\n
![\"%r\"](\"/images/propl-2.webp\" "\\"The")
In order to get a wide set of participants, we've got three different ways you can contribute to the program this year, all of which are listed in the call for papers:
\nThe papers and talks can be submitted using the PROPL HotCRP, and the provocations via an online form. The deadline is the 3rd July 2025 (anywhere on earth), so we hope to see you take part!
\nFor those curious about the first PROPL outing, all of the talk videos are all online on YouTube or our EEG video mirror (ad-free).
\n![\"%c\"](\"/images/propl-3.webp\" "\\"We're")
(Thanks to Lena Yang for spotting typos.)
", "url": "https://anil.recoil.org/notes/propl-at-splash", "title": "2nd Programming for the Planet workshop CFP out", "summary": "Call for papers for PROPL 2025 workshop on programming methods for climate and biodiversity research at ICFP/SPLASH Singapore.", "date_published": "2025-04-03T00:00:00.000000Z", "date_modified": "2025-04-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "service", "functional", "conference", "climate", "biodiversity", "conservation" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2025-npu-bench-1", "content_html": "Josh Millar just released our latest preprint on how to make sense of the growing number of dedicated, ultra-low-power 'neural network accelerators' that are found in many modern embedded chipsets. My interest in this derives from wanting to decouple from the cloud when it comes to low-latency local environments, and this needs fast tensor operations in hardware. Josh found a huge number of interesting NPUs in modern low-cost chips, ranging from ESP32-based boards over to ARM ones. All of these have quite a variety of tradeoffs, from the operations supported (which affects which models can be run on them) to the amount of memory and CPU power. This is the first comparative evaluation and independent benchmarking of several commercially-available micro-NPUs. We developed an open-source model compilation framework to enable consistent benchmarking across diverse hardware, measuring end-to-end performance including latency, power consumption, and memory overhead. The analysis uncovered surprising disparities between hardware specifications and actual performance, including unexpected scaling behaviors with increasing model complexity.
Our recently published LIFE biodiversity metric has just been integrated into a newly recognised Official Statistic from the UK government! This integrates the core LIFE biodiversity metric with food provenance data to track the environmental impacts of our consumption habits.
\nI must admit that I'd not heard of "Official Statistics" before this, so I did a bit of research. The UK Office for Statistics Regulation says that:
\n\n\nOfficial statistics are statistics produced by Crown bodies and other organisations listed within an Official Statistics Order, on behalf of the UK government or devolved administrations.\nThey provide a factual basis for assessment and decisions on economic, social and environmental issues at all levels of society.\n-- OSR Policies
\n
The good folk at the Joint Nature Conservation Committee are responsible for this particular statistic. The JNCC launched their Together for Nature strategy in 2023, and have the remit of turning scientific outcomes into robust evidence-based action for protecting nature worldwide. They've been developing a Global Environmental Impacts of Consumption indicator that provides information to policymakers about the tradeoffs of various consumption actions vs the corresponding global environmental impact.
\n\n\nWhen products arrive in our shops and on our doorsteps, they can look very different to the raw ingredients that were used to make them. Most products are made up of many parts, and these can move thousands of miles through many countries before reaching their final destination.
\nThe average consumer is so far removed from the production process, both physically and conceptually, that it is hard to imagine where their products are from, let alone the environmental impacts resulting from their production. This is also true for the academics and governments working to monitor and reduce the environmental impact of our consumption.\n-- DEFRA Blog
\n
Their GEIC tool, developed jointly with the Stockholm Environmental Institute and our LIFE collaborator Jonathan Green, provides data on spatial biodiversity, water use, and forest cover changes associated with a country consumption.
\n\n![\"%c\"](\"/images/life-statistic-1.webp\" "\\"The")
This metric is updated annually, and this year Jonathan Green and Thomas Ball supplied LIFE+FOOD to additionally map where species are more likely to go extinct as a result of land-use change.
\nThis year's GEIC update is the first where it was recognised by the OSR as being of sufficient stability and quality to "graduate" into Offical Statistic status. A pretty cool feeling, and it's all openly downloadable of course; you can navigate over to the Commodity Footprints LIFE section to explore the metrics for yourself.
\nFrom a planetary computing perspective, what I also found interesting is how the flow of observations and evidence works in practise. The computational processing for LIFE involves crunching petabytes of raster maps from a species habitat pipeline into a global map, which is then published as an aggregate map on Zenodo by Alison Eyres and Michael Dales.
\n![\"%c\"](\"/images/life-statistic-2.webp\" "\\"The")
Thomas Ball and Jonathan Green then worked directly with the policy team at the JNCC to further customise the metric for GEIC needs, and the aggregate result of that is what's actually used in the dashboard.
\nThere's quite a long gap between the original observations and the resulting policy use, with many humans in the loop in between. Computational systems need to capture all this nuance rather than viewing these metrics as "just" dataflow pipelines. However, it's equally important to capture the policy customisations in some sort of code, so that we can reliably issue annual updates. Figuring this pipeline out is part of what we're working on in the Conservation Evidence Copilots project at present. See below for a recent talk I gave on the functional programming aspects of this problem at LambdaDays.
\nAccess to reliable and timely scientific evidence is utterly vital for the practise of responsible policymaking, especially with all the turmoil in the world these days. At the same time, the evidence base on which use to make these decisions is rapidly morphing under our feet; the first entirely AI-generated paper passed peer review at an ICLR workshop today. We held a workshop on this topic of AI and evidence synthesis at Pembroke College last week, to understand both the opportunities for the use of AI here, the strengths and limitations of current tools, areas of progress and also just to chat with policymakers from DSIT and thinktanks about how to approach this rapidly moving area.
\n(The following notes are adapted from jottings from Jessica Montgomery,\nSam Reynolds, Annabelle Scott and myself. They are not at all complete, but hopefully useful!)
\nWe invited a range of participants to the workshop and held it at Pembroke College (the choice of the centuries-old location felt appropriate).\nJessica Montgomery and Neil Lawrence expertly emceed the day, with Bill Sutherland, Sadiq Jaffer and Sam Reynolds also presenting provocations to get the conversation going.
\n![\"%c\"](\"/images/evidence-synth-2.webp\" "\\"Lots")
Jessica Montgomery described the purpose of the workshop as follows:
\n\n\nEvidence synthesis is a vital tool to connect scientific knowledge to areas\nof demand for actionable insights. It helps build supply chains of ideas,\nthat connect research to practice in ways that can deliver meaningful\nimprovements in policy development and implementation. Its value can be seen\nacross sectors: aviation safety benefitted from systematic incident analysis;\nmedical care has advanced through clinical trials and systematic reviews;\nengineering is enhanced through evidence-based design standards. When done\nwell, evidence synthesis can transform how fields operate. However, for every\nfield where evidence synthesis is embedded in standard operating practices,\nthere are others relying on untested assumptions or outdated guidance.\n-- Jessica Montgomery, AI@Cam
\n
One such field that benefits from evidence is conservation, which is what Bill Sutherland and his team have been working away on for years. Bill went on to discuss the fresh challenges that AI brings to this field, because it introduces a new element of scale which could augment relatively slow human efforts.
\n\n\nScale poses a fundamental challenge to traditional approaches to evidence\nsynthesis. Comprehensive reviews take substantial resources and time. By the\ntime they are complete – or reach a policy audience – the window for action\nmay have closed. The Conservation Evidence project at the University of\nCambridge offers an example of how researchers can tackle this challenge. The\nConservation Evidence team has analysed over 1.3M journals from 17 languages\nand built a website enabling access to this evidence base. To support users\nto interrogate this evidence base, the team has compiled a metadataset that\nallows users to explore this literature based on a question of interest, for\nexample looking at what conservation actions have been effective in managing\na particular invasive species in a specified geographic area.\n-- Jessica Montgomery, AI@Cam
\n
The AI for evidence synthesis landscape is changing very rapidly, with a variety of specialised tools now\nbeing promoted in this space. This ranges from commercial tools such as Gemini Deep Research and OpenAI's deep searcher, to\nresearch-focused systems such as Elicit, DistillerSR, and RobotReviewer. These tools vary in their approach, capabilities, and target users, raising questions about which will best serve different user needs. RobotReviewer, for example, notes that:
\n\n\n[...] the machine learning works well, but is not a substitute for human systematic reviewers. We recommend the use of our demo as an assistant to human reviewers, who can validate the machine learning suggestions, and correct them as needed. Machine learning used this way is often described as semi-automation.\n-- About RobotReviewer
\n
The problem, of course, is that these guidelines will often be ignored by\nreviewers who are under time pressure, and so the well established protocols\nfor systematic reviewers are under some threat.
\n![\"%c\"](\"/images/evidence-synth-4.webp\" "\\"Sadiq")
Sadiq Jaffer and Sam Reynolds then talked about some of the computing approaches required to achieve a more reliable evidence review base.\nThey identified three key principles for responsible AI integration into evidence synthesis:
\nAlex Marcoci pointed out his recent work on AI replication games which I found fascinating. The idea here is that:
\n\n\nResearchers will be randomly assigned to one of three teams: Machine, Cyborg\nor Human. Machine and Cyborg teams will have access to (commercially\navailable) LLM models to conduct their work; Human teams of course rely only\non unaugmented human skills. Each team consists of 3 members with similar\nresearch interests and varying skill levels. Teams will be asked to check for\ncoding errors and conduct a robustness reproduction, which is the ability to\nduplicate the results of a prior study using the same data but different\nprocedures as were used by the original investigator.\n-- Institute for Replication
\n
These replication games are happening on the outputs of evidence, but the\ninputs are also rapidly changing with today's announcement of a fully generated AI papers passing peer\nreview. It's hopefully now clear\nthat AI is a huge disruptive factor in evidence synthesis.
\n![\"%c\"](\"/images/evidence-synth-3.webp\")
We first discussed how AI could help in enhancing systematic reviews.\nAI-enabled analysis can accelerate literature screening and data extraction,\ntherefore helping make the reviews more timely and comprehensive. The\nopportunity ahead of us is to democratise access to knowledge synthesis by\nmaking it available to those without specialised training or institutional\nresources, and therefore getting wider deployment in countries and\norganisations without the resources to commission traditional reviews.
\nHowever, there are big challenges remaining in gaining access to published research papers and datasets.\nThe publishers have deep concerns over AI-generated evidence synthesis, and more generally about the use of generative AI involving their source material. But individual publishers are already selling their content to the highest bidder as part of the data hoarding wars and so the spread of the work into pretrained models is not currently happening equitably or predictably.\nNeil Lawrence called this "competitive exclusion", and it is limiting communication and knowledge diversity.
\nThe brilliant Jennifer Schooling then led a panel discussion about the responsible\nuse of AI in the public sector. The panel observed that different countries\nare taking different approaches to the applications of AI in policy research.\nHowever, every country has deep regional variances in the application of\npolicy and priorities, which means that global pretrained AI models always need\nsome localized retuning. The "one-size-fits-all" approach works particularly\nbadly for policy, where local context is crucial to a good community outcome\nthat minimises harm.
\nPolicymakers therefore need realistic expectations about what AI can and cannot do in evidence synthesis.\nNeil Lawrence and Jennifer Schooling came up with the notion that "anticipate, test, and learn" methods must guide AI deployment in policy research; this is an extension of the "test and learn" culture being pushed by Pat McFadden as part of the Labour plan to reform the public sector this year. With AI systems, Alex Marcoci noted that we need to be working with the end users of the tools to scope what government departments need and want. These conversations needs to happen before we build the tools, letting us anticipate problems before we deploy and test them in a real policy environment. Neil Lawrence noted that policy doesn't have a simple "sandbox" environment to test AI outcomes in, unlike many other fields where simulation is practical ahead of deployment.
\nLucia Reisch noted that users must maintain critical judgement when using these\nnew AI tools; the machine interfaces must empower users towrads enhancing their\ncritical thinking and encouraging reflection on what outputs are being created\n(and what is being left out!). Lucia also mentioned that her group helps run\nthe "What Works" summit, which\nI've never been to but plan on attending next it rolls around.
\nThe energy requirements for training and running these large scale AI models\nare significant as well, of course, raising questions about the long-term\nmaintenance costs of these tools and their environmental footprint. There was\nwide consensus that the UK should develop its own AI models to ensure\nresilience and sovereignty, but also to make sure that the regional finetuning\nto maximise positive outcomes is under clear local control and not outsourced\ngeopolitically. By providing a single model that combines UK national data, we would also not waste energy with lots of\nsmaller training efforts around the four nations.
\n![\"%c\"](\"/images/evidence-synth-1.webp\" "\\"Sadiq")
Thanks Annabelle Scott for such a stellar organisation job and to Pembroke for hosting and all for\nattending, and please do continue the discussion about this on LinkedIn\nif you are so inclined.
I've been an avid user of GitHub since its launch, and it really has revolutionised how communities come together to work on open source. In recent years though, I find myself utterly overwhelmed by its notifications and want to experiment with alternative workflows. This experimentation also has a more serious undertone due to the increasing need for data sovereignty and so I'm starting to move my source code to self-hosted solutions that are less reliant on centralised services.
\nThis has also come up persistently over the years in the OCaml community, with questions over why participation in packaging requires a GitHub account ever since the early days of opam. I've never found a good answer... until now, with the launch of an exciting new service that's built over the same protocol that Bluesky uses.\nAs I noted a few weeks ago, the ATProto can be used for more than just microblogging. It can also be an identity layer, across which other applications can be built which reuse the social fabric from Bluesky accounts.
\n"Tangled" is a new service launched (just yesterday!) by opilli and icyphox to manage Git repositories. I'm having a lot of fun trying it out, even in its early alpha stages! The coolest thing about Tangled is that you can self-host your own knots, which control where the source code repositories are actually stored.
\nI set up one of the first knots on the network on git.recoil.org, and can now directly share my source code online without depending on GitHub! For example, this is the knot-docker container config which you can use to deploy your own version of this.
![\"%c\"](\"/images/tangled-ss-1.webp\")
It looks pretty similar to GitHub doesn't it? The first key difference is the login on the top-right, which is the same as my anil.recoil.org account. Once you're logged in, the other difference shows up when creating a new Git repository.
\n![\"%c\"](\"/images/tangled-ss-2.webp\")
As you can see, you can not only select the name of the repository, but also where it's going to be stored. I can either put it on the central Tangled knot, or stick it on my own Recoil one. After this, the user experience of cloning is as simple as:
\ngit clone https://tangled.sh/@anil.recoil.org/knot-docker\ngit clone git@git.recoil.org:anil.recoil.org/knot-docker\nIn the first case, the central tangled web server proxies the Git contents over HTTP, and for SSH I can just connect directly to my own server. Inside my Knot container, we can see where the Git repositories are stored:
\n/home/git # ls -1\ndid:plc:nhyitepp3u4u6fcfboegzcjw\nknotserver.db\nknotserver.db-shm\nknotserver.db-wal\nlog\nThe did: directory is actually my 'decentralised identifier' from the ATProto, which we can verify by looking up the DNS atproto TXT record for my domain:
$ dig txt _atproto.anil.recoil.org\n;; ANSWER SECTION:\n_atproto.anil.recoil.org. 10799 IN TXT "did=did:plc:nhyitepp3u4u6fcfboegzcjw"\nAnd then if we navigate into that directory, we can see there are just normal bare git repositories stored on my server.
\n/home/git/did:plc:nhyitepp3u4u6fcfboegzcjw/knot-docker # ls -la\ntotal 24\ndrwxr-sr-x 4 git git 4096 Mar 8 19:02 .\ndrwxr-sr-x 4 git git 4096 Mar 8 18:23 ..\n-rw-r--r-- 1 git git 21 Mar 8 18:01 HEAD\n-rw-r--r-- 1 git git 36 Mar 8 18:01 config\ndrwxr-sr-x 17 git git 4096 Mar 8 19:02 objects\ndrwxr-sr-x 4 git git 4096 Mar 8 18:01 refs\nThis makes the core of Tangled very safe to use, even if the service disappears: I maintain the actual git repositories myself, so I can (e.g.) mirror them to GitHub via a simple cron script.
\nTangled has only been out for about a day, so I coopted fellow Recoiler Nick Ludlam to create an account. I added his handle over to the Recoil knot, and that's all it took for him to be able to create repositories on our server.
\n![\"%c\"](\"/images/tangled-ss-5.webp\")
I can also just add people directly to a particular repository, as you can see from the one below on his profile.
\n ![\"%c\"](\"/images/tangled-ss-3.webp\")
The real lockin to code repository management though, is the metadata around the repository; things like issues, comments and so on. Tangled makes it possible to decentralise where is this stored without needing a central Forge, by relaying it all via the ATProto.\nLet's take a look at how this works.
\nI created an issue on knot-docker, and it looks very similar to a GitHub issue. Zicklag on #tangled pointed me to the PDSLS public ATProto browser with which you can browse the actual ATProto records. I can start from my DID record and look for the sh.tangled.repo.issue collection, and find the issue URL from earlier. I then prodded Nick Ludlam to leave a comment on the issue, and you can see his sh.tangled.repo.issue.comment in the relay as well.
![\"%c\"](\"/images/tangled-ss-4.webp\")
Even the repository stars are on the relay; see for example this entry for knot-docker that I did. The Tangled developers just added support for stars a few hours ago, and that changeset is a nice way to see how to add a new lexicon entry.
\nSamuel Wedgwood then reminded me of his project a few years ago to run git pull requests over Matrix chat. It would indeed be very cool if the pull request model on Tangled evolved into something more message-oriented like git-send-email, in order to let us try out more personalised workflows than GitHub PRs.
\nThe ATProto developers also released their roadmap for early 2025 today, and it aligns really well with some of the productions features I would need to completely shift over to a service like Tangled.
\nThe first, and most vital one, is auth scopes to control the permissions of an app password to only certain operations. Once this is in the protocol, then a client to manage Tangled repositories could use a differently privileged password from the main social client.
\nSecondly, privately shared data and encrypted DMs using MLS point to how private code repositories could work. Srinivasan Keshav and I were discussing the difficulty of access-controlled replication over the Internet just yesterday, and I'm starting to believe that ATProto has the right balance of ergonomics and good design to make solving this problem much, much easier.
\nIf you'd like to try this out, then the Knot Docker repository welcomes your issues!
\n\nMany thanks to Zicklag and icyphox on tangled IRC for helping me out with debugging the Knot setup and fixing bugs in real-time. 12th Mar 2025: updated with Samuel Wedgwood comments.
\nSrinivasan Keshav organised this week's EEG group discussion on what AI tools we use for our daily work. I was immediately struck by how few tools there are that are actually making us more productive, so I jotted down notes as the discussion was going on.
\nThen there's a long list of things that people aren't using because they suck. LLM-driven searches are pretty inaccurate, as many people noted; I use Kagi but only because I love their AI-filtered search results, not because of their assistant!. I've turned off Apple Intelligence on all my devices, not because of privacy concerns, but because it's just utter crap -- the summaries are actually incorrect half the time. I find the autocorrect features similarly distracting and wrong most of the time, and normal spellcheckers do a better job in practise.
\nOur discussion then into developing news of emerging tools and techniques, since the field overall is just moving incredibly fast. Two things I've been reading this week are:
\nSo, I guess the TL;DR of our discussion was that current AI tools are the first generation, but we're heading rather rapidly into new frontiers of discovery, so there's only going to be more of them coming up...
I started pushing OCaml Docker images over to the Docker Hub in around 2017, to support the burgeoning automated build infrastructure around the use of the language. Back then, OCaml 4.06 was the latest release, and so I wrote an ocaml-version library to track the release metadata. It has been a bit of a success disaster, as that library now tracks every release of OCaml in the modern era, and also backs the automatic building of a huge array of compiler versions and variants across Linux and Windows.
\nThe problem is...we're now building the full set of images from OCaml 4.02 onwards through to the latest OCaml 5.3.0 release, which is unsustainable for obvious reasons; despite the hosting being kindly sponsored by Docker, we must also consider the carbon footprint of our infrastructure.\nSo the question for the OCaml community: are there are any remaining users who still need images earlier than OCaml 4.08 or can we can stop pushing those now?
\nHannes Mehnert lead an effort to deprecate compilers earlier than 4.08 in the opam-repo, and now Mark Elvers is asking the same question on the OCaml discussion forum about the Docker image infrastructure. The latter lags the opam repository since there still may be operational usecases of industrial users who depend on older compilers, even if they don't use the latest package repository. So if you are using a really old OCaml and depend on our infrastructure, we'd appreciate you chiming in on the forum thread or just contact Mark Elvers or myself directly to let us know.
\nOn another note, it's also quite difficult on the central Docker Hub to get statistics per-tag as to how many people are using each image. Does anyone have any recommendations on whether we should deploy our own "proxy registry" before pushing through to the central Docker Hub, or alternative open source registries to run our own?
", "url": "https://anil.recoil.org/notes/deprecating-ocaml-408", "external_url": "https://discuss.ocaml.org/t/docker-base-images-and-ocaml-ci-support-for-ocaml-4-08/16229", "title": "Are you still using OCaml 4.08 or earlier? If so, we need to know", "summary": "OCaml users: share your needs regarding older versions to help determine support for OCaml 4.08 and earlier.", "date_published": "2025-03-05T00:00:00.000000Z", "date_modified": "2025-03-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/aecmt-k3h39", "content_html": "Yaron Minsky nerdsniped me last week into getting OCaml to drive the 80s-retro RGB Matrix displays. I grabbed one from the local Pi Store and soldered it together with help from Michael Dales. But instead of writing OCaml bindings by hand, we thought we'd try out the latest agentic CLI called Claude Code released last week to see if we could entirely autogenerate the bindings.
\n\nTL;DR: Claude Coder generated working OCaml code almost from scratch, ranging from C bindings to high-level OCaml interface files and even Cmdliner terms, but needs a more sophisticated sandboxing model before something goes horribly wrong. So much potential and so much danger awaits us. Coincidentally Cyrus Omar and Patrick Ferris and I wrote about this a few months ago. Read on...
\nThe RGB Matrix display has a very nice C++ rpi-rgb-led-matrix library, so I fired up my Raspberry Pi 4 to get an OCaml development environment going by using that. The included demo immediately gave me a disappointingly noisy display, but my larger-than-usual 64x64 display turned out to just need a jumper soldered.
\n![\"%c\"](\"/images/rgb-matrix-hat-ocaml-2.webp\" "\\"Deploying")
As soon that was soldered, the examples worked great out of the box, so I could get on with some agentic OCaml coding. Thanks Michael Dales and CamMakespace!
\nYaron Minsky and I first played around with using ocaml-ctypes to build the bindings by hand, but quickly switched over to trying out Claude Sonnet 3.7, first in VSCode and then directly on the Pi CLI via Claude Code. The latter fires up an interactive session where you not only input prompts, but it can also run shell commands including builds.
\nThe very first hurdle was sorting out the build rules. This is the one place where Claude failed badly; it couldn't figure out dune files at all, nor the intricate linking flags required to find and link to the C++ library. I made those changes quickly by hand, leaving just a stub librgbmatrix_stubs.c that linked successfully with the main C++ library, but didn't do much beyond that. I also added a near-empty rgb_matrix.ml and rgb_matrix.mli interface files to have a place for the OCaml side of the interface.
![\"%c\"](\"/images/claude-coder-ss-1.webp\" "\\"The")
After that, it was just a matter of "asking the Claude Code CLI" via a series of prompts to get it to fill in the code blanks I'd left. The VSCode Copilot editing mode has to be told which files to look at within the project for its context, but I didn't have to do that with the Claude Code CLI.
\nInstead, I just prompted it to generate C stubs from the led-matrix-c.h C interface (so it didn't get distracted attempting to bind C++ to OCaml, which isn't a winning proposition). It duly generated reasonable low-level bindings, along with the right OCaml interface files by suggesting edits to the files I'd created earlier. At this point, I got a very basic "hello world" circle going (with the test binary also built by Claude).
\n![\"%c\"](\"/images/rgb-matrix-hat-ocaml-3.webp\" "\\"The")
Although the binding generation built fine, they did segfault when I first ran the test binary! Claude 3.7 bound some C/OCaml functions with more than 5 arguments, which are a special case in OCaml due to differing bytecode and native code ABIs. Although Claude almost got it right, it subtly mixed up the order of the external binding on the OCaml side. The correct version is:
external set_pixels_native :\n t -> int -> int -> int -> int -> Color.t array -> unit =\n "caml_led_canvas_set_pixels_bytecode" "caml_led_canvas_set_pixels"\nThe bytecode C stub comes first, and the native code second, but Claude swapped them which lead to memory corruption. This mixup would ordinarily be rather hard to spot, but the valgrind backtrace lead me to the problem very quickly (but only because I'm very familiar with the OCaml FFI!). I couldn't convince Claude to fix this with prompting as it kept making the same mistake, so I swapped the arguments manually and committed the results by hand.
\nOnce the basics were in place, I then asked it to then refine the OCaml interface to be higher-level; for example instead of a string for the hardware mode, could it scan the C header file, find the appropriate #defines, and generate corresponding OCaml variant types? Incredibly, it not only did this, but also generated appropriate OCamldoc annotations for those types from the C header files.
![\"%c\"](\"/images/claude-coder-ss-2.webp\" "\\"These")
The Claude Code CLI then helpfully summarises all the changes, and also offers execute dune to check the result works! This is starting to get a bit mad...
\n![\"%c\"](\"/images/claude-coder-ss-3.webp\" "\\"Claude")
![\"%c\"](\"/images/claude-coder-ss-4.webp\" "\\"It")
![\"%c\"](\"/images/claude-coder-ss-5.webp\" "\\"The")
The OCaml interfaces generated here required a little iteration to get right, with some manual tweaks. Claude, for some reason, generated duplicate entries for some type definitions, which OCaml doesn't permit. I fixed those manually very quickly, and then asked Claude Code to commit the changes to git for me. It generated a good summary commit message. The interfaces were all documented with docs from the C header file, such as:
\ntype multiplexing =\n | DirectMultiplexing (* 0: Direct multiplexing *)\n | Stripe (* 1: Stripe multiplexing *)\n | Checker (* 2: Checker multiplexing (typical for 1:8) *)\n | Spiral (* 3: Spiral multiplexing *)\n | ZStripe (* 4: Z-Stripe multiplexing *)\n | ZnMirrorZStripe (* 5: ZnMirrorZStripe multiplexing *)\n | Coreman (* 6: Coreman multiplexing *)\n | Kaler2Scan (* 7: Kaler2Scan multiplexing *)\n | ZStripeUneven (* 8: ZStripeUneven multiplexing *)\n | P10MapperZ (* 9: P10MapperZ multiplexing *)\n | QiangLiQ8 (* 10: QiangLiQ8 multiplexing *)\n | InversedZStripe (* 11: InversedZStripe multiplexing *)\n | P10Outdoor1R1G1_1 (* 12: P10Outdoor1R1G1_1 multiplexing *)\n | P10Outdoor1R1G1_2 (* 13: P10Outdoor1R1G1_2 multiplexing *)\n (* ...etc <snipped> *)\n | Custom of int (* Custom multiplexing as an integer *)\nPretty good! After that, I couldn't resist pushing it a bit further. I asked the CLI to generate me a good command-line interface using Cmdliner, which is normally a fairly intricate process that involves remembering the Term/Arg DSL. But Claude aced this; it generated a huge series of CLI converter functions like this:
\n(* scan_mode conversion *)\n let scan_mode_conv =\n let parse s =\n match String.lowercase_ascii s with\n | "progressive" -> Ok Progressive\n | "interlaced" -> Ok Interlaced\n | _ -> Error (`Msg "scan_mode must be 'progressive' or 'interlaced'")\n in\n let print fmt m =\n Format.fprintf fmt "%s"\n (match m with\n | Progressive -> "progressive"\n | Interlaced -> "interlaced")\n in\n Arg.conv (parse, print)\nThese are not entirely what I'd write, as Cmdliner.Arg.enum would suffice, but they're fine as-is and could be refactored later. I even got it to complete the job and generate a combined options parsing function for the (dozens) of command-line arguments, which would have been very tedious to do by hand:
\n(* Apply options from command line to Options.t *)\nlet apply_options options\n ~rows ~cols ~chain_length ~parallel ~hardware_mapping ~brightness \n ~pwm_bits ~pwm_lsb_nanoseconds ~pwm_dither_bits ~scan_mode ~row_address_type \n ~multiplexing ~disable_hardware_pulsing ~show_refresh_rate ~inverse_colors\n ~led_rgb_sequence ~pixel_mapper_config ~panel_type ~limit_refresh_rate_hz \n ~disable_busy_waiting =\n Options.set_rows options rows;\n Options.set_cols options cols;\n Options.set_chain_length options chain_length;\n Options.set_parallel options parallel;\n Options.set_hardware_mapping options hardware_mapping;\n Options.set_brightness options brightness;\n Options.set_pwm_bits options pwm_bits;\n Options.set_pwm_dither_bits options pwm_dither_bits;\n Options.set_scan_mode options scan_mode;\n Options.set_pixel_mapper_config options pixel_mapper_config;\n Options.set_panel_type options panel_type;\n Options.set_limit_refresh_rate_hz options limit_refresh_rate_hz;\n Options.set_disable_busy_waiting options disable_busy_waiting;\n (* ...etc <snipped> *)\n options\nOnce this compiled, I asked for a rotating 3D cube demo, and it duly used the bindings to give me a full command-line enabled generator which you can see below. I just ran:
\nrotating_block_generator.exe --disable-hardware-pulsing -c 64 -r 64 --hardware-mapping=adafruit-hat --gpio-slowdown=2\nand I had a spinning cube on my display! The code model had no problem with the matrix transformations required to render the cool spinning effect.
\n\nOf course, I had to pay the piper for the truckload of GPUs that drove this code model. At one point, the Claude Code agent got into a loop that I had to manually interrupt as it kept oscillating on a code fix without ever finding the right solution. This turned out to have sucked up quite a lot of money from my Claude API account!
\n![\"%c\"](\"/images/claude-coder-ss-6.webp\" "\\"This")
Overall, I'm impressed. There's clearly some RL or SFT required to teach the code model the specifics of OCaml and its tooling, but the basics are already incredible. Sadiq Jaffer, Jon Ludlam and I are having a go at this in the coming months.
\nThe obvious downside of this whirlwind binding exercise is that while the NPM-based Claude Code asks nicely before it runs shell commands, it doesn't have to ask. I happened to run it inside a well-sandboxed Docker container on my rPi, but most people probably won't. And in general, we need a more sophisticated security model; running the agent within a coarse sandbox that limits access to the file system, the network, and other sensitive resources is too restrictive, as we want to provide access to these resources for certain agentic tasks!
\nSo in a happy coincidence, this leads to a line of research that Cyrus Omar and Patrick Ferris started last year with something we presented at HOPE 2024. We explored how to express more precise constraints on what an AI can do by the use of the scary-sounding Dijkstra monad. It's far easier to understand by perusing the slides of the talk, or watch Cyrus Omar's great video presentation.
\nWe're mainly concerned with situations where the AI models are running over sensitive codebases or datasets. Consider three scenarios we want to handle, which are very logical extensions from the above agentic coding one:
\nFor each of these, we could use a capability security model where access to sensitive data and effects can occur only via unforgeable capabilities granted explicitly. And the generation of that specification could also be done via code LLMs, but needs to target a verification friendly language like Fstar. The prototype Patrick Ferris built looks like this:
\nmodule type CapDataAccess (readonly : list dir, writable : list dir)\n (* abstract monad *)\n type Cmd a\n val return : a -> Cmd a\n val bind : Cmd a -> ( a -> Cmd b ) -> Cmd b\n (* only allows access to given directories *)\n val readfile : path -> Cmd string\n (* only allows writes to writable dirs *)\n val writefile : path -> string -> Cmd ()\nAnd then you can use this rich specification to add constraints, for example see this JSON parsing example from the Fstar prototype:
\n(* Following IUCN's Globally Endangered (GE) scoring *)\nlet datamap = [\n"iberian-lynx.geojson", O [ "rarity", Int 2 ];\n"bornean-elephant.geojson", O [ "rarity", Int 3 ]\n]\n\n(* We add some additional predicates on the files allowed to be used *)\n@|-1,9 +1,10 ==========================================\n| (ensures (fun _ -> True))\n| (requires (fun _ _ local_trace ->\n| dont_delete_any_file local_trace /\\\n+| all_paths_are_not_endangered readonly /\\\n| only_open_some_files local_trace readonly))\n|}\nOnce you have this specification, then it's a matter of implementing fine-grained OS-level sandboxing policies to interpret and enforce them. Spoiler: we're working on such a system, so I'll write about that just as soon as it's more self-hosting; this area is moving incredibly fast.
\n\nThanks to Michael Dales for help soldering. For the curious, here's the PR with the code, but it shouldn't go anywhere near any real use until we've had a chance to review the bindings carefully. There needs to be a new, even more buyer-beware no-warranty license for AI generated code!
\nThe IETF announced their new AI Preferences Working Group (AIPREF), which will "work on standardizing building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence models". This is quite well timed; the IETF tries not to standardise too early before there is running code but also needs to move before it's too late and a bad defacto standard is chosen. The AI world seems to be at that nexus point right about now, with GPT 4.5 seemingly hitting a scaling wall and possibly triggering the start of a renewed data scraping frenzy.
\nI've found when developing my own website there are a number of approaches to interacting with automated data crawlers. For the record, over 90% of the traffic to this site is from automated sources, so it's a material concern for ##selfhosting infrastructure.
\nrobots.txt doesn't give us enough control to usefully rate-limit across all of these, unfortunately. You need to regenerate the file every time there are new URLs on the site that don't fit a longest-prefix match. This, combined with having a mega sitemaps file, is a lot of non-cacheable metadata that's just adding to my serving load.The IETF has a valuable role to play here to establish a consensus around what a sensible, usable protocol for exchanging data on our websites might look like, rather than mandating any specific backend technology or storage format.\nThere is a lot of nuance around sharing content over HTTP: it supports authentication, caching, access control, rate limiting, and many other features hidden behind a seemingly simple request-response specification.
\nI'm hoping that the AIPREF process will end up with something that gives me something closer to 5) above than 1). I need an HTTP-based mechanism by which I can express my preferences for AI crawling, and cooperate with the crawlers so that I can ensure maximum collective benefit to both people and bots visiting my site, rather than withdrawing behind a gated community of humans only. However, I think that this requires the establishment of a protocol to help sequence the HTTP requests together and not just a single static file like llms.txt or sitemap.xml.
Back in the 90s, I worked on NetApp/NetCache with John Martin. Bandwidth used to be expensive and so we deployed edge caches that could modify website content with local modifications to common global content. Consider, for example, a local news website that might want to show mostly cached global news, but also modify the HTML to include local news content. You can do that today via JavaScript, but back then the only way to have a protocol to modify the static HTML. The Internet Content Adaptation Protocol was the IETF's answer to creating a structured HTTP-like protocol to allow edge modifications from proxy servers:
\n\n\nICAP is, in essence, a lightweight protocol for executing a "remote procedure call" on HTTP messages. It allows ICAP clients to pass HTTP messages to ICAP servers for some sort of transformation or other processing ("adaptation"). The server executes its transformation service on messages and sends back responses to the client, usually with modified messages. Typically, the adapted messages are either HTTP requests or HTTP responses.\n-- RFC3507, IETF
\n
One of the coolest features of ICAP is that is didn't mandate the transformation mechanism, just the protocol. The proxies deployed at the edge networks would get a vector into transforming the data stream. NetCache implemented an implementation of ICAP, and Squid still supports it. What would a similar approach look like for allowing crawlers into your site's content, but leaving lots of freedom for the details of this to be delegated to the crawlers and servers?
\nAntoine Fressancourt identifies the main problem facing AIPref:
\n\n\nGiven the reports that some current LLM models have been trained on data corpus obtained illegally, I have some doubts that AIPref will be respected.
\n
This is true for the current generation of data crawlers, but also where the opportunity lies for AIPref. Without a systematic way to support replication of non-public data, the situation will get even worse as custom apertures are created into data silos without any integrity underlying them.
\nThe main reason for having a protocol-based solution is that we could support the strong authentication and identification of bots. If (for example) the GoogleBot supplied a token with every HTTP request to fetch my content, I could track its use and perhaps even get compensation for the bandwidth costs. The current methods of bot verification all seem quite weak; they are just IP based checks for example.
\nThis would in turn open a path to the disciplined negotiation for access controlled data bilaterally between crawlers and hosters. More and more content publishers are signing various exclusive deals with AI training companies. Irrespective of your opinion on such deals, a protocol to make it easier to authenticate bots strongly would make the establishment (and ongoing negotiation) of those mechanisms far easier to handle.
\nWe are also seeing rapid adoption of the the Model Context Protocol released a few months ago. This establishes a JSON-RPC specification for LLM clients and data providers to talk to each other locally. It seems odd to me that we'd have a rich "local" specification for data exchange like this for RAG-like systems, but not have one in the wide area across the Internet. As the chair of the AIPREFS group Mark Nottingham notes, platform advantages are not just network effects, so there may be deep repurcussions into the economics of AI here:
\n\n\nIn short: there are less-recognised structural forces that push key Internet services into centralized, real-time advertising-supported platforms. Along with factors like network effects and access to data, they explain some of why the Internet landscape looks like it does.\n-- Mark Nottingham
\n
Just substitute "advertising-supported" with "AI" above and the trend becomes clear. The protocol designs we chose today will form structural forces that decide the future of what the post-advertising driven Internet culture and content architecture looks like. It would be a nice outcome to establish open protocols that are somewhere in between the MCP clients and HTTP servers to facilitate a more equitable outcome rather than pooling all the data to a few big players.
\nThe other consideration is here is that such an open protocol could have utility far beyond "just" managing AI training bots and address the general problem we have that replicating datasets with access control is difficult. This would help the good folk at Archive.org to manage restricted access data sets that might want to become eventually open. There are also geospatial datasets such as biodiversity data that need help managing how they are mirrored, but with access restrictions for geopolitical reasons.
\nLuckily, the IETF do a lot of things over email, so I've signed up to the AIPREF mailing list to learn more as it develops and hopefully participate!
\n\nChangelog. Mar 1st 2024: Thanks to Michael Dales for spotting typos, and Antoine Fressancourt for helpful clarifying questions on Bluesky.
\nThis week I've been reading three really nice pieces of work by my\ncolleagues, in the form of a review paper on biodiversity and AI,\na benchmark for 3D forest reconstruction using laser scanners and a mobile app for measuring the width of tree trunks. A real bonanza for forest lovers!
\nA paper on 'Harnessing AI to fill global shortfalls in biodiversity knowledge' just came out in Nature Biodiversity today (via Oisin Mac Aodha). They start with the baseline present uses of AI (camera traps, acoustic monitoring and improved data analysis) which are pretty well known to anyone in the field, but then introduce a lovely diagram of future uses of AI for biodiversity which includes:
\nThose interested in this may also want to look at our recent horizon scan on AI and conservation from a few months ago. The field is moving so quickly that I wouldn't be surprised if both of these were obsolete a year from now!
\n ![\"%c\"](\"/images/umzc-4.webp\" "\\"If")
And then out in MEE is a comprehensive benchmark from a collection of forestry researchers on a benchmark for tree species classifiction from proximal laser scanners. Their FOR-species20k dataset is on Zenodo, and has tons of tree point clouds taken using a variety of laser scanning techniques (TLS, MLS and ULS).
\nAs Emily Lines notes:
\n\n\nMost importantly, we demonstrate that community efforts and open science are the only way to make significant progress in this important task. With more researchers publishing and sharing high quality 3D forest datasets, I hope we see an end of single-site studies and that proper and broad benchmarking of all new 3D forest deep learning methods becomes the standard.\n-- Emily Lines on LinkedIn
\n
I've been learning more about tree species identification for tropical species last year, so I'm looking forward to delving more into laser scanning techniques soon from this work.
\nAnd last but not least, I was delighted to see that my colleagues Srinivasan Keshav and Frank Feng released the extremely cool mobile phone app they've been working on for some time along with a paper in Ecological Informatics. Their app is a simple and elegant mobile phone app that can measure the diameter of a tree trunk (more specifically, the DBH) just using standard cameraphone hardware on most modern-ish Android phones.
\nI was lucky enough to beta test this and try it out on my recent trip to India, and the GreenLens is also now open source as well.
\n\n\nOther apps for measuring forest plots are available [...] but those for Android phones tend not to perform as well as ours, while those designed for the iPhone require the purchase of a high-end phone that is not affordable for researchers in the Global South.
\nWe believe ours is the only app to sit in the 'sweet spot' of offering high quality for low cost.\n-- Frank and Keshav on cam.ac.uk
\n
![\"%c\"](\"/images/pups-india-1.webp\" "\\"I")
Over the past year, Sadiq Jaffer and I have been getting an object lesson in how the modern Internet handles researcher access to data, as we've been downloading tens of millions of research papers towards our Conservation Evidence project. This is legally possible via our institutional subscriptions that give us license to fulltexts, and the incredibly helpful head of electronic services at the University Library who wields encyclopedic knowledge of each of our agreements with the hundreds of publishers out there. My thoughts on this then segwayed into recent conversations I've been having about the emerging National Data Library and also with the UK Wildlife Trusts...
\nIn late 2023, once we got past the legal aspects of downloading closed access papers[1] it was still remarkably difficult to actually gain access to the actual paper datasets themselves. For instance, a select few hurdles include:
\nThe situation is better for open access downloads, where projects such as Core offer easier bulk access and large metadata databases like OpenAlex use 'downloader pays' S3 buckets. And in other domains like satellite data, there is still a lot of complexity in obtaining the data, but programming wrappers make implementing the (often terabyte-level) downloads much more palatable. For our recent LIFE biodiversity maps, we also make them available on services like Zenodo as they are open.
\nThe lesson I took away from this is that it's really difficult to deal with large sensitive datasets where selective access control is required, and also that sort of data is rarely mirrored on the open web for obvious reasons. But in the current climate, it's utterly vital that we move to protect human health or biodiversity data gathered over decades that is irreplaceable once lost. And beyond data loss, if the data is present but not accessible, then what's the point in gathering it in the first place? It's also really important not to blame the existing publishers of these datasets, who are getting overwhelmed by AI bots making huge numbers of requests to their infrastructure. So I'm getting energised by the idea of a cooperative solution among all the stakeholders involved.
\nYou can imagine my excitement late last year when I got a call from the Royal Society to show up bright and early for a mysterious speech by Rishi Sunak. He duly announced the government's AI summit that mostly focussed on safety, but a report by Onward caught my eye by recommending that "the Government should establish a British Library for Data – a centralised, secure platform to collate high-quality data for scientists and start-ups". I wasn't down for the "centralised" part of this, but I generally liked the library analogy and the curation it implied.
\n![\"%c\"](\"/images/rishi-sunak-rs-ai-1.webp\" "\\"Seeing")
Then in 2025, with Sunak dispatched back to Richmond, Labour took up the reigns with their AI Action Plan. While this report started predictably with the usual need for acres of GPU-filled datacenters, it continued onto something much more intriguing via the creation of a "National Data Library":
\n\n\n\n
\n- Rapidly identify at least 5 high-impact public datasets it will seek to make available [...] Prioritisation should consider the potential economic and social value of the data, as well as public trust, national security, privacy, ethics, and data protection considerations.
\n- Build public sector data collection infrastructure and finance the creation of new high-value datasets that meet public sector, academia and startup needs.
\n- Actively incentivise and reward researchers and industry to curate and unlock private datasets.\n-- AI Opportunities Action Plan, Jan 2025
\n
This takes into account much more of the nuances of getting access to public data. It identifies the need for data curation, and also the costs of curating such private datasets and ensuring correct use. The announcement spurred on a number of excellent thoughts from around the UK web about the implications, particularly from Gavin Freeguard who wrote about how we should think about an NDL. Gavin identified one particularly difficult element of exposing private data:
\n\n\n[...] analogy with the National Data Library suggests that there might be some materials available to everyone, and some restricted to specialist researchers. There may be different access models for more sensitive material. There may be better and worse options — bringing together all the data in one place for accredited researchers to access [...] would be a logistical and security nightmare [...] may be possible to keep the data where it already is, but provide researchers with the ability to access different systems.\n-- Gavin Freeguard
\n
Others also identified that the centralised library analogy only goes so far, and that we should focus on building trustworthy data services instead and on the real lifecycle of the data usage:
\n\n\n[...] this means that the latest data is already there in the "library" [...] researchers don't first need to work with the data owners to create it [...] bodies of knowledge around using these complex datasets can be built up over time.
\nResearchers can share code and derived data concepts, so the researchers that come after can iterate, refine, and build on what has gone before. None of this was possible with the previous "create and destroy" model of accessing these types of datasets, which was hugely inefficient\n-- Administrative Data Research UK
\n
Gosh, this network effect sounds an awful lot like what I experienced as a Docker maintainer, which had its incredible popularity fuelled by tapping into users to building and sharing their own software packaging rather than depending on third parties to do it for them. If we could unlock the power of crowds here but go one step further and enforce privacy constraints on the underlying data and code, then the technical solution could be both usable and secure. I'm still not quite sure what that balance of UI would look like, but we're working on it spearheaded by Patrick Ferris, Michael Dales and Ryan Gibb's research areas.
\nThe Wellcome and ESRC have also put together a series of whitepapers about the challenges and potential approaches behind the NDL (via Nick McKeown). I'm still going through them in detail, but the modular approach paper makes sensible observations about not trying to build one enormous national database and to not outsource it all to one organisation to build. Instead, they espouse a federated architectural approach.
\n ![\"%c\"](\"/images/federated-ndl-ss-1.webp\" "\\"Sourced")
Since their primary (but not only) usecase focuses on health data, there is an emphasis on moving the computation and data around rather than pooling it:
\n\n\nThe project's overlay mesh network dynamically and securely connects all the required resources. The\nmesh network creates a transient, project-specific, secure network boundary such that all the project’s\ncomponents are within one overarching safe setting\n-- A federated architecture for a National Data Library
\n
This isn't a million miles away from how we set up overlay networks on cloud infrastructure, but with the added twist of putting in more policy enforcement upfront.
\nHowever, it's not all plain sailing, as there is also mega-controversy ongoing with the UK government's surprising demands for an encryption backdoor into iCloud, leading to even more of a geopolitical tangle with the US. Irrespective of what happens with this particular case, it's clear that any end-to-end encryption in these federated systems will need to deal with the reality that jurisdictions will have different lawful decryption needs, so end-to-end encryption may be at an end for initiatives like the NDL. Add onto this the flagrant disregard for licensing in current pretrained language models but also the movement to revise copyright laws to legislate around this, and it's clear that technology will need to be fluid in adapting to matters of provenance tracking as well.
\nThere's definitely a rich set of academic literature in this space, combined with interesting constraints, and so I'll pull this together into an annotated bibtex soon!
\nTo get some more inspiration on a technical solution, I've been looking to users of such an infrastructure to understand what easy-to-use interfaces might look like.
\nMy colleague Neil Lawrence over at AI@Cam co-lead a recent report into case studies for the NDL which is very much worth a read. From a conservation perspective, Sadiq Jaffer and Alec Christie both gave input about the importance of having such infrastructure for evidence-driven landuse.
\n\n\nWhat would be helpful, according to Dr Jaffer, is more\nstandardisation between publishers for Open Access material\nunder permissive licences.\n[...] having a coherent archive for OA materials that are licensed\nin such a way that they can be used for data mining without\nany technical hurdles would be the ideal scenario for this kind\nof research, as well as for a National Data Library,\n-- Access to Data for Research, AI@CAM
\n
![\"%c\"](\"/images/ai-cam-data-library.webp\" "\\"The")
Another very different group I talked to back in 2023 via Rosalind Goodfellow as part of her CSaP fellowship was the Geospatial Commission who began work on a National Underground Asset Register. The NAUR was initially restricted to "safe dig" usecases and not exposed more widely for security and other concerns. In 2024, they subsequently reported great interest in expanded usecases and are doing a discovery project on how to expose this information via APIs. This seems like an ideal usecase for some of the access control needs discussed above, as it's not only a lot of data (being geospatial) but also updated quite frequently and not necessarily something to make entirely public (although satellite pipeline monitoring is perhaps obsoleting this need).
\nAnd a month ago after reading our horizon scan for AI and conservation paper, Sam Reynolds David Coomes Emily Shuckburgh and I got invited by Craig Bennett to a remarkable dinner with the assembled leaders of all 46 of the UK's wildlife trusts. They are a collective of independent charities who together maintain wildlife areas across the UK, with most people living near one of their 2300+ parks (more than there are UK McDonald's branches!). Over the course of dinner, we heard from every single one of them, with the following gist:
\nThe trusts are operating on a fairly shoestring budget already, so they're a great candidate to benefit from a collective, federated National Data Library. In particular, if the NDL can nail down a collective bargaining model for data access to big tech companies, this could finance the collection costs among smaller organisations throughout the four nations. The same holds true for thousands of small organisations around the UK that could benefit from this infrastructure and kickstart more sustainable growth.
\n![\"%c\"](\"/images/wildlife-trusts-homerton.webp\" "\\"The")
I'm organising a get-together on the topic of planetary computing next month with Nate Foster and a number of colleagues from around the world, so stay tuned for more updates in this space in the coming months! Your thoughts, as always, are most welcome.
\n\n(Thanks Sam Reynolds for the notes on what we discussed with the Wildlife Trusts)
\n\nThis largely involved talking to individual publishers and agreeing not to directly train generative AI models and to keep them private to our own research use. Fairly reasonable stuff.
\n ↩We've uploaded a revised preprint on our ongoing work on quantifying the biodiversity cost of global food consumption, lead by Thomas Ball. This is based on the recently published LIFE metric, combined with supply chain data and provenance modeling.
\nOf particular interest may be the new analysis in the supplementary material which analyses the biodiversity cost of common food types against their size as a "functional unit" (that is, how much is typically consumed). The results don't come too differently at a macro scale, but the breakdown is interesting to see.
\n ![\"%c\"](\"/images/food-functional-service-units-preprint-v2.webp\")
Richard Mortier also pointed out some related work that he did on this topic back in 2015 at Horizon, where they did an ethanographic study on "Understanding food consumption lifecycles using wearable camera". It would be an extremely cool followup project to:
\n\n\nbuild a tool (app, whatever) that incentivised collection and enabled [privacy-preserving] sharing of such data. "A Social Food Network".\n-- Richard Mortier, personal communication, 2025
\n
I'll write that up in my ideas page when I get a chance. Any other comments or questions on the preprint are, as always, very welcome! In the meanwhile, please find the preprint details below.
The terms carbon credits and carbon offsets are often used interchangeably,\nbut are in fact two distinct concepts. I've spent a nice Sunday morning\nreading up on some recent articles that Bhaskar Vira sent me which introduce a\nthird term, known as "carbon contributions". Rather than this adding confusion, I\nfound it helped me clarify my own thoughts on the matter, which I\nnote down here in draft form. (Update 7th Feb: I've revised this several times after many discussions this week, especially with David Coomes and Srinivasan Keshav, with full list of credits in the end)
\nA carbon credit aims to quantify the net climate benefit resulting an\nintervention that alters some CO2 emissions that would otherwise have gone into\nthe atmosphere in a business-as-usual counterfactual scenario. While there are many\ndifferent categories of carbon credits, I'll focus on nature-based solutions. For example,\nwe could fund an intervention which provides an alternative livelihood to cutting down tropical rainforests,\nand then calculate the area of rainforest saved (and therefore, the amount of avoided carbon emitted into the atmosphere) as a result\nof this action.
\nThe carbon credit therefore measures the additional amount of CO2 avoided as a result of the specific intervention,\nadjusted for negative externalities and the impermanence of\nthe action into the future if it's at risk of being reversed. We can monitor the measurements using spaceborn sensing to\nestablish global baselines against which to calculate the counterfactual impacts of positive actions.[1] Carbon credits are nowadays their own asset class, both legally and fiscally.
\nA carbon offset [2] is then a way to account for the net climate benefits that one entity brings to another. The "benefits" are the amount of CO2e avoided or removed via the carbon credit, and the "costs" are the amounts of CO2e being emitted by the other party. The origin of this accounting can be traced back to the UN's net-zero goals:
\n\n\nNet-zero means cutting carbon emissions to a small amount of residual emissions that can be absorbed and durably stored by nature and other carbon dioxide removal measures, leaving zero in the atmosphere.\n-- UN Net Zero coalition
\n
The theory behind offsetting is that we can never get to a complete net zero state due to the residual CO2 emissions that will remain in even the most optimal decarbonised societies. For these residual emissions, we need to offset them with corresponding climate benefits in order to balance the books on how much carbon is in the atmosphere and how much is being absorbed by the planet's biosphere. And one of the main sources of CO2 absorption that we must protect in the biosphere are rainforests:
\n\n\nCarbon sinks have increased in temperate and tropical regrowth forests owing to increases in forest area, but they decreased in boreal and tropical intact forests, as a result of intensified disturbances and losses in intact forest area, respectively. The global forest sink is equivalent to almost half of fossil-fuel emissions. However, two-thirds of the benefit from the sink has been negated by tropical deforestation.\n-- The enduring world forest carbon sink, Nature 2024
\n
Since tropical rainforests are so crucial for both CO2 absorption and biodiversity, my own recent research has largely focussed on reliable accounting for quantifying carbon credits accurately for avoided deforestation projects in these regions. This work been progressing steadily, and we're increasingly confident in the quantification methods used behind measuring the carbon sequestration impact of nature-based credits.
\nHowever, what has been dragging down carbon credits is how they are used after they are verified and purchased, which is predominately via carbon offsetting. Let's first examine the problems with carbon offsetting, and then examine an emerging concept of "carbon contributions" might provide a better way forward for carbon credits.
\nCarbon offsets are currently mostly voluntary, where private actors can purchase carbon credits towards reducing their emissions targets. The obvious problem with offsetting is that it can give bad actors a license to spend money to continue to pollute, while breaking their emissions pledges. And the harsh reality is that if we don't engage in immediate and real emissions reductions, we're screwed in the coming decades.
\nUnfortunately, we need to balance this with the short-term reality that many of these businesses have to emit to remain competitive, for example in the AI sector (Deepseek notwithstanding!).\nAmazon highlighted the difficulty of forecasting their emissions in their annual sustainability report in 2023:
\n\n\n[...] our progress toward a net-zero carbon business will not be linear, and each year as our various businesses grow and evolve, we will produce different results [...] These results will be influenced by significant changes to our business, investments in growth, and meeting the needs of our customers.\n-- Amazon Sustainability Report 2023
\n
As did Google, who gave up on 'real time net zero' last year, preferring instead to aim for the comfortably distant 2030:
\n\n\n[...] starting in 2023, we're no longer maintaining operational carbon neutrality. We're instead focusing on accelerating an array of carbon solutions and partnerships that will help us work toward our net-zero goal [...]\n-- Google Environment Report 2024
\n
Your heart may not be bleeding for these tech companies finding it difficult to forecast how they'll make their next trillion dollars, but there is the undeniable reality that they need to break emissions pledges in response to global competitive pressure on their core businesses. But given this, is there still any point in all the precise accounting frameworks for net-zero carbon offsetting?
\nA December article in the FT argues that there needs to be a fundamental shift in our approach to carbon credits for this reason. They observed that the use of carbon offsets for emissions trading in the EU will probably only apply to removal projects that suck carbon from the air and not to the nature-based deforestation avoidance schemes I described above.
\n\n\nCorporate funding for nature conservation has a useful role to play — but as a contribution to the public good, not for use in tonne-for-tonne emissions offsetting calculations.\n-- Simon Mundy, "It's time for a shift in approach to carbon credits", FT
\n
And there is the critical distinction between carbon "credits" and "offsets" I was looking for! Simon acknowledges the crucial importance of generating forest carbon credits to advance the extremely urgent problem of tackling tropical deforestation, but notes that corporations should not be giving to this pot as part of a complex accounting scheme tied to the vagaries of their ever-shifting business strategies. Forests are too important to our continued existence to be left to the mercies of a volatile stock market.
\nInstead, we need to come up with a scheme for spending carbon credits whose incentives are aligned towards keeping the focus on emissions reductions and behavioural change. So, let's next firmly decouple carbon credits from carbon offsets, and examine how organisations that wish to do the right thing can...contribute...instead.
\nAn article last year by a former Cambridge Gates Scholar Libby Blanchard and colleagues made a very clear case how and why we might replace carbon offsetting with "carbon contributions", and especially so for forest protection. She observed that the integrity crisis in the offsets market has quite rightly lead to the exposure of many poor quality schemes, but is also drying up crucial funding for the good actors who are working hard under very adverse conditions to launch forest protection schemes in the global south and north.
\n\n\nOne way to channel forest finance away from bad offsets toward more productive outcomes is, simply, to stop claiming that forests offset fossil fuel emissions. Companies could, instead, make "contributions" to global climate mitigation through investments in forests.
\nThis change in terminology may seem small, but it represents a fundamentally different approach. For one thing, not allowing companies to subtract carbon credits from their direct emissions into a single net number, as offsetting does, refocuses priorities on direct emissions reductions. Companies would no longer be able to hide inaction behind offset purchases.\n-- Libby Blanchard, Bill Anderegg and Barbara Haya, Instead of Carbon Offsets, We Need 'Contributions' to Forests, Jan 2024
\n
This approach is radically more accessible for a good actor who has been scared away from offsets and is entangled in complex SBTI-style accounting frameworks!
\nFirstly and most importantly, it removes the incentive to purchase the cheapest credits on the market at the lowest price possible. Since the organisations are no longer racing to hit a net-zero target, they can afford to find the highest quality and highest impact carbon projects available, and put their money towards those instead.
\nSecondly, a contributions model focussed on quality means that more organisations can safely participate. In the current voluntary market, there is a market for lemons situation where it is very difficult to distinguish junk credits from worthwhile credits, since the market price is not a reliable indicator of quality. This means that the vast majority of organisations withdraw from participating in the (voluntary) market due to the reputational risks, leaving only two sorts of participants: very good actors who really want to do the right thing, and very bad actors who are blatantly greenwashing. It's a very odd party if the only two sorts of people left are the sinners and the saints!
\nLet's look more closely at each of these points, as I think it fundamentally changes the dynamics of the use of carbon credits.
\nThere are a vast array of carbon avoidance, reduction and removal schemes; how do we chose between them? The current carbon markets focus on secondary trading as a price proxy, but this is a poor indicator of the underlying reliability and human and biodiversity cobenefits of any given intervention. In 2021, the University of Cambridge Offset Working Group commissioned a comprehensive report on how we might compare project quality and cobenefits first, and then figure out a suitable price for each. This methodology (dubbed "PACT") allows us to compare diverse credit types such as direct-air-capture and nature-based solution projects as apples-to-apples. Here's an excerpt from that report:
\n![\"%c\"](\"/images/pact-table.webp\" "\\"Table")
The important column is the £PACT one, which shows the adjusted costs per ton of carbon of purchasing those credits. The Climeworks direct-air-capture comes in at £900/tonne [3] whereas a tropical rainforest project in Sierra Leone ranks in at £73/tonne, even after impermanance is adjusted for! That's an absolutely mind-blowing price difference for a market that's allegedly more efficient due to the existence of secondary trading. Yet there is an order-of-magnitude price difference between tropical forest protection and direct air capture, and that's before taking into account the obvious co-benefits of forest protection such as biodiversity and livelihood improvements.
\nBlanchard's earlier article identifies the key benefits of a contributions model here:
\n\n\nFreeing companies from the pressure of "offsetting" by switching to a "contributions" frame lessens the incentive to minimize costs at the expense of quality, allowing them to focus on contributing to higher-quality projects.\n-- Libby Blanchard, Bill Anderegg and Barbara Haya
\n
Since the University is not planning on spending these carbon credits on accounting towards a net-zero goal, it is free to search the market for the highest quality impact -- in this case, tropical rainforest avoidance credits that are hugely undervalued -- and also filtering based on important co-benefits such as biodiversity and livelihood impacts. And by sharing our knowledge about high quality carbon credit projects, we could hopefully find many other organisations that want to similarly contribute, and drive up the price of rainforest credits to their true value.[4]
\nWith a contributions model, we no longer care what the absolute price we're paying for the credits are: our contributions only reflect a fraction of our total climate damage anyway, and we want the carbon credits that we do purchase to reflect the highest available impact out of the spectrum of compensation efforts that we could engage in. There's still one important consideration we'll talk about next though: how should an organisation account for these contributions, if not as part of a net-zero mechanism?
\nThe primary sustainability focus of any organisation must be on decarbonisation via direct emissions reduction. With carbon contributions, we can focus on this without the distractions of race-to-the-bottom carbon offset accounting.
\nFor example, consider the area of international air travel. There are plenty of things to do to reduce emissions here as a matter of urgent policy change. My University's sustainable travel policy is sensible and dictates that it must be a trip of last resort to fly; we must use trains or other land travel where available, such as for European trips. There is also plenty of science to invest in to reduce the impact of aviation; ranging from electrified planes and contrails and optimised routing. But, while all this is going on, sometimes there is only one practical way to get somewhere internationally, such as for an annual conference. We need all the emissions reductions strategies to be deployed first, and while these are taking effect we also need to also augment them with voluntary contribution towards the last-resort travel that's happening while they are being rolled out or researched. Or indeed, also compensate for past travel emissions, as CO2e affects the climate for longer than Stonehenge has existed!
\nAnother similarly topical emissions reduction area is on how to reduce our ruminant meat consumption. More and more research is showing how damaging this is for tropical forest destruction but also from a biodiversity angle. But it turns out that nudging consumers such as Cambridge students and staff towards less damaging choices by default is entirely practical:[5]
\n\n\nA study of over 94000 cafeteria meal choices has found that doubling the vegetarian options – from 1-in-4 to 2-in-4 – increased the proportion of plant-based purchases by between 40-80% without affecting overall food sales.\n-- Veg nudge. Impact of increasing vegetarian availability on meals (paper / followup)
\n
For both of these emissions reductions initiatives, we could tag on a voluntary contribution whenever some damaging action (long-haul flying, importing ruminant meat, etc) is taken. This is an contribution of last resort ("I am a grad student presenting a paper and have to go to abroad for this conference"). In annual sustainability reports, the primary focus of reporting would remain firmly on the emissions reductions initiatives themselves. But the contributions gathered from these schemes could be pooled, and treated as a collective (but voluntary) carbon tax on the damages to nature and the atmosphere.
\nAnd how do we spend this carbon tax? On the highest quality carbon projects we can find in the big wide world, as I described earlier! Each individual reductions scheme doesn't worry about what the compensation mechanisms are; groups similar to the COWG could regularly assess projects worldwide. By publically sharing their results to allow other organisations to participate in supporting them, they would also help reinforce the emerging core carbon principles championed by the IC-VCM.
\nThis contributions model places the emphasis back where it should be -- on behavioural and systemic reductions of our environment impacts -- rather than on being a "license to pollute", as carbon offsets have often been used as. It allows us to pragmatically identify high-impact areas where we have policies in place to reduce emissions, purchase carbon credits from those projects, and then account for their expenditure via our emissions reductions activities.
\nAn explicit non-goal is to use credits towards a big net-zero target of claiming carbon neutrality; they just reflect our collective contribution towards mitigating environmental damage that we've judged that we had to do.\nEllen Quigley succinctly summarises this: "a contribution is an acknowledgement of harm rather than its expiation".
\nJulia P.G. Jones also applies this approach to biodiversity credits in a recent piece:
\n\n\nUsing biodiversity credits to quantify contributions toward nature recovery, rather than to directly offset specific negative impacts, is a key way to reduce some of the risks we highlight. This is referred to in the forest carbon world as a "contribution" model. Instead of buyers of forest carbon credits claiming that the credits can offset emissions to achieve "net zero", they instead make a "contribution" to global climate mitigation through investments in forests.
\nWhile this may seem like a small change in terminology, it represents an important difference. If carbon credits cannot be subtracted from a company's emissions to produce a single net number, they cannot be used as a license to continue emitting. This also lessens the incentive for buyers to focus on quantity rather than quality in purchased credits. Some biodiversity credit operators are already promoting this approach [...]\n-- Hannah Wauchope et al, What is a unit of nature? Measurement challenges in the emerging biodiversity credit market, Royal Society 2024
\n
I couldn't agree more! Julia also highlights eloquently the urgency of the situation in her commentary in Nature in response to a recent Panorama program on the BBC:
\n\n\nHowever, dramatically more finance is urgently needed to stop the ongoing loss of forests and the vital services that they provide. REDD+ credits that cover the true cost of reducing deforestation in an effective and equitable way can help to provide that finance. If they are only used to offset residual emissions after substantial reductions, they could also contribute to the transition to net zero. The bottom line is that failure to conserve our carbon-rich forests and the life they support would be a dramatic and catastrophic failure for humanity.\n- Julia P.G. Jones, Scandal in the voluntary carbon market must not impede tropical forest conservation, Nature
\n
While we're still in early days of working through the details, Thomas Swinfield, Srinivasan Keshav, David Coomes and I have been framing a three-step checklist that organisations could apply towards the implementation of a carbon contributions model:
\nWith these principles, it should be possible for an organisation to contribute to carbon credit financing without adverse incentives. While there is some concern that this contributions mechanism has no built-in incentive to force organisations to contribute, I believe that it could bring a lot more people into the fold than voluntary offsetting has (which, as I noted earlier, has only mainly the best and the worst participants remaining now with the majority of people stepping back from it due to all the controversies). However, we still need to see if this is a strong enough incentive to get more organisations to participate voluntarily; this concern has been raised by several colleagues in response to this article and I will think on it further.
\nThe stakes cannot be higher right now for tropical rainforests, and we do not have the collective luxury of time to remain locked in the offset-or-not debate without an immediate alternative. The carbon contributions model could be just what we need to push forward! My hope is that this model makes it easier and safer for many organisations that have decided against offsetting to still contribute towards nature protection and restoration.
\nOther universities also grappling with this topic include Brown and UPenn, so I plan to circulate this article to them to gather wider opinions. The good folks at Native also published a piece about this shift from a compensation mindset to a contributions one.
\nAs noted at the beginning, I am updating this article regularly and would greatly welcome any other thoughts from you, the reader! I am grateful to David Coomes, Srinivasan Keshav, Thomas Swinfield, Eleanor Toye Scott, Robin Daniels, Emily Shuckburgh, Rachael Garrett, Isobel Cohen, Simon Zadek, Bhaskar Vira, Melissa Leach, Andrew Balmford, Michael Dales, Harriet Hunnable, Elliot Kinsey, Ellen Quigley, Jon Pierre, Julia P.G. Jones and many others for their thoughts. This article includes their input but is not endorsed by them and any mistakes are mine alone.
\nThomas Swinfield, Srinivasan Keshav and I decided it might be instructive to run a NotebookLM summary of some of our discussions, which you can find as in (AI-generated) podcast format below.
\n\nChangelog: 2nd Feb 2025 was original article. 5th Feb 2025 refined draft principles. 12th Feb 2025 added note about Native.eco article via Robin Daniels, note on incentives via Emily Shuckburgh. 20th Feb 2025 fixed typo in Ellen Quigley quote, via Srinivasan Keshav.
\nSrinivasan Keshav has an excellent video explainer series of the work 4C has been doing towards this.
\n ↩From the Wikipedia article to carbon credits and offsets.
\n ↩The Climeworks price seems to have gone up since 2022, and the subscription site now shows £1100/tonne.
\n ↩There's a nice article from Vice that explains the paper more accessibly.
\n ↩As an aside, I've been purchasing sustainable Gola rainforest chocolate from the RSPB. Srinivasan Keshav gave me some of their truffles for Christmas and they were consumed rapidly by my family.
\n ↩While Bluesky is taking off like a rocket, a number of us moving towards self sovereign digital infrastructure have been looking at how to use the Bluesky network for other uses than just short-form notes. This is possible because of my colleague Martin Kleppmann's hard work on the "AT Protocol" that underpins the Bluesky network. Martin recently gave us a deep-dive into the AT proto in the Cambridge security group, which made me look into other uses of it more closely. As background, you may wish to read his paper on the subject which explains the technical architecture extremely clearly.
\n ![\"%c\"](\"/images/atproto-paper-ss-1.webp\")
One of the key problems this solves is one I'm having with using my ActivityPub-based services at the moment. Each of these services (like my video or microblog sites) do not share a common authentication system, and so each account is different. Nick Ludlam and I are also thinking of renaming all of our services to go under the cleaner recoil.org domain rather than a subdomain, but this involves a fairly error-prone migration that lacks resilience to domain change changes since they are baked into the ActivityPub protocol messages. The AT Protocol underpinning Bluesky deals with all these by decoupling the underlying authentication and identity system, and the content that's flowing over the network.
This strikes a nice balance between pure self-hosting and longevity while bootstrapping the network; I chuckled, for instance, reading Q's post from 2012 about how a new social network called "app.net", which described itself as "your real-time feed, a home for meaningful conversation, where you control your data", but is now a long expired domain-squatted adfest. The AT Proto model seems more pragmatic in that it builds up a centralised bootstrap, but the underlying protocol itself admits innovation for other apps, and so permits evolution.
\nSo let's look at some of these alternative apps are already cropping up:
\nThen there are a bunch of "alternative clients" that do specific forms of media, such as photos or videos. This is less about using the underlying protocol than about building a new client, but it's still pretty neat that it's so accessible:
\nThis is just the tip of the iceberg for the open web, of course. Excitingly, there are experiments ongoing to run ActivityPub over AT Proto which describes how complementary these ecosystems are.\nAnd most excitingly from my personal perspective, is Patrick Ferris successfully posting from an up-and-coming OCaml ATProto implementation. I'm looking forward to hacking in this ecosystem in 2025!
\n(Thanks David Gageot for spotting typos!)
We've been happy users of Matrix for our group communications in the EEG. Today we've been bringing in more members of the wider group to using it instead of Slack. As part of that, I've set up a cool bot called Hookshot which allows Matrix to be connected to external services such as GitHub and Atom/RSS feeds. This is a test post to demonstrate to the members of the EEG how Matrix and Atom work!
\nThe basic idea behind Hookshot is to provide a bridging service to communications rooms hosted on Matrix, in such a way that it can exert administrative control over a room to intercept requests for services (such as adding an Atom feed).
\nThe setup for Hookshot can be a little involved as there are lots of encryption keys flying around. In a nutshell, I have a Docker container running it with a Yaml config of this nature:
\nbridge:\n domain: recoil.org\n url: http://synapse:8008\n port: 9993\n bindAddress: 0.0.0.0\npassFile:\n /data/hookshot.pem\nexperimentalEncryption:\n storagePath: /state\nexperimental_features:\n msc3202_device_masquerading: true\n msc3202_transaction_extensions: true\n msc2409_to_device_messages_enabled: true\nlogging:\n level: debug\n colorize: true\n json: false\n timestampFormat: HH:mm:ss:SSS\nThis gives me a healthy amount of debug logging, and uses my personal Matrix server at recoil.org as the "home" for the bot. Ryan Gibb set up our EEG Matrix server completely separately over in a VM in the Computer Lab, at eeg.cl.cam.ac.uk. It's pretty cool that Matrix allows for this sort of decentralised communucation pretty seamlessly!
After this worked and was tested, I now have an active bot user on the Matrix (in this case, it's llama:recoil.org. I then configured GitHub on Hookshot so that the bot can monitor GitHub via its API.
github:\n auth:\n id: 861482\n privateKeyFile: /data/github-key.pem\n webhook:\n secret: <secret>\n oauth:\n client_id: <client-id>\n client_secret: <client-secret>\n redirect_uri: https://recoil.org/hookshot/oauth/\n defaultOptions:\n showIssueRoomLink: false\n hotlinkIssues:\n prefix: "#"\nThis bit of Yaml takes some configuring via the GitHub OAuth API to get the client-id and secrets. Once that's done, the bot can then be instructed to monitor certain repositories just by issing some commands from within Matrix!
\n![\"%c\"](\"/images/hookshot-ss-1.webp\" "\\"The")
After this, the bot can be configured for a variety of other service. For instance it can monitor Atom feeds to keep track of what the whole group is writing. For this, it's as simple as:
\n!hookshot feed https://anil.recoil.org/news.xml (as an example for my feed)![\"%c\"](\"/images/hookshot-ss-2.webp\" "\\"It")
![\"%c\"](\"/images/hookshot-ss-3.webp\" "\\"But")
Hookshot supports a variety of other services to bridge to as well, including webhooks for arbitrary services. One of the most fun student projects I've supervised recently is "Decentralised Capability-based Code Collaboration using Matrix" in which Samuel Wedgwood built Git-patches-over-Matrix. If anyone wants to pick up on that and build a "real" version, perhaps we could use this for peer-to-peer coding! It might work really well with coding copilots, as they have a chat based interface anyway...
", "url": "https://anil.recoil.org/notes/enter-the-matrix-hookshot", "title": "Entering the Matrix with Hookshot", "summary": "Hookshot integrates Matrix with external services like GitHub and Atom feeds for enhanced group communications.", "date_published": "2025-02-07T00:00:00.000000Z", "date_modified": "2025-02-08T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "selfhosting", "matrix", "networking", "computerlab" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/bxnyj-v6f40", "content_html": "With the vast amount of data we have these days for our planetary computing processing, it's naturally tempting to use more hardware offload. The obvious choice, GPGPUs, are not a great fit for the problem due to the difficulty of unlocking high data parallelism for geospatial data. So it's back to an old technology I worked on twelve years ago in the form of FPGAs!
\nFPGAs are a very flexible way to execute boolean combinatorial logic, but are notoriously difficult to program. We have two possible angles to explore to address this. One is to design more declarative DSLs for data processing that compile to the FPGAs, such as Michael Dales work on Yirgacheffe or Omar Tanner's work on in-memory compressive computation. The other angle is to work on the low-level API to programming the FPGAs, to get away from Verilog and program in our favourite high-level language...OCaml! KC Sivaramakrishnan and I have started making a list of resources for programming FPGAs in OCaml for our own education.
\nHardCaml was originally a side project by Andy Ray. He gave a great presentation about it at ORConf 2015. Later on in the project's lifecycle, he moved it to being maintained by Jane Street, where is used in production and is open source. The first two resources to learn about HardCaml are to listen to the Signals and Threads episode with Andy, and then to read the 2023 paper:
\n\n\nUnlike high level synthesis (HLS), Hardcaml allows for low level control of the underlying hardware for maximum productivity, while abstracting away many of the tedious aspects of traditional hardware definition languages (HDLs) such as Verilog or VHDL. The richness of OCaml’s type system combined with Hardcaml’s fast circuit elaboration checks reduces the chance of user-introduced bugs and erroneous connections with features like custom type defining, type-safe parameterized modules and elaboration-time bit-width inference and validation.
\nHardcaml tooling emphasizes fast feedback through simulation, testing, and verification. It includes both a native OCaml cycle-accurate and an event-driven simulator. Unit tests can live in the source code and include digital ASCII waveforms representing the simulator’s output. Hardcaml also provides tools for SAT proving and formal verification. Hardcaml is industrially proven, and has been used at Jane Street internally for many large FPGA designs.
\n
Let's look at the source code repository next to see some more code.\nHardCaml is easily installable via opam, so there appears to be few barriers to getting the software up and running. For the development lifecycle, there are a few other packages to ease the interfacing with the FPGA hardware:
\nLet's look at some examples. There is a hardcaml_circuits repository with some interesting designs in HardCaml. Picking some at random:
\nI was curious to see what HardCaml's been used for recently. Most notably, it took home awards at the ZPrize in 2022, winning the multi-scalar multiplication track. So this thing is right up there with other HDLs in terms of producing high performing circuits!
\nThere are two good blog posts about each of the implementations:
\n![\"%c\"](\"/images/hardcaml-webterm-1.webp\" "\\"The")
More relevantly to my interested in geospatial processing, there is a JPEG decoder in HardCaml which looks rather exciting. It implements the JPEG baseline profile with arbitrary huffman tables for encoding, along with a more work-in-progress decoder. A GeoTIFF implementation would be a fun starter project to port to HardCaml!
\nMoving on from prizes, there is also a MIPS processor in HardCaml designed by a couple of students at Penn State. They've also written a series of great blog posts about their adventures in learning HardCaml as students.
\nSadiq Jaffer and I have also been discussing the possibility of using computational SSDs to accelerate vector databases, which would be a game-changer for the huge datasets we're throwing around at the moment.
\nI'm going to continue to explore this further, and will update this note with any more resources I found. Please do send me any ideas you might have! (Update 2025/02/07): Immediately after posting this, two interesting responses came up:
\n\n\nThe VGA controller is here and the hardcaml output works nicely with yosys and open lane tooling and verilator. So far it seems to work in simulation and on an FPGA (output recording video, see bottom of this on how it got recorded).
\nYet to find out whether it'll work in a physical chip (they say the tape out will be done in April). I particularly like the waveforms in source code for unit test (see the above VGA example).
\n
Thanks also to Andy Ray and Andrew W. Moore for feedback and corrections to this post.
Sadiq Jaffer sent along this piece in The Conversation last week about the remarkable number of academic papers that are now AI generated. The numbers of these papers are probably underestimated:
\n\n\nThese papers are absorbed into the worldwide library of research faster than they can be weeded out. About 119,000 scholarly journal articles and conference papers are published globally every week, or more than 6 million a year. Publishers estimate that, at most journals, about 2% of the papers submitted – but not necessarily published – are likely fake, although this number can be much higher at some publications.\n-- Frederik Joelving et al, The Conversation
\n
What caught my eye in this article is their development of the Problematic Paper Screener, which the good folks at Retraction Watch developed. It works with high precision to detect papers issued by grammar-based generators. They noted in another article that over 764,000 articles cited papers that could be unreliable, further illustrating the creeping unreliability. Sadiq Jaffer and I are planning to run this over our growing paper corpus, but I can't find the source code to their system, just the hosted version.
\nMeanwhile, datasets are also under similar threat of causing recursive model collapse. The Wordfreq team announced in September 2024 that they would discontinue updating their corpus because generative AI has polluted the data and information that used to be free has became expensive. Patrick Ferris also noted the related problem of dataset versioning becoming unreliable across science in "Uncertainty at scale: how CS hinders climate research", but for different reasons -- large datasets are inherently difficult to version and reproduce (it's quite hard to share a terabyte of data over the Internet easily, even in this day and age).
\n![\"%c\"](\"/images/conversation-fakeai-1.webp\" "\\"Wayne")
Another big development this week was the release of OpenAI's Deep Research feature, which goes off and really mines a literature corpus for information. I've grudgingly updated to their expensive Pro to try this out and will report my findings in a future post. The ability to generate papers has moved well beyond just the grammar generators that the Problem Paper Screener can filter out, so this arms race is unlikely to end well if we're pinning our hopes on detecting AI-generated papers. The current publish-or-perish model has already died; at least our Cambridge promotion process is more enlightened than "just" looking at paper counts!
There's a letter in Science today from a bunch of well known remote sensing researchers that make the unusual point that modern satellite resolution is getting too good to be accurate for forest carbon estimation.
\n\n\nMany new satellites can resolve fine features on the landscape, and even some individual trees outside forests, but this resolution (0.3-5m) is too high for mapping forest carbon. Forest carbon has a natural resolution constraint: the size of an individual tree. To create these maps, tree data from the ground are required because there is no direct measure of tree carbon nor any way to accurately divide trees into smaller components from space.\n[...]\nBecause most carbon in a forest is stored in large trees, map resolutions should at minimum exceed the crown diameter of a typical large tree, which ranges from about 10m for temperate forests to about 20m for tropical forests\n--- Laura Duncanson et al, Spatial resolution for forest carbon maps, Science
\n
The lead author Laura Duncanson is a remote sensing scientist at Maryland who works on the incredible GEDI instrument on the International Space Station. In her recent EEG seminar talk, she noted that their instrument is so sensitively calibrated that they can detect when astronauts on the space station are flushing the loo!
\n\nDavid Coomes further notes that we shouldn't think of either field data or GEDI footprints as sole ground truths, but rather factor in the combined uncertainties in both ground and remote sensing data. This 2018 Geosciences paper goes through the details of how this error propagation works in Borneo rainforests:
\n\n\nBy combining ALS imagery with data from 173 permanent forest plots spanning the lowland rainforests of Sabah on the island of Borneo, we develop a simple yet general model for estimating forest carbon stocks using ALS-derived canopy height and canopy cover as input metrics. An advanced feature of this new model is the propagation of uncertainty in both ALS- and ground-based data, allowing uncertainty in hectare-scale estimates of carbon stocks to be quantified robustly.
\n[...] Since the 1970s Borneo has lost more than 60% of its old-growth forests, the majority of which have been replaced by large-scale industrial palm oil plantations.
\nWith the view of halting the further deforestation of carbon-dense old-growth forests and generating the necessary knowledge to better manage its forests into the future, in 2016 the Sabah state government commissioned CAO to deliver a high-resolution ALS-based carbon map of the entire state. The regional carbon model we develop here underpins this initiative [...]\n-- Tommaso Jucker, David Coomes et al, Estimating aboveground carbon density and its uncertainty in Borneo’s structurally complex tropical forests using airborne laser scanning
\n
Michael Dales and Thomas Swinfield are just starting to refresh our PACT methodology spec, so this yet another timely warning to not race ahead with the latest satellite data without careful consideration of what it is we are actually measuring (in our case, forest carbon for ##carboncredits).
Srinivasan Keshav posted a link to this excellent deepdive by Prasad Raje of Udemy into the advances that\nDeepSeek R1 has made from a perspective of the core\ntechnology.
\n\n\n
\n- Multi-headed Latent Attention (MLA). In the famous Google "Attention is all you need" paper, the attention block is responsible for a lot of the magic of LLMs but is also compute heavy [...] Deepseek has innovated here with Multi-headed latent attention - which essentially reduces the size of matrix multiplication applied to generate the K,V vectors that are inputs into the attention block. Combined with KV Caching, this reduces the memory needs [...]
\n- Mixture of Experts (MoE). The key idea here is that instead of feeding each token through one massive FFN, break down the single FFN into a number of smaller FFNs and route each token through a subset of these FFNs. [...] each of these smaller FFNs will learn during training something specific about how to transform each token, hence becoming an "expert". Deepseek took MoE to this 670B parameter scale that no one had done before [...] and created 256 FFNs and routes each token through only 8 of these.
\n- Multi-token prediction (MTP): [...] you compute more than 1 token and send the aggregate error to back propagate. The intuition is that you get more changes made to the model weights in each training step, thus reducing the total training steps needed [...] Deepseek took this idea further, added innovations of their own (Sequential vs parallel MTP) and used this to reduce training time.\n -- Prasad Raje
\n
Now that I've switched to a new website, I'm working on open-sourcing its components. I've got a lot of small OCaml scripts that are all work-in-progress, and so not quite suitable to be published to the central opam-repository but I still need be able to run them conveniently on my own self-hosted infrastructure.
\nI mainly use a variety of macOS and Linux hosts[1] and I want a workflow as simple as "brew install avsm/ocaml/srcsetter" and have it install a working binary version of my CLI utility. In this case, it's srcsetter, a simple tool I knocked up to generate the responsive images on this website. Luckily, Homebrew has made this really easy for us! They have a BrewTestBot that integrates with GitHub Actions to automate the compilation of binary packages for us, all from a convenient PR-like workflow.
First, we need to set up a GitHub Homebrew "tap" repository. Mine is avsm/homebrew-ocaml which allows for the tap to be referred to as avsm/ocaml (Homebrew special-cases these to expand to the full GitHub repository). We then add in a couple of GitHub Actions to activate the testbot:
pr-pull label to a successful PR and have it be merged automatically by the bot.Secondly, we need to create a Homebrew package for the opam package. For this, I just added a very simple script to the srcsetter repository called .opambuild.sh which builds a local binary using a temporary opam installation. In the future, we should be able to use dune package management to remove the need for this script, but I'm blocked on some teething issues there in the short-term.
\nexport OPAMROOT=`pwd`/_opamroot\nexport OPAMYES=1\nexport OPAMCONFIRMLEVEL=unsafe-yes\nopam init -ny --disable-sandboxing\nopam switch create . \nopam exec -- dune build --profile=release\nOnce this is present in the repository we're building, I just need to open a pull request with the Homebrew formula for my CLI tool.
\nclass Srcsetter < Formula\n desc "Webp image generator for responsive HTML sites"\n homepage "https://github.com/avsm/srcsetter/"\n url "https://github.com/avsm/srcsetter.git", branch: "main"\n version "0.0.1"\n license "ISC"\n\n depends_on "gpatch"\n depends_on "opam"\n\n def install\n system "bash", "./.opambuild.sh"\n bin.install "_opam/bin/srcsetter"\n end\nend\nThe formula is fairly self-explanatory: I just point Homebrew at the source repository, give it some descriptive metadata, and tell it to invoke the binary build script and make the sole resulting binary available as the contents of the package. At this point, the BrewBot will run against the PR and report any build failures on both macOS and Ubuntu. Most of these were swiftly fixed by running brew style (as instructed in the build failures) to take of fairly minor issues.
![\"%c\"](\"/images/gh-brewbot-screen.webp\")
When the PR went green, all I then had to do was to add the pr-pull label, and the bot takes care of uploading the binary artefacts to my homebrew tap repo and merging the PR. It also takes care of adding checksums to the merged Formula, so what actually got merged is:
class Srcsetter < Formula\n desc "Webp image generator for responsive HTML sites"\n homepage "https://github.com/avsm/srcsetter/"\n url "https://github.com/avsm/srcsetter.git", branch: "main"\n version "0.0.1"\n license "ISC"\n\n bottle do\n root_url "https://github.com/avsm/homebrew-ocaml/releases/download/srcsetter-0.0.1"\n sha256 cellar: :any_skip_relocation, arm64_sequoia: "b3e1289965d8bcf086db06b18e6c2865f9949a9e1202b8fafa640f3e363b6bd4"\n sha256 cellar: :any_skip_relocation, ventura: "9b61e8e4be5f777e3ef98672f275909a80c3cc3f82d6886ca1a90b66ea7bb9f8"\n sha256 cellar: :any_skip_relocation, x86_64_linux: "d8279f11f30edf865368a3c6f63d811d31c1a9ca019ef86e93afeb6624232850"\n end\n\n depends_on "gpatch"\n depends_on "opam"\n\n def install\n system "bash", "./.opambuild.sh"\n bin.install "_opam/bin/srcsetter"\n end\nend\nThe end result is that brew install avsm/ocaml/srcsetter now works, without me having to cut a release of the tool more centrally. I'd love to incorporate some aspects of this workflow into the OCaml opam-repository, as users are currently responsible for the checksumming generation themselves via dune-release or opam-publish. It's an interesting twist to automate this part of the process and let the humans focus on the core package metadata instead. Thanks for all the help, Brewbot!
My colleagues Thomas Swinfield and Eleanor Toye Scott lead the publication of a comprehensive\nreport of the steps the voluntary carbon market needs to take\nto restore its scientific credibility, with input from many of us in 4C and beyond.
\n\n\n\n
\n- establishing common standards for carbon quantification and accounting, to cover additionality, leakage and permanence.
\n- avoiding perverse incentives and align the motivations of all stakeholders with high-integrity outcomes. [...]
\n- issuing all carbon credits based on trusted primary observations.
\n- making all the data needed to reproduce carbon calculations available in standard file formats.
\n- [...] reporting social and biodiversity dimensions of projects separately from carbon calculations.
\n- integrating DMRV methods into carbon and biodiversity accounting standards to reduce the financial and administrative burdens on nature-based projects and the local communities participating in or affected by them.
\n
This paper represents three years of hard work from the team on trying to blend remote sensing with carbon quantification. For more reading on the topic, you may also wish to browse the full 4C publication list for the firehose of activity from the centre.
", "url": "https://anil.recoil.org/notes/credible-credit-principles", "external_url": "https://www.cambridge.org/engage/coe/article-details/679385946dde43c9082f7009", "title": "Position paper on scientifically credible carbon credits", "summary": "Experts outline steps to restore scientific credibility in the voluntary carbon market.", "date_published": "2025-01-30T00:00:00.000000Z", "date_modified": "2025-01-30T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "carboncredits", "sensing", "economics", "conservation", "nbs" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/0r62w-c8g63", "content_html": "This website has been through quite a few iterations over the years. The first version in 1998 was written in Perl and hosted on OpenBSD; the second was rewritten in 2000 when I got commit access to PHP; the third rewrite became a hybrid OCaml/PHP/Perl special in 2004 in Blosxom; then the forth rewrite around 2013 got turned into a unikernel in MirageOS; then the fifth in 2019 then transitioned to an OCaml static site generator hosted on a prerelease multicore OCaml webserver. So the sixth generation now needs something to continue the grand Rube Goldberg tradition of helping me learn the latest and greatest in systems technology.
\nAnd so here it is! The site is now written in a bleeding-edge unreleased variant of OCaml with extensions based around Rust-like type system features activated, including rather exciting data-race freedom work that just won a best paper award at POPL 2025. It's normally difficult to work on continuously moving compilers, but Diana Kalinichenko did a tremendous amount of work into making it usable with opam out of the box, and this post documents the journey to getting this website live.
\nFirstly, we did some groundwork a few months ago by adding support into the opam-repository for bootstrap versions of dune, menhir and ocamlfind. These are used to build the Jane Street version of the OCaml compiler, which is published as an opam-repository#with-extensions.
\nThe extensions there are straightforward for those familiar with opam. On a clean system you can run:
\n$ opam init\n$ opam switch create 5.2.0+flambda2 \\\n --repos with-extensions=git+https://github.com/janestreet/opam-repository.git#with-extensions,default\n$ eval $(opam env)\nThis creates a new opam switch known as 5.2.0+flambda2, and we can then verify it's running the variant compiler.
$ ocaml\nOCaml version 5.2.0+jst\nEnter #help;; for help.\n# let () =\n let local_message : string @@ local = "Hello, World" in\n print_endline local_message\n ;;\nError: This value escapes its region.\nThat last bit is the new region magic which I'm keen to start experimenting with for this website! But before that, we need to get the rest of the ecosystem packages needed for the website working under this compiler.
\nI decided to build the new site based on a content manager I've been designing\n(and scrapping) for a few years, codenamed Bushel. The basic idea behind\nBushel is to extend Markdown sufficiently with rich contextual data (such as\ncontacts, papers, projects, ideas and so on), and allow for cross-referencing\nto other sites that also follow the Bushel protocol. I'll talk about that in\nmore detail in future posts, but for now that means that I need a more dynamic\nwebsite than the static one I used for the past few years.
\nSince the Jane Street compiler doesn't yet support the effect system from OCaml 5, I couldn't use my own Eio-based webserver. So after some discussion with Michael Dales who is also porting his site to OCaml, I took the opportunity to learn the the excellent Dream server, which is based on Lwt. I also used Daniel Bunzli's cmarkit library for Markdown parsing, and my own Jekyll_format and yaml libraries.
\nAmazingly, all of these libraries worked out of the box on the Jane Street\ncompiler, except for one snag: the parsetree internals have changed in their\nbranch. This means that PPX\nextensions will not work out-of-the-box. Thankfully, there is an abstraction\nlibrary called ppxlib which\nhas been ported to the variant compiler, and the differences in the parse tree\nwere easy to fix up (thanks Nathan Reb and Patrick Ferris for your recent ppxlib work!)
\nAfter forking and fixing just two libraries that were using ppx (and not part of the\nJane Street core libraries that were already ported), all I had to do was to pin them\nand add them to my development environment.
\nopam pin add ppxlib 0.33.0+jst\nopam pin add dream-httpaf 1.0.0~alpha4\nopam pin add hpack https://github.com/avsm/ocaml-h2.git#js-extensions-fixes\nopam pin add lwt_ppx https://github.com/avsm/lwt.git#js-extensions-fixes\nAnd this then installs the overridden version of packages that I needed,\nwith the pins making sure that the right dependencies were also present.\nAfter that, it was plain sailing! I've now compiled up a native code version\nof my webserver code, deployed it into a Docker container, and\ndeployed it on Linux.
\nIn the future, I hope to use dune package management to ease the deployment\nof the site, but it didn't work in its current preview form due to a problem\nwith depopts. Just teething\nproblems with a preview, so I'll post more about that when I get it working!\nI also have a half-finished port of the variant compiler to OpenBSD, so that\nI can shift my website back to its familiar home rather than running on Linux.
\nI haven't yet actually taken advantage of any of the new extensions in the\nJane Street variant, since I wantd to get this site up and running first.\nI'll tidy up the code, open source it in the coming weeks, and then we can\ndive into some region extensions and see how far I get!
", "url": "https://anil.recoil.org/notes/bushel-lives", "title": "Arise Bushel, my sixth generation oxidised website", "summary": "Learn about my sixth generation oxidised website built with a bleeding-edge OCaml variant.", "date_published": "2025-01-29T00:00:00.000000Z", "date_modified": "2025-01-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "recoil", "selfhosting", "bushel" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-ce-llm-2", "content_html": "We have just updated our preprint on using LLMs for evidence decision support with more evaluation results and corrections from peer review.
\n\n\nOur findings suggest that, with careful domain-specific design, LLMs could potentially be powerful tools for enabling expert-level use of evidence syntheses and databases. However, general LLMs used "out-of-the-box" are likely to perform poorly and misinform decision-makers. By establishing that LLMs exhibit comparable performance with human synthesis experts on providing restricted responses to queries of evidence syntheses and databases, future work can build on our approach to quantify LLM performance in providing open-ended responses.
\n
See also the fantastic EEG seminar talk that the student group who worked on this over the summer gave towards the end of last year.
After some years of hard work, our Mapping LIFE on Earth biodiversity metric was published today in a special issue of the Royal Society Philosophical Transactions B! The idea behind LIFE is that although human-driven habitat loss is known to be the greatest cause of the biodiversity crisis, we do not yet have robust spatially explicit metrics that quantify the relative impacts of human actions on species extinctions. And that's what LIFE provides: a way to compare the relative impacts of some landuse anywhere in the world, in a manner that is globally applicable.
\nThere are lots of limitations in this first version: it does not yet cover plants (but we're working on it), and some other taxa such as freshwater (working on that too!), but it's a very solid start. We've uploaded the LIFE datasets to Zenodo, and the full pipeline source code is available too.
\nThe release of LIFE is also discussed further in an article on Mongabay.
\n\n\nNoodling around with the team’s maps can reveal noticeable variations in the\nrisk of extinction to various species in different regions, with implications\nfor sourcing the goods we use every day. For example, clearing a hectare of\nforest in the Congo Basin will nudge far more species toward extinction than\ndoing the same in northern Europe — the former is much more biodiverse than\nthe latter. Companies could also use these maps to boost their sustainability\nby sourcing goods from places where extinctions are less likely. Or,\nindividual consumers might use them to understand how their consumption\nchoices affect species’ habitats.\n -- John Cannon
\n
The Stockholm Environment Institute also covered it:
\n\n\nIn a major step forward, the authors of this paper have expanded the potential applications of the "persistence score" approach to create LIFE: they have introduced high-performance computing and brought in data for over 30 000 vertebrate species. The tool can now create global maps of extinction risk probability for 30 875 species of terrestrial vertebrates at 1 arc-minute resolution (3.4 km2 at the equator).
\nThe technical leap means that, for the first time, map users can access powerful quantitative data regarding the expected number of extinctions (whether that is an increase or decrease) caused by the conversion of natural vegetation to agriculture, or restoring farmland to natural habitat.
\n
If you'd like to see a talk about LIFE, I discussed it in my recent LambdaDays keynote:\n
Here are the various repos used to create the interactive teaching environment\nwe use for 1A Foundations of Computer Science in Cambridge. It may be useful to\nother professors who are using OCaml in their courses.
\nPing me if you want to know more, and other people who have worked\non this with me are Jeremy Yallop, David Allsopp and Jon Ludlam, with Jon\nbeing the currently active additional lecturer on the course as of 2024/2025.
", "url": "https://anil.recoil.org/notes/focs", "title": "Foundations of Computer Science", "summary": "Resources for teaching Foundations of Computer Science with OCaml and Jupyter notebooks.", "date_published": "2018-09-02T00:00:00.000000Z", "date_modified": "2025-01-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "teaching", "cambridge", "computerlab", "pembroke", "compsci", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-ai-conhorizon-1", "content_html": "Back in July 2024, a large group of conservation and computer scientists got together in the CCI to prioritise the storm of AI-related projects that have been kicking off around the world. Our key goal was to harness AI to accelerate the positive impact of conservation efforts, while minimising harm caused through either the direct or indirect use of AI technologies.
\nThe first horizon scan resulting from this has just been published in Trends in Ecology and Evolution. If you're looking for a gentle introduction to some of the terms in AI from a non-experts perspective, the first section does a good job of defining a glossary as well. The panel identified 21 key ideas ranging from species recognition for uncovering 'dark diversity' to multimodal models for improving biodiversity loss predictions, monitoring wildlife trade, and addressing human-wildlife conflict. Importantly, we also considered the potential negative impacts of AI adoption, including AI colonialism and the loss of essential conservation skills. The scan provides guidance on how the conservation field can adapt to harness AI's benefits while mitigating its risks.
\n![\"%c\"](\"/images/2024-ai-horizon-scan-group.webp\" "\\"The")
Customers of online services may want to take carbon emissions into account\nwhen deciding which service to use, but it's currently difficult to do so due\nto the lack of reliable emissions data that is comparable across online\nservices. There's a lot of muddled data out there, and calculating accurate\ncarbon emissions across a computing pipeline involves a number of stakeholders,\nnone of whom are incentivised to accurately report their emissions for\ncompetitive reasons!
\nIn this LOCO paper, Jessica Man lead our\nexploration of mechanisms to support verifiable and privacy-preserving\nemissions reporting across a chain of energy suppliers, cloud data centres,\nvirtual machine hosting services providers and cloud services providers. The\nidea is that all of this can ultimately be exposed to APIs that can be consumed\nby client devices in order to let consumers make direct choices about their\ndecisions based on relative environmental impacts.
All the work we've been doing on biodiversity (such as LIFE) comes at\na fairly large computation and storage cost due to the amount of data that we\nchurn through. This gets worse when you consider the exploratory nature of\nscience -- we sometimes just need to mess around with the large dataset to test\nhypotheses which are often shown to be wrong. So then, when the\nLOCO conference came around, we wrote\nup our thoughts on what a frugal Linux userspace might look like.
\nThe key insight is that the Linux kernel already exposes a number of namespace\nmechanisms (that we use in Docker, for example), and so we explore a new OS\narchitecture which defaults to deterministic, reusable computation with the\ncareful recording of side-effects. This in turn allows Linux to guide complex\ncomputations towards previously acquired intermediate results, but still\nallowing for recomputation when required by the user. We're putting this\ntogether into a new shell known as "Shark", and this first abstract describes\nour early results.
", "url": "https://anil.recoil.org/notes/2024-loco-shark-1", "title": "Towards a frugal userspace for Linux", "summary": "LOCO paper presenting Shark shell for deterministic, reusable Linux computation using kernel namespaces to reduce computational costs.", "date_published": "2024-12-01T00:00:00.000000Z", "date_modified": "2024-12-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "shark", "life", "carbon", "linux", "docker", "loco" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2024-loco-shark.pdf", "mime_type": "application/pdf", "title": "Lineage first computing: towards a frugal userspace for Linux" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-loco-carbonres-1", "content_html": "Ryan Gibb and I have been thinking about how the current Internet architecture fails to treat the carbon emissions\nassociated with networked services as a first-class metric. So when the LOCO conference came up, we tried extending the DNS with load balancing techniques to consider the carbon cost of scheduling decisions. A next step was then to build a custom DNS server written in OCaml to actively wake machines running networked services as a side effect of the name\nresolution.
\nExtending DNS means that we maintain compatibility with existing Internet\ninfrastructure, unlocking the ability for existing applications to be\ncarbon-aware. This is very much a spiritual follow on to the\nSignposts project that I worked on back in 2013, and\nhave always wanted to return to!
", "url": "https://anil.recoil.org/notes/2024-loco-carbonres-1", "title": "Prototyping carbon-aware domain name resolution", "summary": "Extending DNS with carbon-aware load balancing and OCaml-based DNS server for sustainable Internet infrastructure presented at LOCO.", "date_published": "2024-12-01T00:00:00.000000Z", "date_modified": "2024-12-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "selfhosting", "dns", "distributed", "carbon", "signpost", "loco" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2024-loco-carbonres.pdf", "mime_type": "application/pdf", "title": "Carbon-aware Name Resolution" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-loco-terracorder-1", "content_html": "Josh Millar and I have been having great fun designing embedded systems for\ncooperative biodiversity monitoring. Josh presented our work over at LOCO\n2024 with an abstract on the\nTerracorder project. Read more if you enjoy a combination of machine learning\nand ESP32 hacking.
", "url": "https://anil.recoil.org/notes/2024-loco-terracorder-1", "title": "Cooperative Sensor Networks for Long-Term Biodiversity Monitoring", "summary": "LOCO 2024 presentation on Terracorder project combining machine learning and ESP32 for cooperative biodiversity monitoring networks.", "date_published": "2024-12-01T00:00:00.000000Z", "date_modified": "2024-12-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "terracorder", "biodiversity", "sensing", "embedded", "ai", "qlearning", "loco" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2024-loco-terracorder.pdf", "mime_type": "application/pdf", "title": "Cooperative Sensor Networks for Long-Term Biodiversity Monitoring" } ], "_references": [] }, { "id": "https://doi.org/10.59350/0znpc-fw825", "content_html": "I got invited to join the Royal Society and DeepMind to a summit on\nhow AI is revolutionising scientific discovery and trotted along with\nJon Crowcroft. This event is hot on the heels of the\nexcellent RS report on Science in the Age of AI\nand, of course, the Nobel prize for Demis Hassabis which was the first ever\nfor my department!\nThe event was held at the BAFTA today, and what follows are my quick livenotes\nas there was just so much to absorb. The RS and Deepmind will have the\nfull sessions online sometime soon, so I'll update this with those more polished\noutputs when they're out! Update: Proper notes now available from Google and the Royal Society.
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-4.webp\" "\\"Hannah")
The summit was a day-long exploration of how artificial intelligence is\ntransforming science and society, and the overall theme (including four Nobel\nlaureates!) was on how we are in a golden age of scientific discovery,\nespecially in the biological sciences. The emcee for the event was Hannah Fry,\nwho simply dazzled with her rapid summarisation of complex discussions\ninterspersed with very dry humour about the proceedings!
\nThe consistent message was how ongoing synergy between science, technology, and\nsociety is essential to setting the stage for an exploration of transformative\ndiscoveries powered by AI that would benefit everyone in society. Missing that\nsynergy would lead to unequal benefit or dangerous crossings of boundaries.
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-8.webp\" "\\"Busy")
The first session had James Manyika interviewing Jennifer Doudna, Nobel Laureate and co-inventor of CRISPR, shared how gene editing has moved from science fiction to an essential tool for societal improvement. Some key points:
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-1.webp\" "\\"James")
The panel next was Eric Topol chairing a discussion with Pushmeet Kohli, Fiona H. Marshall and Alison Noble. The theme was on how huge number of foundation\nmodels are coming out for LLLMs (large language life models) at a dizzying\npace.
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-2.webp\" "\\"Eric,")
Pushmeet Kohli first explained how deciphering the genome accelerates\nbiological discoveries by orders of magnitude. AI tools like AlphaFold (which\njust got the Nobel prize) exemplify breakthroughs that transform biology into a\npredictive science from a wetlab driven discipline. On other fronts,\nDeepMind's GraphCast model enables near-term weather forecasting in minutes,\nwhich previously required days of supercomputer time to do an equivalent\nforecast (and now\nNeuralGCM is doing even\nbetter with mechanistic models combined with data). Pushmeet then noted how\nGNNs for materials science identified over 400k (or 200k? couldnt catch it) new\nstable inorganic crystals, with potential applications like high-temperature\nsuperconductors which were just scifi before.
\nThen Fiona H. Marshall from Novartis emphasized how AI identifies new drug\ntargets using genomic and population-level data. In drug development,\npredictive safety is absolutely crucial. Pharmaceutical companies have decades’\nworth of histological data, such as rodent testing, stored on hundreds of\nthousands of glass slides that are now being digitized. Once this data is\nprocessed, we can use it to make various predictions. Sharing this data across\nthe pharmaceutical industry would benefit everyone. One of their partner\ncompanies is developing scanning algorithms, and once these are operational\nthey will be made open, the entire industry will gain from the resulting\ndataset. Generative chemistry (like AlphaFold) now designs drugs faster while\npredictive toxicology ensures safer medicines.\nInterestingly, the data scientists are in the prime author\nposition as they are dictating the experimental procedures vs the wetlab\nscientists a few decades ago. This change in incentives drives change within a\nlarge org towards more data driven methods.
\nAnother valuable source of data is population-level information, such as Our\nFuture Health (a UK-based NHS initiative).\nProper management of nomenclature will ensure that this project generates a\nmassive, usable dataset vs what we have anywhere else. Eric noted that they\nrely heavily on the UK Biobank, which, with its 15,000 participants, is one of\nthe largest in the world and the Our Future Health program is a huge leap\nforward with 5m participants. The NIH in the United States is hesitant to\nengage in public-private partnerships, and so the UK is leading the way in this\ndomain (Anil notes: with privacy concerns about the data sharing).
\nFiona also noted that drug discovery is also transforming clinical trials, not\njust the discovery process itself. Typically, it takes around 10 years for a\ncandidate molecule to move to the clinical trial phase. One major bottleneck is\npatient enrollment. By leveraging vast demographic databases, which include\ninformation on global populations, their diseases, medications, and hospital\naffiliations, we can drastically improve recruitment efficiency. For example,\na clinical trial protocol targeting "women aged 30-50 who are not taking drug X\nor estrogen modifiers" can utilize these databases to identify and enroll\npatients quickly. This approach can reduce recruitment time from three years to\njust six months, significantly accelerating the process of getting drugs to\nmarket.
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-12.webp\" "\\"Sneaking")
Alison Noble discussed how deep learning has revolutionized ultrasound imaging.\nAI now guides users on probe placement, reducing training requirements for\nmedical professionals. However, we're now moving so fast that we need to be\ncareful; even the notion of what a scientist is is changing. In the RS report\non Science in the Age of AI a number of scientists around the UK were\nconsulted and this concern of reproducibility and data access came up loud and\nclear. When we publish results, we need to make sure theyh are sound and that\npeer review is possible. Openness is a deliberate choice however and not always\nappropriate when sensitive data is involved (e.g. healthcare) but requiring a\nrigor in evaluation is essential for good science. Scientists need to rethink\nin the age of AI how we present our work, and how we train new scientists in\nthis environment. So we have some wonderful early examples like AlphaFold, but\nwe need to understand the societal/incentive impacts on our new generation of\nscientists.
\nEric also noted that one of the greatest challenges in genomics is\nunderstanding variance, and\nAlphaMissense has\ntackled this head-on. However, there is a significant data shortage. Without\nHelen Birman and the creation of the protein data\nbank, AlphaFold\nwouldn’t have been possible. This raises the critical question: where do we\nsource the necessary inputs? Pushmeet responded that intelligence doesn’t\nemerge in isolation; it relies on experiential datasets. Models can derive this\nexperience from real-world input or interactions within simulations.\nHigh-fidelity simulations, in particular, provide models with valuable\nintuition about future outcomes. Experimental data is also critical, as it\nintegrates with simulations to complete the picture. When dealing with\nunlabeled data, prediction labels generated by the model itself can be used for\nfurther training. However, it's essential to discard incorrect labels to ensure\naccuracy, which makes this technique effective for bridging data gaps.\nCrucially, a pure and uncontaminated test set is vital to ensure the\nreliability of the system. For example, AlphaMissense was trained in an\nunsupervised manner and successfully identified cancer mutations.
\nThe discussion was quite wide ranging, but overall the two key challenges were:
\nAs AI transforms science, public-private partnerships and interdisciplinary\ncollaboration are essential (like the Our Future Health) program. Transparency\nand openness are deliberate choices in science, but regulation must keep up\nwith the pace of innovation. Alison Noble also noted there is a culture change\ngoing on for these public/private partnerships within academia. While\ncompetition drives innovcation, we need to make sure that the academic reward\nsystem keeps up; if there are 50 people on a paper then how is this attractive\nfor young scientists to enter a field and make their own name?
\nSiddhartha Mukherjee, a cancer physician and Pulitzer Prize winner (and\npersonally speaking, author of one of my favourite medical\nbooks ever), began\nthe discussion with a warning about potential breaches of data privacy and the\ndangers they pose. He emphasized the risk of AI being weaponized for\nmisinformation, calling it a frontier challenge. These issues highlight the\nurgent need to develop mitigation strategies while continuing to advance the\ncapabilities of AI in their respective fields.\nSiddhartha emphasized that data is the critical missing link in advancing AI.\nIssues of access, quality, integration, equity, and privacy must be addressed.\nThe complexity of data ownership in AI raises ethical and commercial concerns,\nas data aggregators often benefit disproportionately.
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-9.webp\" "\\"Siddhartha")
Janet Thornton, from the European Molecular\nBiology Lab, shared her perspective on protein structures. She highlighted how\nAI has transformed the field—from modeling just 20 protein structures in the\nearly days to now over 214 million. Structural biologists worldwide are using\nthis data to validate their findings, explore ligand binding, and venture into\nuncharted territories of protein functionality. Anna delved into her work as a\ncell biologist studying membrane proteins and their organization within the\nbody. She shared a case from Cyprus, where mysterious kidney disease affected\ncertain families for decades. AI-driven image recognition was used to identify\na genetic mutation, leading to a therapeutic solution. The issue stemmed from a\nmisshapen protein caused by a nodal molecule that traps faulty proteins,\nultimately causing cell death. This discovery is now being applied to other\nconditions, such as Alzheimer’s and blindness, offering hope for broader\ntherapeutic applications.
\nJanet also spoke about her time as the director of the European\nBioinformatics Institute, which manages data repositories like the Worldwide\nProtein Data Bank. She described the cultural shift required to encourage data\nsharing, noting it took 20 years for crystallographers to agree to mandatory\ndata deposition before publication. She stressed that medical data,\nparticularly in clinical contexts, must undergo a similar transformation.\nPublicly funded data must eventually reach the commons, especially when such\nsharing has the potential to save lives. The UK’s NHS, with its secure data\nenvironments, provides a strong model for this approach. However, the health\nsector needs to move faster than the crystallography community did, requiring\nbuy-in from both patients and medical professionals, as emphasized in Kathy\nSudlow’s recent report on the UK’s health data landscape.
\nAnna Greka, a pathologist and head of a new institute focusing on women’s\ncancer at the Broad Institute, discussed her work on building AI tools to identify and facilitate the\ndevelopment of disease mechanisms. Anna Greka added that millions of human\ngenomes have been sequenced and aggregated into databases usable by scientists\nworldwide. If pathology labs globally pooled their data, AI training models\nwould benefit immensely. She suggested anonymizing the data while preserving\nmetadata and federating results across countries to protect privacy and enhance\nglobal collaboration.
\nAnne Vincent-Salomon, head of diagnostic and theranostic medicine at the\nInstitute Curie, stressed the importance of multidisciplinary science and\neffective communication. She emphasized the need to educate the public,\nreducing fear and fostering understanding of scientific advancements.
\nAnna concluded by underscoring the importance of understanding the "unit of\nlife" to progress in biology. She argued for the creation of a high-quality\nperturbation dataset for cells, akin to the Protein Data Bank’s role in\nAlphaFold. Skeptical of synthetic data, she emphasized the need for\nexperimentally derived data as a foundation for future breakthroughs. She\ncalled this effort a moonshot for the next five years -— a grand challenge to\ndeepen our understanding of life itself! (Anil: see also this great TED talk from Anna last year)
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-5.webp\" "\\"Thomas")
The (epic) next speaker was Thomas L. Friedman, who explored the the interplay of three critical "scaling laws" in the modern era:
\nBetween 2017, with the introduction of transformer algorithms, and 2020, when\nadvancements in microchips and GPUs took off, artificial intelligence has\nexpanded dramatically. This period reflects a "scaling law" in action.\nPolymathic AI—AI capable of addressing a broad range of problems—now seems\nwithin reach. In just three years, AI-driven science has evolved from science\nfiction to reality and become accessible to many (albeit often with some\nlimitations on free access). To address the challenges AI presents, we need\nhigher-dimensional thinking for higher-dimensional problems.
\nAt the same time, we're seeing a scaling law in carbon emissions. Levels of CO₂\nand methane in the atmosphere, including methane from livestock, are causing\ndestructive climate change. The seven warmest years on record occurred between\n2015 and 2021, resulting in what’s called "global weirding"—where hot regions\ngrow hotter, wet regions grow wetter, and the effects become increasingly\ndestructive.
\nIn parallel with these scaling points in carbon and silicon (AI hardware),\nwe’re facing a scaling point in disorder—the erosion of civic structures.\nGovernments worldwide have over-promised on the benefits of industrial\ndemocracies, such as healthcare, retirement plans, and infrastructure, yet are\nstruggling to deliver. Societies are aging, educational attainment has\nstagnated, and productivity growth has slowed.
\nWe're also witnessing growing national security challenges and the dissolution\nof the great power balance that defined the post-Cold War era. Electric\ntransportation, healthcare, and employment systems are under strain, leading to\nincreased migration. Today, there are 56 active conflicts globally—the highest\nnumber since World War II—and more displaced people than at any point in\nhistory.
\nWe need a game-changer.
\nTo solve these interconnected crises, we must adopt higher-dimensional\napproaches that blend solutions across disciplines and scales. The future\nstability of our planet—and the well-being of the next generation—depends on\npresenting holistic, interconnected solutions. Friedman calls this the "Polycene" era.
\nNever before has politics needed science more. Science must enable leaps\nforward in education and provide buffers against disruptive climate change.\nSimilarly, politics must create the frameworks and systems to allow science to\nthrive and deliver meaningful solutions.
\nIn That Used to Be Us,\nFriedman argued that "average is over"; the rapid acceleration of technology\nmeans the American Dream -- once achievable for many -- is no longer guaranteed.
\nHowever, technology can flatten access to resources. For instance, an Indian\nfarmer can now access advanced insights about crop planting, watering\nschedules, and fertilizers directly on a smartphone. For the first time, those\nwithout access to "average" resources now have access to them through AI.\nThanks to AI, "average" as a benchmark is over—and that gives Friedman optimism.
\nHowever, there’s a caveat: science and politics must work together. The\nalignment problem between these fields is real and will become more pressing as\nwe approach AGI. As a species, we’ve become more godlike than ever before,\ncreating systems and intelligence that resemble a larger, more powerful brain.\nHow we use this power will determine our future.
\nThe speakers here were Paul Hofheinz, Asmeret Asefaw Berhe, Fabian J. Theis and Bosun Tijani.
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-6.webp\" "\\"Paul")
Berhe began by noting that we are at an inflection point -- how do we avoid\nrepeating mistakes from the past, ensuring we don’t leave segments of human\nsociety behind or widen the digital divide further? In previous programs such\nas exascale computing, they insisted as part of the program that they must\nhave explicit sustainability goals. While these goals may have seemed\nunrealistic initially and were criticised, in retrospect they have shown they\ncan be achieved. An example of the next thing theyre working on is the\nHigh-Performance Data Facility, recognizing that the Office of Science produces\nmore scientific data than any other entity in the world (e.g particle physics,\ngenomic labs). We need to rethink how we handle such huge amounts of data,\naddressing concerns around privacy and integrity.
\nMinister Tijani then talked about how in Nigeria, there is a direct correlation\nbetween science and economic growth, yet in the Global South, we often fail to\napply science effectively to societal issues. The answers we think we have\noften got to shift solutions with context; for instance, policies from the UK don’t transplant\ncleanly to Nigeria where the poplulation is growing hugely faster.
\nKey points included:
\nTo address these challenges, the Nigerian government did a few things. Over\nthe past 30 years, publications and a name library were correlated to identify\nAI-focused Nigerian scientists. This effort brought 100 of them together to\ndevelop a national AI strategy for Nigeria. An AI Trust was created with a\ncommunity of trustees to support this strategy. And then a Talent Attraction\nProgram was launched, supported by Google and the government, alongside local\nprivate investment. This is one of the largest talent accelerators globally.\nNigeria aims to become a net exporter of talent, much like India’s success in\nthis area.
\nFabien then talked about how many scientists are driven by natural curiosity,\nand it's vital to nurture that environment. The "holy trinity" of AI consists\nof data compute and algorithms need to be completed together. Ten years ago,\ncomputer vision flourished due to ImageNet’s availability and now we’re\nentering a new era with foundation models for cell biology. These models\nrequire algorithmic innovations to merge datasets and techniques like\nmultiscale modeling mixed with self-supervised learning to succeed.
\nWe're at a tipping point where we can build generalizable models that can be\nadapted for specific tasks around the world (a reference to the Nigerian\nusecases earlier)
\nSome key points discussed were:
\nBosun Tijani also noted that the Global South isnt a charity case when it comes\nto AI. By the end of this century, Nigeria’s population will be half a billion,\nmaking it crucial in a highly connected world. There's a strong business case\nfor not missing out. Nigeria is investing $2 billion to deploy dark fiber\ninfrastructure nationwide. This connectivity will empower people to contribute\nmeaningfully to the global digital economy. Governments in the Global South\nmust expand their capacity to think strategically about AI and its potential.\nUnlike academic institutions, which often drive innovation, governments in\nthese regions need to strengthen their ability to lead and cant rely on a large\nuniversity pool like Europe does.
\nThe speakers here were Lila Ibrahim, Ilan Gur, Dame Angela McLean and Sir Paul Nurse.
\nThe first question about around how the speakers have experienced changes in science and how it have evolved?
\nPaul Nurse noted that we live in an increasingly complex scientific world. The\nexpansion of science has led to greater complexity, which, in turn, has created\nmore silos across disciplines. To address this, we need more interaction —- not\nnecessarily immediate collaboration -— but increasing permeability between\nfields. There are also important social science aspects to consider, such as\nhow to structure training and foster interdisciplinary to work effectively.
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-7.webp\" "\\"Lila,")
Angela McClean: we’ve transitioned from an era of "big science" projects -—\nlarge, centrally organized efforts with clear, command-and-control structures\n-— towards distributed collectives. These collectives pool together the\nnecessary data to tackle significant questions, such as those addressed by\nAlphaFold. Unlike a single centralized project, AlphaFold’s success came from a\nclear, well-defined question that encouraged competition and fostered winning\nstrategies.
\nToday, we need the big research projects to define what success looks like and\nthen innovate towards new ways for people to contribute collectively without a\nbig central structure.
\nDisciplines can generally be divided into two categories. Those with "countable\nquestions"; for example, solving the Riemann hypothesis might win you a Nobel\nPrize! Then we have unstructured disciplines: fields like biology, where there\nisn’t a single list of questions to solve. As Paul Nurse put it, "biology is a\nbunch of anarchist scientists!". He contined that we need more unfocused\nresearch organizations that keep track of unstructured problems and help refine\nthem into focused scientific questions. This kind of work is essential for\nachieving progress in disciplines that don't have clear or countable goals.
\nIlan Gur then introduced ARIA, the Advanced Research Intelligence Agency, that\nhas a mission to build on the UK’s rich scientific ecosystem by pursuing\nbreakthroughs that may not yet have immediate or obvious consequences. ARIA\nfocuses on identifying the right people, their environments, their incentives, and\nhow their work can ultimately benefit society.\nARIA’s method begins by gathering program manager scientists to develop a thesis about\nwhere to focus efforts. This doesn’t involve just one research project but\nrather a constellation of efforts that can cross technology readiness\nlevels and organizational boundaries to achieve a focused target.\nExamples of ARIA initiatives include scaling AI via compute inspired by nature, and\nanother project observing that formal mathematics should be better integrated\ninto AI research to create more robust models. By guaranteeing bounds on\ninputs, we could use AI in critical applications with confidence in its\noutcomes.
\nAngela McClean then talked about how the UK government (under Labour) has outlined missions for addressing key societal challenges, such as\ngrowth, safer streets, opportunities for all, clean energy, and better health.\nThese missions are a great starting point for research and problem-solving.\nHowever, the structure of Whitehall (government departments) tends to remain\nsiloed. To succeed, we need to draw expertise from across departments to\naddress these challenges.
\nPaul Nurse noted that science operates on a spectrum between discovery and\napplication. Discovery is anarchic and unpredictable but applications are more\ndirected and goal-oriented. We need end-to-end funding that supports the\nentire spectrum, from discovery to application, while embracing diversity in\napproaches. A one-size-fits-all method won’t work. At the Francis Crick\nInstitute, departments were abolished, allowing disciplines to mix; turnover\nwas encouraged after a limit of tenure to keep staffing dynamic (including at\nsenior levels) and a high level of technical expertise was made available to\neveryone. Mixing people from different disciplines and using social scientists\nto understand the cultural structures within organizations is key to fostering\ninnovation.
\nAt the Crick Institute, the space encourages serendipitous conversations* This\nincluded inviting external guests and creating opportunities for unexpected\ninteractions. Tools like Slack’s "lunch roulette" feature could similarly\nencourage serendipity and collaboration.\n(Anil note: Cambridge Colleges do a great job here, but our departments are\nmore siloed but events like Rubik's 50 are a great example of how different disciplines come together)
\nAngela McClean also noted that we need to find ways to communicate the\nimportance of breakthroughs like AlphaFold outside of the scientific community.\nFor example, when AlphaFold was introduced, the Ministry of Defence (MoD)\ndidn’t grasp why the science mattered -— they lacked the necessary context. Even\namong highly educated people in the UK, there's a gap in understanding just how\nmuch AI is transforming society. By better connecting experts and amplifying\ntheir insights, we can and must help bridge this gap.
\nPaul Nurse also noted that the public must be informed about science advances;\ne.g. the fiasco around GM crops happened because noone trying to introduce GM\ncrops had bothered to infrm the public explainign what the issues are and\ngetting feedback. The main answer from the public smaple about "why not GM crops" is because they\ndidnt want to eat food with genes in it, and thats what bothered them. So when\nwe're thinking about AI and its implications, lets go out and talk to the\npublic and find out what worries them and then think about how to communicate.
\nDemis Hassabis then reflected on AlphaFold and the future of AI-Driven science.\nAlphaFold, which has been cited over 28,000 times already and by open-sourcing it (including AlphaFold 3 with open weights for non-commercial use), its impact has been profound. Some standout applications include:
\nAlphaFold is described as a "root node" problem within Deepmind -— once solved, it unlocks entirely new branches of scientific discovery. Its success in determining protein structures has validated this potential. What's next?
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-10.webp\" "\\"Hannah")
Material Design (Gnome Project) is the next frontier of material design, which shares characteristics with protein folding:
\nApplying AI to mathematics also offers exciting possibilities, including solving major conjectures that have eluded mathematicians.
\nInspired by mentorship from Paul Nurse, the aim is to simulate an entire\nvirtual cell -— a "Mount Everest" of biology. AlphaFold 2 solves static protein\nstructures, while AlphaFold 3 models interactions, taking us closer to\nsimulating an entire cell (e.g., a yeast cell as the model organism). Within\n5–10 years, this ambitious goal may well be achievable.
\nQuantum Computing is accelerating and offers exciting intersections with AI; simulating quantum systems to generate synthetic data or addressing challenges like error-correcting codes. However, classical Turing machines have proven more capable than initially thought:
\nDavid Deutsch called this approach "crazy, but the right kind of crazy" when Demis talked to him about it. Demis thinks that every natural phenomenon has inherent structure, which machine learning can model to efficiently search for optimal solutions. So quantum may not be necessary for this, and classical computing used with machine learning sufficient to solve the hugely complex underlying problem.
\nMeanwhile they also launched Isomorphic Labs to rethinking the drug discovery\nprocess from the ground up, leveraging AI for one of the most impactful use\ncases: curing diseases. AlphaFold is a powerful tool for fundamental research,\nand Isomorphic works on adjacent usecases need for practical drug discovery\n(helping design chemical compounds, test for toxicity, and minimize side\neffects, etc). Isomorphic aims to cure diseases with AI, and generate revenue\nto reinvest in fundamental research, so striking a balance between societal\nimpact and profitability.
\nDemis then commented that we stand on the brink of a golden era of scientific\ndiscovery, driven by interdisciplinary collaboration with domain experts and\nlimitless possibilities for applying AI to new fields and improving AI itself\n(approaching exponential improvement). The scientific method is humanity's\ngreatest invention and remains the foundation of modern civilization. In an era\nof transformative AI, its useful to go beyond simple A/B testing and treat AI\ndevelopment as a scientific method test. We need to understand the emergent\nproperties of AI systems and improve interpretability. Techniques from\nneuroscience (e.g fMRI for studying brains) could inspire ways to study neural\nnetworks and make them explainable rather than just being black boxes. The\napproach is to build the artifact of interest first, then decompose it through\ntargeted experiments to understand it once it has proven useful. Artificial\nsystems like neural networks can be as complex as natural systems, requiring\nsimilar rigor to understand.
\nScience is increasingly expensive and complex, leading to slower progress in\ncertain areas. However, interdisciplinary work will drive significant advances\nin the next decade. DeepMind, originally founded at the intersection of\nneuroscience and computer science, exemplifies how crossing disciplines\naccelerates innovation.
\nTo support these efforts, Google.org just announced a $20 million fund for\ninterdisciplinary\nresearch,\nfurther enabling breakthroughs at the intersection of fields. (Anil's note: let's hope that sustainability is on the list here!)
\nThe last panel had all four Laureates on stage to answer questions, moderated\nby Hannah Fry: Jennifer Doudna, Sir Demis Hassabis, John Jumper and Sir Paul\nNurse.
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-11.webp\" "\\"What's")
The discussion opened by asking the panelists how they first felt when they\nmade their prize winning discoveries.
\nJohn Jumper: when you release groundbreaking work, it’s fascinating to see the\nimmediate responses. I remember refreshing Twitter and seeing graduate students\nexclaiming, “How did they get my structure? It hasn’t even been published!”\nThere was a special issue of Science related to the nuclear pore complex, and\nthree out of four studies had heavily used AlphaFold without me even knowing\nit. It was amazing to see how our tools are empowering researchers.
\nJennifer Doudna: In the fall of 2011, while working on CRISPR (a bacterial\nimmune system), we realized it was an RNA-guided system that targets DNA for\ncleaving. It was one of those "aha" moments—bacteria had figured out how to do\nthis, and now we could understand and manipulate DNA using the same principle.\nA year later, when we published our findings, we could feel the momentum\nbuilding in the scientific community.
\nPaul Nurse: In 1985 (much older than the others!), I was working on yeast and\nmy lab had identified the genes responsible for the cell cycle—how one cell\nreproduces into two. We wondered whether these findings could apply to humans,\neven though this was well before human genome mapping. Using the first human\ncDNA library ever made, we introduced human genes into defective yeast cells.\nIf a human gene could replace the defective yeast gene and restore function, it\nmeant the discovery was transferable. Remarkably, 1.5 billion years of\nevolutionary divergence didn’t stop this experiment from working.
\nQ: What would you say to your 18-year-old self?
\nDemis Hassabis: I actually had this plan when I was 18! The amazing thing is that it worked out, but I’d tell myself to enjoy the journey a bit more.
\nJohn Jumper: My career has been more of a random walk, driven by doing good science in the moment and being open to new opportunities. My advice is to focus on doing good science now and let the interesting paths unfold naturally. It’s almost the opposite of Demis’s advice.
\nJennifer Doudna: Follow your passion, never give up, and don’t listen to naysayers.
\nPaul Nurse: Coming from a non-academic background, I couldn’t believe that I could be paid to follow my curiosity. Even now, it still feels like a privilege.
\n![\"%r\"](\"/images/ai-for-science/ai-for-science-2024-3.webp\" "\\"Hideo")
Q: AI gives answers but struggles with mechanistic insights. How big a barrier is this to public trust, and when can we expect true mechanistic insights?
\nDemis Hassabis: AI is an engineering science. First, we need to build systems that are worthy of study. Once built, we can break them down and understand them mechanistically over time. Early systems weren’t worth this effort, but now we’re developing tools that are, and they’re improving themselves. Unlike physics, biology can’t always be explained by universal laws, but simulations that can be tested and probed are better suited. Neuroscience techniques, like those used to study real brains, can also help us understand artificial neural networks.
\nQ: Is attention still all we need?
\nJohn Jumper: AlphaFold isn’t just an off-the-shelf transformer. While attention is an important component, many other innovations were added to change the structure of the network significantly. Fundamental research continues to unlock insights into both new data and previously unexamined data. AlphaFold has revealed new knowledge about data that had been available for years.
\nDemis Hassabis: The transformer architecture has been incredible but isn’t sufficient on its own. We’ll need several more breakthroughs of that scale to reach full AGI.
\nQ: What are the current challenges in biology data?
\nJennifer Doudna: Biology faces issues with both the quality and quantity of data for training AI models. We need to educate researchers on how to collect data both sparsely and smartly. Sparse but broad data is critical to creating robust platforms for training. This ultimately comes down to asking the right questions.
\nQ: What about people who are skeptical of these breakthroughs? Could society reject them?
\nPaul Nurse: Keeping the public on board is critical. This isn’t the first time new technology has faced resistance, and every time it happens, there’s concern. Special interest groups often hijack these conversations, so we need to find better ways to engage with the public and explain the science behind the breakthroughs.
\nQ: Africa will have the largest population of young adults by 2050. How can Africans be included in this global scientific revolution?
\nJennifer Doudna: The Innovative Genomics Institute has an ongoing effort in Kenya to work with scientists and help them understand CRISPR. This initiative has fostered a network of researchers, and I’d like to see more of that happening.
\nDemis Hassabis: DeepMind has been actively working in Africa, with events like the Deep Indaba conference serving as key convening points for African talent. There’s still a lot more to be done, but it’s a hugely important area of focus.
\nQ: How do we encourage the next generation of scientists?
\nPaul Nurse: In today’s world, journals are dominated by big data studies. While there’s value in this, we must ensure that creativity doesn’t get lost. There’s enormous potential in big data if approached with creativity, and we need to foster this mindset in our colleagues and students.
\nDemis Hassabis: Encouraging the next generation is crucial. One of my heroes is Richard Feynman. Every schoolchild should read Surely You’re Joking, Mr. Feynman! It shows how exhilarating it is to work at the frontier of knowledge. Science is incredible and fun, and we need to expose people to that joy.
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-13.webp\" "\\"Ray")
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-15.webp\" "\\"Interactive")
\n![\"%c\"](\"/images/ai-for-science/ai-for-science-2024-14.webp\" "\\"Glitzy")
These conclude my live notes! Beyond the notes here, the corridor conversations were incredibly\nuseful for me: I have lots of connections to make next. Any errors in these\nnotes are all mine, of course; I mainly took them for myself, but I hope it's\nuseful for you to have put them online as well.
Sophie Chapman lead an effort to explore a novel legal framework for forest carbon credits\nthat separates carbon tenure (i.e. title and associated property rights to the\nland and trees which store the carbon) from the carbon rights (i.e. title and\nassociated rights to monetise and manage the credits which\nsymbolically represent the carbon stored in the trees), while also specifying\nthe relationship between the carbon tenure and the carbon rights.
\nThe resulting paper has just been published in the Climate\nand Carbon Law Review journal, and is available as open access for your perusal.
Josh Millar presented our work on biodiversity sensing over at ACM Sensys 2024 in China. The full set of papers and demos has a range of impressive work on sensor networks, and some that stood out to me follow.
\nWe have just uploaded a preprint on using LLMs for conservation evidence, based on our work on large-scale crawling of the academic literature. Well done in particular to Radhika Iyer for having done the bulk of the evaluation on this as part of a very productive summer internship with us! This work evaluates whether LLMs can facilitate evidence-based decision support for conservation by testing ten different LLMs against human experts on multiple choice questions about conservation interventions. We found that with careful design, open-book LLM performance was competitive with human experts on filtered questions, both in correctly answering them and retrieving the source documents. However, general LLMs used "out-of-the-box" performed poorly, highlighting the importance of domain-specific design to avoid misinforming decision-makers.
Smita Vijayakumar went along to Seattle to SOCC 2024 to present her PhD research on Murmuration. This is a new scheduler for Kubernetes that allows for 15%--25% faster job completion times than the default scheduler for different job arrival characteristics in datacenters that are very busy. The key insight is that existing schedulers impose large wait times on tail tasks in highly utilized clusters, leading to long job completion times. Murmuration employs multiple communicating schedulers to schedule tasks such that their start times are as close together as possible, ensuring small tail task completion time. Our evaluation shows it scales to workloads with millions of tasks, and with queue re-ordering enhancements, achieves up to 100x better median job completion time than current schedulers on industry workloads.
\nUnfortunately, the videos from SOCC don't seem to be online yet (I could only find SOCC 2020), but I'll update this if they do show up.
I got invited by Sertaç Sehlikoglu to deliver a lecture to the Masters students down at the UCL Institute for Global Prosperity. I talked about the recent work on planetary computing, with an overview of the LIFE and FOOD papers.
After some time away from cloud computing (due to my new focus on conservation research), I served on the ACM SOCC 2024 program committee. It was quite interesting seeing the massive shift away from "traditional" cloud research (such as consensus protocols) towards many submissions aimed at accelerating machine learning workloads.
\nI also had a paper accepted there on decentralised scheduling, thanks to my former PhD student Smita Vijayakumar and her hard work on Murmuration!
I'm at the Royal Society this morning for the 2 day programme on "How does ecological risk related to commercial risk?", and am reporting on the morning session. The full program is being livestreamed so please do dial in if the below notes seem interesting to you. I put this note up almost live, so any errors below are my own.\n(Update: partial day 2 notes now available below)
\nThe summit kicked off with a keynote by economist Sir Partha Dasgupta. The focus was on the intersection of nature and economics, covering how markets fail to account for the ecosystems that sustain them. His landmark report covered ecosystem services, freshwater, tipping points, and physical risk, bringing to light the urgent need to reframe economic activities around the services provided by nature.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-1.webp\" "\\"Sir")
He began by distinguishing between two types of market activities:
\nIn the language of economics, there is a missing link here for the markets of\nprocesses, which makes for inefficiency, but more alarmingly a crisis in the\nmanagement of natural resources. The action is on the activities we undertake\nwhich affect our landscape -- and the huge number of species affected therein.\nOur knowledge is extremely limited, but keeping them intact is in our interest.
\nThere is a good deal of work on option values which economists have tried to\nuncover, but research has somewhat stalled in recent years. The risks that\ncompanies face as a consequence of this inefficiency are correlated, which is\nextremely dangerous for market stability.
\nHe tried to address this in the review of the Economics of Biodiversity\nreport,\nbut it needs a lot more thinking. A feature of the human condition is that\nfragmented ecosystems lose their productivity. The sum of the productivity of\nan ecosystem vs the sum of the individual parts has a big difference. The\nlate\nTom Lovejoy did a lot of work on this in the context of the Amazon rainforest\necosystem. Similarly, we are looking at a fragmented nature ecosystem today in\na global context as more and more habitats get truncated.
\nThe background extinction rates of organisms are hugely growing - the option\nvalue of organisms suggests we are losing enormous amounts of value in the form\nof unknown lifeforms. The correlation risk here is something that whole\ngovernments need to take on board -- it's too vast for any single organisation\nto take on board!
\nJane Lutchenko chaired the panel session. She is the administrator of the NOAA, "on loan" to the White House to work on nature and technology policy.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-2.webp\" "\\"Jane")
We have not always been looking at nature through a financial perspective. The\njourney of Natural England in the 1940 began more an ethical and moral\nperspective, and the scientific and beauty value of landscapes. It's only\nrecently been that practical impacts of nature and its benefit to humanity is\nbeing considered.
\nThe Millenium Ecosystem\nAssessment\ncommissioned in 2001 by Kofi Annan was a stocktake of the earth's natural\ncapital assets, and remarkable for how it made it clear how if we didn't\nreverse the decline of nature then we would be unable to meet humanity's needs\nsuch as ending poverty. A few years later in 2007 the G8 commissioned\n"Economics of Ecosystems and\nBiodiversity"\nthat worked until 2011 and helped reset the understanding of the human world\nand is fundamental to how we conduct our ecosystem. Later on the\n"Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem\nServices" (IPBES) commissioned by the UN general\nassembly to consider the contribution of nature to people.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-3.webp\" "\\"Tony")
So as nature rises in the political eye, there are multiple national programs\nall trying to assess the state of natural assets. In 2020 there was a Treasury\norganisation to bring natural capital into national accounting in the UK.\nDasgputa's landmark report established that nature and the economy are\nintertwined, which was a huge change in thinking. We no longer need to assume\nthat degradation of nature is the price of progress, and the new reality are\nthat the range of ecosystem services are critical to how we need to conduct our\necosystem services into the future. Pollination, carbon capture, biomimicry,\nnutrient capture, when all added up are worth more than the UK's GDP, but we\nstill struggle to how to measure this as part of our conventional economies!
\nAfter 20 years of expert studies and carefully constructed datasets, we\nstill struggle to do the right thing, but why? No one company is quite able to\nfully embrace the scope of the problem. There are 1000s of medium to large\ncompanies that depend on agriculture, but they are all dependent on 25 billion\ntons of water moving around intercontinental distances for the global water\ncycle. Any particular one company might make a small difference to reduce their\ndeforestation footprint but without collective action they will be unable to\nmake a global change. There must be a collective force to bring things\ntogether, and also better understanding of the connectivity across these\nfactors.
\nHowever, there still isn't a prominent intellectual place for the connections\nbetween ecology and economy and still isn't being taught in undergraduate\ncourses and so not making it through to high political decision making. There\nis still a lot of emphasis on financial growth, but not much discussion of\nnature: we should be factoring in that nature is the mechanism by which we can\nachieve financial growth. It's quite hard to meaure biodiversity vs "tons of\ncarbon". Environment regulation must move from stopping harm, but also plot\npathways to nature recovery. So the link between ecological and commercial\nrisk is very real and must be measured and actioned.
\nHer first ever "proper job" was working at the Nature Conservancy as their first hydrologist. Freshwater is at the centre of the triple planetary crisis today. As a result of these challenges, what we see is an intensification of the water cycle:
\nOur lack of investment in proper waste recycling infrastructures is now coming\nback to haunt us. They impact hugely on freshwater availability and quality. It\ndisrupts commercial supply chains, and the quality of the supply for both water\nand ecosystem services for which we heavily depend.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-4.webp\" "\\"Louise")
Two satellites (GRACE) spinning around earth from 2002-2017 that can detect\nchanges at a centimetre scale of changes in water mass. The image shows how\nconnected the world is, and the scale of the changes at the poles and greenland\nis incredible. Some research just finished up in Greenland, and the rate of\nwarming in the arctic is roughly 2x the global warming rate. Greenland is\nlosing almost 260 gigatons of ice per year, which is a million Olympic-sized\nswimming pools a year -- an awful lot of water to move around into the ocean\nsystems. The ocean cycles aren't on the slide, but they will impact.
\nOnly 1% of the total water supply is freshwater, and its appropriation among human\nactivities is quite unalanced. We have green water to blue water to white water\nto grey water to black water. We have messed up the blue and black water\nmanagement. We are producing 360 billions of cubic metres of waste water a year\nand only 3% is recycled. A significant percentage of the world population (2\nbillion people) are dependent on black water, and a lot only have access to\ngrey or black water.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-5.webp\" "\\"Water")
We need to treat management of surface and groundwater systems holistically.\nThe freshwater living planet index is dropping precipitously: some causes are flow\nreduction, introduction of invasive species and nitrogen/acidification peaks in the\n1970s/80s/90s which all causes major changes in freshwater biodiversity. We\nalso introduced laws on long range transport regulation about acidification. In\n1991 we had the EU urban wastewater directive and in 2000 the EU water\nframework directive, but none of this seems to have made much of a difference (see graph above).
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-6.webp\" "\\"Freshwater")
Are we looking a tipping point in terms of waste water systems? We dont have\nclear metrics about freshwater biodiversity, and without the metrics we dont\nhave a way of valuing them as part of commercial systems. We have new emerging\npollutants (microplastics etc) which we have to deal with multi-sectoral\nchanges to make a difference, which is lacking despite the legislation in\nplace.\nHowever, the growth in urbanisation (56% of UK population lives in cities, increasing to\n70% by 2050) which makes a huge difference to where waste comes from. When we\nmove food around the word, there is a water cost which is passed onto other\ncountries. (Note: see also our preprint on Food impacts on species extinction risks can vary by three orders of magnitude)
\nWhen we move to decarbonisation, the last 20% of action (the really difficult bit) is\nrelated to land use in particular. But if we join up the practises around good\nwater use and decarbonisation then we have a bunch of innovative interventions,\nfor example natural flood management. There are also cobenefits around\ninfrastructure and there is a real challenge with our UK underinvestment. The\nlast report in 2022 there are some excellent propositions for how to deal with\nwater and atmospheric pollution and biodiversity. We need to act on this\nreport, but progress is slow: e.g. Thames Water is a long way away from\nactioning this.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-7.webp\" "\\"Water")
In the 1980s, ADAS was the wing of farming and food ministry worked on how to\nrecycle waste better. Metals were the big problem back then, but now there are\nloads of extra contaminants such as plastics and forever chemicals, We must\nmove towards reusing waste we produce towards reusable end products for farming\nand dramatically drop our water use.
\nWe have polycrises with climate change, biosphere pressure, intertwined\nsystems, interacting shocks, and many tipping points. Tipping points are a\nshock that can move a system from a local maxima into a new stable state, but also\nthe gradual loss resilience can cause a system to shift suddenly. There are a\nnumber of drives that cause these tipping points, and they are not just a\nrandom aside: they should be a key part of any investment strategy.
\nA lot of people live in areas that are at risk of ecological tipping points,\nand the map seems very correlated to the earlier water change map shown by\nLouise! 56% of human CO2e have been taken up by the biosphere so far (1430\nGtCO2 from 1850-2019) which has been happening for "free" thanks to nature, and\nthis might come to an abrupt end soon which is a big source of worry in the\nscientific community.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-8.webp\" "\\"Prof")
The connectivity of various tipping points is becoming clearer. There are just\na few major actors that shape some of these (such as the Amazon rainforest) and\nthey are often quite far away geographically by virtual of their influence over\nmarkets.
\nThe current idea is that social tipping ("social norms as solutions") is the\nway to alter our value systems. See "Operationalising positive tipping points\ntowards global\nsustainability"\nby Tim Lenton et al. (Note: Simon Sharpe also authored this, see our CSaP\nreading\ngroup\non his excellent "Five Times Faster" book.)
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-9.webp\" "\\"Exposure")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-10.webp\" "\\"Interconnected")
So we need to prepare for tranformation, navigate the sudden transition and\nthen build resilience in the new norm after the tipping point. But our window\nof opportinty is happening right now so action must happen urgently or we miss\nthe window and enter the tipping point unprepared. So "corporate biosphere\nstewardship" is a new business logic with the purpose of shepherding and\nsafeguarding the resilience of the biosphere for human well-being, and\nfostering the sustainability of a rapidly changing planet. Rather than viewing\nnature as a compliance question, we should view it as humanity's greatest\nbusiness opportunnity! For example, Seafood Business for Ocean Stewardship\n([seabos.org](Seafood Business for Ocean Stweardship)) is codifying this\napproach for marine foodstocks. There is an increasing focus on how to report\nthis stuff. There are three good books to read more on this:
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-11.webp\" "\\"The")
Paul Polman is the former CEO of\nUnilever. He started by noting that we have to make sure that we dont just\nspend our time talking to each other, but we have to get the message out in the\nform of action to the wider world! When they developed the SDGs, Ban Ki Moon\nrequested Paul to represent the private sector. He was described as "the\nproblem walking into the room" when the politicans first met him, but luckily\nthey ended up in a happier place! We must find a balance into a sustainable\neconomy into the future. Most businesses, although they dont behave entirely\nsustainabiltly now, they do understand the need for a planet. When he ran\nUnilever for a decade, they encoutered hundreds of millions of dollars worth of\nnature-related interruptions tot heir business.
\n4 billion people in the world depend on natural medicine, and nature governs\nhuge amounts of the human ecosystem. Nature supports peace and national\nsecurity - many of the world geopolitical events train back to inequality in\naccess to natural resources and this is the foundation for many economics. The\nWEF report recently calculated thaat $44tn of the world economy depends on\nnature -- but this is a huge understatement given that our entire life depends\non it! Changes in one ecosystem affects others, and so our conversation must\naddress the interrelationship of all this that makes it so complex.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-12.webp\" "\\"Paul")
We must not see ourselves at the top of the pyramid -- biomimicry and\ngeoengineering is an arrogant approach as our survival relies on cooperation\nwith the biosphere -- when we destroy nature we destroy ourselves. (paraphrased quote) "Man is the\nmost insane species - he worships an invisible god and destroys the visible\nnature, not realising that the invisible god he worships is the visible nature\nhe destroys" (original: Hubert Reeves). Extinctions are 10-100x the average of the previous\ncenturies.
\nFood and land use is about 30% of our global emissions, and yet we have the\naudacity to keep our famers in poverty. Every $1 invested in changing nature\nand regenerative farming approaches give us $16 return. The food companies are\nreally exposed if they don't act, much as we criticiise fossil fuel companies\nright now.
\nWe are making withdrawals much faster than we are depositing in the bank of\nplanetary boundaries, and there are millions of people losing their lives and\nbillions being displaced because of these choices. The Carribean has lowered\nits tourism by a significant percentage due to beach erosion. The Amazon this\nyear has had unprecedented wildfires (around the size of Italy) as an example.\nWe name our tropical storms (we name them as if they are our friends); would it\nbe different if we named them Exxon Chevron etc? We must absolutely link the\ncommercial sector to nature as companies like Unilever must become\nborder-positive and nature positive as they are hugely global.
\nLeadership is increasingly centred in europe with many business regulations\nthat are nature positive. There is a bunch of business interaction happening,\nbut the estimation is that loss of biodiversity has cost us over $10tn. In\nagriculture we are destroying $12tn of value, but if we turn that around its a\n$4tn opportunity. This just doesn't make business sense to not act. Covid\nshowed us that infinite growth on a finite planet is unsustainable.
\nWhat can business do next then?
\nPriorities for business action:
\nNature needs business, and nature needs business, and the time to act is now!
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-13.webp\" "\\"Breaching")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-14.webp\" "\\"Key")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-15.webp\" "\\"The")
Partial notes from the audience Q&A follow.
\nQ: To what extent have we made progress in the transition towards a nature positive future -- is the awakening fast enough?
\nTony: what we have discussed here is not very well known in the majority of the\nworld's population. Most people get in their car and go to work, and their food\nis processed and disconnected from nature. So the insight that most people have\ninto their dependencies on nature isnt very well known. David Attenborough has\nmade it clear that if we take steps towards repairing the natural world then\npeople will need to be exposed to nature so that they understand what they're\ntrying to save. This was disconnected in the beginning of the industrial\nrevolution, and we need to urgently reconnect it, or people will simply not\nappreciate what we're about to lose.
\nLouise: is quite optimistic because the younger generation is very engaged in\nthis, and we need to ensure they have the levers for action. It's the older\ngenerations challenge to bring the right levers to them.
\nPaul: the science is still evolving at a fast pace, and will continue to, and\nif we dont convert it to fast practical steps we'll be trouble. Everything\nstarts and ends with educaiton, but we are also out of time. The urgency isnt\nwell understood -- a survey of 4000 people about this skewed to about 30% of\npeople where they left a company because it wasnt aligned with their values.\n60% were considering leaving but didnt. We must get businesses to move out of\nthe current gridlock where multilateral institutions cannot change easily, and\nwe must cooperate across boundaries. The EU has a nature restoration law which\nwouldnt have happened without businesses getting involved to push it over the\nline for example -- without that it wouldnt have moved. And tipping points mean\nthat only 4-5% change is required to get us to a new stable state, so this is\nboth an opportunity and a risk. The biggest risk right now is the US election,\nas its a global vote but decided by a tiny minority.
\nQ: What examples have you seen about nature flows in the real world?
\nPaul: the economic forces must be made to work; people arent skeptical but the\neconomic realities must work. But there are pressures; the average tenure of a\nCEO has dropped to 4-5 years and so their actions are aligned with reelection\ncycles just like poliitics! So we have to make capital and money flow work much\nmore transparently. With a relatively small investment, we can get to 50%\nregenerative agriculture; this is happening in the USA (thanks to IRA). Many\nof the major food companies (from Pepsi to Unilever) are committed to $15bn\ntowards soil health and regenerative agriculture. And there is nothing better\nthan healthy soil towards preserving yields in the face of climate change. The\naverage age of farmers is ~50 now and so they only have 10 seasons left before\nthey retire, so they need to be paid to shift proactively for ecosystem\nservices. It needs a proper farmer rewards to do the shift quickly and hedge\nthe risk. Luckily, the speed of nature restoration is faster than we predicted.
\nCarl: it has become a strategic issues in companies, as opposed to a\ngreenwashing protection. They run an executive course for companies and they\nare factoring in risks proactively now in a way they didnt before. They are\ndemanding satellite data and other risk management data products to quantify\nthe uncertainties in nature. Earth SYstem Indicators is their system to combine\nmultoiple planetary boundaries into one actionable indicator. It is a bit like\nan avalanche now where the reaction is awakening, but we are probably in a\ntipping point right now so we must act in the right direction.
\nJane: social systems are highly non linear and also characterised by social\nsystem tipping points. Water seems to be an obvious one that underpins the\noperation of many companies.
\nLouise: Universities and research must turn into businesses (of innovation).\nOne thing we failed to do is to find the problems but dont find the solutions,\nand research must come up with solutions. "Ecologists turn over stones looking\nfor problems". We must take a position on coming up with solutions and\ncommunicating them clearly, and the only way to do this is to work together\nacross disciplines. We are looking to social scientists to combine civic change\nwith environmental scientists to propose interventions.
\nJane: there has been a huge transformation in the US towards finding solutions,\nand that means partnering with government agencies and NGOs in a way we haven't\nseen before. (Note: this describes the Cambridge Conservation Initiative model perfectly!).
\nTony: even when the solutions exist, a number of entities want to keep the\nstatus quo: e.g. Exxon and Shell spent more than two decades pretending it\nwasnt a problem to protect their interests, and they're not alone. And so we\nhave to establish a social tipping point (just 3-4%) was just the idea of\nExctinction Rebellion to overload the court system and get to the point where\nwe could shift the discussion, But it didnt work, so what will work? We need to\ntry, try and try again on this. Otherwise we're rather just talking to\nourselves.
\nQ: Water related financial risks: TCFD galvanised focus on climate,\nspeciifcally carbon, and one of the barriers is this focus on regulations that\nfocus on the cause (carbon) but not the impacts. We mainly feel these impacts\nthrough the water cycle, so how can we make the focus on the phsyical risks of\nclimate change being on the impacts and not the cause?
\nLouise: we need a systems approach to this; tipping points sort of get there,\nbut how do we get the evidence to get over the top?
\nQ: Why is it that the business and finance community isn't moving faster? The\nsuspicion is that not all businesses are part of the solution, and do we need\nto be clearer on that? If we combine some of the social aspects and linking of\npeople to nature, does that supply chain need to be shorter?
\nPaul: we are slightly bending the curve linearly, but the gap is getting bigger\nexponentially. If we dont solve the problems for the world, we can be great as\na company but it doesnt save the planet for the next generation. Business alone\ncannot do it, but what we can do is to make it more transparent. The more\ntransparent we are the more we change behaviours and social norms. Companies\nusing AI across supply chains are doing surprisingly well at bringing this\ntransparency around. In every transition in the history of mankind, there will\nbe people resisting change, but now the noise globally is becoming louder is a\nbig deal and is a sign that the process is unstoppable and has begun. The noise\nwill only increase in the next five years. In the food chain there are\nherbicide, pesticide producers that are doing really well on food speculation,\nbut at the expense of many lives. We need a leader to change them, and if they\ndont change they will obsolete their business models.
\nQ: What is the single most important action businesses can take today to address the crisis?\nQ: Why not spin this around to look at this at a very local level (e.g. flooding for business crisis)?\nQ: Unless we can localise analysis with improved precision, can we improve decision making?
\nTony: Localisation of issues is critical. The founding slogan of Friends of the Earth was to "think globally act lcoally" which was a very good slogan. Natural England is working with local authorities on spatial planning exercises on where the good remaining nature exists and where it might usefully be repaired (cleaning up peatlands etc). The targeting of the resources like this might help us bend the curve. This must be linked with our need to build 100k new houses and balance natural and business.
\nPaul: COP30 is likely to involve businesses heavily (COP29 is a writeoff due to location). But businesses are getting behind it all as countries cant implement their policies without business help. A large percentage of indigenous populations are planetary guardians and so we must support their efforts to protect existing natural capital against tipping points. Science without impact is useless when we have a crisis at this level.
\nEcosystems are capital assets; we now use the term natural capital to include\nthe biosphere in the full stock. This raises the question of asset management,\nand the questions discussed today represent shortcomings in asset management.
\nWith natural capital, property rights are hard to enforce -- nature is always\non the move and "mobile" and the commodity changes when it moves. This leads to\nan underpricing of many forms of natural capital (not all, but many) which\nleads to the overuse of it, which in turns leads to deterioration of the asset,\nwhich implies a runaway tipping point of decline. This all leads to a circular\ndecline in the value of natural capital; if there is a heightened risk of an\nasset collapsing, the accounting value is naturally reduced. So this is a\nvicious cycle at work that leads to the inevitable decline.
\nGlobal GDP has increased hugely as opposed to natural capital, which increases\nthe pressure to overspend on natural capital. What are the arbitrage conditions\n(e.g. risk adjusted rate of return) on the portfolio assets that comprise\nnatural assets? Because of imperfect pricing, there is a big misalignment of\nany portfolios. For example, the Ganges is the most polluted river in the world\nand under the Ganges action plan to remedy this there was a social cost benefit\nanalysis -- there was a 15% rate of return calculated. The rate of return on\ngovernment bonds was roughly 5%. There was a gap of 10% between these two assets,\nand so if the structure was efficiently organised then the Ganges value as a stock\nshould be decreasing by 10%. But this is perverse since the reverse should hold\nsince the river quality has been decreasing not increasing!
\nSo from the firm's point of view we are looking at their balance sheets (from\nthe natural capital POV). But GDP was not constructed for this purpose -- instead\nit was constructed in the post war period to calculate progress towards getting\nout of economic depression resulting from lack of economic activity. But somehow\nafter WWII it became a long-term goal but it has no social benefit justification.\nGDP is a flow and so not a future predictor like stocks. You can have GDP growth\nand the national package of accounts, but it just won't take natural capital\ninto account. There is a gap between portfolio management and GDP therefore.
\nBut countries are moving towards wealth accounting bit by bit, including for natural\ncapital aspects of wealth. Most of the attention has been towards human capital,\nincluding the attention of investors. This now needs to shift quickly towards\nnatural capital. There have been studies attempting to assess the market demand\nfor natural capital. Think of the biosphere as a massive fishery -- the natural\nanalogy is how much we take out of it, and what the regeneration rate is. The question\nis what the overreach is and there are projects working on this.\nWe need to take the outputs of these projects and treat them as underestimates due\nto the huge extinction pressure on organisms.
\nEven if firms are competing in the market, they still need to cooperate on the\nunderlying natural capital accounting to avoid a "market crash". In academia,\nthere is both cooperation and competition. There is a race for paper publication,\nbut also a huge amount of sharing towards global scientific co-creation. There is a\ngood deal that companies could learn from this to cooperate and communicate towards\nnatural capital accounting. More on day 2 tomorrow about how we can get there!
\n\n
Kat Bruce (founder of NatureMetrics) opened the session by noting just how quickly\nthe biodiversity space is moving, and how encouraging it is to see so many businesses\nengaging with environmental scientists.
\nIs going to cover terrestrial metrics only. There are several sorts of metrics we could measure:
\nWe do need to understand biodiversity risk in regional detail so that\nbusinesses can use this to influence their actions. There is also the need for\nclear target setting to know how much opportunity cost to "spend" on\nbiodiversity vs other actions.
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-16.webp\" "\\"Neil")
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-17.webp\" "\\"Categorising")
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-18.webp\" "\\"Metrics")
UNEP-WCMC have a huge database about all the various metrics (~573!) that\ncover biodiversity. If you even add other things like marine, the number\nbreaks 600 (see picture). There are around 23 useful ones for business use;\nthere are few in this list that check benefits to people or discuss genetic\nchanges, but there are lots of on diversity, habitat area and so on.\n(Neil noted that our LIFE: A metric for mapping the impact of land-cover change on global extinctions paper isnt listed as its not published yet\ntill the end of the year, but it will be!)
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-19.webp\" "\\"What")
There is quite a lot of work needed to make the metrics usable, and a pipeline.\nHe also noted the importance of incremental pipelines and mentioned the\nincremental pipelines that Michael Dales and Patrick Ferris are working on for these.\nAnd this is something I'd like to advance with Planetary Computing...
\nThe main tool that pulls this together is IBAT\nwhich is a paid-for model to support its continued developed. There is also\nthe free ENCORE nature platform. For supply chains,\nthere is also tools to help check the country to country. There is a big emphasis\non the need for open sharing between platforms as well due to the sheer\ncomplexity of biodiversity worldwide, which is a key thing to keep the platforms\nsustainable both financially and equitably.
\nAndy's career as a scientist started 35 years, and he is now at the Natural\nHistory Museum as a senior researcher. The mission there is not just to track\nthe decline of the planet, but also to advocate towards a healthier nature\npositive society as well.
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-20.webp\" "\\"Why")
We have a million or so species of animal and plants threatened by species\nextinction. Our job as scientists is to come up with defensible statistics,\nbut they also don't convince anyone. So stories about individual species\n(like the white rhino) are vital to build public awareness about the real\nimpact.
\nIs there one index to rule them all? Andy is against that idea of a single\nmetric to describe the world's biodiversity. Any indicator combining some of\nthe metrics has an "exchange rate" between extinctions and human wellbeing. In\nother words, it creates an "extinction market".
\nDecision grade data are derived very carefully from very biased raw data. And\nthe data collection requires a huge amount of expertise, and is painstaking\nto conduct. This has to be funded, and not even the collection of the raw data\nis very well funded right now. To illustrate the data bias, there are a huge\namount of birds...and mallards...in there, which is clearly not representative\nof global species spread.
\nWe can plot ecosystem function (resource capture, biomass production, decomposition\nand nutrient recycling) on the axis of biological diversity (variation in genes\nand species and functional traits). Across this, there is a huge spectrum\nof ecosystem services that this supports, of which a few are really useful to humans.\nAnd the BII index\nis a statistical model relating nature to human pressures that produces both\nhigh resolution temporal and spatial models.
\nWhat should we do in terms of ecosystem health?
\nIt requires a model, has to be global, and measure both species persistence and\necosystem health. It has to be vs counterfactuals otherwise the cost is too\nhigh for any individual organisation vs society taking action collectively.
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-21.webp\" "\\"Why")
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-22.webp\" "\\"Ecosystem")
\n![\"%r\"](\"/images/rs-ecorisk24/rs-ecorisk-23.webp\" "\\"Defining")
We need to combine models with monitoring to give us a "sat nav" for nature.\nThere is a need to monitor drivers as well biodiversity, which\nrelies on data being available to improve the models. And if the platforms are\nopen then this is possible. There are platforms in the form of Geo BON, IBAT, BII ha a data license with\nBloomberg, and some TNFD tools.
\nTake home messages:
\nAndy: if we go back to a system with the original biodiversity, but the actions to get it there don't have an immediate effect (there is a timelag). There's no standardisation effort yet across many metrics, but we are moving towards ensembles of models with alternative sets of inputs for the environmental rasters in order to normalise the uncertainty in both the environmental and geographic space.
\nNeil: UNEP-WCMC uses BII a lot! In terms of metrics, a species and ecosystem metric and something on genes would cover the three dimensions of biodiversity, but others also need to be factored in. There is an effort to use AI/geospatial to get higher resolution landuse data (is this a forest?) and also probabilistic SDMs, but no standardisation.
\nKat: being able to reverse degradation is a secret weapon for nature as it has an incredible ability to bounce back if pressures are reduced due to its innate resilience. The aspect of the data we often forget about is its ability to tell these stories and drive uptake, and this is powerful.
\nNeil: There are criteria (peer reviewed, published, etc) with lots of feedback from the co-authors of the paper (to appear later this year). There are strongly held opinions in the biodiversity metric space. But it's actually not a huge and manageable list of metrics.
\nAndy: trying to avoid false claims of nature positive requires verification. We're going to get better at this by estimating the net gain and how certain its positive. The models will improve when there is a connecting pipeline to monitoring data and needs verification. So any payment for nature-positive claims needs to be staged and ex-ante.
\nI failed to capture notes for the middle sessions as I was engrossed in conversations, but here's a gallery of some of the fantastic speakers! They're on the livestream video if you want to catch the details.
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-24.webp\")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-25.webp\")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-26.webp\")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-28.webp\")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-29.webp\")
\n![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-30.webp\")
See the equation below. The right hand side is G (nature's regeneration rates) and is a function of the state of the biosphere (the S).\nThe left hand side is the human demand (human activity) which is a per-capita adjusted population and the efficiency of provisioning goods (recall what those are from the start of day 1 notes).\nMost of the discussion in these two days have been on the right hand side. A huge amount of the literature is on the alpha, and how technological progress can raise alpha (e.g. via cleaner energy) and help rebalance the inequality. How fast can alpha change, as the faster it grows, the faster GDP and natural capital grows.
![\"%c\"](\"/images/rs-ecorisk24/rs-ecorisk-31.webp\" "\\"Sir")
How might we advocate for change? The total we pay ourselves is 2-3% of global GDP for subsidising our assault on nature. Removing those subsidies is the equivalent of raising alpha. Adam Smith's classic book in the 18th century was the "Wealth of Nations" and not the "GDP of Nations". For us, wealth includes natural capital. Even as early as 60 years ago, human capital didn't appear in the economic literature. Most national accountants include statements about the increase in human capital, and for our purposes we must include natural capital in the notion of wealth. We must shift accounting away from GDP into calculations of stock and inequalities, as shown in the equation.
\nWhat the discussions missed didn't discuss invasive species and the "transfer of natural capital" through the fact that goods and services are traded. This is represented in the capital N in the equation. So that's future work!
This concludes my rapid note taking. Do join the livestream and follow for the remaining day and a half if you have a spare moment!
\nEdit: upon being prodded by Alec Christie, I've uploaded a NotebookLM generated podcast summary of the morning. It's surprisingly entertaining, but I'm sure I'm going to regret this for some reason...
\nI was on the program committee for HotNets\n2024 this year, which was a\nthoroughly enjoyable experience. The list of accepted\npapers is now out,\nand it's a diverse program -- with my personal favourites being the ones on\nspace communications networks using low earth orbit satellites.
\nWell done to Behnaz\nArzani and Nate\nFoster for really excellent general\nchairing and ensuring the PC maintained a constructive, positive tone while\ndoing the difficult job of selecting papers from a crowded set of submissions.\nThe structure of of the program committee was also somewhat novel, and one\nI'd like to replicate in other conferences I organise in the future.
\n![\"%c\"](\"/images/hotnetspc-view-2024.webp\" "\\"The")
![\"%c\"](\"/images/hotnetspc-anil-2024.webp\" "\\"Me")
I'm noting down the HotNets as a potentially really good way to run the next\nProgramming for the Planet, which is due in 2025. More\nnews on that soon! In the meanwhile, get your papers into LOCO\n2024 which is due in a couple of\ndays...
", "url": "https://anil.recoil.org/notes/hotnets-pc-2024", "title": "Being on the HotNets 2024 program committee", "summary": "Serving on HotNets 2024 program committee was a great experience with a novel structure.", "date_published": "2024-09-22T00:00:00.000000Z", "date_modified": "2024-09-22T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "networks", "functional", "hotnets", "janestreet", "service" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/mitigating-nbs-risk-paper", "content_html": "Many of the questions around our recent Nature Sustainability commentary on NbS credits revolve around\nhow to finance new projects if credible credits need to be ex-post. Our\nlatest paper published in Carbon Management on "Mitigating risk of credit reversal in nature-based climate solutions by optimally anticipating carbon release" tries to address this.
\nThe problem with selling ex-ante (future) carbon credits for (e.g.) a\ndeforestation avoidance scheme is that project reversals can happen in the\nfuture ("deforestation has increased") thus rendering any credits issued\npreviously useless. On the flip side though, an overly conservative view of the\nfuture ("the entire forest will disappear overnight!") is clearly so\nconservative that it doesn't serve the best interests of the project developer.\nSo ideally, a project would make realistic but conservative ex-ante predictions\nthat is safe for both project developer (who gets more funds upfront) and a\ncarbon credit purchasers (who needs to account for impermanence of nature\ncredits).
\nOur paper shows how to do this by calculating a "release schedule" to predict\nfuture drawdowns, and then issuing extra credits when the release at some\nfuture date is less than predicted by the release schedule. We use verified\nex-post observations to construct these release schedules, and design them to\nbound the risk of the project becoming negative overall (that is, net drawdown\nis negative) and thus failing.
\nThe paper evaluates this process with both theoretical and real projects to\nassess how well it balances the tradeoff between generating permanent nature\ncredits and bounding the risk of project failure in the future. As a nice side\neffect, our method removes the need for buffer pools entirely, which do not\ncurrently base the sizing on an empirical assessment of reversal risks, and are\nusually cancelled at project end (wasting potential credits). Read the full\nopen access paper, lead expertly by E.-Ping Rau Srinivasan Keshav and David Coomes,\nthat just came out in Carbon Management for details: Mitigating risk of credit reversal in nature-based climate solutions by optimally anticipating carbon release
\nThere's still plenty of future work to be done -- we focus on avoided\ndeforestation projects in this paper, but afforestation projects could also be\nmodelled on similar principles. Do get in touch if you'd like to help assess\nour methods!
\nA very fun talk at ACM HOPE 2024\non some new work with Cyrus Omar and Patrick Ferris on how we can formally specify\nsystems to be robust to code generation by AI agents. For instance, if you were\nto ask GitHub Copilot to generate you code to filter endangered animals out of\na folder of images, it might interpret that as to delete the image, or to move\nit to another folder (which might be public), or just remove it from the index.\nAny of those options are potentially valid, so what do we do? Our idea is to\nuse F* to specify a rich set of allowable behaviours which can then be\ndynamically enforced in less expressive languages, and thus offer layers of\nprotection against over-eager (or rogue) AI agents.
\nWe'll increasingly need these sort of protections in our systems as natural\nlanguage interfaces get adopted for programming, and the traditional 'yes or\nnot' access control policies we use right now will be insufficient.
\nRead more in:
\nThere's a new teaching page with my past and present courses, and links\nto the associated teaching materials. One of the nice things about most Cambridge\ncourses is that all the teaching materials are public, except for video recordings of\nthe lectures themselves.
", "url": "https://anil.recoil.org/notes/new-teaching-page", "external_url": "https://anil.recoil.org/teaching", "title": "New teaching page with my Computer Science courses", "summary": "Explore my new teaching page featuring Computer Science courses and materials.", "date_published": "2024-09-03T00:00:00.000000Z", "date_modified": "2024-09-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "teaching", "cambridge", "computerlab" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/ukri-grant-terra", "content_html": "I don't normally announce funded grants (preferring to focus on outcomes), but I'm really excited by this one and couldn't resist! Myself and my colleagues Srinivasan Keshav (from computer science), David Coomes (from Plant Sciences), Andrew Balmford (from Zoology) and Neil Burgess (the Head of Science at UNEP-WCMC) have just received a £1.2m grant from the UKRI to work on building foundation models for planetary intelligence.
\nNow, normally a grant isn't news, but I wanted to highlight the scheme that it came under. UKRI announced an interdisciplinary program specifically for projects that don't normally get funded by just one research council. In our case, this work usually falls between the cracks of EPSRC ("too much nature") or NERC ("too much engineering") or STFC ("not enough satellites"). But this interdisciplinary program expressly assembled a panel across all these areas, and collectively gave us a shot. I really hope this scheme continues to gather steam within the UKRI.
\nAs to what we're doing? There'll be the evolution of the work described in Remote Sensing of Nature and Mapping LIFE on Earth, with lots of domain knowledge that we're pulling together with our partners at UNEP-WCMC (especially Neil Burgess and Ian Ondo) on plant and animal species distributions across the globe.
\n![\"%c\"](\"/images/2024-clr-scotland.webp\" "\\"Us")
You can read more both in the UKRI announcement today and in the Cambridge Computer Science coverage about what we're up to. Some exciting preprints about our work in this space so far:
\nOur paper on ex-ante projection for nature-based solutions has been published in the Journal of Carbon Management. I also wrote up some long-form thoughts on it here. The publication represents the culmination of work on how to properly account for credit reversal risk in nature-based climate solutions through optimal carbon release anticipation. E-Ping did excellent work leading the methodology development and analysis. The approach we developed incentivizes project performance while resolving the fundamental trade-off between a credit's permanence rating and risk reduction, providing a pragmatic solution to one of the key challenges facing the effectiveness of nature-based climate solutions.
We have just released the Tropical Moist Forest v2.1 specification, to follow up the now-expired v2.0 from six months ago. The key updates are a new high-level explainer, as well as clarifications for buffer zones and base tiles. The high-level explainer is particularly useful as it makes the methodology more accessible to a broader audience beyond just technical specialists. We've also refined the technical details around how to handle edge cases with buffer zones and base tiles in the remote sensing data, which came from feedback from teams trying to implement the methodology. These incremental improvements are important for ensuring the specification can be reliably applied in practice.
Our commentary on nature-based credits has been published in Nature\nSustainability,\nlead expertly by my colleagues Thomas Swinfield and Sophus zu Ermgassen.
\nIn our view the carbon credits markets are vitally important for forest\nconservation, but the key is to only transact these credits after they have\nbeen proven to be demonstrably additional using robust statistical techniques,\nso that we know before a sale that each credit represents real gains that would\nnot otherwise have occurred without the carbon finance.
\nA more scientific approach that supports transparent, third-party validation\ncould absolutely transform these markets. And given the rapid rate of tropical\nforest loss, such upscaling of credibility is vitally necessary to raise\ninvestor confidence in protecting nature, since we can now be confident that\nevery "credit" sold is resulting in real climate benefit. There are real\nquestions remaining about this reform, of course.
\n![\"%c\"](\"/images/naturecrossroads-method.webp\")
Since projects can no longer\nsell ex-ante credits (i.e. future credits which may not be real), then we\nneed to come up with financing models that embrace the upfront risk. This\nalready happens in other areas such as oil and gas; as Siddarth Shrikanth notes:
\n\n\n<..>speculative efforts like mining or oil exploration, we’ve still managed to build large industries out of uncertain (but potentially very valuable) payoffs. The challenge here will be to figure out which archetype different projects fall into, and create enough trust that the output will be real and valuable enough to someone to justify the up front investments<..>\n-- Siddarth Shrikanth via LinkedIn
\n
Lead author Sophus zu Ermgassen comments as well that:
\n\n\nSociety has made huge policy commitments to upscale carbon & biodiversity offsetting.\nBut, carbon credit markets have suffered serious hits to their credibility & nascent biodiversity markets risk inheriting shortcomings. Impact evaluations have shown that these markets have systematically underdelivered additionality.\n-- Sophus zu Ermgassen via LinkedIn
\n
We've been working on this aspect in 4C, since ex-ante predictions of outcomes are necessary for project developers to be able to forecast financing. See the paper "Mitigating risk of credit reversal in nature-based climate solutions by optimally anticipating carbon release" for our latest work on that, lead by E.-Ping Rau and Srinivasan Keshav.
\nSophus zu Ermgassen also gave a fantastic talk at the CCI ont his topic a few months ago that is a must watch for anyone working on carbon or biodiversity markets.
\nIt's also not enough to "just" show that a given project is additional from a satellite perspective, but also that they do not result in justice and equity concerns for the local populations. Current reporting practices often require only superficial descriptions of how projects approach justice and equity issues, which are challenging to verify and lack consistency and transparency. So our group has also been working on a framework for assessing justice and equity impacts, started by Miranda Lam. I've also been working with Sophie Chapman and Eleanor Toye Scott on the Legal perspectives on integrity issues in forest carbon. Please do get in touch if you have thoughts on this aspect of project development.
Our preprint on the Terracorder ground sensing platform I've been working with Josh Millar at Imperial on is now available on arXiv. It's a heady combination of ESP32 very low power hardware, combined with Q-learning to build cooperative networks of them that can run for long periods of time without wasting energy on redundant operations. The Terracorder is a versatile multi-sensor device designed for biodiversity monitoring in remote environments. Josh's clever on-device reinforcement learning scheduler captures more than 80% of events at less than 50% of the number of activations of the best-performing fixed schedule. We also explore how a collaborative scheduler can maximize useful operation across a network of devices, improving overall power consumption and robustness - crucial for long-term deployment in places like tropical forests.
Our commentary on nature-based credits has been published in Nature Sustainability. I wrote some thoughts about it here as well. This piece argues that nature-based credit markets are at a critical crossroads - recent impact evaluations showing disappointing results have threatened investor confidence. We make the case that these markets need fundamental reform to adopt the latest scientific understanding on additionality, leakage, and permanence. The key proposal is releasing credits ex-post only after proven demonstrable additionality relative to statistically-derived counterfactuals, and making credit estimation methods robust to rather than resistant to scientific improvements. Without these reforms, we risk losing one of our most promising tools for drawing private investment into conservation.
I've refreshed the set of project ideas for incoming CST Part II and MPhil and PhD student projects for 2024-2025.
\nThese are not an exhaustive list, but intended to kickstart conversations for things we could work on together. Do get in touch if you're an incoming student and see something that grabs your interest.
", "url": "https://anil.recoil.org/notes/student-ideas", "external_url": "/ideas", "title": "New set of ideas for incoming students", "summary": "Explore new project ideas for CST Part II, MPhil, and PhD students starting in 2024-2025.", "date_published": "2024-07-15T00:00:00.000000Z", "date_modified": "2024-07-15T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "cambridge", "research" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/p7kck-5bt81", "content_html": "This is a trip report of ACM COMPASS 2024 held in New Delhi, which had a novel track of "Research to Impact Collaboratives" that drew me in. The general chair, Aadi Seth wrote a fantastic book on "Technology and Disempowerment" a few years ago, and he organised one RIC session on the CoRE Stack -- a climate adaptation stack for rural communities. This was a must-visit for me as it is closely related to the work we've been doing on Remote Sensing of Nature and Planetary Computing. The following notes are somewhat raw as they have only been lightly edited, but please refer to the more polished documents on the agenda for ACM COMPASS RIC and the overall CoRE Stack initiative on commoning technologies for resilience and equality
\nThe conference itself was held at IIIT-D in New Delhi, right at the cusp of the monsoon season and after record-breaking temperatures. Luckily, as always, the hospitality and welcoming nature of New Delhi overrode all the climate discomfort!
\n![\"%c\"](\"/images/compass24/compass24-17.webp\" "\\"Arriving")
The main focus of this report is the one-day RIC held on the 8th July 2024. The RIC had around 60 attendees in person and 40 online, and was a mix of presentations and discussions on the CoRE stack and how it could be used to address climate adaptation in rural communities. The day was divided into two sessions, with the first being a series of scene setting presentations by practitioners and researchers, and the second being a series of breakout discussions on how the CoRE stack could be used in different contexts.
\nData driven approaches enable new approaches to social ecological system health, but need to be grounded in community based approaches, and the scope is too vast for any one group to handle. The CoRE stack (Commoning for Resilience and Equality) is being architected as a digital public infrastructure consisting of datasets, pre-computed analytics, and tools that can be used by rural communities and other stakeholders to improve the sustainability and resilience of their local landscapes. It will enable innovators to build upon and contribute their own datasets, use APIs for third-party apps, and track and monitor socio-ecological sustainability through a systems approach. The CoRE stack broadly consists of four layers.
\n![\"%c\"](\"/images/compass24/compass24-19.webp\" "\\"Getting")
The broad approach is bottom-up usecase discovery, and picking a digital public infrastructure approach to work with civic services with, and to do distributed problem solving across stakeholders in academia, government and business.\nAadi noted the need to balance between standards and design and end-to-end servicing, and the overheads of collaboration across so many people; see the notes on RIC collaboration across people.
\nAadi then described the CoRE stack is a logical layered architecture:
\nTo enable this, Aadi motivated the need to work together with networked co-creation and a\ndigital commons and build on top of it with open licenses. We need to overcome\nfault lines not only in terms of new climate change problems but also\nsocio-ecological barriers. And ultimately we need to go to scale and work with\ngovernment departments to make urgent improvements.
\nAn example of this is water security, via WellLabs Jaltol which allows for\nlandscape characterisation for action pathways and side validation decision\nsupport tools, but also builds community based capacity for social accountability.\nE.g. given a drainage network, if you were to construct a new water body at this\npoint, what would the effect be on downstream water bodies and the communities that depend on it?
\n![\"%c\"](\"/images/compass24/compass24-2.webp\" "\\"The")
Aadi states the goals for this RIC:
\nSlides\n![\"%r\"](\"/images/compass24/compass24-3.webp\" "\\"Abhijeet")
The speaker highlighted the importance of scalability in approaches, particularly in the context of technological applications. Applications must remain simple, grounded in community needs, and usable by the general public. A key problem discussed was the extraction of Above-Ground Biomass (AGB) using smartphone cameras while traversing forested areas. Traditional Lidar-based systems, though effective in providing detailed depth information, are deemed impractical due to the specialised equipment required.
\nThe proposed solution involves creating a Self-Supervised Learning (SSL) model that utilises mobile phones to conduct real-time segmentation of individual trees as one walks through a forest. This approach leverages a pre-trained segmentation model alongside advanced modelling and tagging processes.
\nThe development involves three distinct pipelines, which could be integrated into a single application in the future. Consideration must be given to the UI design to ensure accessibility and effectiveness by rural populations. Advancements in data collection, benchmarking, and pipeline development suggest that such technology could support large-scale forest management initiatives, particularly in public policy contexts. The initial testing phase of this model is being conducted under controlled conditions, including specific lighting and seasonal factors, with plans to extend its applicability.
\nDuring the discussion, a question was raised regarding the allocation of funds for tree planting initiatives and identifying a starting point. Answer: it was suggested that bamboo, a valuable resource for biofuel production, could be a focal point. The Indian landscape has sufficient bamboo to meet current biofuel demand, and directing Corporate Social Responsibility (CSR) funds towards this effort could significantly expedite progress.
\nDuring a break later I showed Srinivasan Keshav's GreenLens mobile app for estimating DBH from a mobile phone image (see app download).
\n![\"%r\"](\"/images/compass24/compass24-4.webp\" "\\"Snehasis")
\nThe presentation by Snehasis Mukherjee focused on plantation monitoring using drone imagery, addressing the limitations of satellite images, esp. their inaccessibility to farmers. The workflow involves using drones at lower altitudes to capture detailed field imagery. The process begins with printing satellite images of a village onto paper, collaboratively marking land use with the locals, and proposing interventions. These are then imported into QGIS by a technical team, followed by field trips to gather further data using GeoODK, which is also integrated into QGIS. This iterative process is intended to inform local policy decisions at the Gram Sabha level.
For drone imagery, the low-cost DJI Mini 2 with an RGB camera was chosen. Heights of 50-100m proved effective for capturing plantation images with sufficient resolution. The use cases include crop area estimation, classification, and monitoring plantation health. The first field trip occurred in Aug 2023 in Vittalpur village near Hyderabad, resulting in 253 usable images at ~50m (mainly of plantations).
\nImage annotation was labor-intensive, with 100 images annotated by the team and 150 outsourced for ₹1000, resulting in approximately 9000 annotations. The Xception and ResNet50 models showed promising results with reduced overfitting, and 2000 acres have been mapped now with multiple tree varieties. The challenge remains on how to supplement limited drone imagery with lower-resolution satellite images, since flying drones is expensive.
\n![\"%r\"](\"/images/compass24/compass24-5.webp\" "\\"Ashish")
\nAshish is building a community participation model to scale agroforestry, aiming to create a feedback/knowledge loop with locals. Goal is to promote tree planting outside traditional forestry areas and restore degraded common lands. The approach involves identifying degraded areas and building a toolkit to recommend suitable tree species.
The project includes several modules: Species Distribution Modelling (SDM), water management, carbon sequestration, and economic analysis. Water management is particularly critical and is informed by research from the Kolar district, which has experienced declining groundwater levels since the 1990s and exacerbated by increasing demand. Remote sensing data shows significant variation in water usage depending on plant type and location (e.g., mango vs eucalyptus).
\nTheir work utilised the SSEBOP evapotranspiration product, accessed via Google Earth Engine (GEE), to analyse water use and its implications for agroforestry efforts.
\n![\"%r\"](\"/images/compass24/compass24-6.webp\" "\\"Siddharth")
Focussing on detecting riverbed sand mining activities using satellite imagery, particularly in areas where on-site visits are impractical. It turns out that sand is the second most extracted material globally after water, and its mining is a significant environmental concern especially for river communities. The project aims to develop a machine learning model to detect such mining activities using S1/S2 (didn't catch which, or both) satellite data.
\nIndia Sand Watch, an open data platform developed with Ooloi Labs, aims to collect annotate and archive data related to sand mining in India. This emerged due to the high costs associated with using detailed satellite imagery and processing and the need to understand sand mining comprehensively. The project covers the entire sand mining process, from discovery and land auctions to clearances and mining, and includes a 'sites of violence' framework that identifies intervention points.
\nSignificant challenge identified was the readability of documents associated with planning, which can be difficult even for humans let alone LLMs, making digitisation and structuring of data crucial. The transition from discovery to the actual mining site often involves navigating poorly accessible documents, highlighting the need for better evidence pipelines. Note to self: just like our Conservation Evidence Copilots project!
\nThey are in collaboration with Berkeley with the aim to develop a machine learning model that predicts mining activity using low-resolution imagery (thus saving costs), covering vast areas (up to 10000 km2+) with Sentinel-1/2 as base maps. Their goal is to combine this data to create large-scale evidence that can then be used to drive large-scale action. This approach has been validated in court, where the data was accepted as evidence by the National Green Tribunal (NGT).
\nQ: is the community getting involved? A: The initiative began with community action, reflecting concerns over sand mining's impact on ecosystems, as sand is the second most extracted material globally after water.
\n![\"%r\"](\"/images/compass24/compass24-7.webp\" "\\"R.")
\nR. Nandakumar (recently retired from ISRO but still working on this space) proposed a new interactive electronic publication standard aimed at improving the quality of information products in communicating research results more interactively. He seeks to integrate code with data, ensuring version control while addressing security and privacy concerns. The current business model, which relies on distracting advertisements, exacerbates the digital divide especially with rural communities and hampers effective communication.
He highlighted several issues with existing formats; inadequate representation of images, maps, infographics, and spreadsheets, and the absence of interactive features like running commentaries during visualisation animations. Also, there is a lack of fine printing and zoom capabilities and flexible authorisation mechanisms.
\nHis proposal suggests evolving existing standards (like PDFs) into more interactive and self-contained formats that include code. First phase would extend 2D image maps to support animations and metadata while embedding free and open-source software within the PDF. The second phase could expand this to include 3D models.
\nThe end goal is to standardise interactions across various formats—image maps, spreadsheets, infographics, animations, and audiovisual content—using the ISO/IEC 25010 square standard, which provides a comprehensive framework for functionality, performance, compatibility, usability, reliability, security, maintainability, and portability. (see slides for more details on each of these)
\nMy mad idea: might we build a WASM interpreter in JavaScript so that it can run inside the existing PDF JS interpreter and work with existing docs? WASI for PDF! I've got a project idea relevant to this that can perhaps be extended or forked; see Using wasm to locally explore geospatial layers.
\n![\"%r\"](\"/images/compass24/compass24-8.webp\" "\\"Craig")
There is an overload of data and algorithms in all directions, so we want to accelerate development of better data and algorithms rather than quantity. How do we increase trust and reduce friction in the source data and eventual results with rural communities?\nExisting domain specific standards do exist, but they either dont exist or aren't widely adopted (see previous talk), especially for natural resource managemen where it can be of different modalities/resolution and some commonality exists but also sector specific extensions are required from current standards to deal with local variability.
\nSo they are surveying data standards and algorithm standards. To consider data standards first, the most successful is Open Street Map. For algorithm standards, there is rapidly adopted services like HuggingFace. But what is the combination of both so that they can be coupled to real outcomes?
\nHow do we compare the performance of data standards and build guiding principles of which ones to pick?
\nAn existing example is how to share a LULC dataset using existing open geospatial standards (STAC). The data standard creates a simple JSON file which has metadata for that module. The data user can then access to eh latest version of the data via either an API or the STAC browser.
\nTODO for myself: Look at mapping these metrics onto our TMF pipeline (in Trusted Carbon Credits) and investigate a possible user study with some CCI data. Also is STAC relevant to TMF/LIFE/FOOD publishing pipeline in Mapping LIFE on Earth as we need to publish the various layers there soon.
\nMy talk, I was speaking, so no notes! I'll upload the slides later and edit this section.
\nGood question from the audience about healthcare management and its relevance to planetary computing -- it seems to share a lot of the problems involving data sensitivity and the need for spatially explicit data sharing.
\nAnupan introduced the main questions across the rural farming cycle including:
\nSo onto the notion of "Agricultural Computing", which:
\nUrban heat islands are a significant problem detectable with geospatial data. He noted the reference of paper by\nMohajerani, Abbas, Jason Bakaric, and Tristan Jeffrey-Bailey. "The urban heat island effect, its causes, and mitigation, with reference to the thermal properties of asphalt concrete." Journal of Environmental Management 197 (2017): 522-538.
\nNote to self: Send to Andres Zuñiga-Gonzalez re Green Urban Equity: Analyzing the 3-30-300 Rule in UK Cities and Its Socioeconomic Implications.
\nQ: For marginalised communities, should there be standards for interactions to obtain feedback iteratively, reducing the shock of policy changes? A: There is a need for significant groundwork engineering right now to provide immediate feedback, helping communities adapt more smoothly to changes.
\nSlides\n![\"%r\"](\"/images/compass24/compass24-9.webp\" "\\"Prachi")
Prachi gave a perspective from the farmer's fields, with a study aiming to group relatively homogenous regions based on soil, climate, and physiography, focusing on moisture availability periods for soil and the length of the growing season. Their approach uses simple moisture sensors at various depths to measure soil resistivity, providing farmers with real-time information on whether to irrigate. This system can map dry spells and their duration, offering actionable insights for crop management.
\nThe Navadhanya system is a traditional cropping method with specific design and crop geometry, which can be analysed for soil moisture as a multidimensional system—both spatially and temporally. Different crops have varying maturity and root depth cycles, making soil moisture critical for establishing and protecting these crops. A fallow period during a critical stage can lead to crop loss and so highlights the importance of consistent moisture.
\nNavadhanya bridges traditional crop mixing knowledge with modern scientific sensor methods as described in the talk. Navadhanya offers nutritional security through crop variety though farmers typically sell a reliable monocrop in the market. Their analysis suggests a need to consider soil use regimes both in the short and long term, challenging the practice of forcing farmers to switch crops (e.g., from rice to bajra) based on short-term profitability.
\nQ: How can this tool assist with monsoon management? A: The tool can map soil moisture and integrate it with traditional knowledge, enabling the development of combined solutions for managing monsoon impacts.
\nSlides\n![\"%r\"](\"/images/compass24/compass24-10.webp\" "\\"Atanu")
\n![\"%r\"](\"/images/compass24/compass24-11.webp\" "\\"The")
Atanu switched tack to the business side of things, focused on switching Farmer Producer Organisations (FPOs), of which there are 10000+ in India, to adopt climate-smart practices. The incentive based approach includes:
\nThe key requirements for obtaining an FPO Land Parcel Rating with their method are:
\nFor climate data, their approach involves using neural network-based chaos forecasting to provide weather predictions in a format useful to farmers. The second half of the presentation went into great detail into their ensemble methods to predict weather patterns, which I didn't note in detail, but see Diffusion models for terrestrial predictions about land use change.
\n![\"%r\"](\"/images/compass24/compass24-12.webp\" "\\"Himani")
\nGroundwater monitoring in India faces significant data scarcity, with only 4886 wells having long-term data in the whole country, averaging just 7 wells per district. To address this 150+ organisations collaborated a few years ago to create an Android app for crowdsourcing groundwater data. Starting with 5000 villages, the project has now expanded to 11000+ villages and is used both pre- and post-monsoon and is revealing substantial fluctuations in water levels.
The app enables users to generate village-level groundwater maps, correlating water level data with geological information to create comprehensive groundwater flow maps, even within individual villages. The process involves measuring water depth from three wells per village, using GPS and mobile devices, and rendering the data on an online platform.
\n![\"%c\"](\"/images/compass24/compass24-sm-ss.webp\" "\\"Soil")
\nThe crowdsourcing presents challenges in data quality, requiring post-processing and filtering. Despite this, the analysis has been highly effective, and the Jaldoot scheme now covers 450000+ villages as of 2023, following extensive lobbying with the Indian government who are now supporting it directly.
In addition to groundwater monitoring, efforts are also focused on community-based ecological health monitoring, including biodiversity, biomass assessment, and pollinator/insect tracking. Four sample watersheds with detailed socio-ecological-economic indicators and over 150 annual monitoring sites are used to track changes in vegetation and species over time. These assessments both reveal valuable insights (e.g., the increased presence of a rare frog in specific watersheds) and are resource-intensive and challenging to scale. Potential solutions include GIS-based platforms, remote sensing, and tools for tracking changes in standing biomass, carbon stock, and biodiversity.
\nNote to self: Possible connection with the iRecord team in the UK to explore applicability of biodiversity data collected?
\nThe project also maps highly infested areas by invasive species, such as the Lantana camara, to focus restoration efforts abd is drawing on data from 150+ sites.
\nQ: what are the next steps? A: going to withdraw the Android app in the next few years, so the government is taking over next after creating a similar app. Declaring the project a success! Q: But will the data remain open for the communities once the government takes over? A: There is a growing widening of the dataset collection (e.g. biodiversity) to refine the datasets for things not yet considered such as ecosystem services. Not clear on the future of the government-run data provenance.
\n![\"%r\"](\"/images/compass24/compass24-13.webp\" "\\"Athithiyan")
\nTheir "LifeLands" system is designed to unlock the productive potential of degraded lands, aiming to mitigate climate impacts through better land use. The digital planning tool they built utilises satellite imagery, public databases, and AI modelling to assess land suitability for regenerative purposes such as solar energy, sustainable water management, or ecological restoration.
The system integrates geospatial and socioeconomic data layers, along with public datasets, to produce an interactive map and report, determining whether land is unused and suitable for intervention. Data collection is facilitated through a mobile app that traces land boundaries using GPS, captures four site photos and a video, and gathers information on land ownership and existing vegetation (shrubs and trees).
\n![\"%r\"](\"/images/compass24/compass24-14.webp\" "\\"Aila")
\nCitizens and community stewards need to be able to understand, analyse and apply various concepts and data around climate change to understand intricacies of socio-economic changes. So how might be simplify complex systems and data to encourage data driven decision making through these interventions? To be successful this needs to be participatory decision making and a reclamation of agency of each of the stakeholders within the system.\nIt is essential for citizens and community stewards to comprehend, analyse, and apply complex concepts and data. The goal is to simplify these systems and data, fostering participatory decision-making and empowering stakeholders to reclaim their agency within the system.
Broad research approach:
\nTo enhance understanding, environmental education and curriculum design can incorporate semi-fictional "case studies" that place users in relatable contexts. This approach increases adoption by breaking the system into modules and using gamification to test concepts. For example, users can explore the concept of 'climate change' as it pertains to their own land and prosperity.
\nIn the analysis phase, it’s crucial to not only graph data but also describe it in ways that participants can relate to their own landscapes. The decision-making process must integrate data-driven insights with existing frameworks. Generative images and brainstorming sessions are used to develop innovative ways to visualise complex data, such as precipitation and climatic variables, in a simple and understandable form.
\nExample Activity: "Set a 15-minute timer and brainstorm all possible ways to present data simply." Consider descriptors like terrain, slopes, plains, rainfall, surface water, MNREGA projects, and agriculture to see how users can better utilise this information.
\nQ: Is 'making data actionable' a priority, and how do we address the tragedy of the commons? A: Yes, systems thinking and collaboration are essential to prevent resource depletion and ensure shared benefits.\nQ: Can this approach scale from smaller to larger communities? A: Yes, by developing microwatershed data and village-level datasets, even large communities can work at much smaller, more precise resolutions.
\n![\"%c\"](\"/images/compass24/compass24-group1.webp\" "\\"The")
After this, we split into groups to discuss the following topics roughly as follows:
\nI'll follow up with more analysis about the outcomes soon, as I'm in touch with Aadi and hopefully we will be working on a project together in the future. But for now, I'll conclude this trip report with great appreciation for Aadi and the hard working volunteers at COMPASS 2024 that made attendance such a pleasure!
\n![\"\"](\"/images/compass24/compass24-18.webp\" "\\"Glorious")
\n![\"\"](\"/images/compass24/compass24-21.webp\" "\\"Spotted")
\n![\"\"](\"/images/compass24/compass24-22.webp\" "\\"Made")
We have made an update to the LIFE biodiversity metric based on reviewer feedback, and are very pleased that it has been accepted for publication early next year as part of a special issue from the Royal Society. Any comments would be most welcome before we submit the final proofs in a few months. The revisions incorporated feedback from peer reviewers and represent an improved version of the methodology. Getting accepted for the Royal Society special issue is particularly exciting as it means the LIFE metric will be published alongside other important contributions to biodiversity science. This metric has already proven useful in our work on food systems and will hopefully become a widely-used tool for quantifying biodiversity impacts across diverse applications.
Bill Sutherland organised a workshop at the CCI on how to bring about an Effectiveness Revolution for transforming conservation into an evidence-driven discipline.
\n\n\nThe aim was to discuss the "Evidence Emergency" (The Wildlife Trusts' term), the urgent need to embed evidence into decision-making and to create additional evidence to fill the considerable gaps in the evidence base, to improve conservation practice.
\n
I gave a talk about our early results with the conservation copilots work.
I talked to the AI@Cam team to discuss our AICN\nproject and what we're planning to do in the Conservation Evidence Copilots team.
\n\n", "url": "https://anil.recoil.org/notes/aicam-interview-ce", "external_url": "https://ai.cam.ac.uk/blog/harnessing-the-power-of-ai-to-help-save-our-planet", "title": "Interview with AI@CAM about conservation", "summary": "Discussing conservation with AI@CAM and using AI to transform biodiversity conservation efforts.", "date_published": "2024-06-09T00:00:00.000000Z", "date_modified": "2024-06-09T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ai", "cambridge", "computerlab", "biodiversity", "evidence", "interview" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-hyper-tropical-mapping-1", "content_html": "Over the last two decades, the University of Cambridge-based project Conservation Evidence has screened more than 1.6 million scientific papers on conservation, as well as manually summarising 8,600+ studies relating to conservation actions. However, the current project’s work is limited by the specialised skills needed to screen and summarise relevant studies. It took more than 75 person years to manually curate the current database and only a few 100 papers can be added each year. By accelerating these efforts, AI has the potential to transform the impact this database has on biodiversity conservation.
\nWhat we’re aiming to do through the ai@cam project – bringing together an interdisciplinary team from across the fields of computer science, ecology, climate and conservation – is to build up models of the world that are really detailed and that can be queried by policy makers to help make informed decisions.
\n-- AI@Cam
\n
A preprint on using hyperspectral sensors to perform tree species identification across the tropics is now available on bioarxiv.
\n\nThis study introduces a new approach for mapping tree species linking a multi-temporal implementation of the CNN method detectree2 to segment tree-crowns from aerial photographs to machine learning classifiers to identify species from hyperspectral data.
\n
I was invited by Mary Sheeran to deliver a keynoted at Lambda Days, and I decided to go along to talk about my work on Programming for the Planet. The conference was a really vibrant crowd and I would definitely go along in future years. It's best summarised via an interview video they took of all the speakers.
", "url": "https://anil.recoil.org/notes/d592bf17-c835-435f-9469-f0f65e926975-1", "title": "Programming for the Planet", "summary": "Planetary computing keynote at LambdaDays conference featuring interview video on biodiversity and satellite sensing work.", "date_published": "2024-05-27T00:00:00.000000Z", "date_modified": "2024-05-27T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", ":plancomp", "interview", "biodiversity", "sensing", "satellites", "fp", "sweden" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-food-life-1", "content_html": "Submitted preprint on quantifying the biodiversity cost of global food consumption for peer review. This work links the LIFE biodiversity metric with food consumption and production data to quantify how different types of food and their production locations impact species extinctions. We discovered that the impact varies widely both across and within foods - in many cases by more than an order of magnitude. Using an opportunity-cost framing, we can estimate the marginal changes in expected extinctions from converting natural vegetation to agriculture or restoring farmland to natural habitat. Despite marked differences in per-capita impacts across countries, there are consistent patterns that could inform everything from national policies to individual dietary choices.
Emily Morris did some great MPhil work here in her Masters on using CNNs with satellite data to do species predictions in South Africa better. She presented it at the ICLR CCAI workshop in Vienna, and is now off to do a PhD at Oxford!
\n\n", "url": "https://anil.recoil.org/notes/2024-sdm-sa-1", "title": "Predicting species using machine learning at CCAI", "summary": "MPhil research on using CNNs with satellite data for species distribution modeling presented at ICLR CCAI workshop in Vienna.", "date_published": "2024-05-01T00:00:00.000000Z", "date_modified": "2024-05-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "machine-learning", "conservation", "climate", "cnn", "remote-sensing" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2024-sdm-sa.pdf", "mime_type": "application/pdf", "title": "Towards Scalable Deep Species Distribution Modelling using Global Remote Sensing" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/swseng", "content_html": "Species distribution models are crucial tools that predict species locations by interpolating observed field data with environmental information. We develop an improved, scalable method for species distribution modelling by proposing a dataset pipeline that incorporates global remote sensing imagery, land use classification data, environmental variables, and observation data, and utilising this with CNN models to predict species presence at higher spatial and temporal resolutions than well-established species distribution modelling methods.
\n
We're still reeling from the shocking and unexpected passing of Ross Anderson\nlast week. You can read a lovely tribute to him by Satnam Singh, and I still getting my thoughts together on all the guidance, advice and prods in the right direction that Ross has given me over the years.
\nIn pragmatic news, I'll be emergency lecturing part of Ross' 1A Software and Security Engineering course here at Cambridge, along with my colleagues Martin, Alastair, Mort and Rob who have all stepped up at short notice. I have absolutely no idea how we'll live up to Ross' standard, but we'll do our very best in his memory!
", "url": "https://anil.recoil.org/notes/swseng", "title": "Teaching 1A Security and Software Engineering", "summary": "Lecturers step in to teach Software and Security Engineering course after Ross Anderson's passing.", "date_published": "2024-04-04T00:00:00.000000Z", "date_modified": "2024-04-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "teaching", "cambridge", "computerlab", "security" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-cc-blockchain-1", "content_html": "Paper on smart contracts for carbon credits at ICBC 2024 in Dublin. This work proposes the PACT stablecoin, which addresses concerns about credibility, scalability, and liquidity in voluntary carbon markets. We combine remote sensing data, modern econometric techniques, and blockchain-based certification and trading to create digital carbon assets against which offsetting claims can be transparently verified. The key innovation is creating a reproducible computational pipeline that not only quantifies CO2 emissions but also allows credits to be pooled based on co-benefits like biodiversity and jurisdictional attributes, increasing liquidity through fungibility. We implemented it on the Tezos blockchain, which is designed for low-cost transactions with minimal environmental impact.
I've done a redesign of my site after about 20 years since the last one back in 2003.\nThe site design is based on my upcoming Bushel content manager, which I'll post about more once I get the data model in place and try it out properly using this site as a guinea pig.
\nNick Ludlam also refreshed his website since we were chatting about how outdated our web presences were, and he also put up a main recoil.org page for the main server.
", "url": "https://anil.recoil.org/notes/bushel-step1", "external_url": "/", "title": "Rolling out a new site design", "summary": "New site design launched after 20 year redesign featuring Bushel content manager preview.", "date_published": "2024-04-01T00:00:00.000000Z", "date_modified": "2024-04-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "selfhosting", "website", "ui", "recoil" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-nbs-risk-1", "content_html": "A new preprint is available on our work on ex-ante pricing models for nature-based solutions. It is currently under review, so any feedback is most welcome! This paper tackles the challenge of variability in the performance of nature-based climate solutions by developing optimal release schedules that balance generating credits with higher permanence ratings against limiting the risk of negative additionality. We use Monte Carlo simulations on both theoretical and real-life projects to show how conservatively anticipating carbon release and issuing additional credits when reality is less pessimistic than projections can provide pragmatic insurance against the inherent uncertainty in these systems.
Abstract on urban biodiversity and human health at EGU 24. This work analyzes the 3-30-300 rule for urban green spaces across major UK cities - the rule states that every resident should be close to at least three trees, every neighborhood should have 30% canopy cover, and every citizen should have access to a public green space within 300m. We employed remote sensing imagery, census data, and machine learning to assess implementation of this rule and found significant disparities, particularly in impoverished areas. The findings emphasize the need for strategic urban planning that fosters both social equity and environmental sustainability in our cities.
Revision of planetary computing preprint. We updated the paper with additional insights and refinements based on feedback from the research community. This revision strengthens the case for planetary-scale infrastructure that can handle global environmental data with proper traceability and reproducibility guarantees. The computing challenges we face in environmental science are fundamentally different from traditional big data problems - we need to handle continuously evolving datasets with complex provenance requirements and access controls. This revision better articulates these unique requirements and the opportunities for the systems research community to contribute to addressing the climate and biodiversity crises.
We won the AI@CAM challenge that was sent out\nUniversity wide to find research projects that use AI to tackle society's biggest challenges.\nOur project on using AI for climate and nature\nis one of the five selected.
\n\n\nThe twin climate and biodiversity crises are two of the world’s most complex challenges to tackle. This project aims to develop AI approaches for bringing together a wide range of datasets and accelerating the collation of information.
\nThis work will provide up to date, relevant and robust information for researchers and decision-makers working on climate and biodiversity conservation – opening up the possibility for more targeted and effective solutions to some of our world’s most pressing climate and biodiversity challenges.
\nAnil Madhavapeddy, AI-deas challenge co-lead, said: 'Mitigating the impacts of climate change while maintaining and restoring biodiversity demands urgent, evidence-based action. We're excited to bring together an interdisciplinary team across computer science, ecology, climate and conservation to use AI to empower decision-makers to equitably tackle the biggest challenge of our generation.'
\n-- AI@CAM
\n
This project is a collaboration between lots of friendly people at Cambridge Zero, the Cambridge Conservation Initiative, Conservation Evidence, the Institute for Computing for Climate Science, Conservation Research Institute, Centre for Landscape Regeneration, Cambridge Centre for Carbon Credits and Cambridge Centre for Earth Observation.
\n![\"%c\"](\"/images/aicn-team-feb24.webp\" "\\"Team")
Paper on uncertainty in climate science in Undone CS. This workshop paper examines how computer science approaches to uncertainty can both help and hinder climate research practices. Patrick led this critical reflection on our own work, exploring the tensions between the way computer scientists typically handle uncertainty - often through abstraction and simplification - and the complex, multifaceted nature of uncertainty in climate and environmental science. It's part of the "Undone Computer Science" workshop series that encourages reflection on the limitations and impacts of computing approaches, which is particularly important when working at the intersection of CS and climate science.
", "url": "https://anil.recoil.org/notes/2024-uncertainty-cs-1", "title": "Uncertainty at scale: how CS hinders climate research", "summary": "Paper examining how computer science approaches to uncertainty impact climate research practices at Undone CS workshop.", "date_published": "2024-02-01T00:00:00.000000Z", "date_modified": "2024-02-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "climate", "uncertainty", "methodology", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2024-uncertainty-cs.pdf", "mime_type": "application/pdf", "title": "Uncertainty at scale: how CS hinders climate research" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/royal-society-newton", "content_html": "I joined the Royal Society Newton International Fellowships\ncommittee to help with selecting bright new scientists from abroad who wish to conduct research in the UK.
\n\n", "url": "https://anil.recoil.org/notes/royal-society-newton", "external_url": "https://royalsociety.org/people/anil-madhavapeddy-36582/", "title": "Joined the Royal Society fellowships committee", "summary": "I joined the Royal Society's Newton International Fellowships committee to help select international researchers for UK-based projects.", "image": "https://anil.recoil.org/images/rs-car-1.768.webp", "date_published": "2023-12-02T00:00:00.000000Z", "date_modified": "2023-12-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "service", "royalsociety", "funding" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-pact-tmf-2", "content_html": "The Newton International Fellowship (NIF) programme provides support for outstanding early career researchers to make a first step towards developing an independent research career through gaining experience across international borders. The fellowships enable researchers to access expertise, gain new perspectives and build long-lasting collaborative relationships.\n -- The Royal Society
\n
We have just released the Tropical Moist Forest v2.0 specification, to update the v1.1 released earlier in the year. There are significant updates to the methodology to better match the scheme described in Realizing the social value of impermanent carbon credits. This revision integrates the permanence valuation framework from our Nature Climate Change paper into the operational methodology. It represents an important evolution as we refined our approach based on both peer review feedback and early implementation experiences. The updates ensure that the practical specification aligns with the theoretical foundation we published, creating a more coherent and scientifically robust system for evaluating forest carbon projects.
On the BBC briefly about the Dawn supercomputer. This was an unexpected media appearance when the new Cambridge supercomputer was announced at the 2023 AI Summit. Dawn represents a significant investment in computational infrastructure for AI research in the UK. While I didn't expect to end up on the BBC discussing supercomputing, it was a good opportunity to highlight the importance of computational resources for advancing both AI research and applications like the environmental and conservation work we're doing. These kinds of facilities are crucial for training large models and processing the massive datasets we work with in planetary computing.
", "url": "https://anil.recoil.org/notes/48a7ab10-3f49-4978-a00f-c26b64c2cae7-1", "title": "BBC report on the new Cambridge supercomputer (\"Dawn\") announced at the 2023 AI Summit", "summary": "BBC interview about the Dawn supercomputer announced at 2023 AI Summit.", "date_published": "2023-11-02T00:00:00.000000Z", "date_modified": "2023-11-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ai", "supercomputing", "interview", "hpc" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-hotnets-sns-1", "content_html": "Paper on spatial networks on DNS at HotNets 2023. Ryan Gibb led this work proposing the Spatial Name System (SNS) that extends DNS to support location-based names and resolution mechanisms. The key insight is that the existing Internet architecture lacks proper support for naming physical locations and resolving them to the diverse addressing mechanisms we use beyond IP addresses. By building on DNS infrastructure, SNS enables integration of spatial names into existing applications and opens up exciting possibilities for sensor networks and augmented reality applications. It's a neat piece of work bridging the gap between cyberspace and physical space.
Our paper on valuing impermanent carbon credits has been published at Nature Climate Change. It has received a bunch of press coverage, including phys.org, cam.ac.uk, and Mirage. The publication follows our July preprint and represents a significant milestone in bringing scientific rigor to carbon credit markets. The press coverage has been really encouraging, with science journalists picking up on the key insight that our framework allows for like-for-like comparisons of diverse carbon projects while generating incentives for safeguarding already-credited carbon. The methodology we developed integrates three substantial advances that together provide a path forward for credible nature-based climate solutions.
The first preprint on our new LIFE metric for global biodiversity is now available. It is under review, so feedback would be very welcome. LIFE stands for Land-cover change Impacts on Future Extinctions and represents a significant computational achievement - we generated global maps at 1 arc-minute resolution covering 29,153 terrestrial vertebrate species. The metric quantifies the marginal changes in expected extinctions from either converting natural vegetation to agriculture or restoring farmland to natural habitat. By coupling the persistence score approach with high-performance computing, we've created a tool that integrates information on species richness, endemism, and past habitat loss, offering unprecedented opportunities to estimate extinction impacts from scales ranging from individual dietary choices to global protected area development.
I was on stage in New York for Mission Possible\nduring NYC Climate Week. I was there with Emily Shuckburgh and we met with a lot of Cambridge alumni who\nare all engaged with climate change related activities -- either directly in their careers, or through a side interest.
\nThe major highlights on the discussions with alumni centred around agency: a lot of them were wondering how to combine the evidence coming\nout Cambridge research and combine it with real policy action. A number of the alumni are obviously highly successful in their individual\ncareers, and so the University helping to glue this together would potentially result in valuable actions that might not otherwise come together.
\nThis reminded me strongly of the discussions we had in Pembroke a little while back when Emily Shuckburgh chaired my talk about "Who's in Charge?" for the William Pitt Seminar where we had very similar discussions at dinner afterwards.
\n\n(See also Cambridge Zero notes on the event, and thanks to CISL.)
", "url": "https://anil.recoil.org/notes/mission-possible", "external_url": "https://www.cam.ac.uk/news/cambridge-zero-highlights-university-efforts-at-climate-week-nyc", "title": "Cambridge Zero highlights University efforts at Climate Week NYC", "summary": "Cambridge University highlights climate efforts at NYC Climate Week.", "date_published": "2023-10-18T00:00:00.000000Z", "date_modified": "2023-10-18T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "computerlab", "cambridge", "climate", "outreach" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-raid-deluminator-1", "content_html": "Paper on DIFC Deluminator interface at RAID 2023. Zahra led this work on information flow tracking for heterogeneous compartmentalized software environments. The key contribution is recognizing that modern systems increasingly use diverse compartmentalization mechanisms - processes, SGX enclaves, TrustZone Trusted Apps, and intra-address space compartments - but existing abstractions assume single-compartment models. Deluminator provides OS abstractions and a userspace framework to enable extensible, fine-grained information flow tracking across these heterogeneous compartments. We implemented it on Linux for both ARM and x86-64, with evaluation showing reasonable overhead (7-29% on average) that makes it practical for real-world use.
Paper on MicroGuards memory API at ACNSW with Zahra Tarkhani. MicroGuards provides lightweight kernel modifications and APIs for fine-grained in-process memory protection and privilege separation in multithreaded applications. Taking advantage of tagged memory support in modern CPUs, MicroGuards enables compartmentalization even on resource-constrained mobile devices with minimal overhead (less than 3.5%) - addressing the challenge of securing applications without requiring heavyweight isolation mechanisms.
Keynoted at ICFP 2023 on Functional Programming for the Planet, giving the opening keynote in Seattle. I discussed how functional programmers could contribute to addressing the climate and biodiversity crises through planetary computing. The talk covered our work on satellite sensing, forest monitoring, and how the principles of functional programming - immutability, composability, type safety - could help build reliable systems for environmental monitoring and conservation at scale.
", "url": "https://anil.recoil.org/notes/981c00b5-32c0-4cac-a387-6c945dfa9934-1", "title": "Functional Programming for the Planet", "summary": "Keynote presentation at ICFP 2023 on functional programming for the planet.", "date_published": "2023-09-05T00:00:00.000000Z", "date_modified": "2023-09-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "fp", "conservation", "biodiversity", "icfp", "climate" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-ocaml-platform-1", "content_html": "We deliver the annual presentation about the OCaml Platform in the OCaml Workshop at ICFP 2023.
\n\n", "url": "https://anil.recoil.org/notes/2023-ocaml-platform-1", "title": "State of the OCaml Platform 2023", "summary": "Annual OCaml Workshop presentation reviewing a decade of progress from opam 1.0 to modern toolchain including Dune and LSP server.", "date_published": "2023-09-01T00:00:00.000000Z", "date_modified": "2023-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "devtools", "testing" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2023-ocaml-platform.pdf", "mime_type": "application/pdf", "title": "State of the OCaml Platform 2023" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-ocaml-eio-1", "content_html": "This paper reflects on a decade of progress and developments within the OCaml Platform, from its inception in 2013 with the release of opam 1.0, to today where it stands as a robust toolchain for OCaml developers. We review the last three years in detail, emphasizing the advancements and innovations that have shaped the OCaml development landscape and highlighting key milestones such as the migration to Dune as the primary build system, and the development of a Language Server Protocol (LSP) server for OCaml.
\n
An update on the OCaml EIO library at the OCaml Workshop 2023. Tom Leonard led this presentation on the release of Eio 1.0, which brings effects-based IO to OCaml 5. This is a big deal for the OCaml ecosystem as it provides a modern approach to concurrent programming using algebraic effects rather than monads or callbacks. The library has been under development for a while and reaching 1.0 is a significant milestone. It's particularly exciting because it takes advantage of OCaml 5's new effect handlers to provide a clean, composable interface for IO operations that feels natural in the language.
", "url": "https://anil.recoil.org/notes/2023-ocaml-eio-1", "title": "Eio 1.0 – Effects-based IO for OCaml 5", "summary": "OCaml Workshop 2023 update on the Eio effects-based IO library for OCaml 5.", "date_published": "2023-09-01T00:00:00.000000Z", "date_modified": "2023-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "eio", "effects", "fp", "concurrency" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2023-ocaml-eio.pdf", "mime_type": "application/pdf", "title": "Eio 1.0 – Effects-based IO for OCaml 5" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-ncc-permanence-1", "content_html": "We have uploaded a preprint of our 4C paper on valuing impermanent carbon credits, by using the Social Cost of Carbon as a basis for a discount function into the future. Comments and feedback are most welcome. This work tackles one of the biggest challenges in nature-based climate solutions: how to value carbon sequestration that might not be permanent. We developed a novel framework that conceptualizes permanence as additionality over time relative to a counterfactual baseline, uses risk-averse estimation of future carbon release, and deploys post-credit monitoring to correct for overly pessimistic forecasts. Our preliminary comparisons suggest that even after fully adjusting for impermanence, nature-based interventions may offer less costly ways of reducing climate damages than more technological solutions.
I was honoured to be included in the OCaml team that won the ACM Programming Languages Software Award for 2023.
\n\n\nThe Association for Computing Machinery (ACM), the world's largest association of computing professionals, today gave the 2023 SIGPLAN Award to a group of developers for their work on the functional programming language OCaml.
\nThe award was presented at the annual SIGPLAN Programming Language Design and Implementation Conference to a group of researchers and developers including our colleague Anil Madhavapeddy, Professor of Planetary Computing here.
\nThe prestigious Programming Languages Software Award is given annually "to an institution or individual(s) to recognise the development of a software system that has had a significant impact on programming language research, implementations, and tools," ACM says.
\n\n
See also the main ACM Award Page citation:
\n\n", "url": "https://anil.recoil.org/notes/acm-sigplan-award", "external_url": "https://www.cst.cam.ac.uk/news/acm-programming-languages-software-award-goes-ocaml-researchers", "title": "OCaml wins the ACM Programming Language Software award", "summary": "OCaml wins 2023 ACM Programming Languages Software Award for its impact on research and tools.", "date_published": "2023-06-19T00:00:00.000000Z", "date_modified": "2023-06-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "award", "acm", "icfp" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2023-pact-tmf-1", "content_html": "The OCaml Compiler Distribution is the reference implementation of the OCaml language, a dialect of ML that aims to be pragmatic, both in language features and implementation, encouraging a simple programming style that yields good performance and usability. It has a large user base in industry, research, and education throughout the world, and was used to implement a number of other impactful systems, notably in verification: Coq proof assistant, CompCert verified compiler, Why3 verified programming environment, Frama-C, Astrée and Gillian static analyzers, Infer, Hack and Flow projects at Meta, SLAM/SDV and F* at Microsoft, etc.\n-- ACM SIGPLAN
\n
We have just published the Tropical Moist Forest v1.0 specification, which is a detailed description of the methodology we are using for counterfactual dynamic baselines to calculate the additionality, leakage and permanence behind REDD+ projects. I explained some of the background behind this in a seminar last year. This specification operationalizes the theoretical framework we developed for the Cambridge Center for Carbon Credits (4C), translating research into a practical methodology that can be applied to real-world forest conservation projects. It's a comprehensive document that lays out how to use satellite data and econometric techniques to establish what would have happened to a forest in the absence of a conservation project, which is crucial for determining whether carbon credits genuinely represent additional climate benefits.
\nOur perspective in Science magazine appeared this week on the credibility of carbon credits and its importance for tropical forest protection. This was a collaborative effort with colleagues from across conservation science, economics, and computer science to address one of the most pressing issues in climate finance. We argue that improving the quantification methods behind carbon credits is not just a technical issue but an existential one for forest protection. If credits don't accurately represent their environmental benefits, the entire voluntary carbon market risks collapse, which would be catastrophic for tropical forests that depend on this financing. The piece calls for robust, scientifically-grounded methodologies to ensure that carbon credits can genuinely contribute to climate mitigation rather than providing false assurance.
\n\nAddressing global warming requires increased investment in conserving and restoring carbon-dense natural habitats. Some companies that emit carbon have turned to certified carbon credits to offset their environmental impact. However, the effectiveness of carbon credits depends on the methods used to quantify them. If carbon credits do not accurately represent their environmental benefits, relying on them could exacerbate climate change. To ensure that carbon credits are robust, the methods used to calculate them must be improved.
\n
Discussion with Mantle Labs about carbon credits at the Forestry and Agriculture Summit in London. On stage with Jon Pierre, we discussed how scientific innovation and AI could help scale carbon markets. The conversation explored satellite-driven approaches to calculating additionality and permanence in forest carbon projects, addressing the credibility challenges facing the voluntary carbon market. This work aims to use technology to create more transparent and verifiable carbon credits.
", "url": "https://anil.recoil.org/notes/ce64a918-ff52-4116-b1ee-256f08e6e7f1-1", "title": "Leveraging Scientific Innovation and AI to Scale Carbon Markets", "summary": "Discussion with Mantle Labs exploring applications of AI to carbon credits markets.", "date_published": "2023-03-07T00:00:00.000000Z", "date_modified": "2023-03-07T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "carbon-credits", "ai", "conservation", "climate" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2024-planetary-computing-1", "content_html": "Preprint of planetary computing paper. Patrick led this work making the case for infrastructure to handle the ingestion, transformation, analysis, and publication of global data products for environmental science and policy-making. Drawing on our experiences working with environmental scientists on forest carbon and biodiversity preservation, we classify existing solutions by their flexibility in processing geospatial data and their support for building trust through traceability and reproducibility. The paper identifies research gaps around handling continuously changing datasets collected across decades that require careful access control. It's a call to action for the computing community to build better infrastructure for planetary-scale environmental data.
I recap the OCaml community progress in 2022, which covers a number of bases ranging from\nthe release of OCaml 5.0, the launch of a new website with integrated documentation for 20000+ packages, prototyping new developer workflows that are better integrated into editors, and the launch of ActivityPub based services such as https://watch.ocaml.org.
", "url": "https://anil.recoil.org/notes/recapping-ocaml-22", "external_url": "https://discuss.ocaml.org/t/ocaml-org-recapping-2022-and-queries-on-the-fediverse/11099/1", "title": "OCaml.org: recapping 2022 and queries on the Fediverse", "summary": "Recapping OCaml's 2022 progress and exploring Fediverse integration.", "date_published": "2023-01-02T00:00:00.000000Z", "date_modified": "2023-01-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "opensource", "selfhosting", "fediverse" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/bc9da6fc-9419-4f18-9db9-c13b1a4a859f-1", "content_html": "Wednesday seminar on financing forests using carbon credits at the Cambridge Computer Lab. I presented our work at 4C on applying computer science and satellite sensing to tropical deforestation. The talk explored how technology could help create credible carbon credit markets by using satellite data to measure additionality and permanence, potentially providing financial mechanisms to help avert both the biodiversity and carbon crises.
", "url": "https://anil.recoil.org/notes/bc9da6fc-9419-4f18-9db9-c13b1a4a859f-1", "title": "Financing Forests: A Credible Approach towards Halting Tropical Deforestation", "summary": "Wednesday seminar presentation on using carbon credits to finance tropical forest conservation.", "date_published": "2022-11-16T00:00:00.000000Z", "date_modified": "2022-11-16T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "carbon-credits", "conservation", "forests", "climate" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/a26475b5-c169-478e-b88e-be5cd1f2aff8-1", "content_html": "I opened the 17th William Pitt Seminar at Pembroke College on climate change with a brief talk about the status of the world's biodiversity, and how we have more agency than ever before to take matters into our own hands.
", "url": "https://anil.recoil.org/notes/a26475b5-c169-478e-b88e-be5cd1f2aff8-1", "title": "17th William Pitt Seminar - Who's in Charge?", "summary": "Opening talk at Pembroke College's climate change seminar on global biodiversity status and individual agency for conservation action.", "date_published": "2022-11-01T00:00:00.000000Z", "date_modified": "2022-11-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "conservation", "biodiversity", "climate", "seminar", "agency" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/rwo-2", "content_html": "I'm delighted to report that the second edition of Real World OCaml is now available from Cambridge University Press! It's also freely available online, and CUP also kindly agreed that the PDF version could be freely available online thanks to sponsorship from Tarides.
\n![\"%c\"](\"/images/rwo-window-1.webp\" "\\"The")
![\"%c\"](\"/images/rwo-fans-1.webp\" "\\"I")
![\"%c\"](\"/images/rwo-fans-2.webp\" "\\"I")
As always, if you have any feedback about the book, please post it on https://github.com/realworldocaml/book.
Paper on our incremental computation DSL OCurrent presented at OCaml Workshop 2022 with Tim McGilchrist, David Allsopp, Patrick Ferris, and others. OCurrent provides a declarative way to express CI/CD pipelines as incremental computations that automatically track dependencies and cache results. Combined with OBuilder for reproducible builds, it enables homogeneous build infrastructure across different platforms - the foundation powering OCaml-CI and other OCaml ecosystem infrastructure.
", "url": "https://anil.recoil.org/notes/2022-oud-ocurrent-1", "title": "Homogeneous Builds with OBuilder and OCaml", "summary": "Paper on OCurrent incremental computation DSL presented at OCaml Workshop 2022.", "date_published": "2022-09-01T00:00:00.000000Z", "date_modified": "2022-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "build-systems", "ci-cd", "tooling" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/4c-1", "content_html": "With the recent controversies over low-integrity carbon credits, I spoke to Vox magazine\nabout my skepticism about Adam Neumann's new startup.
\n\n", "url": "https://anil.recoil.org/notes/4c-1", "title": "Trusted Carbon Credits", "summary": "Interview in Vox magazine expressing skepticism about Adam Neumann's carbon credit startup, emphasizing supply shortage over trading efficiency.", "date_published": "2022-05-06T00:00:00.000000Z", "date_modified": "2022-05-06T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "carbon-credits", "conservation", "interview", "climate" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/multicore-monthly-mar22", "content_html": ""The problem with the current markets is nothing to do with how we can trade these more effectively," said Anil Madhavapeddy, who is an associate professor of computer science and technology at Cambridge University and the director of the Cambridge Center for Carbon Credits. "We just do not have enough supply."\n-- Vox
\n
We're getting closer to a stable release of OCaml 5.0, including reenabling support for the BSDs and introducing ARM64 multicore support.
", "url": "https://anil.recoil.org/notes/multicore-monthly-mar22", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-march-2022/9692", "title": "OCaml Multicore Monthly: heading towards OCaml 5.0", "summary": "OCaml 5.0 approaches with BSD support and ARM64 multicore.", "date_published": "2022-04-19T00:00:00.000000Z", "date_modified": "2022-04-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "multicore" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/multicore-monthly-jan22", "content_html": "After we got the massive OCaml 5.0 pull request merged, we've taken some time to consolidate the trunk branch of OCaml and start down the release path towards getting OCaml 5.0 out of the door.
", "url": "https://anil.recoil.org/notes/multicore-monthly-jan22", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-january-2022-and-post-merge-activity/9294", "title": "OCaml Multicore Monthly: post merge activites", "summary": "OCaml 5.0 merge complete, focusing on release preparation.", "date_published": "2022-02-09T00:00:00.000000Z", "date_modified": "2022-02-09T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "multicore", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2022-enhancing-brain-security-1", "content_html": "Preprint on security vulnerabilities in brain-computer interfaces with Zahra Tarkhani, Lorena Qendro, and colleagues from Cambridge. This fascinating work analyzed security threats to wearable BCI devices from both an OS and adversarial ML perspective, discovering over 300 vulnerabilities across six attack vectors in real devices like Muse, NeuroSky, and OpenBCI. We introduced Argus, an information flow control system that mitigates these attacks with acceptable overhead - critical for protecting users' brainwave data and preventing remote attackers from compromising BCI-assisted devices.
We've been working hard on OCaml multicore support, and went over to Paris to sit down with some core developers from Inria and work through code review of our proposed patches.
", "url": "https://anil.recoil.org/notes/multicore-monthly-dec21", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-november-2021-with-results-of-code-review/8934", "title": "OCaml Multicore Monthly: code review complete with Inria", "summary": "OCaml multicore support code review completed with Inria developers.", "date_published": "2021-12-21T00:00:00.000000Z", "date_modified": "2021-12-21T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "multicore", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/4c-launch", "content_html": "I launched 4C recently, and Pembroke College covers the launch with an interview with me.
\n\n", "url": "https://anil.recoil.org/notes/4c-launch", "external_url": "https://www.pem.cam.ac.uk/college/corporate-partnership/25th-anniversary-corporate-partnership-programme/25th-anniversary-11", "title": "Launching the Cambridge Centre for Carbon Credits", "summary": "Introducing the Cambridge Centre for Carbon Credits to combat climate change and biodiversity loss.", "date_published": "2021-11-04T00:00:00.000000Z", "date_modified": "2021-11-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "pembroke", ":4c", "carboncredits", "interview" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/signals-and-threads", "content_html": "The world is facing a large-scale environmental crisis. Two parallel and related strands of this are, first, the crisis in biodiversity and the rapid extinction of many species, recently addressed at the COP15 UN Biodiversity Conference in October, and second, the threat of climate change, the topic of last month’s COP26 summit in Glasgow. Pressure is growing on governments to execute nature-based solutions which will offset some of the most damaging impacts of these crises. While COP26 built some momentum, there is still a long way to go to turn promises into lasting change. More engagement with the private sector is urgently needed.
\nThe solution to the crisis is two-pronged: we must engage in behaviour change to reduce unnecessary harmful emissions, and also invest in nature-based solutions at global scales to not only reduce, but ultimately reverse the effects of climate change and biodiversity loss.\n-- Pembroke College
\n
I am the latest person to feature on the first season of the Signals and\nThreads podcast\nhosted by Yaron Minsky (you may recognise him as my co-author on Real World OCaml).
\n\n\nAnil Madhavapeddy is an academic, author, engineer, entrepreneur, and OCaml aficionado. In this episode, Anil and Ron consider the evolving role of operating systems, security on the internet, and the pending arrival (at last!) of OCaml 5.0. They also discuss using Raspberry Pis to fight climate change; the programming inspiration found in British pubs and on Moroccan beaches; and the time Anil went to a party, got drunk, and woke up with a job working on the Mars Polar Lander.\n-- Signals and Threads
\n
I think I might be the first non- Jane Street person to be on their podcast! Quite the honour.
We're making steady progress on getting multicore support merged into OCaml, including some great developer meetings where we achieved consensus with the core team to include support for effect handlers in the 5.0 release.
", "url": "https://anil.recoil.org/notes/multicore-monthly-sep21", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-september-2021-effect-handlers-will-be-in-ocaml-5-0/8554", "title": "OCaml Multicore Monthly: effect handling confirmed for 5.0", "summary": "OCaml 5.0 to include effect handling and multicore support.", "date_published": "2021-10-01T00:00:00.000000Z", "date_modified": "2021-10-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "opensource", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/forests", "content_html": "I track external notes and media articles here on forest preservation and\nrestoration as part of my work on Trusted Carbon Credits. Not complete, just a reading list.
\nNick Ludlam and I have self-hosted recoil.org since around 1996, typically for\nemail and web. These days, there are a number of interesting software stacks\naround decentralised communication that we deploy. This note keeps track of\nthem.
\nIn general, our operating system of choice is OpenBSD (since 1998 or so) with\nAlpine Linux for the more recent things that run on a cloud or haven't been\nported yet.
", "url": "https://anil.recoil.org/notes/decentralised-stack", "title": "Decentralised tech on Recoil", "summary": "Recoil's decentralized tech stack includes email, web, and chat services.", "date_published": "2021-09-19T00:00:00.000000Z", "date_modified": "2021-09-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "recoil", "selfhosting", "openbsd", "opensource", "security" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/roadmap-ocamlorg-v3", "content_html": "After a decade of good service, it's time to overhaul OCaml's online presence\nto more modern technologies. This post lays out the roadmap for the third\nedition of the OCaml.org website.
", "url": "https://anil.recoil.org/notes/roadmap-ocamlorg-v3", "external_url": "https://discuss.ocaml.org/t/v3-ocaml-org-a-roadmap-for-ocamls-online-presence/8368", "title": "Roadmap for OCaml's online presence", "summary": "Overhauling OCaml's online presence with a modern new website.", "date_published": "2021-08-27T00:00:00.000000Z", "date_modified": "2021-08-27T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "selfhosting" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2021-arxiv-forestrycs-1", "content_html": "Preprint about our working notes on how CS might contribute to forest preservation with Gemma Gordon, Amelia Holcomb, Tom Kelly, Srinivasan Keshav and Jon Ludlam. This was a departure from our usual systems work, exploring how computational techniques could aid forest restoration efforts - tackling the interlinked crises of climate change and biodiversity loss. The paper outlined opportunities for computer science to contribute to reforestation, from drone-based sensing to data management for restoration projects.
Paper on programming with effects in OCaml at the OCaml Workshop with Thomas Leonard, Craig Ferguson, Patrick Ferris, Sadiq Jaffer, Tom Kelly and KC Sivaramakrishnan. This shared practical experiences from actually using effect handlers in real systems, now that multicore OCaml with effects was becoming available. The lessons learned helped refine the design and informed best practices for effect-based programming in production code.
", "url": "https://anil.recoil.org/notes/2021-oud-effects-1", "title": "Experiences with Effects", "summary": "Paper on programming with effects in OCaml presented at OCaml Workshop.", "date_published": "2021-08-01T00:00:00.000000Z", "date_modified": "2021-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "effect-handlers", "fp", "programming-languages" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2021-oud-effects.pdf", "mime_type": "application/pdf", "title": "Experiences with Effects" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2021-pldi-retroeff-1", "content_html": "Paper on retrofitting effects in OCaml presented at PLDI 2021 with KC Sivaramakrishnan, Stephen Dolan, Leo White, Tom Kelly and Sadiq Jaffer. This work complemented our retrofitting parallelism paper, showing how algebraic effect handlers could be added to OCaml's runtime while maintaining backwards compatibility. Effect handlers are the key mechanism enabling lightweight concurrency in multicore OCaml, and getting the design and implementation right was critical for the language's evolution.
Paper on Banyan for coordination-free distributed transactions at ASPLAS 2020 with Shashank Shekhar Dubey, KC Sivaramakrishnan and Thomas Gazagnaire. Banyan addressed a key limitation of existing geo-distributed and IoT systems: they couldn't afford coordination latency but still needed transactions. The paper showed how mergeable replicated data types built on Irmin could support distributed transactions without coordination, while remaining composable - something previous CRDT approaches couldn't achieve.
Annual update on the OCaml Platform at the OCaml Workshop, the 2020 edition of our ongoing series. This year's update covered the continued evolution of the toolchain, including improved testing infrastructure, better cross-platform support, and preparations for the coming multicore OCaml changes. These yearly checkpoints help ensure the platform evolves in sync with the language itself.
", "url": "https://anil.recoil.org/notes/2020-oud-platform-1", "title": "The OCaml Platform: 2020", "summary": "Annual OCaml Workshop update on the OCaml Platform development toolchain.", "date_published": "2020-09-01T00:00:00.000000Z", "date_modified": "2020-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "devtools", "platform" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/b11188ba-0f97-4ec4-b372-fa3cea0821ab-1", "content_html": "Talk on the state of the OCaml Platform in 2020, outlining the next steps for OCaml development tools and infrastructure. Delivered online during the pandemic, the talk covered improvements to the OCaml toolchain, IDE support, documentation, and package management. I discussed our ongoing work at OCaml Labs to make OCaml more accessible to developers and strengthen the ecosystem with better developer experience.
", "url": "https://anil.recoil.org/notes/b11188ba-0f97-4ec4-b372-fa3cea0821ab-1", "title": "State of the OCaml Platform 2020", "summary": "Talk on next steps for the OCaml Platform development tools, delivered online during the pandemic.", "date_published": "2020-08-28T00:00:00.000000Z", "date_modified": "2020-08-28T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling", "devtools", "community" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/multicore-monthly-sep20", "content_html": "The big advance in the multicore OCaml branch is that we restored compatibility\nwith the traditional OCaml systhreads. This in turn means that many existing\nsoftware packages just work out of the box on the new runtime.
\n\n", "url": "https://anil.recoil.org/notes/multicore-monthly-sep20", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-september-2020/6565", "title": "OCaml Multicore Monthly: systhreads compatibility merged", "summary": "OCaml Multicore now supports systhreads compatibility for seamless integration.", "date_published": "2020-08-20T00:00:00.000000Z", "date_modified": "2020-08-20T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "opensource", "multicore" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2020-icfp-retropar-1", "content_html": "Big news this month is that the systhreads compatibility support PR has been\nmerged, which means that Dune (and other users of the Thread module) can\ncompile out of the box. You can now compile the multicore OCaml fork\nconveniently using the new opam compiler plugin (see announcement).\n-- me, on the discussion forum
\n
Won best paper award at ICFP 2020 for our paper on retrofitting parallelism onto OCaml! This was the culmination of years of work with KC Sivaramakrishnan, Stephen Dolan, Leo White and the multicore team. The paper presented a mostly-concurrent garbage collector that maintained backwards compatibility for sequential code while enabling true parallelism. The achievement was maintaining both feature compatibility and performance for existing single-threaded programs while scaling admirably on multicore processors - a balancing act that required novel GC techniques and careful engineering.
Paper on how to parallelise OCaml code at the OCaml Workshop with Sadiq Jaffer, Sudha Parimala, KC Sivaramakrishnan and Tom Kelly. As multicore OCaml became available, we needed practical guidance for developers on how to actually use the new parallel features. This paper provided concrete techniques and patterns for parallelizing existing OCaml codebases, helping the community prepare for the multicore transition.
", "url": "https://anil.recoil.org/notes/2020-oud-parallelising-1", "title": "Parallelising your OCaml Code with Multicore OCaml", "summary": "OCaml Workshop paper on techniques for parallelizing OCaml code using multicore features.", "date_published": "2020-08-01T00:00:00.000000Z", "date_modified": "2020-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "multicore", "parallel", "fp", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2020-oud-parallelising.pdf", "mime_type": "application/pdf", "title": "Parallelising your OCaml Code with Multicore OCaml" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2020-oud-ci-1", "content_html": "Presented the new OCaml DSL for continuous integration at the OCaml Workshop with Thomas Leonard, Craig Ferguson, Kate Deplaix and Magnus Skjegstad. OCaml-CI automatically extracts metadata from opam and dune files to determine what to build, uses aggressive caching for speed, and tests across multiple OCaml versions and platforms. Deployed on around 50 GitHub projects, it delivered response times an order of magnitude faster than less integrated CI solutions - demonstrating the benefits of deep integration with the OCaml ecosystem.
", "url": "https://anil.recoil.org/notes/2020-oud-ci-1", "title": "OCaml-CI: A Zero-Configuration CI", "summary": "Presentation of new OCaml DSL for continuous integration at the OCaml Workshop.", "date_published": "2020-08-01T00:00:00.000000Z", "date_modified": "2020-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "ci", "devops", "dsl", "testing" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/multicore-monthly-apr20", "content_html": "In the April OCaml multicore monthly, we have a preprint available of our ICFP submission about the OCaml 5 multicore runtime.\n(Update: This paper actually won the ICFP best paper award later in the year! Read it at "Retrofitting parallelism onto OCaml").
Part 2 of my distinguished lecture series at St Andrews, discussing how unikernels had reached real-world deployment at scale. This talk explored actual deployments reaching billions of users, demonstrating that unikernel technology had moved beyond academic research into production systems. I covered lessons learned from deploying MirageOS in embedded systems and cloud infrastructure, and the future potential for unikernels in mainstream computing.
", "url": "https://anil.recoil.org/notes/c09ed36f-6ad5-4254-a0ce-3ca3398f38a3-1", "title": "The First Billion Real Deployments of Unikernels", "summary": "Discussion of unikernel deployments at scale reaching billions of users.", "date_published": "2020-02-26T00:00:00.000000Z", "date_modified": "2020-02-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "deployment", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/d456e4bc-bce6-45ad-9d2e-102f834ec400-1", "content_html": "Delivered the distinguished seminar series at St Andrews on rebuilding Operating Systems with functional principles. Part 1 of the lecture series explored how functional programming concepts could be applied to systems design, using MirageOS as a case study. I discussed how immutability, type safety, and composability from functional languages could help build more secure and reliable operating systems, moving away from the complexity and vulnerabilities of traditional OS architectures.
", "url": "https://anil.recoil.org/notes/d456e4bc-bce6-45ad-9d2e-102f834ec400-1", "title": "Rebuilding Operating Systems with Functional Principles", "summary": "Distinguished lecture series at University of St Andrews on functional programming principles applied to operating systems design.", "date_published": "2020-02-26T00:00:00.000000Z", "date_modified": "2020-02-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "fp", "systems", "unikernels", "mirageos", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/287364fa-b59c-4b9f-812d-d81cc0c992a5-1", "content_html": "Part 3 of my distinguished lecture series at St Andrews on functional foundations for operating systems. This talk explores the programming challenges we'll face with the coming wave of embedded devices - likely trillions of them over the next decade. The series at St Andrews was a great opportunity to present a comprehensive vision for how functional programming approaches, particularly OCaml and unikernels, can address the unique constraints of embedded systems. These devices need to be secure, resource-efficient, and easy to deploy at scale, which makes traditional OS approaches challenging.
", "url": "https://anil.recoil.org/notes/287364fa-b59c-4b9f-812d-d81cc0c992a5-1", "title": "Programming the Next Trillion Embedded Devices", "summary": "Talk on programming approaches for the coming wave of embedded devices.", "date_published": "2020-02-26T00:00:00.000000Z", "date_modified": "2020-02-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "embedded", "systems", "iot", "programming", "fp" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/multicore-monthly-jan20", "content_html": "We started the process of upstreaming our multicore OCaml branch to mainline OCaml, and so I started posting regular updates to the community forum.
\n\n", "url": "https://anil.recoil.org/notes/multicore-monthly-jan20", "external_url": "https://discuss.ocaml.org/t/multicore-ocaml-january-2020-update/5090", "title": "OCaml Multicore Monthly: starting upstream to OCaml", "summary": "Upstreaming multicore OCaml to mainline OCaml has begun, with opportunities for community contribution through testing and feedback.", "date_published": "2020-01-29T00:00:00.000000Z", "date_modified": "2020-01-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "multicore", "ocaml", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/openbsd-hosting", "content_html": "The most common question we get is how to contribute to the overall multicore effort. As I noted last year, we are now in the process of steadily upstreaming our efforts to mainline OCaml. Therefore, the best way by far to contribute is to test for regressions or opportunities for improvements in the patches that are outstanding in the main OCaml repository.\n-- me, on the discussion forum
\n
I asked on Twitter about hosting options for OpenBSD on cloud providers, so that we could have some alternative options for Recoil. We have a strong preference for bare-metal and not VMs when it comes to OpenBSD. Options that came back were:
\nAnnual update on the OCaml Platform in 2019 with Gemma Gordon at the OCaml Workshop. By this point, the platform had evolved considerably with major improvements to tooling, better IDE support through LSP integration, and growing adoption of modern best practices. These regular updates help track progress and set priorities for the community-driven tooling effort.
", "url": "https://anil.recoil.org/notes/2019-ocaml-platform-1", "title": "The OCaml Platform in 2019", "summary": "Annual update on OCaml Platform development and ecosystem progress.", "date_published": "2019-08-01T00:00:00.000000Z", "date_modified": "2019-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2019-mirage-build-1", "content_html": "Paper on the MirageOS 4 build system at OCaml Workshop with Lucas Pluvinage, Romain Calascibetta and Rudi Grinberg. MirageOS 4 introduced practical build systems for exotic targets like embedded devices and specialized hypervisor backends. The work integrated with dune, OCaml's modern build system, making it much easier to configure and build unikernels for different deployment targets - a major step toward making unikernel development accessible to more developers.
", "url": "https://anil.recoil.org/notes/2019-mirage-build-1", "title": "MirageOS 4: the dawn of practical build systems for exotic targets", "summary": "Paper on the MirageOS 4 build system presented at OCaml Workshop.", "date_published": "2019-08-01T00:00:00.000000Z", "date_modified": "2019-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "ocaml", "build-systems", "unikernels" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2019-mirage-build.pdf", "mime_type": "application/pdf", "title": "MirageOS 4: the dawn of practical build systems for exotic targets" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2019-mirage-functors-1", "content_html": "Preprint on programming unikernels with ML modules, exploring functor-driven development with Gabriel Radanne, Thomas Gazagnaire, Jeremy Yallop, Richard Mortier and others. The paper tackled the configuration matrix problem - realistic unikernel applications depend on hundreds of libraries, each with different requirements across heterogeneous platforms. We showed how OCaml's module system, particularly functors, could cleanly separate configuration from application logic, enabling modular composition and leveraging link-time optimization for efficiency.
Paper on a framework to rearchitect applications for better TEE support at EdgeSys 2019 with Zahra Tarkhani and Richard Mortier. Snape (named after the Harry Potter character who mastered the "dark arts") provided tools for handling heterogeneous trusted execution environments. The framework helped developers partition applications across different TEE technologies like Intel SGX, ARM TrustZone, and others, addressing the practical challenge that edge devices often have diverse security capabilities.
Managing package manager constraints is getting difficult, particularly given the growth of the number of packages in the opam repository. I'm therefore laying out a new mechanism for the OCaml contributors to submit large package sets, such as those from Jane Street.
", "url": "https://anil.recoil.org/notes/ocaml-opam-new-layout", "external_url": "https://discuss.ocaml.org/t/experimenting-with-a-new-opam-repository-release-strategy-for-large-libraries/2918", "title": "New opam repository layout for large libraries", "summary": "Introducing a new opam repository layout for managing large libraries and packages.", "date_published": "2018-11-19T00:00:00.000000Z", "date_modified": "2018-11-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2018-socp-modular-ffi-1", "content_html": "Journal paper on building modular foreign function interfaces in Science of Computer Programming with Jeremy Yallop and David Sheets. This expanded our FLOPS work into a comprehensive treatment of how to structure FFI systems modularly. The approach separated the what (function specifications) from the how (binding mechanisms), enabling the same specification to be used with different backends - a pattern that has since influenced FFI design in other languages.
Paper on the OCaml memory model and underlying theory at PLDI 2018 with Stephen Dolan and KC Sivaramakrishnan. This work was crucial for multicore OCaml, defining the formal memory model that specifies what behaviors are legal for concurrent programs. The paper presented novel techniques for bounding data races in both space and time, providing the theoretical foundation needed to ensure multicore OCaml programs behave predictably while still achieving good performance on modern hardware.
Paper on concurrent systems programming with effect handlers at TFP 2017, expanding on our ML Workshop work. This paper with Stephen Dolan, Spiros Eliopoulos, Daniel Hillerstrom, KC Sivaramakrishnan and Leo White demonstrated that effect handlers in Multicore OCaml could elegantly express difficult concurrent systems programs without performance compromises. We showed that highly concurrent web servers built with effect handlers performed on par with heavily optimized monadic concurrency libraries, while maintaining the simplicity of direct-style code - a significant practical achievement.
Paper on the interspatial networking architecture at HotPOST 2018 with KC Sivaramakrishnan, Gemma Gordon and Thomas Gazagnaire. OSMOSE inverted the typical cloud-centric model by designing an OS for extremely low-latency local computation in physical spaces. The architecture used unikernels and Irmin to provide secure, high-bandwidth connectivity between physical spaces rather than relying on remote datacenters. This addressed the data security, latency and reliability issues inherent in shipping everything to the cloud.
I'm delighted to report that I'm helping my long-time collaborator Thomas Gazagnaire to return to his OCaml roots from Docker. He has just founded Tarides, a startup in Paris with the goal of advancing the open source OCaml ecosystem.
\n\n", "url": "https://anil.recoil.org/notes/founded-tarides", "external_url": "https://discuss.ocaml.org/t/tarides-is-looking-for-software-engineers-to-work-on-mirageos-and-irmin/1690", "title": "Founded Tarides and looking to hire OCaml hackers", "summary": "Tarides, a Paris-based startup, seeks OCaml hackers to build secure network services with MirageOS and Irmin.", "date_published": "2018-03-02T00:00:00.000000Z", "date_modified": "2018-03-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "startups", "france", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2f824dde-e112-4f4f-890d-1825572ea1c4-1", "content_html": "Founded in Paris in early 2018, Tarides helps developers and companies build secure, performant and resource-efficient network and storage services. We are using MirageOS to run applications without the overhead of a traditional operating system and Irmin to create scalable distributed applications. Tarides offers commercial support and commercial development services for companies interested to run MirageOS or Irmin as part of their technology stack.\n -- Thomas Gazagnaire
\n
Talk on the state of the OCaml Platform at the 2017 OCaml Workshop. This was my annual update to the community on the progress of the OCaml Platform - the collection of tools and libraries that make up the OCaml development experience. By 2017, we were seeing real momentum with opam, the package manager, becoming widely adopted, and early work on better editor integration and build tools. These annual talks at the OCaml Workshop serve as both progress reports and opportunities to gather feedback from the community on what tooling improvements matter most to practitioners.
", "url": "https://anil.recoil.org/notes/2f824dde-e112-4f4f-890d-1825572ea1c4-1", "title": "State of the OCaml Platform", "summary": "Talk presenting the current state of the OCaml Platform toolchain and development ecosystem.", "date_published": "2017-09-08T00:00:00.000000Z", "date_modified": "2017-09-08T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling", "devtools", "community" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2017-oud-platform-1", "content_html": "Annual update on the OCaml Platform at ICFP, continuing the tradition of yearly status reports on the OCaml development toolchain. By 2017, the platform had matured significantly with improved editor integration, better documentation tools, and more robust CI infrastructure. These yearly talks help coordinate the community's efforts and ensure the tools evolve together coherently.
", "url": "https://anil.recoil.org/notes/2017-oud-platform-1", "title": "The State of the OCaml Platform: Sep 2017", "summary": "Annual update on OCaml Platform development presented at ICFP.", "date_published": "2017-09-01T00:00:00.000000Z", "date_modified": "2017-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2017-oud-platform.pdf", "mime_type": "application/pdf", "title": "The State of the OCaml Platform: Sep 2017" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2017-ml-effects-1", "content_html": "Paper on how to tackle awkward IO patterns with effect handlers at the ML Workshop with Stephen Dolan, Spiros Eliopoulos, Daniel Hillerstrom, KC Sivaramakrishnan and Leo White. The work addressed what Simon Peyton Jones famously called the "Awkward Squad" - the messy reality of I/O, concurrency, and exceptions in functional programming. We showed how algebraic effects and handlers could elegantly express these programs, introducing the concept of asynchronous effects to solve the interaction between user-level threads and operating system services without compromising performance.
", "url": "https://anil.recoil.org/notes/2017-ml-effects-1", "title": "Effectively tackling the awkward squad", "summary": "Paper on tackling awkward IO patterns using effect handlers.", "date_published": "2017-09-01T00:00:00.000000Z", "date_modified": "2017-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "effects", "fp", "io", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2017-ml-effects.pdf", "mime_type": "application/pdf", "title": "Effectively tackling the awkward squad" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/opening-discuss-ocaml", "content_html": "I opened up a Discourse forum for the OCaml community to use, which is running successfully on https://discuss.ocaml.org. This forum thread collates the feedback and discussions about it.
", "url": "https://anil.recoil.org/notes/opening-discuss-ocaml", "external_url": "https://discuss.ocaml.org/t/discussion-site-status-and-timeline/23", "title": "Opening discuss.ocaml.org for the community", "summary": "Introducing discuss.ocaml.org, a new community forum for OCaml discussion and feedback.", "date_published": "2017-05-13T00:00:00.000000Z", "date_modified": "2017-05-13T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "selfhosting", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2017-snapl-dali-1", "content_html": "Position paper on building databases-as-a-library at SNAPL 2017, the Summit on Advances in Programming Languages. Working with Gowtham Kaki, KC Sivaramakrishnan, Thomas Gazagnaire and Suresh Jagannathan, we argued for a radical rethinking of database architecture. Instead of monolithic database servers, DaLi proposed encapsulating data management as transparent libraries in the same language as the application. This enables the type system and verification tools to enforce application-level invariants across the data layer - essentially extending the unikernel philosophy from computation to data.
", "url": "https://anil.recoil.org/notes/2017-snapl-dali-1", "title": "DaLi: Database as a Library", "summary": "Position paper on building databases-as-a-library presented at SNAPL 2017.", "date_published": "2017-05-01T00:00:00.000000Z", "date_modified": "2017-05-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "databases", "fp", "ocaml", "systems", "library" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2017-snapl-dali.pdf", "mime_type": "application/pdf", "title": "DaLi: Database as a Library" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/725dda70-b12b-4b1a-a8ae-fa9c22683ff2-1", "content_html": "DockerCon talk on unikernels and MirageOS, explaining the integration work we had done to deliver Docker for Desktop using library hypervisor technology. I spoke about how unikernels could work alongside containers, with MirageOS providing the underlying virtualization infrastructure. This talk came after our acquisition by Docker and showed how functional programming and systems research could have real-world impact on tools used by millions of developers.
", "url": "https://anil.recoil.org/notes/725dda70-b12b-4b1a-a8ae-fa9c22683ff2-1", "title": "Unikernels: the rise of the library hypervisor in MirageOS", "summary": "DockerCon talk on integrating MirageOS unikernels with Docker using library hypervisor approach for Docker Desktop.", "date_published": "2016-10-14T00:00:00.000000Z", "date_modified": "2016-10-14T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "unikernels", "docker", "hypervisor", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/mirageos-hack-retreat-2016", "content_html": "", "url": "https://anil.recoil.org/notes/mirageos-hack-retreat-2016", "external_url": "https://mirageos.org/blog/2016-summer-hackathon", "title": "MirageOS Summer 2016 hack retreat", "summary": "MirageOS holds summer 2016 hack retreat to advance its unikernel technology.", "date_published": "2016-06-29T00:00:00.000000Z", "date_modified": "2016-06-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/dbd7546a-95d8-40af-b286-3cf930767682-1", "content_html": "I gave a talk at the Functional Works meetup, held in Jane Street London about how Docker for Mac and Windows use OCaml and unikernels under the hood.
", "url": "https://anil.recoil.org/notes/dbd7546a-95d8-40af-b286-3cf930767682-1", "title": "The functional innards of Docker for Mac and Windows", "summary": "Talk at Functional Works meetup discussing how Docker for Mac and Windows use OCaml and unikernels internally.", "date_published": "2016-06-24T00:00:00.000000Z", "date_modified": "2016-06-24T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "docker", "janestreet", "ocaml", "unikernels" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/8c92d6cf-3e05-429f-8c8e-094f77be61c6-1", "content_html": "Interviewed by The New Stack at OSCON in Austin, Texas alongside Ian Eyberg and Joshua Bernstein. We discussed unikernels and Docker with Alex Williams, exploring how unikernel technology was maturing and the different approaches being taken by the community. The interview covered the acquisition of Unikernel Systems by Docker and what it meant for bringing unikernels to mainstream development workflows.
", "url": "https://anil.recoil.org/notes/8c92d6cf-3e05-429f-8c8e-094f77be61c6-1", "title": "Ian Eyberg, Joshua Bernstein, Anil Madhavapeddy at OSCON in Austin", "summary": "Interview with The New Stack about unikernels and Docker at OSCON in Austin, Texas.", "date_published": "2016-06-06T00:00:00.000000Z", "date_modified": "2016-06-06T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "docker", "interview", "mirageos", "containers" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2016-usenix-flick-1", "content_html": "Paper on application-specific network services at USENIX ATC 2016, a collaboration across multiple universities including Cambridge, Imperial, UCL and Nottingham. FLICK provided a framework for developing network services like custom load balancers and middleboxes using high-level abstractions while achieving good performance. The system automatically translated FLICK programs to efficient parallel task graphs with bounded resource usage, enabling safe concurrent execution of multiple services. We demonstrated it with practical applications including an HTTP load balancer, Memcached router, and Hadoop data aggregator.
", "url": "https://anil.recoil.org/notes/2016-usenix-flick-1", "title": "FLICK: Developing and Running Application-Specific Network Services", "summary": "Paper on framework for developing and deploying application-specific network services at USENIX ATC 2016.", "date_published": "2016-06-01T00:00:00.000000Z", "date_modified": "2016-06-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "networking", "nfv", "middleboxes", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2016-usenix-flick.pdf", "mime_type": "application/pdf", "title": "FLICK: Developing and Running Application-Specific Network Services" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/vpnkit-hyperkit", "content_html": "I announce the release of three big components that form the basis for Docker for Desktop: a hypervisor framework called HyperKit, a networking framework for host translation called VPNKit, and a versioned data management store called DataKit.
", "url": "https://anil.recoil.org/notes/vpnkit-hyperkit", "external_url": "https://www.docker.com/blog/docker-unikernels-open-source/", "title": "Improving Docker with Unikernels", "summary": "Introducing HyperKit, VPNKit, and DataKit to enhance Docker performance.", "date_published": "2016-05-18T00:00:00.000000Z", "date_modified": "2016-05-18T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "ocamllabs", "ocaml", "docker", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2016-flops-cmeleon-1", "content_html": "Paper on declarative approaches to foreign function bindings at FLOPS 2016 with Jeremy Yallop and David Sheets. The work addressed a fundamental problem with FFI systems: they typically tie the specification of foreign functions to a specific binding mechanism, making it hard to switch between dynamic binding and static code generation. Our approach using generic programming allowed developers to write FFI specifications once and then flexibly choose the binding strategy, demonstrating the power of abstraction in systems programming.
Announced that Unikernel Systems is now part of Docker, marking a significant milestone for bringing unikernel technology to mainstream developers. This acquisition meant that our research on MirageOS and library operating systems would directly influence tools used by millions of developers worldwide. The team would work on integrating unikernels into Docker for Mac and Windows, demonstrating how academic systems research could have real commercial impact.
", "url": "https://anil.recoil.org/notes/a612e810-d56c-48af-b43e-2893a96b9120-1", "title": "Unikernel Systems is now part of Docker", "summary": "Announcement that Unikernel Systems has been acquired by Docker.", "date_published": "2016-01-21T00:00:00.000000Z", "date_modified": "2016-01-21T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "docker", "mirageos", "acquisition", "startup" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/docker-buys-unikernel-systems", "content_html": "My startup Unikernel Systems was acquired by Docker, and I'll\nbe joining and setting up a UK branch of Docker along with the rest of my team.
\n\n\n'Just like we did with containers, we are interested is democratizing that technology, making it available and useful to the millions of developers and IT pros out there, said Solomon Hykes, founder and chief technology officer for Docker. 'Unikernels allow you to basically get rid of the operating system, and instead compile into the application the small bits of the operating system it really needs.'\n-- The New Stack
\n
You can also see an announcement from me explaining the background story:
\n", "url": "https://anil.recoil.org/notes/docker-buys-unikernel-systems", "external_url": "https://thenewstack.io/docker-buys-unikernel-systems-plans-bring-unikernels-data-center/", "title": "Unikernel Systems acquired by Docker", "summary": "Docker acquires Unikernel Systems to bring unikernel tech to developers and IT pros.", "date_published": "2016-01-21T00:00:00.000000Z", "date_modified": "2016-01-21T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "docker", "startups", "unikernels", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2015-sosp-sibylfs-1", "content_html": "Paper on formal specification and testing of filesystems at SOSP 2015, the premier operating systems conference. SibylFS with Tom Ridge, David Sheets, Peter Sewell and colleagues created the first comprehensive formal specification of POSIX filesystem behavior, then used it as a test oracle to discover numerous inconsistencies and bugs in real-world filesystems. The work demonstrated how executable specifications could bridge theory and practice, finding genuine bugs in production Linux and BSD filesystem implementations that had existed for years.
Paper on personal databoxes at the one-in-a-decade Aarhus conference on participatory design. This work with Hamed Haddadi, Heidi Howard, Richard Mortier and colleagues proposed that individuals should have direct control over their personal data through a personal networked service - the Databox. The paper argued for a fundamental shift away from centralized data silos toward personal data management infrastructure, presaging many of the concerns that would drive GDPR and the broader privacy movement.
Invited talk at NetPL on Immutable Distributed Infrastructure with Unikernels at NetPL 2015. The talk explored how combining unikernels with immutable infrastructure concepts through Irmin's distributed storage could revolutionize how we build cloud systems. I discussed how MirageOS and Irmin together provided version control and branch consistency for entire distributed systems, enabling new approaches to deployment, rollback, and system management.
", "url": "https://anil.recoil.org/notes/d5fbd6a4-bef2-4fbc-9d02-cb9935e50d8e-1", "title": "Immutable Distributed Infrastructure with Unikernels", "summary": "Invited talk at NetPL on immutable distributed infrastructure using unikernels.", "date_published": "2015-09-29T00:00:00.000000Z", "date_modified": "2015-09-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "distributed-systems", "infrastructure" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/35e1a70d-0fb4-49b1-86ce-dd6266b812de-1", "content_html": "Update on the state of the OCaml Platform at the 2015 OCaml Workshop. This talk covered the evolution of the OCaml development toolchain as we continued building out the platform vision. By 2015, opam had become the standard package manager and we were working on improving the overall developer experience with better documentation tools, testing frameworks, and build systems. It's interesting to look back at these talks and see the progression - many of the tools we were prototyping in 2015 became production-ready in subsequent years and are now fundamental parts of how people write OCaml.
", "url": "https://anil.recoil.org/notes/35e1a70d-0fb4-49b1-86ce-dd6266b812de-1", "title": "The State of the OCaml Platform", "summary": "Update on the state of the OCaml Platform development environment and toolchain.", "date_published": "2015-09-04T00:00:00.000000Z", "date_modified": "2015-09-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling", "devtools", "community" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2015-usenixsec-nqsb-1", "content_html": "Paper on rebuilding TLS securely but practically at USENIX Security 2015, presented with David Kaloper-Mersinjak, Hannes Mehnert and Peter Sewell. Not-Quite-So-Broken TLS was a from-scratch implementation in OCaml that served dual roles: both an executable specification for testing other TLS implementations and a production-ready library. By using a memory-safe language and modular design, we excluded entire classes of security vulnerabilities by construction. The implementation achieved reasonable performance (73-84% of OpenSSL throughput) despite the safety guarantees, and could be compiled into tiny Xen unikernels with a 4% TCB compared to Linux/OpenSSL stacks.
", "url": "https://anil.recoil.org/notes/2015-usenixsec-nqsb-1", "title": "Not-Quite-So-Broken TLS", "summary": "Paper on rebuilding TLS securely but practically presented at USENIX Security 2015.", "date_published": "2015-08-01T00:00:00.000000Z", "date_modified": "2015-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "security", "tls", "cryptography", "ocaml" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2015-usenixsec-nqsb.pdf", "mime_type": "application/pdf", "title": "Not-Quite-So-Broken TLS" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/55852136-843d-4043-98e7-6b46c6d39b01-1", "content_html": "Talk at Esper on functional programming with unikernels to a packed room at their OCaml meetup in California. I covered how MirageOS enables building functional infrastructure by compiling OCaml applications directly into specialized unikernels. The talk demonstrated the integration possibilities with Docker and explored how Irmin's distributed storage capabilities could revolutionize how we build cloud services.
", "url": "https://anil.recoil.org/notes/55852136-843d-4043-98e7-6b46c6d39b01-1", "title": "Unikernels: Functional Infrastructure with Mirage OS", "summary": "Talk at Esper on functional programming with unikernels using MirageOS.", "date_published": "2015-05-12T00:00:00.000000Z", "date_modified": "2015-05-12T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "unikernels", "fp", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/ad4658f5-ca4f-42f3-b61a-58f13dcdeb1a-1", "content_html": "NSDI 2015 talk on Jitsu at USENIX NSDI in Oakland, presenting our system for just-in-time summoning of unikernels. Jitsu addressed network latency by rapidly instantiating local services near users on resource-constrained ARM devices. The system used DNS-triggered launches to boot unikernels in milliseconds, masking boot latency through clever shared memory channels. We demonstrated that lightweight, memory-safe unikernels could provide secure multi-tenant isolation on embedded devices while being highly responsive and power-efficient.
", "url": "https://anil.recoil.org/notes/ad4658f5-ca4f-42f3-b61a-58f13dcdeb1a-1", "title": "Jitsu: Just-In-Time Summoning of Unikernels", "summary": "NSDI 2015 talk on Jitsu system for on-demand unikernel instantiation.", "date_published": "2015-05-04T00:00:00.000000Z", "date_modified": "2015-05-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "systems", "xen" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2015-diynet-kadupul-1", "content_html": "Workshop paper on DIY networking using timelock puzzles, presented at the DIY Networking workshop with Magnus Skjegstad and Jon Crowcroft. Kadupul explored using virtual currencies like Bitcoin to incentivize heterogeneous multihop mesh networking. The key insight was using time-locked puzzles where forwarding nodes compete to solve cryptographic challenges, with the fastest delivery claiming a reward. This created a natural surge pricing model during congestion and could enable low-latency services like video streaming and AR on mesh networks.
Paper on spinning up low-latency unikernels per-connection at NSDI 2015, one of the top systems conferences. Jitsu demonstrated that MirageOS unikernels could boot fast enough (milliseconds) to be instantiated on-demand in response to network traffic, masking boot latency through DNS tricks. Working with Thomas Leonard, Magnus Skjegstad and a large team, we showed this was practical even on resource-constrained ARM devices, enabling secure multi-tenant edge computing years before it became mainstream. The paper won acclaim for demonstrating compelling performance while maintaining type-1 hypervisor isolation guarantees.
", "url": "https://anil.recoil.org/notes/2015-nsdi-jitsu-1", "title": "Jitsu: Just-In-Time Summoning of Unikernels", "summary": "Paper on low-latency unikernel spawning per-connection presented at NSDI 2015.", "date_published": "2015-05-01T00:00:00.000000Z", "date_modified": "2015-05-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "networking", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2015-nsdi-jitsu.pdf", "mime_type": "application/pdf", "title": "Jitsu: Just-In-Time Summoning of Unikernels" } ], "_references": [] }, { "id": "https://doi.org/10.59350/hg4a4-m0a36", "content_html": "The OCaml Labs initiative within the Cambridge\nComputer Laboratory is now just over two years\nold, and it is time for an update about our activities since the last\nupdate at the end of\n2013\nand\n2012.
\nThe theme of our group was not to be pure research, but rather a hybrid\ngroup that takes on some of the load of day-to-day OCaml maintenance\nfrom INRIA, as well as help grow the wider\ncommunity and meet our own research agendas around topics such as\nunikernels. To this end,\nall of our projects have been highly collaborative, often involving\ncolleagues from OCamlPro,\nINRIA, Jane Street,\nLexifi and Citrix.
\nThis post covers our progress in tooling, the compiler and language,\ncommunity efforts, research projects and concludes with our priorities\nfor 2015.
\n![\"%r\"](\"/images/toru-cucl-window.webp\" "\\"OCaml:")
\nToolingAt the start of 2014, we had just helped to release OPAM\n1.1.1 with our\ncolleagues at OCamlPro, and serious OCaml users\nhad just started moving over to using it.
\nOur overall goal at OCaml Labs is to deliver a modular set of of\ndevelopment tools around OCaml that we dub the OCaml Platform. The\nremainder of 2014 was thus spent polishing this nascent OPAM release\ninto a solid base (both as a command-line tool and as a library) that we\ncould use as the basis for documentation, testing and build\ninfrastructure, all the while making sure that bigger OCaml projects\ncontinued to migrate over to it. Things have been busy; here are the\nhighlights of this effort.
\nThe central OPAM repository\nthat contains the package descriptions has grown tremendously in 2014,\nwith over 280 contributors committing almost 10000 changesets across\n3800 pull requests on\nGitHub. The front line of incoming testing has been continuous\nintegration by the wonderful Travis\nCI, who also granted us\naccess to their experimental MacOS\nX build pool. The\nOPAM package team also to expanded to give David Sheets, Jeremy Yallop,\nPeter Zotov and Damien Doligez commit rights, and they have all been\nbusily triaging new packages as they come in.
\nSeveral large projects such as Xapi,\nOcsigen and our own\nMirageOS switched over to using OPAM for\nday-to-day development, as well as prolific individual developers such\nas Daniel Buenzli and Markus\nMottl. Jane\nStreet continued to send\nregular monthly\nupdates\nof their Core/Async suite, and releases appeared from the\nFacebook\nopen-source team as well (who develop\nHack,\nFlow and\nPfff in OCaml).
\n![\"%r\"](\"/images/opam12-contributors-mar14.webp\" "\\"Number")
\n![\"%r\"](\"/images/opam12-packages-mar14.webp\" "\\"Total")
\n![\"%r\"](\"/images/opam12-unique-packages-mar14.webp\" "\\"Total")
We used feedback from the users to smooth away many of the rough edges,\nwith:
\nopam source. This handles any supported OPAM archive, including Git, Mercurial or Darcs remotes.These changes were all incorporated into the OPAM 1.2, along with backwards compatibility shims to keep the old 1.1 metadata format working until the migration is complete. The 1.2.x series has been a solid and usable development manager, and last week’s release of OPAM 1.2.1 has further polished the core scripting engine.
\nOne of the more notable developments during 2014 was the adoption of\nOPAM further up the\necosystem by the Coq theorem prover. This\nbroadening of the community prompted us to create an official OPAM\nblog to give us a central place for new and\ntips, and we’ve had posts about\nXenServer developments,\nthe Merlin IDE\ntool\nand the modern UTop\ninteractive REPL. If you are using OPAM in an interesting or production\ncapacity, please do get in\ntouch so that we can\nwork with you to write about it for the wider community.
\nThe goal of the blog is also to start bringing together the various\ncomponents that form the OCaml Platform. These are designed to be\nmodular tools (so that you can pick and choose which ones are necessary\nfor your particular use of OCaml). There are more details available from\nthe OCaml Workshop presentation at ICFP 2014\n(abstract,\nslides,\nvideo).
\nOPAM has also been adopted now by several\nbig universities\n(including us at\nCambridge!) for\nundergraduate and graduate Computer Science courses. The demands\nincreased for an out-of-the-box solution that makes it as easy possible\nfor new users to get started with minimum hassle. We created a\ndedicated teaching list to\naid collaboration, and a list of teaching resources on\nocaml.org and supported\nseveral initiatives in collaboration with Louis\nGesbert at OCamlPro, as usual with OPAM\ndevelopment).
\nThe easiest way to make things "just work" are via regular binary builds\nof the latest releases of OCaml and OPAM on Debian, Ubuntu, CentOS and\nFedora, via Ubuntu PPAs and the OpenSUSE\nBuild Service\nrepositories. Our industrial collaborators from Citrix, Jon\nLudlam and Dave Scott\nbegan an upstreaming\ninitiative\nto Fedora and sponsored the creation of a CentOS\nSIG\nto ensure that binary packages remain up-to-date. We also contribute to\nthe hardworking packagers on MacOS X, Debian, FreeBSD, NetBSD and\nOpenBSD where possible as well to ensure that binary builds are well\nrounded out. Richard Mortier also assembled Vagrant\nboxes that contain OCaml,\nfor use with VirtualBox.
\n![\"%r\"](\"/images/opam-in-nice.webp\" "\\"Louis")
Within OPAM itself, we applied polish to the handling of external\ndependencies to automate checking\nthat the system libraries required by OPAM are present. Two emerging\ntools that should help further in 2015 are the\nopam-user-setup and\nOPAM-in-a-box plugins that\nautomate first-time configuration. These last two are primarily\ndeveloped at OCamlPro, with design input and support from OCaml Labs.
\nWe do have a lot of work left to do with making the new user experience\nreally seamless, and help is very welcome from anyone who is\ninterested. It often helps to get the perspective of a newcomer to find\nout where the stumbling blocks are, and we value any such advice. Just\nmail opam-devel@lists.ocaml.org\nwith your thoughts, or create an\nissue on how we can improve. A\nparticularly good example of such an initiative was started by Jordan\nWalke, who prototyped CommonML\nwith a NodeJS-style development workflow, and wrote\nup\nhis design document for the mailing list. (Your questions or ideas do\nnot need to be as well developed as Jordan’s prototype!)
\nThe public Travis CI testing does come with some limitations, since it\nonly checks that the latest package sets install, but not if any\ntransitive dependencies fail due to interface changes. It also doesn’t\ntest all the optional dependency combinations due to the 50 minute time\nlimit.
\n![\"%r\"](\"/images/travis-mascot-200px.webp\")
We expanded the OPAM repository testing in several ways to get around\nthis:
\nIndividual Repositories: Thomas Gazagnaire built centralised\nTravis scripts that\ncan be used on any OCaml GitHub repository to easily test code\nbefore it is released into OPAM. These scripts are sourced from a\ncentral\nrepository and\nsupport external, optional and reverse dependency checking across\nmultiple revisions of the compiler. For instance, it just needs one\nfile\nto test all the supported permutations of the\nCoHTTP library.
\nBulk Builds: Damien Doligez and I independently started doing\nlarge-scale bulk builds of the repository to ensure that a single\nsnapshot of the package repository can automatically build as many\npackages as possible. My implementation used the\nDocker container manager to spawn off 1000s of\npackage builds in parallel and commit the results into a filesystem\nThis required building a Dockerfile\neDSL, and the results are\nnow online at\nhttps://opam.ocaml.org/builds.
\nOCamlot: An ongoing piece of infrastructure work is to take the\nbulk build logs (which are around 7GB per daily run), and to store\nand render them using our Irmin Git store. Expect\nto see more around this soon; it has the awesome feature of letting\nany developer clone the build logs for their project locally, to\nmake triage of foreign operating systems as simple as possible.
\nThis ability to do unattended builds of the package repository has also\nimproved the decision making process within the core compiler team.\nSince we now have a large (3000+ package) corpus of OCaml code, it\nbecame a regular occurrence in the 4.02 development cycle to “ask\nOPAM”\nwhether a particular feature or new syntax would break any existing\ncode. This in turn provides an incentive for commercial users to provide\nrepresentative samples of their code; for instance, the Jane Street Core\nreleases in OPAM (with their very modular style) act as an open-source\ncanary without needing access to any closed source code.
\nOne good example in 2014 was the decoupling of the\nCamlp4 macro preprocessor from\nthe main OCaml distribution. Since Camlp4 has been used for over a\ndecade and there are some very commonly used syntax extensions such as\ntype_conv, a simple removal\nwould break a lot of packages. We used OPAM to perform a gradual\nmovement that most users hopefully never noticed by the time OCaml 4.02\nwas released. First, we added a dummy\npackage in OPAM for\nearlier versions of the compiler that had Camlp4 built-in, and then used\nthe OPAM constraint engine to compile it as an external tool for the\nnewer compiler revisions. Then we just had to triage the bulk build logs\nto find build failures from packages that were missing a Camlp4\ndependency, and add\nthem\nto the package metadata.
\nAn interesting\ncomment from\nVincent Hanquez about OPAM is that "OCaml's OPAM is a post-GitHub\ndesign". This is very true, as much of the workflow for pinning git://\nURLs emerged out of being early adopters of GitHub for hosting the\nMirageOS. OCaml Labs supported two pieces of infrastructure integration\naround GitHub in 2014:
OPAM has a compiler switch feature that lets you run simultaneous\nOCaml installations and swap between them easily. I used my GitHub\nAPI bindings to regularly\nconvert every GitHub pull request into a custom compiler\nswitch (see Easily OPAM switching to any OCaml feature request).\nThis lets users reporting bugs try out a patched compiler almost\nimmediately upon a fix becoming available.
\nThe motivation behind this feature was our collaborator Gabriel\nScherer’s\nexperiment\nto enable patch review of OCaml on GitHub, alongside the venerable\nMantis bug\ntracker. We\nsupported this via adding Travis CI support to the main compiler,\nand also helped to migrate a number of support libraries to GitHub,\nsuch as camlp4. These can all be\nfound on the ocaml organisation on\nGitHub.
\nLeo White, David Sheets, Amir Chaudhry and Thomas Gazagnaire led the\ncharge to build a modern documentation generator for OCaml, and\npublished\nan alpha version of codoc 0.2.0\nafter a lot of work throughout 2014. In the 2014 OCaml workshop\npresentation\n(abstract,\nslides,\nvideo),\nwe mentioned the “module wall” for documentation and this attempts to\nfix it. To try it out, simply follow the directions in the README on\nthat repository, or browse some\nsamples of the current, default output\nof the tool. Please do bear in mind codoc and its constituent libraries\nare still under heavy development and are not feature complete, but\nwe’re gathering feedback from\nearly adopters.
\ncodoc's aim is to provide a widely useful set of tools for generating\nOCaml documentation. In particular, we are striving to:
\ncodocWe haven’t yet achieved all of these at all levels of our tool stack but\nare getting close, and the patches are all under discussion for\nintegration into the mainstream OCaml compiler. codoc 0.2.0 is usable\ntoday (if a little rough in some areas like default CSS), and there is a\nblog post that\noutlines the architecture of the new system to make it easier to\nunderstand the design decisions that went into it.
As the amount of infrastructure built around the\nocaml.org domain grows (e.g. mailing lists, file\nhosting, bulk building), it is important to establish a governance\nframework to ensure that it is being used as best needed by the wider\nOCaml community.
\nAmir Chaudhry took a good look at how other language communities\norganise themself, and began putting together a succinct governance\nframework\nto capture how the community around ocaml.org operates, and how to\nquickly resolve any conflicts that may arise in the future. He took care\nto ensure it had a well-defined scope, is simple and self-contained, and\n(crucially) documents the current reality. The result of this work is\ncirculating privately through all the existing volunteers for a first\nround of feedback, and will go live in the next few months as a living\ndocument that explains how our community operates.
One consequence of OCaml’s age (close to twenty years old now) is that\nthe tools built around the compiler have evolved fairly independently.\nWhile OPAM now handles the high-level package management, there is quite\na complex ecosystem of other components that are complex for new users\nto get to grips with: OASIS,\nocamlfind,\nocamlbuild, and\nMerlin to name a few.\nEach of these components (while individually stable) have their own\nmetadata and namespace formats, further compounding the lack of cohesion\nof the tools.
\nThomas Gazagnaire and Daniel Buenzli embarked on an effort to build an\neDSL that unifies OCaml package descriptions, with the short-term aim of\ngenerating the support files required by the various support tools, and\nthe long-term goal of being the integration point for the build, test\nand documentation generation lifecycle of an OCaml/OPAM package. This\nprototype, dubbed Assemblage has\ngone through several iterations and design\ndiscussions over\nthe summer of 2014. Daniel has since been splitting out portions of it\ninto the Bos OS interaction library.
\nAssemblage is not released officially yet, but we are committed to\nresuming work on it this summer when Daniel visits again, with the\nintention of unifying much of our workflow through this tool. If you are\ninterested in build and packaging systems, now is the time to make your\nopinion known!
\nWe also spent time in 2014 working on the core OCaml language and\ncompiler, with our work primarily led by Jeremy Yallop and Leo White.\nThese efforts were not looking to make any radical changes in the core\nlanguage; instead, we generally opted for evolutionary changes that\neither polish rough edges in the language (such as open type and handler\ncases), or new features that fit into the ML style of building programs.
\nThe OCaml 4.02 series was primarily developed and\nreleased in 2014. The\nChangeLog\ngenerated much user\nexcitement,\nand we were also pleased to have contributed several language\nimprovements.
\nOCaml’s try and match constructs are good at dealing with exceptions\nand values respectively, but neither constructs can handle both values\nand exceptions. Jeremy Yallop investigated how to handle\nsuccess\nmore elegantly, and an elegant unified syntax emerged. A simple example\nis that of a stream iterator that uses exceptions for control flow:
let rec iter_stream f s =\n match (try Some (MyStream.get s) with End_of_stream -> None) with\n | None -> ()\n | Some (x, s') -> f x; iter_stream f s'\nThis code is not only verbose, but it also has to allocate an option\nvalue to ensure that the iter_stream calls remains tail recursive. The\nnew syntax in OCaml 4.02 allows the above to be rewritten succinctly:
let rec iter_stream f s =\n match MyStream.get s with\n | (x, s') -> f x; iter_stream f s'\n | exception End_of_stream -> ()\nRead more about the background of this feature in Jeremy’s blog\npost,\nthe associated discussion in the upstream Mantis\nbug, and the final\nmanual\npage in\nthe OCaml 4.02 release. For an example of its use in a real library, see\nthe Jane Street\nusage\nin the s-expression handling\nlibrary (which they use widely to reify arbitrary OCaml values and\nexceptions).
\nA long-standing trick to build universal\ncontainers in OCaml has\nbeen to encode them using the exception exn type. There is a similar\nconcept of a universal type in\nStandard ML, and they were described in the “Open Data Types and Open\nFunctions” paper by Andres\nLöh and Ralf Hinze in 2006.
Leo White designed, implemented and upstreamed support for extensible\nvariant\ntypes in\nOCaml 4.02. Extensible variant types are variant types that can be\nextended with new variant constructors. They can be defined as follows:
\ntype attr = ..\n\ntype attr += Str of string\n\ntype attr +=\n | Int of int\n | Float of float\nPattern matching on an extensible variant type requires a default case\nto handle unknown variant constructors, just as is required for pattern\nmatching on exceptions (extensible types use the exception memory\nrepresentation at runtime).
\nWith this feature added, the OCaml exn type simply becomes a special\ncase of open extensible types. Exception constructors can be declared\nusing the type extension syntax:
type exn += Exc of int\nYou can read more about the discussion behind open extensible types in\nthe upstream Mantis bug.\nIf you’d like to see another example of their use, they have been\nadopted by the latest releases of the Jane Street Core libraries in the\nType_equal\nmodule.
\nA common criticism of OCaml is its lack of support for ad-hoc\npolymorphism. The classic example of this is OCaml’s separate addition\noperators for integers (+) and floating-point numbers (+.). Another\nexample is the need for type-specific printing functions (print_int,\nprint_string, etc.) rather than a single print function which works\nacross multiple types.
Taking inspiration from Scala’s\nimplicits\nand Modular Type\nClasses by\nDreyer et al., Leo White designed a system for ad-hoc polymorphism in\nOCaml based on using modules as type-directed implicit parameters. The\ndesign not only supports implicit modules, but also implicit functors\n(that is, modules parameterised by other module types) to permit the\nexpression of generic modular implicits in exactly the same way that\nfunctors are used to build abstract data structures.
\nFrederic Bour joined us as a summer intern and dove straight into the\nimplementation, resulting in an online\ndemo and ML\nWorkshop presentation\n(abstract,\nvideo and\npaper). Another innovation in how\nwe’ve been trialling this feature is the use of Andy Ray’s\nIOCamlJS to publish an\ninteractive, online notebook that is fully hosted in the browser. You\ncan follow the examples of modular implicits\nonline, or try\nthem out on your own computer via an OPAM switch:
\nopam switch 4.02.0+modular-implicits\neval `opam config env`\nopam install utop \nutop\nSome of the early feedback on modular implicits from industrial users\nwas interesting. Jane Street commented that although this would be a big\nusability leap, it would be dangerous to lose control over exactly what\ngoes into the implicit environment (i.e. the programmer should always\nknow what (a + b) represents by locally reasoning about the code). The\ncurrent design thus follows the ML discipline of maintaining explicit\ncontrol over the namespace, with any ambiguities in resolving an\nimplicit module type resulting in a type error.
In addition to ad-hoc polymorphism, support for parallel execution on\nmulticore CPUs is undoubtedly the most common feature request for OCaml.\nThis has been high on our list after improving tooling support, and\nStephen Dolan and Leo White made solid progress in 2014 on the core\nruntime plumbing required.
\nStephen initially added thread-local\nsupport to the OCaml compiler. This\ndesign avoided the need to make the entire OCaml runtime preemptive (and\nthus a huge patch) by allocating thread-local state per core.
\nWe are now deep into the design and implementation of the programming\nabstractions built over these low-level primitives. One exciting aspect\nof our implementation is much of the scheduling logic for multicore\nOCaml can be written in (single-threaded) OCaml, making the design very\nflexible with respect to heterogenous\nhardware and variable IPC\nperformance.
\nTo get feedback on the overall design of multicore OCaml, we presented\nat OCaml 2014\n(slides,\nvideo and\nabstract), and\nStephen visited INRIA to consult with the development team and Arthur\nChargueraud (the author of\nPASL). Towards the end of the\nyear, KC Sivaramakrishnan finished his PhD studies\nat Purdue and joined our OCaml Labs group. He is the author of\nMultiMlton, and is\nnow driving the completion of the OCaml multicore work along with\nStephen Dolan, Leo White and Mark Shinwell. Stay tuned for updates from\nus when there is more to show later this year!
\nThe Ctypes library started\nas an experiment with GADTs by Jeremy Yallop, and has since ballooned in\na robust, comprehensive library for safely interacting with the OCaml\nforeign function interface. The first release came out in time to be\nincluded in Real World\nOCaml\nin lieu of the low-level FFI (which I was not particularly enamoured\nwith having to explain in a tight page limit).
\nThroughout 2014, Jeremy expanded support for a number of features\nrequested by users (both industrial and academic) who adopted the\nlibrary in preference to manually writing C code to interface with the\nruntime, and issued several updated\nreleases.
\nThe first release of Ctypes required the use of\nlibffi to dynamically load shared\nlibraries and dynamically construct function call stack frames whenever\na foreign function is called. While this works for simple libraries, it\ncannot cover all usecases, since interfacing with C demands an\nunderstanding of struct memory layout, C preprocessor macros, and\nother platform-dependent quirks which are more easily dealt with by\ninvoking a C compiler. Finally, the performance of a libffi-based API\nwill necessarily be slower than writing direct C stub code.
While many other language FFIs provide separate libraries for dynamic\nand static FFI libraries, we decided to have a go at building a\nmodular version of Ctypes that could handle both cases from a single\ndescription of the foreign function interface. The result (dubbed\n“Cmeleon”) remained surprisingly succinct and usable, and now covers\nalmost every use of the OCaml foreign function interface. We submitted a\npaper to ICFP 2015 dubbed “A modular\nforeign function\ninterface”\nthat describes it in detail. Here is a highlight of how simple a generic\nbinding looks:
\nmodule Bindings(F : FOREIGN) = struct\n open F\n let gettimeofday = foreign "gettimeofday"\n (ptr timeval @-> ptr timezone @-> returning int)\nend\nThe FOREIGN module type completely abstracts the details of whether or\nnot dynamic or static binding is used, and handles C complexities such\nas computing the struct layout on the local machine architecture.
The other nice result from functorising the foreign function interface\nemerged when we tried to invert the FFI and serve a C interface from\nOCaml code (for example, by compiling the OCaml code as a shared\nlibrary). This\nwould let us begin swapping out C libraries that we don’t\ntrust with safer\nequivalents written in OCaml.
\nYou can see an\nexample\nof how inverted stubs work via a simple C XML parsing exposed from the\nXmlm library. We can define a C\nstruct by:
(* Define a struct of callbacks (C function pointers) *)\nlet handlers : [`handlers] structure typ = structure "handlers"\nlet (--) s f = field handlers s (funptr f)\n let on_data = "on_data" -- (string @-> returning void)\n let on_start_tag = "on_start_tag" -- (string @-> string @-> returning void)\n let on_end_tag = "on_end_tag" -- (void @-> returning void)\n let on_dtd = "on_dtd" -- (string @-> returning void) \n let on_error = "on_error" -- (int @-> int @-> string @-> returning void)\nlet () = seal handlers\nand then expose this via C functions:
\nmodule Stubs(I : Cstubs_inverted.INTERNAL) = struct\n (* Expose the type 'struct handlers' to C. *)\n let () = I.structure handlers\n\n (* We expose just a single function to C. The first argument is a (pointer\n to a) struct of callbacks, and the second argument is a string\n representing a filename to parse. *)\n let () = I.internal "parse_xml" \n (ptr handlers @-> string @-> returning void) parse\nend\nYou can find the full source code to these snippets on the\nocaml-ctypes-inverted-stubs-example\nrepository on GitHub.
\nWe’ll be exploring this aspect of Ctypes further in 2015 for SSL/TLS\nwith David Kaloper and Hannes Mehnert, and Microsoft Research has\ngenerously funded a PhD\nstudentship\nto facilitate the work.
\nCtypes benefited enormously from several external contributions from the\nOCaml community. From a portability perspective, A. Hauptmann\ncontributed Windows\nsupport, and Thomas\nLeonard added Xen\nsupport to allow\nCtypes bindings to work with MirageOS\nunikernels (which opens up the intriguing\npossibility of accessing shared libraries across virtual machine\nboundaries in the future). C language support was fleshed out by Edwin\nTorok contributing typedef\nsupport, Ramkumar\nRamachandra adding C99\nbools and Peter\nZotov integrating native\nstrings.
\nThe winner of “most enthusiastic use of OCaml Labs code” goes to Thomas\nBraibant of\nCryptosense, who used every\nfeature of the Ctypes library (consider multi-threaded, inverted, staged\nand marshalled bindings) in their effort to hack the\nhackers.\nDavid Sheets comes a close second with his implementation of the FUSE\nbinary protocol, parameterised by\nversion quirks.
\nIf you’re using Ctypes, we would love to hear about your particular use.\nA search on GitHub and OPAM reveals over 20 projects using it already,\nincluding industrial use at Cryptosense and\nJane Street, and ports to Windows, *BSD,\nMacOS X and even iPhone and Android. There’s a getting\nstarted guide, and a\nmailing list available.
\nIn addition to the online community building, we also participated in a\nnumber of conferences and face-to-face events to promote education about\nfunctional programming.
\n![\"%r\"](\"/images/qcon-unikernel-talk.webp\" "\\"Anil")
There has been a huge growth in the number of quality conferences in\nrecent years, making it tough to choose which ones to attend.\nICFP is the academic meeting point that\npredates most of them, and we participated\nextensively\nin 2014 via talks, tutorials and a\nkeynote at the Haskell\nSymposium.
\nI also served on the program\ncommittee and industrial\nrelations\nchair\nand took over as the steering committee chair of\nCUFP. Jeremy Yallop, Thomas Gazagnaire and Leo White\nall served program committees on workshops, with Jeremy also chairing\nthis year’s ML Workshop.
Outside of academic conferences, we participated in a number of\nnon-academic conferences such as QCon,\nOSCON, CCC, New Directions in\nOS,\nFunctionalConf,\nFPX\nand FOSDEM. The vast majority of these talks\nwere about the MirageOS, and slides can be found at\ndecks.openmirage.org.
\nYaron Minsky and I have run OCaml tutorials for ICFP for\na\nfew\nyears,\nand we finally hung up our boots in favour of a new crowd.
\nJeremy Yallop and Leo White stepped up to the mark with their ICFP/CUFP\n2014 Introduction to\nOCaml\ntutorial, which had the additional twist of being taught entirely in a\nweb browser by virtue of using the\njs_of_ocaml and\nIOCamlJS. They decided that a\ngood practical target was the popular\n2048 game that has wasted many\nprogrammer hours here at OCaml Labs. They hacked on\nit over the summertime,\nassisted by our visitor Daniel Buenzli who also released useful\nlibraries such as Vg,\nReact,\nUseri, and\nGg.
\nThe end result is satisfyingly playable\nonline, with the source code\navailable at\nocamllabs/2048-tutorial.
\nThomas Gazagnaire got invited to Bangalore for Functional\nConf later in the year, and he extended the\ninteractive tutorial notebook and\nalso ran an OCaml tutorial to a packed room. We were very happy to\nsupport the first functional programming conference in India, and hope\nto see many more such events spring up! Amir Chaudhry then went to\nBelgium to FOSDEM 2015 where he showed off\nthe 2048 game running as an ARM\nunikernel to a\ncrowd of attendees at the Xen booth.
\n![\"%r\"](\"/images/l23.webp\" "\\"Jeremy")
\n![\"%r\"](\"/images/compiler-hacking-dsyme.webp\" "\\"Compiling")
\n![\"%r\"](\"/images/jeremy-rwo.webp\" "\\"Finding")
Jeremy Yallop and Leo White (with assistance from Alan Mycroft and\nmyself) also led the design of a new graduate course on Advanced\nFunctional Programming at\nthe Computer Laboratory. This ran in the Lent\nTerm and was over-subscribed by\nthree times the number who pre-registered (due to a number of PhD\nstudents and our collaborators from Citrix also\nattending).
\nThe course materials are freely available\nonline and\ncover the theory behind functional programming, and then move onto type\ninference, abstraction and parametricity, GADTs, rows, monads, and\nstaging. We will be running this again in future years, and the lecture\nmaterials are already proving useful to answer mailing list\nquestions.
\nWe also had the pleasure of mentoring up-and-coming functional\nprogrammers via several outreach programs, both face-to-face and remote.
\nWe started the Cambridge Compiler\nHacking sessions in a\nsmall way towards the end of 2013 in order to provide a local, friendly\nplace to assist people who wanted to dip their toes into the\nunnecessarily mysterious world of programming language hacking. The plan\nwas simple: provide drinks, pizza, network and a bug list of varying\ndifficulty for\nattendees to choose from and work on for the evening, with mentoring\nfrom the experienced OCaml contributors.
\nWe continued this bi-monthly tradition in 2014, with a regular\nattendance of 15-30 people, and even cross-pollinated communities with\nour local F# and Haskell colleagues. We rotated locations from the\nCambridge Computer Laboratory to Citrix, Makespace, and the new\nCambridge Postdoc Centre. We posted some\nhighlights\nfrom sessions towards the start of the year, and are very happy with how\nit’s going. There has even been uptake of the bug list across the water\nin France, thanks to Gabriel Scherer.
\nIn 2015, we’d like to branch out further and host some sessions in\nLondon. If you have a suggestion for a venue or theme, please get in\ntouch!
\nThere has been a laudable rise in summer programs designed to encourage\ndiversity in our community, and we of course leap at the opportunity to\nparticipate in these when we find them.
\nOur own students also had the chance to participate in such workshops to\nget out of Cambridge in the summer! Heidi\nHoward liveblogged her experiences at\nthe\nPLMW\nworkshop in Mumbai. Meanwhile, David\nSheets got to travel to the slightly less\nexotic London to liveblog\nOSIO,\nand Leonhard Markert covered ICFP\n2014 as a\nstudent volunteer.
\nOur blog roll\nmaintains the ongoing stream of activity from the OCaml Labs crew, but\nthere were some particular highlights throughout 2014.
\nIt wasn’t all just blogging though, and Jeremy Yallop and Leo White in\nparticular participated in some epic OCaml bug\nthreads about new\nfeatures, and\nexplanations\nabout OCaml semantics on the mailing list.
\nAmir Chaudhry also continued to curate and develop the content on the\nocaml.org website with our external collaborators\nAshish Agarwal, Christophe Troestler and Phillippe Wang.\nNotably, it is now the recommended site for OCaml (with the INRIA\nsite being infrequently updated), and also hosts\nthe ACM OCaml Workshop pages. One\naddition that highlighted the userbase of OCaml in the teaching\ncommunity came from building a map of all of the\nuniversities where the\nlanguage is taught, and this was Yan Shvartzshnaider’s first\ncontribution\nto the site.
\n![\"%r\"](\"/images/ocl-pub.webp\" "\\"Down")
Finally, a really important part of any community is hanging out with\neach other to chat over ideas in a friendly environment. As usual, we\nhad a very steady stream of visitors and interns throughout 2014 to\nfacilitate this.
\nFrederic Bour, Benjamin Farinier and Matthieu Journault joined us as\nsummer interns from their respective universities in France as part of\ntheir Masters programs. Frederic worked on modular implicits and gave a\ngreat\ntalk\nat the OCaml Users group. Benjamin and Matthieu worked on Irmin data\nstructures and complexity (and\nmerge-queues and\nmerge-ropes), and Benjamin had\nhis paper on “Mergeable Persistent Data\nStructures” accepted\nto JFLA 2015, while Matthieu’s work on\nefficient algorithms for synchronising Irmin DAGs is being integrated\ninto the upstream source code.
\nDaniel Buenzli repeated his visit from 2013 and spent a productive\nsummer with us, commenting on almost every project we’re working on. In\nhis own words (edited for brevity):
\n\n\nI started by implementing and releasing\nUucp, a library to provide\nefficient access to a selection of the properties of the latest\nUnicode Character database (UCD). […] As a side effect of the previous\npoint I took time to write an absolute minimal introduction to\nUnicode.\n[…] Since I was in this Unicode business I took the opportunity to\npropose a 31 loc patch to the standard\nlibrary for a type to\nrepresent Unicode scalar values (an Unicode character to be imprecise)\nto improve interoperability.
\nThe usual yearly update to OpenGL was announced at the Siggraph\nconference. This prompted me to update the ctypes-based tgls\nlibrary for supporting the latest\nentry point of OpenGL 4.5 and OpenGL ES 3.1. Since the bindings are\nautomatically generated from the OpenGL XML registry the work is not\ntoo involved but there’s always the odd function signature you\ndon’t/can’t handle automatically yet.
\nSpend quite a bit (too much) time on\nuseri, a small multi-platform\nabstraction for setting up a drawing surface and gather user input\n(not usury) as React events.\nUseri started this winter as a layer on top of SDL to implement a CT\nscan app and it felt like this\ncould be the basis for adding interactivity and animation to Vg/Vz\nvisualizations – js viz libraries simply rely on the support provided\nby the browser or SVG support but Vg/Vz strives for backend\nindependence and clear separations of concern (up to which limit\nremains an open question). Unfortunately I couldn’t bring it to a\nrelease and got a little bit lost in browser compatibility issues and\ntrying to reconcile what browser and SDL give us in terms of\nfunctionality and way of operating, so that a maximum of client code\ncan be shared among the supported platforms. But despite this\nnon-release it still managed to be useful in some way, see the next\npoint.
\nHelped Jeremy and Leo to implement the rendering and interaction for\ntheir ICFP tutorial 2048 js_of_ocaml\nimplementation. This\nfeatured the use of Gg, Vg, Useri and React and I was quite pleased\nwith the result (despite some performance problems in certain\nbrowsers, but hey composable rendering and animation without a single\nassignement in client code). It’s nice to see that all these pains at\ntrying to design good APIs eventually fit together […]
\n
A couple of visitors joined us from sunny\nMorocco, where Hannes Mehnert and David\nKaloper had gone to work on a clean-slate TLS stack. They found the\nMirageOS effort online, and got in touch about\nvisiting. After a very fun summer of hacking, their stack is now the\nstandard TLS option in MirageOS and resulted in the Bitcoin Pinata\nchallenge being issued! Hannes\nand David have since moved to Cambridge to work on this stack full-time\nin 2015, but the internships served as a great way for everyone to get\nto know each other.
\nWe also had the pleasure of visits from several of our usually remote\ncollaborators. Christophe Troestler,\nYaron Minsky, Jeremie\nDiminio and Andy\nRay all visited for the annual OCaml Labs\nreview meeting in\nChrist’s College. There were also many academic talks from foreign\nvisitors in our SRG seminar\nseries, ranging from Uday\nKhedkar from IIT to Oleg\nKiselyov deliver multiple talks on staging and\noptimisation (as well as making a celebrity appearance at the compiler\nhacking session, and Yaron Minsky\ndelivering an Emacs-driven departmental seminar on his experiences with\nIncremental computation.
\nThe OCaml Labs are of course based in the Cambridge Computer Laboratory,\nwhere our day job is to do academic research. Balancing the demands of\nopen source coding, community efforts and top-tier research has be a\ntricky one, but an effort that has been worthwhile.
\n![\"%r\"](\"/images/christs-dinner.webp\" "\\"Dinner")
\n![\"%r\"](\"/images/nsdi-deadline.webp\" "\\"Hacking")
\n![\"%r\"](\"/images/scotty.webp\" "\\"Dave")
Our research efforts are broadly unchanged from\n2013\n(it takes time to craft good ideas!), and this will not be an exhaustive\nrecap. Instead, we’ll summarise them here and point to our\npapers\nthat describe the work in detail.
\nThe MirageOS really found its own feet in\n2014, with a summer 2.0\nrelease\nand an extensive end-of-year\nrecap. The most notable\nthing has been how well the MirageOS research work has melded with\nthe core OCaml Labs efforts, since much of it has been constructing\ngood quality OCaml libraries to plug holes in the ecosystem. It also\nserved to make us use OPAM on a day-to-day basis for our own work,\nthus creating an effective feedback loop between open-source and\nresearch.
\nIn the Trilogy2 and\nUCN EU projects, we built out\nMirageOS features such as the\nJitsu toolstack\nfor the “just-in-time” summoning of unikernels in response to DNS\nrequests. This paper will be presented next month at UlSENIX\nNSDI. It also drove the\ndevelopment of the ARMv7\nport, an\narchitecture for which OCaml has an excellent native code generator,\nas well as more experimental forays into BitCoin incentive\nschemes for distributed systems.
\nThe Irmin Git-like branchable store created by\nThomas Gazagnaire matured, with Dave Scott\nprototyping a complex\nport of the XenStore database\nto Irmin, thus letting us show off debugging systems with\nGit. We had a paper\naccepted on some early datastructures accepted at\nJFLA, and\nThomas Leonard is building the JavaScript backend for running\nin-browser, while Yan Schvartzshnaider is experimenting with graph\nprocessing\nover the DAG representation for privacy-friendly queries. KC is\ninvestigating how to adapt his PLDI 2015 paper on\nQuelea into using\nIrmin as a backend as well.
\nThe Higher kinded\npolymorphism library written by Jeremy Yallop and Leo White was\npublished in FLOPS 2014,\nforming a basis for building more complex use-cases that need the\nflexibility of higher kinded types without requiring functorising\ncode.
\nOur long standing research into personal online\nprivacy led to our next system target that uses\nunikernels: the Databox paper\noutlines the architecture, and was covered in the\nGuardian\nnewspaper. Jon Crowcroft led the establishment of the Cambridge wing of\nthe Microsoft Cloud Computing Research\nCenter to consider the legal aspect of\nthings, and so we have made forays outside of technology into\nconsidering the implications of region-specific\nclouds as well.
\nSome of the most exciting work done in the group as part of the\nREMS and NaaS projects\ncame towards the end of 2014 and start of 2015, with multiple\nsubmissions going into top conferences. Unfortunately, due to most of\nthem being double blind reviewed, we cannot link to the papers yet. Keep\nan eye on the blog and published paper\nset, or\nask us directly about what’s been going on!
\nAs spring breaks and the weather (almost) becomes bearable again, we’re\nsetting our work priorities for the remainder of the year.
\nTooling Cohesion: The entire core team is focussed on fusing\ntogether the individual tools that have been created last year into\na cohesive OCaml Platform release that covers the lifecycle of\ndocumentation, testing and build. This is being managed by Amir\nChaudhry. OPAM remains at the heart of this strategy, and Louis\nGesbert and Thomas Gazagnaire have settled on the OPAM 1.3\nroadmap\n(summary).
\nMulticore: KC Sivaramakrishnan has joined the core\nOCaml Labs fulltime to drive the multicore work into a publically\ntestable form. Leo White recently departed after many productive\nyears in Cambridge to head into a career in industry (but still\nremains very much involved with OCaml development!).
\nLanguage Evolution: Jeremy Yallop continues to drive our efforts\non staged programming, modular implicits, and a macro system for\nOCaml, all of which are key features that make building complex,\nreliable systems more tractable than ever.
\nI’d like to thank the entire\nteam and\nwider community for a wonderfully enjoyable 2014 and start of 2015, and\nam very thankful to the funding and support from Jane Street, Citrix,\nBritish Telecom, RCUK, EPSRC, DARPA and the EU FP7 that made it all\npossible. As always, please feel free to contact any of us directly with\nquestions, or reach out to me personally\nwith any queries, concerns or bars of chocolate as encouragement.
The call for papers for this year’s International Conference on Functional\nProgramming is about to close in two\nweeks, and over a hundred cutting-edge research papers will be submitted on the\ntheory, application, and experiences behind functional programming and type\ntheory. In addition to the main conference, there are also over 10 big\naffiliated workshops that\nrun throughout the week on topics ranging from specific languages\n(Erlang,\nHaskell,\nOCaml), the broader commercial\ncommunity, and even art and\nmusic.
\nThe ICFP conference experience can be a remarkable one for students. Some great\nideas have emerged from random corridor conversations between talks with the\nlikes of Phil Wadler, or from\nrain-soaked discussions with Simon PJ at\nMikeller, or in my case, from being convinced to write a book while in\na smoky Tokyo bar. This year, it will be held in the beautiful city of\nVancouver in the fall.
\nWe’re committed to growing the ICFP community, not just in numbers but also in\ndiversity. The Programming Language Mentoring\nWorkshop has been at capacity since it started\nand will run again. For the first time ever, I am really excited to announce\nthat the Ada Initiative will also be running an\nAlly Skills\nworkshop during the conference.
\nSustaining these activities and responsible growth means that we need to reach\never wider to support the activities of the (not-for-profit) ICFP conference.\nSo as this year’s industrial relations chair, I wish to invite any\norganization that wishes to support ICFP to get in touch with us (e-mail at\navsm2@cl.cam.ac.uk) and sponsor us. I’ve put an abridged version of the\ne-mail solicitation below that describes the benefits. Sponsorship can start as\nlow as $500 and is often tax-deductible in many countries.
\n\nI’m writing to ask if you would be willing to provide corporate financial support for the 20th ACM SIGPLAN International Conference on Functional Programming (ICFP), which takes place in Vancouver, Canada, from August 30th through September 5th, 2015:
\n\nhttp://icfpconference.org/icfp2015/\nCorporate support funds are primarily used to subsidize students – the lifeblood of our community – and in turn serve to raise the community profile of the supporting companies through a high-profile industrial recruitment event.
\nLast year, unprecedented levels of support from you and folks like you at over 25 companies and institutions made it possible for students from all over the world to attend ICFP 2014 in Sweden. The Industrial Reception, open to all attendees, was by all accounts a roaring success. All 2014 sponsoring companies had the opportunity to interact with the gathered students, academics, and software professionals.
\nThis year, let’s build on that success and continue to grow our community, and bring even more students to ICFP 2015 in Vancouver!
\nYour generosity will make it possible for students from all over the world to attend ICFP, the premier conference in functional programming. There, they will meet luminaries in the field, as well as people who’ve built a successful career and/or business on functional programming. They will return home inspired to continue pursuing functional programming in the confidence that exciting future careers await them. For the first time, we will also host an Ally Skills workshop by the Ada Foundation, as well as continue the successful student mentoring workshop from previous years.
\nThis year, we’re continuing a similar system of levels of financial support as last year. Our goal is to enable smaller companies to contribute while allowing larger companies to be as generous as they wish (with additional benefits, in recognition of that generosity).
\nThe support levels, and their associated benefits and pledge amounts and benefits are as follows (costs in US dollars).
\nBronze: $500: Logo on website, poster at industrial reception, listed in proceedings.
\nSilver: $2500: As above plus: logo in proceedings, logo on publicity materials (e.g., posters, etc.)
\nGold: $5000: As above plus: named supporter of industrial reception with opportunity to speak to the audience, and opportunity to include branded merchandise in participants’ swag bag.
\nPlatinum: $10000: As above plus: named supporter of whole event, logo on lanyards, badge ribbon, table/booth-like space available (in coffee break areas), other negotiated benefits (subject to ACM restrictions on commercial involvement).
\nThank you for your time and especially for your generosity! I look forward to seeing you in Vancouver. If you are willing to be a sponsor, it would be helpful to hear back by March 9th to help us plan and budget.
\n
If you are interested, please get in touch with me or any of the organizing committee. If you’re interested in helping out ICFP in a non-financial capacity (for example, as a student volunteer), then there will also be plenty of opportunities to sign up later in the year.
", "url": "https://anil.recoil.org/notes/icfp15-call-for-sponsorships", "title": "ICFP 2015 - a call for sponsorship and how you can help", "summary": "ICFP 2015 seeks sponsors to support students and growth of the functional programming community.", "date_published": "2015-02-18T00:00:00.000000Z", "date_modified": "2015-02-18T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "icfp", "service" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/0bc235e0-b154-4cbf-a84a-61240f16d60a-1", "content_html": "I hopped over to Berlin to give the keynote at BOB 2015 on functional operating systems. The talk focused on how we can apply functional programming principles to build unikernels with MirageOS, compiling OCaml applications directly into specialized virtual machine images that run on the Xen hypervisor. This represents a radical rethinking of the traditional OS stack, eliminating much of the bloat and attack surface of conventional operating systems. If you're in the region, I highly recommend attending BOB as a superbly organised conference with a diverse and interesting crowd of functional programmers.
", "url": "https://anil.recoil.org/notes/0bc235e0-b154-4cbf-a84a-61240f16d60a-1", "title": "Delivered keynote at BOB 2015 on MirageOS", "summary": "Opening keynote at BOB 2015 conference in Berlin on functional operating systems and MirageOS unikernels.", "date_published": "2015-01-23T00:00:00.000000Z", "date_modified": "2015-01-23T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "fp", "unikernels", "ocaml", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2014-sigops-raft-1", "content_html": "Paper on reproducing the Raft consensus protocol, published in ACM SIGOPS Operating Systems Review with Heidi Howard, Malte Schwarzkopf and Jon Crowcroft. We developed a clean-slate OCaml implementation of Raft and built an event-driven simulation framework to test it. The paper proposed several optimizations to the protocol and empirically validated Raft's correctness invariants, while also evaluating whether it lived up to its claims of being more understandable than Paxos.
Paper on mergeable data structures using Irmin (nee Irminsule) at JFLA 2015, the French-language functional programming conference. Working with Benjamin Farinier and Thomas Gazagnaire, we explored the theoretical foundations of mergeable persistent data structures. This formalized the ideas from Irminsule into a more general framework for building eventually-consistent distributed applications with type-safe merge operations.
", "url": "https://anil.recoil.org/notes/2015-jfla-irmin-1", "title": "Mergeable persistent data structures", "summary": "Paper on mergeable data structures using Irmin presented at JFLA 2015.", "date_published": "2015-01-01T00:00:00.000000Z", "date_modified": "2015-01-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "irmin", "ocaml", "database", "data-structures" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2015-jfla-irmin.pdf", "mime_type": "application/pdf", "title": "Mergeable persistent data structures" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2013-cufp-scribe-1", "content_html": "Published the scribe's report for CUFP 2013 in JFP. Continuing our role as scribes for the Commercial Users of Functional Programming workshop series, we documented the 2013 edition held in conjunction with ICFP. This report in the Journal of Functional Programming captures another year of industrial FP adoption, showing how the community continues to grow and functional techniques become more mainstream in production software development. The cumulative record of these workshop reports provides valuable insight into the evolution of practical FP.
Gave Codemesh 2014 talk on Nymote, exploring personal data management with unikernels and self-hosting. The talk focused on how individuals could "git their own cloud" by running secure, lightweight services for managing personal data using MirageOS and Irmin. This was part of our broader vision for giving people control over their own data through easy-to-deploy unikernel applications, moving away from dependence on centralized cloud providers.
", "url": "https://anil.recoil.org/notes/644914a5-a40b-4ef7-bb17-cea43c95dd09-1", "title": "Codemesh 2014: Nymote: Git Your Own Cloud Here", "summary": "Talk on Nymote presented at Codemesh 2014.", "date_published": "2014-12-17T00:00:00.000000Z", "date_modified": "2014-12-17T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "nymote", "cloud", "mirageos", "personal-data", "privacy" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/46968fa0-e5bd-4df8-98e1-3cf88d9b31e5-1", "content_html": "New Directions in Operating Systems talk on Jitsu at a workshop in London. Jitsu demonstrated just-in-time summoning of unikernels - the ability to boot MirageOS unikernels in milliseconds in response to network requests. This was a breakthrough for unikernel practicality, showing that you could have the security and efficiency benefits of unikernels without sacrificing responsiveness. The system worked by keeping unikernels in a suspended state and waking them up on demand, dramatically reducing the overhead compared to traditional VMs. It was a fun piece of work that combined DNS tricks with hypervisor optimizations to create something that felt almost magical in its speed.
", "url": "https://anil.recoil.org/notes/46968fa0-e5bd-4df8-98e1-3cf88d9b31e5-1", "title": "Jitsu: Just-in-Time Summoning of Unikernels (new directions in operating systems)", "summary": "Talk on Jitsu at New Directions in Operating Systems conference.", "date_published": "2014-11-25T00:00:00.000000Z", "date_modified": "2014-11-25T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "systems", "cloud", "jitsu" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2014-regional-clouds-1", "content_html": "Report on regional cloud computing law available as Cambridge tech report TR-863. This interdisciplinary work with Jatinder Singh, Jean Bacon, Jon Crowcroft and legal scholars examined the technical considerations needed to comply with regional data protection laws. The report explored how cloud infrastructure could be designed to respect jurisdictional boundaries - prescient work given the subsequent explosion of GDPR and data sovereignty concerns.
At the Xen Summit speaking about branch consistency for Xen Stub Domains, explaining how MirageOS 2.0 could leverage Irmin for distributed systems. The talk demonstrated how combining Irmin's branch-consistent storage with MirageOS could build reliable stub domains for Xen hosts. This work showed how functional data structures and version control concepts could be applied to building more reliable virtualization infrastructure.
", "url": "https://anil.recoil.org/notes/c9273fa0-802f-4d2b-8f0d-db383943564e-1", "title": "MirageOS 2.0: branch consistency for Xen Stub Domains", "summary": "Xen Summit talk on implementing branch consistency for MirageOS 2.0 Xen Stub Domains.", "date_published": "2014-10-17T00:00:00.000000Z", "date_modified": "2014-10-17T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "xen", "unikernels", "distributed", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/4390c1d0-ed4f-4c01-9e10-dab2a3faed7a-1", "content_html": "Talk on the OCaml Platform reaching v1.0 at the 2014 OCaml Workshop. This was a significant milestone - declaring version 1.0 of the OCaml Platform meant we had a stable, coherent set of tools that developers could rely on. This included opam for package management, core libraries, and documentation infrastructure. Reaching v1.0 was important for signaling to the community that the platform was production-ready and we were committed to maintaining backwards compatibility. It marked the transition from experimental tooling to something that could support serious industrial and academic use of OCaml.
", "url": "https://anil.recoil.org/notes/4390c1d0-ed4f-4c01-9e10-dab2a3faed7a-1", "title": "OCaml 2014: The OCaml Platform v1.0", "summary": "Talk on the OCaml Platform reaching v1.0.", "date_published": "2014-09-05T00:00:00.000000Z", "date_modified": "2014-09-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/ed84b2eb-1b93-4dc3-b746-63a4af13d4ea-1", "content_html": "Gave Haskell Symposium 2014 Keynote on functional OS design, somewhat nervously talking to a room full of Haskellers about OCaml modules. The keynote explored how functional programming principles could be applied to operating systems design, using MirageOS as a concrete example. I discussed the benefits of OCaml's module system for building composable OS components and how type-driven development could improve systems security and reliability.
", "url": "https://anil.recoil.org/notes/ed84b2eb-1b93-4dc3-b746-63a4af13d4ea-1", "title": "Haskell Symposium 2014 Keynote on functional OS design", "summary": "Keynote presentation at Haskell Symposium 2014 on functional operating system design.", "date_published": "2014-09-05T00:00:00.000000Z", "date_modified": "2014-09-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "haskell", "fp", "systems", "unikernels", "mirageos" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2014-oud-platform-1", "content_html": "Paper on the OCaml Platform status, marking the release of version 1.0. This was a major milestone showing how the community-driven toolchain had matured over the year since v0.1. Working with Jeremie Dimino, Louis Gesbert, Mark Shinwell and others, we consolidated the core development tools that OCaml programmers rely on daily, including improvements to opam, the package manager.
", "url": "https://anil.recoil.org/notes/2014-oud-platform-1", "title": "The OCaml Platform v1.0", "summary": "Paper on OCaml Platform version 1.0 status and features.", "date_published": "2014-09-01T00:00:00.000000Z", "date_modified": "2014-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling", "devtools", "community" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2014-oud-platform.pdf", "mime_type": "application/pdf", "title": "The OCaml Platform v1.0" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2014-oud-multicore-1", "content_html": "First paper on multicore OCaml's design at the OCaml Workshop, presented with Stephen Dolan and Leo White. This initial work laid out the vision for adding parallelism to OCaml while maintaining backwards compatibility - a challenging goal that would take several more years of research and development to achieve. The paper introduced the core concepts that would eventually lead to OCaml 5.0's multicore support.
", "url": "https://anil.recoil.org/notes/2014-oud-multicore-1", "title": "Multicore OCaml", "summary": "First paper on multicore OCaml's design presented at the OCaml Workshop.", "date_published": "2014-09-01T00:00:00.000000Z", "date_modified": "2014-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "multicore", "concurrency", "fp", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2014-oud-multicore.pdf", "mime_type": "application/pdf", "title": "Multicore OCaml" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2014-oud-irminsule-1", "content_html": "Paper at the OCaml Workshop introducing Irmin (then called Irminsule), a branch-consistent distributed library database that brings Git-like operations to application data. This work with Thomas Gazagnaire and collaborators introduced mergeable persistent data structures that could be used to build eventually-consistent distributed systems. Irmin has since become a foundational component of MirageOS and other distributed OCaml applications.
", "url": "https://anil.recoil.org/notes/2014-oud-irminsule-1", "title": "Irminsule: a branch-consistent distributed library database", "summary": "Paper on Irmin branch-consistent distributed database library supporting Git-like operations at OCaml Workshop 2014.", "date_published": "2014-09-01T00:00:00.000000Z", "date_modified": "2014-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "irmin", "ocaml", "distributed", "databases", "mirageos" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2014-oud-irminsule.pdf", "mime_type": "application/pdf", "title": "Irminsule: a branch-consistent distributed library database" } ], "_references": [] }, { "id": "https://doi.org/10.59350/p1v4c-7h249", "content_html": "It's the ever-exciting week of the International Conference on\nFunctional Programming again in Sweden,\nand this time OCaml Labs has a variety of talks,\ntutorials and keynotes to deliver throughout the week. This post\nsummarises all them so you can navigate your way to the right session.\nRemember that once you register for a particular day at ICFP, you can\nmove between workshops and tutorials as you please.
\n![\"%r\"](\"/images/gothenburg.webp\" "\\"Gothenburg,")
\nQuick links to the below in date order:
The first round of talks are about improvements to the core OCaml\nlanguage and runtime.
\nLeo White and Frederic Bour have been taking inspiration from Scala\nimplicits and Modular Type\nClasses by\nDreyer et al, and will describe the design and implementation of a\nsystem for ad-hoc polymorphism in OCaml based on passing implicit module\nparameters to functions based on their module type.
\nThis provides a concise way to write functions to print or manipulate\nvalues generically, while maintaining the ML spirit of explicit\nmodularity. You can actually get get a taste of this new feature ahead\nof the talk, thanks to a new facility in OCaml: we can compile any OPAM\nswitch directly into an interactive JavaScript notebook thanks to\niocamljs by Andy\nRay.
\n\nCurrently, threading in OCaml is only supported by means of a global\nlock, allowing at most one thread to run OCaml code at any time. Stephen\nDolan, Leo White and Anil Madhavapeddy have been building on the early\ndesign of a multicore\nOCaml runtime that they started in January, and now have a (early)\nprototype of a runtime design that is capable of shared memory\nparallelism.
\nLeo White has been working with Jacques\nGarrigue on adding support\nfor module aliases into OCaml. This significantly improves the\ncompilation speed and executable binary sizes when using large libraries\nsuch as\nCore/Async.
\nAlan Mycroft has been working with Tomas Petricek and Dominic Orchard on\ndefining a broader notion of context than just variables in scope. Tomas\nwill be presenting a research paper on developing a generalized coeffect\nsystem with annotations indexed by a correct shape.
\nWe released Mirage OS\n2.0 in July,\nand there will be several talks diving into some of the new features you\nmay have read on the blog.
\nSince MirageOS is a\nunikernel\nwritten entirely in OCaml, it makes perfect sense to describe it in\ndetail to our friends over at the Haskell\nSymposium and reflect on\nsome of the design implications between Haskell type-classes and OCaml\nfunctors and metaprogramming. Anil Madhavapeddy will be doing just that\nin a Friday morning keynote at the Haskell Symposium.
\nHannes Menhert and David Kaloper have been working\nhard on integrating a\npure OCaml Transport Layer Security stack into Mirage OS. They’ll talk\nabout the design principles underlying the library, and reflect on the\nnext steps to build a TLS stack that we can rely on not to been more\ninsecure than telnet.
\nHannes will also continue his travels and deliver a couple of talks the\nweek after ICFP on the same topic in Denmark, so you can still see it if\nyou happen to miss this week’s presentation:
\nIrmin is an OCaml library to persist\nand synchronize distributed data structures both on-disk and in-memory.\nIt enables a style of programming very similar to the Git workflow,\nwhere distributed nodes fork, fetch, merge and push data between each\nother. The general idea is that you want every active node to get a\nlocal (partial) copy of a global database and always be very explicit\nabout how and when data is shared and migrated.
\nThis has been a big collaborative effort lead by Thomas Gazagnaire, and\nincludes contributions from Amir Chaudhry, Anil Madhavapeddy, Richard\nMortier, David Scott, David Sheets, Gregory Tsipenyuk, Jon Crowcroft.\nWe’ll be demonstrating Irmin in\naction, so please come\nalong if you’ve got any interesting applications you would like to talk\nto us about.
\nMirage OS lets the programmer build modular operating system components\nusing a combination of OCaml functors and generative metaprogramming.\nThis ensures portability across both Unix binaries and Xen unikernels,\nwhile preserving a usable developer workflow.
\nThe core Mirage OS team of Anil Madhavapeddy, Thomas Gazagnaire, David\nScott and Richard Mortier will be talking about the details of the\nfunctor combinators that make all this possible, and doing a live\ndemonstration of it running on a tiny ARM\nboard!
\nLeo White and Jeremy Yallop (with much helpful assistance from Daniel\nBuenzli) will be giving a rather different OCaml tutorial from the usual\nfare: they are taking you on a journey of building a variant of the\npopular 2048 game in pure\nOCaml, and compiling it to JavaScript using the\njs_of_ocaml compiler. This is a\nvery pragmatic introduction to using statically typed functional\nprogramming combined with efficient compilation to JavaScript.
\n\n\nIn this tutorial, we will first introduce the basics of OCaml using an\ninteractive environment running in a web browser, as well as a local\ninstall of OCaml using the OPAM package manager. We will also explore\nhow to compile OCaml to JavaScript using the js_of_ocaml tool.
\n
The tutorial is focused around writing the 2048 logic, which will then\nbe compiled with js_of_ocaml and linked together with a frontend based\non (a pre-release version of) Useri, React, Gg and Vg, thanks to Daniel\nBuenzli. There’ll also be appearances from OPAM, IOCaml, Qcheck and\nOUnit.
\nThere will also be a limited supply of special edition OCaml-branded USB\nsticks for the first tutorial attendees, so get here early for your\nexclusive swag!
\nThe group here has been working hard all summer to pull together an\nintegrated demonstration of the new generation of OCaml tools being\nbuilt around the increasingly popular OPAM\npackage manager. Anil Madhavapeddy will demonstrate all of these pieces\nin the OCaml Workshop, with guest appearances of work from Amir\nChaudhry, Daniel Buenzli, Jeremie Diminio, Thomas Gazagnaire, Louis\nGesbert, Thomas Leonard, David Sheets, Mark Shinwell, Christophe\nTroestler, Leo White and Jeremy Yallop.
\n\n\nThe OCaml Platform combines the OCaml compiler toolchain with a\ncoherent set of tools for build, documentation, testing and IDE\nintegration. The project is a collaborative effort across the OCaml\ncommunity, tied together by the OCaml Labs group in Cambridge and with\nother major contributors.
\n
Thomas Leonard will also be delivering a separate talk about\ncross-platform binary installation via his\n0install library, which works on\na variety of platforms ranging from Windows, Linux and MacOS X. He\nrecently rewrote it in OCaml from\nPython,\nand will be sharing his experiences on how this went as a new OCaml\nuser, as well as deliver an introduction to 0install.
\nHeidi Howard and Leonhard Markert are acting as student volunteers at\nthis years ICFP, and assisting with videoing various workshops such as\nCUFP Tutorials, Haskell Symposium, the Workshop on Functional\nHigh-Performance Computing and the ML Family Workshop. Follow their live\nblogging on the Systems Research Group\nSysBlog and leave comments about any\nsessions you’d like to know more about!
\nAnil Madhavapeddy is the ICFP industrial relations chair and will be\nhosting an Industrial Reception on Thursday 4th September in the Museum\nof World\nCulture\nstarting from 7pm. There will be wine, food and some inspirational talks from the ICFP\nsponsors that not only make the conference possible, but provide an\navenue for the academic work to make its way out into industry (grad\nstudents that are job hunting: this is where you get to chat to folk\nhiring FP talent).
\nThis list hasn’t been exhaustive, and only covers the activities of my\ngroup in OCaml Labs and the Systems Research\nGroup at Cambridge. There are\nnumerous other talks from the Cambridge Computer Lab during the week,\nbut the artistic highlight will be on Saturday evening following the\nCUFP talks: Sam Aaron\nwill be doing a live musical\nperformance\nsometime after 8pm at 3vaningen. Sounds like\na perfect way to wind down after what’s gearing to up to be an intense\nICFP 2014. I look forward to seeing old friends and making new ones in\nGothenburg soon!
", "url": "https://anil.recoil.org/notes/ocaml-labs-at-icfp-2014", "title": "Talks from OCaml Labs during ICFP 2014", "summary": "OCaml Labs talks at ICFP 2014, covering language improvements & MirageOS", "date_published": "2014-08-31T00:00:00.000000Z", "date_modified": "2014-08-31T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "icfp", "livenotes" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/5cdf2eef-9053-428e-b8b3-ab5ae274c129-1", "content_html": "Appeared on FLOSS Weekly 302 about Open Mirage, where Randal Schwartz and Simon Phipps interviewed me about the MirageOS project. We discussed how unikernels bring functional programming principles to systems programming, the benefits of using OCaml for building secure network services, and the open source community growing around the project. It was a great opportunity to introduce MirageOS to a broader open source audience.
", "url": "https://anil.recoil.org/notes/5cdf2eef-9053-428e-b8b3-ab5ae274c129-1", "title": "FLOSS Weekly 302: Open Mirage", "summary": "Interview with Randal Schwartz and Simon Phipps about MirageOS on FLOSS Weekly podcast episode 302.", "date_published": "2014-07-23T00:00:00.000000Z", "date_modified": "2014-07-23T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "unikernels", "ocaml", "podcast" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/announcing-mirageos-2", "content_html": "This is a big release for us; after the first version came out earlier in the year, we added in support for ARM devices, a new storage subsystem called Irmin and even a pure OCaml TLS stack.
", "url": "https://anil.recoil.org/notes/announcing-mirageos-2", "external_url": "https://mirageos.org/blog/announcing-mirage-20-release", "title": "MirageOS v2.0: a recap of new features", "summary": "MirageOS v2.0 adds ARM support, Irmin storage and OCaml TLS stack.", "date_published": "2014-07-22T00:00:00.000000Z", "date_modified": "2014-07-22T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/announcing-mirageos-1-2", "content_html": "I announce a point release of MirageOS 1.x, and the exciting run up to the major MirageOS 2.0 release which has lots of new features. The number of Mirage users is growing steadily!
", "url": "https://anil.recoil.org/notes/announcing-mirageos-1-2", "external_url": "https://mirageos.org/blog/mirage-1.2-released", "title": "MirageOS v1.2 released and the runup to 2.0", "summary": "MirageOS v1.2 released, paving way for MirageOS 2.0 with new features.", "date_published": "2014-07-08T00:00:00.000000Z", "date_modified": "2014-07-08T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "mirageos", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/d5411e25-7845-41e8-b3ec-ab3c33ce13c8-1", "content_html": "Appeared on SE Radio Episode 204 about Mirage and OCaml, talking with Robert Blumen on the Software Engineering Radio podcast. We discussed the Mirage cloud operating system architecture, how OCaml's type system and functional features made it ideal for systems programming, and the practical benefits of unikernels for cloud deployment. The conversation covered both the technical foundations and real-world applications of our work.
", "url": "https://anil.recoil.org/notes/d5411e25-7845-41e8-b3ec-ab3c33ce13c8-1", "title": "SE Radio Episode 204: Anil Madhavapeddy on the Mirage Cloud Operating System and the OCaml Language", "summary": "Software Engineering Radio podcast episode discussing Mirage cloud operating system and OCaml language.", "date_published": "2014-05-01T00:00:00.000000Z", "date_modified": "2014-05-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "ocaml", "unikernels", "podcast", "interview" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/codio-now-has-opam-support", "content_html": "I noticed an offhand tweet from Phil Tomson about Codio adding OPAM support, and naturally had to take a quick look. I was really impressed by the whole process, and ended up building the Mirage Xen website unikernel directly from my web browser in less than a minute, including registration!
\ngit project and pointed it at the mirage-www repository.\n\nI notice Codio supports OCaml and opam on the server side now.\n— phil tomson (@philtor)\nMarch 26, 2014
\n
$ parts install opam\n$ opam init -a\n$ eval `opam config env`\n$ opam install mirage-www -y\n$ make MODE=xen\nThen have a cup of coffee while the box builds, and you have a mir-www.xen, all from your web browser! Codio has a number of deployment options available too, so you should be able to hook up a Git-based workflow using some combination of Travis or other CI service.
This is the first time I’ve ever been impressed by an online editor, and might consider moving away from my beloved vi...
", "url": "https://anil.recoil.org/notes/codio-now-has-opam-support", "title": "Codio: build Mirage unikernels from a browser", "summary": "Build Mirage unikernels from a browser with Codio's OPAM support and interactive Ubuntu shell.", "date_published": "2014-03-26T00:00:00.000000Z", "date_modified": "2014-03-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "ocaml", "web" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/ocaml-github-and-opam", "content_html": "Gabriel Scherer announced an\nexperiment to\nhost OCaml compiler pull requests on\nGitHub for six months. There is\na general feeling that GitHub would be a more modern hosting platform\nthan the venerable but reliable\nMantis setup that has\nin place for over a decade, but the only way to find out for sure is by\ntrying it out for a while.
\nOne of the great benefits of using GitHub is their excellent\nAPI to easily automate workflows\naround issues and pull requests. After a suggestion from Jeremy Yallop\nand David Sheets over lunch, I decided to use this to make it easier to\nlocally apply compiler patches. OPAM has a great compiler\nswitch\nfeature that lets you run simultaneous OCaml installations and swap\nbetween them easily.
\nFor instance, the default setting gives you access\nto:
\n$ opam switch\nsystem C system System compiler (4.01.0)\n-- -- 3.11.2 Official 3.11.2 release\n-- -- 3.12.1 Official 3.12.1 release\n-- -- 4.00.0 Official 4.00.0 release\n-- -- 4.00.1 Official 4.00.1 release\n-- -- 4.01.0 Official 4.01.0 release\n-- -- 4.01.0beta1 Beta1 release of 4.01.0\nI used my GitHub API bindings to\nknock up a script that converts every GitHub pull request into a custom\ncompiler switch. You can see these by passing the --all option to\nopam switch, as follows:
$ opam switch --all\n-- -- 4.02.0dev+pr10 Add String.{split,rsplit}\n-- -- 4.02.0dev+pr13 Add String.{cut,rcut}.\n-- -- 4.02.0dev+pr14 Add absolute directory names to bytecode format for ocamldebug to use\n-- -- 4.02.0dev+pr15 replace String.blit by String.unsafe_blit\n-- -- 4.02.0dev+pr17 Cmm arithmetic optimisations\n-- -- 4.02.0dev+pr18 Patch for issue 5584\n-- -- 4.02.0dev+pr2 Parse -.x**2. (unary -.) as -.(x**2.). Fix PR#3414\n-- -- 4.02.0dev+pr20 OCamlbuild: Fix the check of ocamlfind\n-- -- 4.02.0dev+pr3 Extend record punning to allow destructuring.\n-- -- 4.02.0dev+pr4 Fix for PR#4832 (Filling bigarrays may block out runtime)\n-- -- 4.02.0dev+pr6 Warn user when a type variable in a type constraint has been instantiated.\n-- -- 4.02.0dev+pr7 Extend ocamllex with actions before refilling\n-- -- 4.02.0dev+pr8 Adds a .gitignore to ignore all generated files during `make world.opt'\n-- -- 4.02.0dev+pr9 FreeBSD 10 uses clang by default, with gcc not available by default\n-- -- 4.02.0dev+trunk latest trunk snapshot\nTesting the impact of a particular compiler switch is now pretty\nstraightforward. If you want to play with Stephen Dolan’s optimized\narithmetic operations, for\ninstance, you just need to do:
\n$ opam switch 4.02.0dev+pr17\n$ eval `opam config env`\nAnd your local environment now points to the patched OCaml compiler. For\nthe curious, the scripts to generate the OPAM pull requests are in my\navsm/opam-sync-github-prs\nrepository. It contains an example of how to query active pull requests,\nand also to create a new cross-repository pull request (using the git\njar binary from my GitHub\nbindings). The scripts run daily for now, and delete switches once the\ncorresponding pull request is closed. Just run opam update to retrieve\nthe latest switch set from the upstream OPAM package\nrepository.
The call for papers for this year’s International Conference on Functional Programming has just closed, with around a hundred cutting-edge research papers submitted on the theory, application, and experiences behind functional programming. This marks just the beginning of sorting out the program, as there are also over 10 big affiliated workshops that run throughout the week on topics ranging from specific languages (Erlang, Haskell, OCaml), the broader commercial community, and even art and music.
\nThe ICFP conference experience can be a remarkable one for students. Some great ideas have emerged from random corridor conversations between talks with the likes of Phil Wadler, or from rain-soaked discussions with Simon PJ at Mikeller, or in my case, from being convinced to write a book while in a smoky Tokyo bar.
\nFunctional programming worldwide has been growing ever more popular in 2014 (and lucrative). We’re committed to growing the ICFP community, not just in numbers but also in diversity. We had a record number of sponsors in 2013, and sustaining the growth means that we need to reach ever wider to support the activities of the (not-for-profit) conference.
\nSo as this year’s industrial relations chair, I thought I’d throw the gates open and invite any organization that wishes to support FP to get in touch with us (e-mail at avsm2@cl.cam.ac.uk) and sponsor us. I’ve put an abridged version of the e-mail solicitation below that describes the benefits. Sponsorship can start as low as $500 and is often tax deductible in many countries.
\n\nI’m writing to ask if you would be willing to provide corporate financial support for the 19th ACM SIGPLAN International Conference on Functional Programming (ICFP), which takes place in Gothenburg, Sweden, from September 1st through 3rd, 2014:
\nhttp://icfpconference.org/icfp2014/
\nCorporate support funds are primarily used to subsidize students – the lifeblood of our community – and in turn serve to raise the community profile of the supporting companies through a high-profile industrial recruitment event.
\nLast year, unprecedented levels of support from you and folks like you at over 25 companies and institutions made it possible for students from all over the world to attend ICFP 2013 in Boston. The Industrial Reception, open to all attendees, was by all accounts a roaring success. All 2013 sponsoring companies had the opportunity to speak to the gathered students, academics, and software professionals.
\nThis year, let’s build on that success and continue to grow our community, and bring even more students to ICFP 2014 in Sweden!
\nYour generosity will make it possible for students from all over the world to attend ICFP, the premier conference in functional programming. There, they will meet luminaries in the field, as well as people who’ve built a successful career and/or business on functional programming. They will return home inspired to continue pursuing functional programming in the confidence that exciting future careers await them.
\nThis year, we’re continuing a similar system of levels of financial support as last year. Our goal is to enable smaller companies to contribute while allowing larger companies to be as generous as they wish (with additional benefits, in recognition of that generosity).
\nThe support levels, and their associated benefits and pledge amounts and benefits are as follows (costs in US dollars).
\nBronze: $500: Logo on website, poster at industrial reception, listed in proceedings.
\nSilver: $2500: As above plus: logo in proceedings, logo on publicity materials (e.g., posters, etc.)
\nGold: $5000: As above plus: named supporter of industrial reception, opportunity to include branded merchandise in participants’ swag bag.
\nPlatinum: $10000: As above plus: named supporter of whole event, logo on lanyards, badge ribbon, table/booth-like space available (in coffee break areas), other negotiated benefits (subject to ACM restrictions on commercial involvement).
\n
If you are interested, please get in touch with me or any of the organizing committee.\nIf you’re interested in helping out ICFP in a non-financial capacity (for example as a student volunteer), then there will also be plenty of opportunity to sign up later in the year.
", "url": "https://anil.recoil.org/notes/icfp-call-for-sponsorships", "title": "ICFP 2014 - a call for sponsorship and how you can help", "summary": "Support functional programming community growth by sponsoring ICFP 2014, starting at $500.", "date_published": "2014-03-03T00:00:00.000000Z", "date_modified": "2014-03-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2010-iswp-dustclouds-1", "content_html": "Paper on building dust clouds for anonymous communication. This work with Richard Mortier, Theodore Hong, and others explored how cloud computing platforms like Amazon EC2 could facilitate anonymous communications. By dynamically leasing diverse collections of virtual machines spread across the world, we could enhance systems like Tor with the flexibility and geographic diversity of cloud infrastructure. The paper discusses both the opportunities this creates for privacy-preserving communications and the unique challenges that arise from using commercial cloud platforms for anonymity.
The Communications of the ACM have just published an article that Dave Scott and I wrote providing a broader background on the concept of Unikernels that we’ve been working on since about 2003, when we started building Melange and the Xen toolstack. You can read either the print article (requires an ACM subscription) or the open access version on the ACM Queue.
\n![\"%r\"](\"/images/acm-queue-unikernels-ss.webp\")
\nThere's been some interesting discussion about it already online:
Two of the most interesting bits of feedback for me personally came from Butler Lampson (via Jon Crowcroft) and Robert Harper, two computer scientists who have made key contributions to operating systems and programming languages and provided some broader perspective.
\nButler Lampson points out (edited for the web):
\n\n\nI found the Mirage work quite interesting: a 21st-century version of things that we did at Xerox in the 1970s. Of course, the application domain is quite different, and so is the whole-program optimization. And we couldn’t afford garbage collection, so freeing storage was not type-safe. But there are lots of interesting parallels.
\nThe “OS as libraries” idea was what made it possible to fit big applications into the Alto’s 128k bytes of memory:
\nLampson and Sproull, An open operating system for a single-user machine, ACM Operating Systems Rev. 11, 5 (Dec. 1979), pp 98-105. ACM.
\nThe use of strong type-checking and interfaces for an OS was pioneered in [Mesa](http://en.wikipedia.org/wiki/Mesa_(programming_language%29) and [Pilot](http://en.wikipedia.org/wiki/Pilot_(operating_system%29):
\nLauer and Satterthwaite, The impact of Mesa on system design, Proc. 4th ICSE, Munich, Sep. 1979, pp 174-182.
\nRedell et al, Pilot: An Operating System for a Personal Computer, Comm. ACM 23, 2 (Feb 1980), pp 81-92 (from 7th SOSP, 1979). ACM.
\n
Robert Harper correctly points out some related work that was missing from our CACM article:
\nBoth Ensemble and FoxNet made strong echoes throughout the design of Mirage (and its precursor software such as Melange in 2007). The Mirage command-line tool uses staged computation to build a concrete application out of functors, and we are making this even more programmable via a new combinator-based functor types library that Thomas Gazagnaire built, and also experimenting with higher kinded polymorphic abstractions.
\nMy thanks to Butler Lampson and Robert Harper for making me go re-read their papers again, and I’d like to leave you with Malte Schwarzkopf’s OS Reading Group papers for other essential reading in this space. Many more citations immediately relevant to Mirage can also be found in our ASPLOS 2013 paper.
", "url": "https://anil.recoil.org/notes/unikernels-in-cacm", "title": "Unikernels, and the Rise of the Virtual Library Operating System", "summary": "Unikernels and virtual library operating systems are on the rise, changing the face of cloud computing.", "date_published": "2014-01-13T00:00:00.000000Z", "date_modified": "2014-01-13T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "cacm", "interview" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/dxt23-hxf18", "content_html": "This time last year in 2012, I had just\nannounced\nthe formation of a new group called OCaml\nLabs in the Cambridge\nComputer Lab that would combine research and\ncommunity work towards the practical application of functional\nprogramming. An incredible year has absolutely flown by, and I’ve put\ntogether this post to summarise what’s gone on, and point to our future\ndirections for 2014.
\nThe theme of our group was not to be pure research, but rather a hybrid\ngroup that would take on some of the load of day-to-day OCaml\nmaintenance from INRIA, as well as help grow the\nwider OCaml community. To this end, all of our projects have been highly\ncollaborative, often involving colleagues from\nOCamlPro, INRIA,\nJane Street, Lexifi\nand Citrix.
\nThis post covers progress in tooling, the compiler and\nlanguage, community efforts,\nresearch projects and concludes with our\npriorities for 2014.
\nAt the start of 2013, OCaml was in the interesting position of being a\nmature decades-old language with a small, loyal community of industrial\nusers who built mission critical applications using it. We had the\nopportunity to sit down with many of them at the OCaml\nConsortium meeting and prioritise\nwhere we started work. The answer came back clearly: while the compiler\nitself is legendary for its stability, the tooling around it (such as\npackage management) was a pressing problem.
\nOur solution to this tooling was centered around the\nOPAM package manager that\nOCamlPro released into beta just at the end of\n2012, and had its first stable release in March 2013. OPAM differs from\nmost system package managers by emphasising a flexible distributed\nworkflow that uses version constraints to ensure incompatible libraries\naren’t mixed up (important for the statically-typed OCaml that is very\ncareful about dependencies). Working closely with\nOCamlPro we developed a git-based workflow to\nmake it possible for users (both individual or industrial) to easily\nbuild up their own package repositories and redistribute OCaml code, and\nstarted curating the package\nrepository.
\nThe results have been satisfying: we started with an initial set of\naround 100 packages in OPAM (mostly imported by the 4 developers), and\nended 2013 with 587 unique packages and 2000 individual versions, with\ncontributions from 160 individuals. We now have a curated central\npackage repository for anyone\nto submit their OCaml code, several third-party remotes are maintained\n(e.g. the Xen Project\nand Ocsigen). We also\nregularly receive releases of the Core\nlibraries from Jane Street, and updates from sources as varied as\nFacebook,\nCoherent\nPDF,\nto the Frenetic\nSDN research.
\n![\"\"](\"/images/opam11-contributors-dec13.webp\" "\\"Number")
\n![\"\"](\"/images/opam11-packages-dec13.webp\" "\\"Total")
\n![\"\"](\"/images/opam11-unique-packages-dec13.webp\" "\\"Total")
A notable contribution from OCamlPro during this time was to\nclarify the\nlicensing on the package repository to be the liberal\nCC0, and also to pass\nownership to the OCaml organization on\nGitHub, where it’s now jointly maintained by OCaml Labs, OCamlPro and\nanyone else that wishes to contribute.
\nIt’s been quite interesting just watching all the varied code fly into\nthe repository, but stability quickly became a concern as the new\npackages piled up. OCaml compiles to native code on not just x86, but\nalso PowerPC, Sparc and\nARM\nCPUs. We kicked off various efforts into automated testing: firstly\nDavid Sheets built the\nOCamlot\ndaemon that would schedule builds across all the exotic hardware. Later\nin the year, the Travis service launched support\nfor testing from GitHub pull requests, and this became the front line of\nautomated\nchecking for\nall incoming new packages to OPAM.
\nA major headache with automated testing is usually setting up the right\nbuild environment with external library dependencies, and so we added\nDocker support\nto make it easier to bulk-build packages for local developer use, with\nthe results of builds available\npublically for anyone to help\ntriage. Unfortunately fixing the bugs themselves is still a very manual\nprocess, so more\nvolunteers are always welcome to help out!
\n\nWe’re going to be really seeing the rewards from all this effort as\nOCaml 4.02 development proceeds, since we can now adopt a data-driven\napproach to changing language features instead of guessing how much\nthird-party code will break. If your code is in OPAM, then it’ll be\ntested as new features such as module\naliases,\ninjectivity\nand extension\npoints show up.
The venerable\nOCamlDoc\ntool has done an admirable job for the last decade, but is increasingly\nshowing its age due to a lack of support for cross-referencing across\npackages. We started working on this problem in the summer when Vincent\nBotbol visited us on an internship,\nexpecting it to be a quick job to come up with something as good as\nHaskell’s excellent Haddock online\ndocumentation.
\nInstead, we ran into the "module wall": since OCaml makes it so easy to\nparameterise code over other modules, it makes it hard to generate\nstatic documentation without outputting hundreds of megabytes of HTML\nevery time. After some hard work from Vincent and Leo, we’ve got a\nworking prototype that lets you simply run\nopam install opam-doc && opam doc core async to generate package\ndocumentation. You can see the results for\nMirage online, but expect to see this\nintegrated into the main OCaml site for all OPAM packages as we work\nthrough polishing up the user interface.
The other behind-the-scenes effort for OPAM has been to keep the core\ncommand-line tool simple and stable, and to have it install OCaml\nlibraries that can be interfaced with by other tools to do\ndomain-specific tasks. Thomas Gazagnaire,\nLouis Gesbert and David\nSheets have been steadily hacking away at\nthis and we now have opamfu to\nrun operations over all packages, and an easy-to-template\nopam2web that generates the live\nopam.ocaml.org website.
\nThis makes OPAM easier to deploy within other organizations that want to\nintegrate it into their workflow. For example, the software\nsection of the OCaml\nLabs website is regularly generated from a search of all OPAM packages\ntagged ocamllabs. We also used it to rewrite the entire OPAM\nrepository in one epic\ndiff to add\nexternal library dependencies via a command-line\nshim.
All of this effort is geared towards making it easier to maintain\nreusable local OPAM installations. After several requests from big\nuniversities to help out their teaching needs, we’re putting together\nall the support needed to easily redistribute OPAM packages via an\n“OPAM-in-a-Box” command\nthat uses Docker containers to let you clone and do\nlightweight modifications of OCaml installations.
\nThis will also be useful for anyone who’d like to run tutorials or teach\nOCaml, without having to rely on flaky network connectivity at\nconference venues: a problem we’ve suffered\nfrom too!
\n![\"%r\"](\"/images/compiler-hacking.webp\" "\\"Compiling")
\nStarting to work on a real compiler can often be a daunting prospect,\nand so one initiative we started this year is to host regular compiler\nhacking\nsessions\nwhere people could find a curated list of\nfeatures to work\non, with the regular developers at hand to help out when people get\nstuck, and free beer and pizza to oil the coding wheels. This has worked\nout well, with around 20 people showing up on average for the three we\nheld, and several\npatches\nsubmitted upstream to OCaml. Gabriel\nScherer and Damien\nDoligez have been helping this\neffort by tagging junior\njobs\nin the OCaml Mantis bug tracker as they are filed.
Leo White started the year fresh out of\ncompleting his PhD with Alan Mycroft,\nand before he realized what he’d gotten himself into was working with\nAlain Frisch on the future of syntax\ntransformations in OCaml. We started off our first\nwg-camlp4 working group on\nthe new lists.ocaml.org host, and a spirited\ndiscussion\nstarted\nthat went\non\nand\non\nfor several months. It ended with a very satisfying design for a simpler\nextension points mechanism which Leo\npresented at\nthe OCaml 2013 workshop at ICFP, and is now merged into OCaml\n4.02-trunk.
\nNot all of the working groups were quite as successful in coming to a\nconclusion as the Camlp4 one. On the Platform mailing list, Gabriel\nScherer started a discussion on the design for\nnamespaces\nin OCaml. The resulting discussion was useful in separating multiple\nconcerns that were intermingled in the initial proposal, and Leo wrote a\ncomprehensive blog\npost on a\nproposed namespace design.
\nAfter further discussion at ICFP\n2013 with Jacques Garrigue later\nin the year, it turns out adding support for module\naliases would solve much\nof the cost associated with compiling large libraries such as\nCore, with no backwards compatibility\nissues. This solution has now been integrated into OCaml 4.02.0dev and\nis being tested with Core.
\nJeremy Yallop joined us in April, and he and Leo also leapt into the\ncore compiler and started triaging issues on the OCaml bug\ntracker. This seems unglamorous in the\nbeginning, but there rapidly turned out to be many fascinating threads\nthat shed light on OCaml’s design and implementation through seemingly\nharmless bugs. Here is a pick of some interesting threads through the\nyear that we’ve been involved with:
\nopam switch 4.00.1+open-types.This is just a sample of some of the issues solved in Mantis; if you\nwant to learn more about OCaml, it’s well worth browsing through it to\nlearn from over a decade of interesting discussions from all the\ndevelopers.
\nWhile OCamlPro was working on their reentrant OCaml\nruntime, we took a different tack by\nadding thread-local\nstorage to the\nruntime instead, courtesy of Stephen Dolan. This\nis an important choice to make at the outset of adding multicore, so\nboth approaches are warranted. The preemptive runtime adds a lot of code\nchurn (due to adding a context parameter to most function calls) and\ntakes up a register, whereas the thread-local storage approach we tried\ndoesn’t permit callbacks to different threads.
\nMuch of this work isn’t interesting on its own, but forms the basis for\na fully multicore runtime (with associated programming model) in 2014.\nStay tuned!
\n![\"%r\"](\"/images/c.webp\")
\nOne other complaint from the Consortium members was quite surprising:\nthe difficulty of using the OCaml foreign function interface safely to\ninterface with C code. Jeremy Yallop began working on the\nctypes library that had the\ngoal of eliminating the need to write any C code at all for the vast\nmajority of foreign bindings.
Instead, Ctypes lets you describe any C function call as an OCaml value,\nand provides various linkage options to invoke that function into C. The\nfirst option he implemented was a dlopen interface, which immediately\nbrought us the same level of functionality as the\nPython or\nHaskell Ctypes\nequivalents. This early code was in itself startlingly useful and more\npleasant to use than the raw FFI, and various folk (such as David\nSheets’ libsodium\ncryptography bindings) started adopting it.
At this point, I happened to be struggling to write the Foreign Function\nInterface chapter of Real World OCaml\nwithout blowing through our page budget with a comprehensive explanation\nof the existing system. I decided to take a risk and write about Ctypes\ninstead, since it let new users to the language have a far more\nproductive experience to get started. Xavier Leroy pointed out some\nshortcomings of the\nlibrary in his technical book review, most notably with the lack of an\ninterface with C macros. The design of Ctypes fully supports alternate\nlinking mechanisms than just dlopen though, and Jeremy has added\nautomatic C stub generation support as well. This means that if you use\nCtypes to build an OCaml binding in 2014, you can choose several\nmechanisms for the same source code to link to the external system.\nJeremy even demonstrated a forking model at OCaml 2013 that protects the\nOCaml runtime from the C binding via process separation.
The effort is paying off: Daniel Bünzli ported\nSDL2 using ctypes,\nand gave us extensive\nfeedback about any\nmissing corner cases, and the resulting bindings don’t require any C\ncode to be written. Jonathan Protzenko even used\nit to implement an OCaml controller for the Adafruit Raspberry Pi RGB\nLCD!
\nOur community efforts were largely online, but we also hosted visitors\nover the year and regular face-to-face tutorials.
\nWhile the rest of the crew were hacking on OPAM and OCaml, Amir\nChaudhry and Philippe\nWang teamed up with Ashish Agarwal and\nChristophe Troestler to redesign and relaunch the OCaml\nwebsite. Historically, OCaml’s homepage has been the\ncaml.inria.fr domain, and the\nocaml.org effort was begun by Christophe and Ashish\nsome years\nago to\nmodernize the web presence.
\nThe webpages were already rather large with complex scripting (for\nexample, the 99\nProblems page runs\nthe OCaml code to autogenerate the output). Philippe developed a\ntemplate DSL that made\nit easier to unify a lot of the templates around the website, and also a\nMarkdown parser that we could link to as\na library from the rest of the infrastructure without shelling out to\nPandoc.
\nMeanwhile, Amir designed a series of interactive wireframe\nsketches and\ngathered feedback on it\nfrom the community. A local design agency in\nCambridge helped with visual look and feel, and finally at the end of\nthe summer we began the\nmigration to the\nnew website, followed by a triumphant\nswitchover in\nNovember to the design you see today.
\nThe domain isn’t just limited to the website itself. Leo and I set up a\nSVN-to-Git mirror of the\nOCaml compiler Subversion\nrepository on the GitHub\nOCaml organization, which is proving\npopular with developers. There is an ongoing effort to simplify the core\ncompiler tree by splitting out some of the larger components, and so\ncamlp4 is also now hosted on that\norganization, along with OASIS. We\nalso administer several subdomains of ocaml.org,\nsuch as the mailing lists and the OPAM\nrepository, and other services such as the\nOCaml Forge are currently migrating over.\nThis was made significantly easier thanks to sponsorship from Rackspace\nCloud (users of XenServer\nwhich is written in OCaml). They saw our struggles with managing\nphysical machines and gave us developer accounts, and all of the\nocaml.org infrastructure is now hosted on Rackspace. We’re very grateful\nto their ongoing help!
\n![\"%r\"](\"/images/rackspace.webp\")
\nIf you’d like to contribute to infrastructure help (for example, I’m\nexperimenting with a GitLab mirror),\nthen please join the\ninfrastructure@lists.ocaml.org\nmailing list and share your thoughts. The website team also need help\nwith adding content and international\ntranslations, so head\nover to the website issue\ntracker and start proposing\nimprovements you’d like to see.
The floodgates requesting features opened up after the launch of the new\nlook and feel. Pretty much everyone wanted deeper OPAM integration into\nthe main website, for features such as:
\nMany of these features were part of the original\nwireframes but\nwe’re being careful to take a long-term view of how they should be\ncreated and maintained. Rather than building all of this as a huge\nbloated opam2web extension, David\nSheets (our resident relucant-to-admit-it web expert) has designed an\noverlay directory scheme that permits the overlaying of different\nmetadata onto the website. This lets one particular feature (such as\nblog post aggregation) be handled separately from the others via Atom\naggregators.
\n\nA big effort that took up most of the year for me was finishing and\npublishing an O’Reilly book called Real World\nOCaml with Yaron\nMinsky and Jason Hickey. Yaron\ndescribes how it all started in his blog\npost, but I learnt a lot from\ndeveloping a book using the open commenting\nscheme\nthat we developed just for this.
In particular, the book ended up shining a bright light into dark\nlanguage corners that we might otherwise not have explored in OCaml\nLabs. Two chapters of the book that I wasn’t satisfied with were the\nobjects and\nclasses chapters,\nlargely since neither Yaron nor Jason nor I had ever really used their\nfull power in our own code. Luckily, Leo White decided to pick up the\nbaton and champion these oft-maligned (but very powerful) features of\nOCaml, and the result is the clearest explanation of them that I’ve read\nyet. Meanwhile, Jeremy Yallop helped out with extensive review of the\nForeign Function\nInterface\nchapter that used his\nctypes library. Finally,\nJeremie Diminio\nat Jane Street worked hard on adding several features to his\nutop toplevel that made it compelling\nenough to become our default recommendation for newcomers.
\nAll in all, we ended up closing over 2000\ncomments\nin the process of writing the book, and I’m very proud of the result\n(freely available online, but do buy a\ncopy\nif you can to support it). Still, there’s more I’d like to do in 2014 to\nimprove the ease of using OCaml further. In particular, I removed a\nchapter on packaging and build systems since I wasn’t happy with its\nquality, and both Thomas Gazagnaire and I\nintend to spend time in 2014 on improving this part of the ecosystem.
\n![\"%r\"](\"/images/pfff.webp\" "\\"Julien")
\nWe had a lively presence at ICFP 2013 this\nyear, with the third iteration of the OCaml\n2013 held there, and\nStephen Dolan presenting a paper in the main conference. I liveblogged\nOCaml\n2013\nand CUFP\n2013\nas they happened, and all the\ntalks we gave are\nlinked from the program. The most exciting part of the conference for a\nlot of us were the two talks by Facebook on their use of OCaml: first\nfor program analysis using\nPfff and\nthen to migrate their massive PHP codebase using an OCaml\ncompiler.\nI also had the opportunity to participate in a panel at the Haskell\nWorkshop on whether Haskell is too big to fail\nyet;\nlots of interesting perspectives on scaling another formerly academic\nlanguage into the real world.
Yaron Minsky and I have been\ngiving tutorials on OCaml at ICFP for several years, but the release of\nReal World OCaml has made it significantly easier to give tutorials\nwithout the sort of labor intensity that it took in previous years (one\nmemorable ICFP 2011 tutorial that we did took almost 2 hours to get\neveryone installed with OCaml. In ICFP 2013, it took us 15 minutes or so\nto get everyone started). Still, giving tutorials at ICFP is very much\npreaching to the choir, and so we’ve started speaking at more\ngeneral-purpose events.
\n![\"%r\"](\"/images/marius-yaron-icfp.webp\" "\\"Marius")
\nOur first local effort was FPDays in\nCambridge, where Jeremy Yallop and Amir Chaudhry ran the tutorial with\nhelp from Phillipe Wang, Leo White and David Sheets. The OCaml session\nthere ended up being the biggest one in the entire two days, and Amir\nwrote up their experiences.\nOne interesting change from our ICFP tutorial is that Jeremy used\njs_of_ocaml to teach OCaml\nvia JavaScript by building a fun Monty\nHall game.
![\"%r\"](\"/images/thomas-nycoug-2013.webp\" "\\"Thomas")
\nSince OCaml Labs is a normal group within the Cambridge Computer\nLab, we often host academic visitors and\ninterns who pass through. This year was certainly diverse, and we\nwelcomed a range of colleagues:
We were also visited several times by Wojciech\nMeyer from ARM, who was an OCaml developer who\nmaintained (among other things) the\nocamlbuild system\nand worked on DragonKit\n(an extensible LLVM-like compiler written in OCaml). Wojciech very sadly\npassed away on November 18th, and we all fondly remember his\nenthusiastic and intelligent contributions to our small Cambridge\ncommunity.
\nWe also hosted visitors to live in Cambridge and work with us over the\nsummer. In addition to Vincent Botbol (who worked on OPAM-doc as\ndescribed earlier) we had the pleasure of having Daniel\nBünzli and Xavier Clerc\nwork here. Here’s what they did in their own words.
\nXavier Clerc took a break from his regular duties at INRIA to join us\nover the summer to work on\nOCaml-Java and adapt it to the\nlatest JVM features. This is an incredibly important project to bridge\nOCaml with the huge Java community, and here’s his report:
\n\n\nAfter a four-month visit to the OCaml Labs dedicated to the\nOCaml-Java project, the time has\ncome for an appraisal! The undertaken work can be split into two\nareas: improvements to code generation, and interaction between the\nOCaml & Java languages. Regarding code generation, several classical\noptimizations have been added to the compiler, for example loop\nunrolling, more aggressive unboxing, better handling of globals, or\npartial evaluation (at the bytecode level). A new tool, namely\nocamljar, has been introduced allowing post-compilation optimizations.\nThe underlying idea is that some optimizations cannot always be\napplied (e.g. depending whether multiple threads/programs will\ncoexist), but enabling them through command-line flags would lead to\nrecompilation and/or multiple installations of each library according\nto the set of chosen optimizations. It is thus far more easier to\nfirst build an executable jar file, and then modify it according to\nthese optimizations. Furthermore, this workflow allows the ocamljar\ntool to take advantage of whole-program information for some\noptimizations. All these improvements, combined, often lead to a gain\nof roughly 1/3 in terms of execution time.
\nRegarding language interoperability, there are actually two directions\ndepending on whether you want to call OCaml code from Java, or want to\ncall Java code from OCaml. For the first direction, a tool allows to\ngenerate Java source files from OCaml compiled interfaces, mapping the\nvarious constructs of the OCaml language to Java classes. It is then\npossible to call functions, and to manipulate instances of OCaml types\nin pure Java, still benefiting from the type safety provided by the\nOCaml language. In the other direction, an extension of the OCaml\ntyper is provided allowing to create and manipulate Java instances\ndirectly from OCaml sources. This typer extension is indeed a thin\nlayer upon the original OCaml typer, that is mainly responsible for\nencoding Java types into OCaml types. This encoding uses a number of\nadvanced elements such as polymorphic variants, subtyping, variance\nannotations, phantom typing, and printf-hack, but the end-user does\nnot have to be aware of this encoding. On the surface, the type of\ninstances of the Java Object classes is\n
\njava'lang'Object java_instance, and instances can be created by\ncalling Java.makeObject().While still under heavy development, a working prototype is\navailable, and bugs can be\nreported. Finally, I would like to thank the\nOCaml Labs for providing a great working environment.
\n
Daniel joined us from Switzerland, and spent some time at Citrix before\njoining us in OCaml Labs. All of his\nsoftware is now on OPAM, and is seeing\never-increasing adoption from the community.
\n\n\nReleased a first version of Vg […]\nI’m especially happy about that as I wanted to use and work on these\nideas since at least 2008. The project is a long term project and is\ncertainly not finished yet but this is already a huge step.
\nAdjusted and released a first version of\nGg. While the module was already\nmostly written before my arrival to Cambridge, the development of Vg\nand Vz prompted me to make some changes to the module.
\n[…] released Otfm, a module to\ndecode OpenType fonts. This is a work in progress as not every\nOpenType table has built-in support for decoding yet. But since it is\nneeded by Vg’s PDF renderer I had to cut a release. It can however\nalready be used to implement certain simple things like font kerning\nwith Vg, this can be seen in action in the
\nvechobinary installed by\nVg.Started to work on Vz,\na module for helping to map data to Vg images. This is really\nunfinished and is still considered to be at a design stage. There are\na few things that are however well implemented like (human)\nperceptually meaningful color\npalettes\nand the small folding stat module (
\nVz.Stat). However it quickly\nbecame evident that I needed to have more in the box w.r.t. text\nrendering in Vg/Otfm. Things like d3js entirely rely on the SVG/CSS\nsupport for text which makes it easy to e.g. align things (like tick\nlabels on such\ndrawings). If you\ncan’t rely on that you need ways of measuring rendered text. So I\ndecided to suspend the work on Vz and put more energy in making a\nfirst good release of Vg. Vz still needs quite some design work,\nespecially since it tries to be independent of Vg’s backend and from\nthe mechanism for user input.Spent some time figuring out a new “opam-friendly” release workflow in\npkgopkg. One of my problem is that by designing in the small for\nprogramming in the large — what a slogan — the number of packages I’m\npublishing is growing (12 and still counting). This means that I need\nto scale horizontally maintenance-wise unhelped by the sad state of\nbuild systems for OCaml. I need tools that make the release process\nflawless, painless and up to my quality standards. This lead me to\nenhance and consolidate my old scattered distribution scripts in that\nrepo, killing my dependencies on Oasis and ocamlfind along the way.\n(edited for brevity, see\nhere)
\n
![\"%r\"](\"/images/daniel-presentation-vg.webp\")
\nDaniel also left his bicycle here for future visitors to use, and the\n“Bünzli-bike” is available for our next visitor! (Louis Gesbert even\ndonated lights, giving it a semblance of safety).
![\"%r\"](\"/images/xenserver.webp\")
\nMost of our regular funding bodies such as EPSRC\nor EU FP7 provide funding,\nbut leave all the intellectual input to the academics. A compelling\naspect of OCaml Labs has been how involved our industrial colleagues\nhave been with the day-to-day problems that we solve. Both Jane Street\nand Citrix have senior staff regularly visiting our group and working\nalongside us as industrial fellows in the Computer Lab.
![\"%r\"](\"/images/js.webp\")
\nMark Shinwell from Jane Street\nEurope has been working on improving the state of native\ndebugging in OCaml, by\nadding extended DWARF debugging information to the compiler output.\nMark is also a useful source of feedback about the forthcoming\ndesign of multicore, since he has daily insight into a huge\nproduction codebase at Jane Street (and can tell us about it without\nus requiring access!).
Dave Scott is the principal architect of\nXenServer at Citrix in Cambridge. This year\nhas been transformative for that project, since Citrix open-sourced\nXenServer\nto GitHub and fully adopted OPAM into their workflow. Dave is the\nauthor of numerous libraries that have all been released to OPAM,\nand his colleagues Jon Ludlam and Euan\nHarris\nare also regular visitors who have also been contributors to the\nOPAM and Mirage ecosystems.
\nThe other 100% of our time at the Labs is spent on research projects.\nWhen we started the group, I wanted to set up a feedback loop between\nlocal people using OCaml to build systems, with the folk developing\nOCaml itself. This has worked out particularly well with a couple of big\nresearch projects in the Lab.
\nMirage is a library operating\nsystem written in\nOCaml that compiles source code into specialised Xen microkernels,\ndeveloped at the Cambridge Computer Lab, Citrix and the Horizon Digital\nEconomy institute at Nottingham. This year saw\nseveral years of effort culminate in the first release of Mirage\n1.0 as a self-hosting entity. While Mirage\nstarted off as a quick\nexperiment into\nbuilding specialised virtual appliances, it rapidly became useful to\nmake into a real system for use in bigger research projects. You can\nlearn more about Mirage here, or read the\nCommunications of the\nACM\narticle that Dave Scott and I wrote to close\nout the year.
\nThis project is where the OCaml Labs “feedback loop” has been strongest.\nA typical Mirage\napplication consists of\naround 50 libraries that are all installed via OPAM. These range from\ndevice drivers to protocol\nlibraries for HTTP or\nDNS, to filesystems such as\nFAT32. Coordinating regular\nreleases of all of\nthese would be near impossible without using OPAM, and has also forced\nus to use our own tools daily, helping to sort out bugs more quickly.\nYou can see the full list of libraries on the OCaml Labs software\npage.
\nMirage is also starting to share code with big projects such as\nXenServer now, and we have been working with\nCitrix engineers to help them to move to the\nCore library that Jane Street has\nreleased (and that is covered in Real World\nOCaml). Moving production codebases this\nlarge can take years, but OCaml Labs is turning out to be a good place\nto start unifying some of the bigger users of OCaml into one place.\nWe’re also now an official Xen Project incubator\nproject,\nwhich helps us to validate functional programming to other Linux\nFoundation efforts.
\n![\"%r\"](\"/images/nymote.webp\")
\nThe release of Mirage 1.0 has put us on the road to simplifying embedded\nsystems programming. The move to the centralized cloud has led to\nregular well-publicised privacy and security threats to the way we\nhandle\nour digital infrastructure, and so Jon\nCrowcroft, Richard\nMortier and I are leading an effort to\nbuild an alternative privacy-preserving infrastructure using embedded\ndevices as part of the User Centric\nNetworking project, in collaboration\nwith a host of companies led by Technicolor\nParis. This work also plays on the strong points of OCaml: it already\nhas a fast ARM\nbackend,\nand Mirage can easily be ported to the new Xen/ARM target as hardware\nbecomes available.
One of the most difficult aspects of programming on the “wide area”\nInternet are dealing with the lack of a distributed identity service\nthat’s fully secure. We published our\nthoughts on this\nat the USENIX Free and Open Communications on the Internet workhsop, and\nDavid Sheets is working towards a full implementation using Mirage. If\nyou’re interested in following this effort, Amir Chaudhry is blogging at\nthe Nymote project website, where we’ll talk about\nthe components as they are released.
\nAt the other extreme from embedded programming is datacenter networking,\nand we started the\nNetwork-as-a-Service\nresearch project with Imperial\nCollege\nand\nNottingham.\nWith the rapid rise of Software Defined\nNetworking\nthis year, we are investigating how application-specific customisation\nof network resources can build fast, better, cheaper infrasructure.\nOCaml is in a good position here: several other groups have built\nOpenFlow controllers in OCaml (most notably, the Frenetic\nProject), and Mirage is specifically\ndesigned to assemble such bespoke infrastructure.
\nAnother aspect we’ve been considering is how to solve the problem of\noptimal connectivity across nodes. TCP is increasingly considered\nharmful in high-through, high-density clusters, and George\nParisis\nled the design of\nTrevi, which is\na fountain-coding based alternative for storage networking. Meanwhile,\nThomas Gazagnaire (who joined OCaml Labs in\nNovember), has been working on a branch-consistent data store called\nIrminsule which supports scalable\ndata sharing and reconciliation using Mirage. Both of these systems will\nsee implementations based on the research done this year.
\nJeremy Yallop and Leo White have been developing an approach that makes\nit possible to write programs with higher-kinded polymorphism (such as\nmonadic functions that are polymorphic in the monad they use) without\nusing functors. It’s early days yet, but there’s a\nlibrary available on\nOPAM that implements the\napproach, and a draft\npaper that\noutlines the design.
\n![\"%r\"](\"/images/camel.webp\")
\nThis year has been a wild ride to get us up to speed, but we now have a\nsolid sense of what to work on for 2014. We’ve decided on a high-level\nset of priorities led by the senior members of the group:
These are guidelines to choosing where to spend our time, but not\nexcluding other work or day-to-day bugfixing. Our focus on collaboration\nwith Jane Street, Citrix, Lexifi, OCamlPro and our existing colleagues\nwill continue, as well as warmly welcoming new community members that\nwish to work with us on any of the projects, either via internships,\nstudentships or good old-fashioned open source hacking.
\nI appreciate the whole\nteam's feedback in\nediting this long post into shape, the amazing professorial support from\nJon Crowcroft, Ian\nLeslie and Alan\nMycroft throughout the year, and of\ncourse the funding and support from Jane Street, Citrix, RCUK, EPSRC,\nDARPA and the EU FP7 that made all this possible. Roll on 2014, and\nplease do get in touch with me with any\nqueries!
\n![\"%c\"](\"/images/fpdays2013-04.webp\" "\\"A")
Now that OCaml 4.01 has been released, there is a frenzy of commit\nactivity in the development trunk of\nOCaml as the new features for 4.02 are all integrated. These include\nsome enhancements to the type system such as\ninjectivity,\nmodule aliases and\nextension\npoints as a\nsimpler alternative to syntax extensions.
\nThe best way to ensure that these all play well together is to test\nagainst the ever-growing OPAM package database as early as possible.\nWhile we’re working on more elaborate continuous\nbuilding\nsolutions, it’s far easier if a developer can quickly run a bulk build\non their own system. The difficulty with doing this is that you also\nneed to install all the external dependencies (e.g. libraries and header\nfiles for bindings) needed by the thousands of packages in OPAM.
\nEnter a hip new lightweight container system called\nDocker. While containers aren’t quite as secure as\ntype-1 hypervisors such as\nXen, they are brilliant for spawning lots of\nlightweight tasks such as installing (and reverting) package\ninstallations. Docker is still under heavy development, but it didn’t\ntake me long to follow the documentation and put together a\nconfiguration file for creating an OCaml+OPAM image to let OCaml\ndevelopers do these bulk builds.
\nI started by spinning up a fresh Ubuntu Saucy VM on the Rackspace\nCloud, which has a recent enough kernel version\nto work out-of-the-box with Docker. The installation\ninstructions\nworked without any problems.
\nNext, I created a\nDockerfile\nto represent the set of commands needed to prepare the base Ubuntu image\nwith an OPAM and OCaml environment. You can find the complete repository\nonline at\nhttps://github.com/avsm/docker-opam.\nLet’s walk through the Dockerfile in chunks.
FROM ubuntu:latest\nMAINTAINER Anil Madhavapeddy <anil@recoil.org>\nRUN apt-get -y install sudo pkg-config git build-essential m4 software-properties-common\nRUN git config --global user.email "docker@example.com"\nRUN git config --global user.name "Docker CI"\nRUN apt-get -y install python-software-properties\nRUN echo "yes" | add-apt-repository ppa:avsm/ocaml41+opam11\nRUN apt-get -y update -qq\nRUN apt-get -y install -qq ocaml ocaml-native-compilers camlp4-extra opam\nADD opam-installext /usr/bin/opam-installext\nThis sets up a basic OCaml and OPAM environment using the same Ubuntu\nPPAs as the Travis\ninstructions I\nposted a few months ago. The final command adds a helper script which\nuses the new depexts feature in OPAM 1.1 to also install operating\nsystem packages that are required by some libraries. I’ll explain in\nmore detail in a later post, but for now all you need to know is that\nopam installext ctypes will not only install the ctypes OCaml\nlibrary, but also invoke apt-get install libffi-dev to install the\nrelevant development library first.
RUN adduser --disabled-password --gecos "" opam\nRUN passwd -l opam\nADD opamsudo /etc/sudoers.d/opam\nUSER opam\nENV HOME /home/opam\nENV OPAMVERBOSE 1\nENV OPAMYES 1\nThe next chunk of the Dockerfile configures the OPAM environment by\ninstalling a non-root user (several OPAM packages fail with an error if\nconfigured as root). We also set the OPAMVERBOSE and OPAMYES\nvariables to ensure we get the full build logs and non-interactive use,\nrespectively.
We’re now set to build a Docker environment for the exact test that we\nwant to run.
\nRUN opam init git://github.com/mirage/opam-repository#add-depexts-11\nRUN opam install ocamlfind\nENTRYPOINT ["usr/bin/opam-installext"]\nThis last addition to the Dockerfile initializes our OPAM package set.\nThis is using my development branch which adds a massive\ndiff to populate\nthe OPAM metadata with external dependency information for Ubuntu and\nDebian.
Building an image from this is a single command:
\n$ docker build -t avsm/opam github.com/avsm/docker-opam\nThe ENTRYPOINT tells Docker that our wrapper script is the “root\ncommand” to run for this container, so we can install a package in a\ncontainer by doing this:
$ docker run avsm/opam ctypes\nThe complete output is logged to stdout and stderr, so we can capture\nthat as easily as a normal shell command. With all these pieces in\nplace, my local bulk build shell script is trivial:
\npkg=`opam list -s -a`\nRUN=5\nmkdir -p /log/$RUN/raw /log/$RUN/err /log/$RUN/ok\nfor p in $pkg; do\n docker run avsm/opam $p > /log/$RUN/raw/$p 2>&1\n if [ $? != 0 ]; then\n ln -s /log/$RUN/raw/$p /log/$RUN/err/$p\n else\n ln -s /log/$RUN/raw/$p /log/$RUN/ok/$p\n fi\ndone \nThis iterates through a local package set and serially builds\neverything. Future enhancements I’m working on: parallelising these on a\nmulticore box, and having a linked\ncontainer\nthat hosts a local package repository so that we don’t require a lot of\nexternal bandwidth. Stay tuned!
", "url": "https://anil.recoil.org/notes/docker-and-opam", "title": "Using Docker to bulk-build OPAM packages on Linux", "summary": "Build OPAM packages in bulk on Linux using Docker containers.", "date_published": "2013-11-15T00:00:00.000000Z", "date_modified": "2013-11-15T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "unikernels", "docker", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/cf9fcf6b-de5d-4a23-a00d-cceadea5b668-1", "content_html": "MirageOS and XAPI project update at XenSummit, announcing the MirageOS 1.0 release. This was a major milestone for the project, showing how MirageOS had matured into a usable system for building unikernels. I discussed the integration work with Xen's XAPI toolstack and how MirageOS could be used to build secure, specialized virtual machines for cloud infrastructure.
", "url": "https://anil.recoil.org/notes/cf9fcf6b-de5d-4a23-a00d-cceadea5b668-1", "title": "MirageOS and XAPI project update at XenSummit", "summary": "Update presentation on MirageOS and XAPI project development at XenSummit.", "date_published": "2013-11-13T00:00:00.000000Z", "date_modified": "2013-11-13T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "xen", "virtualization", "xapi" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/liveblog-plos-2013", "content_html": "I co-chaired the Programming Languages and Operating Systems workshop at SOSP 2013, and made livenotes about the (many) papers presented there.
", "url": "https://anil.recoil.org/notes/liveblog-plos-2013", "external_url": "https://web.archive.org/web/20140813215919/http://www.syslog.cl.cam.ac.uk/2013/11/03/liveblog-from-programming-languages-and-operating-systems-2013/", "title": "Notes from PL and OS 2013 workshop", "summary": "Notes from the 2013 Programming Languages and Operating Systems workshop at SOSP.", "date_published": "2013-11-03T00:00:00.000000Z", "date_modified": "2013-11-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "systems", "fp", "livenotes" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/rise-of-libos-1", "content_html": "Article on the Communications of the ACM on unikernels is published, co-authored with Dave Scott. This was a significant milestone in bringing unikernels to a broader computer science audience through ACM Queue. The article explained how library operating systems could revolutionize cloud computing by replacing traditional OS virtualization with lightweight, specialized virtual machines. We discussed how unikernels could address security, performance, and resource efficiency challenges in multi-tenant cloud environments.
Paper on fountain coding for datacenter networking at HotNets 2013. The Trevi work explored using fountain codes to address storage hotspots in datacenter networks. By applying erasure coding techniques from reliable multicast to the datacenter context, we investigated how to "water down" hotspots and improve load distribution across storage systems. The paper demonstrates how techniques from one domain (networking) can be creatively applied to solve problems in another (distributed storage), providing new approaches to datacenter resource management.
The 1st Edition of Real World OCaml by O'Reilly associates has been released! There have been flurry of signing events, including an upcoming one at OSCON in Austin. This 513-page book, co-authored with Yaron Minsky and Jason Hickey, aimed to make OCaml accessible to working programmers by focusing on practical applications and real-world systems. The book covered everything from functional programming fundamentals to building concurrent systems, with examples drawn from industrial use at Jane Street and academic research in systems programming.
Published the scribe's report for CUFP 2012. Following our work documenting CUFP 2011, we continued serving as scribes for the Commercial Users of Functional Programming workshop. This report, published in the Journal of Functional Programming, captures the talks and experiences shared by industrial practitioners using functional programming. The workshop series provides an invaluable record of how FP is being applied in production systems across different industries and problem domains.
We held the first Human Data Interaction workshop over in Cambridge, with lots of discussion about social networks and the state of play with decentralising them.
", "url": "https://anil.recoil.org/notes/hdi-workshop-2013-liveblog", "external_url": "https://web.archive.org/web/20140624105911/http://www.syslog.cl.cam.ac.uk/2013/10/02/liveblogging-the-first-human-data-interaction-workshop/", "title": "Notes on the first Human Data Interaction workshop", "summary": "Notes on the first Human Data Interaction workshop discussing social networks and decentralization.", "date_published": "2013-10-02T00:00:00.000000Z", "date_modified": "2013-10-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "livenotes", "computerlab", "ubicomp" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/ocaml-2013-liveblog", "content_html": "I attended the OCaml 2013 workshop and took live notes of the event. There was a lot going on here, which you can learn more about in the "Reviewing the first year of OCaml Labs in 2013" roundup as well that I published later in the year.
The Commercial Uses of Functional Programming workshop is one of the best industry/academia crossover workshops to attend, and these are my livenotes from the 2013 edition.
", "url": "https://anil.recoil.org/notes/cufp-2013-liveblog", "external_url": "https://web.archive.org/web/20140328072614/https://www.syslog.cl.cam.ac.uk/2013/09/22/liveblogging-cufp-2013/", "title": "Liveblogging CUFP 2013", "summary": "Live notes from Commercial Uses of Functional Programming 2013 workshop", "date_published": "2013-09-22T00:00:00.000000Z", "date_modified": "2013-09-22T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "livenotes", "cufp", "icfp" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/opam-1-1-beta", "content_html": "Thomas Gazagnaire just announced the availability of the\nOPAM beta release. This has been a huge\namount of work for him and Louis, so I’m\nexcited to see this release!
\nAside from general stability, the main\nhighlights for me are:
\nA switch to the\nCC0\npublic-domain-like license for the repository, and LGPL2+linking\nexception for OPAM itself. The cutover to the new\nlicense was\nthe first non-gratuitous use of GitHub’s fancy issue lists I’ve\nseen, too! As part of this, we’re also beginning a transition over\nto hosting it at opam.ocaml.org, to underline our committment to\nmaintaining it as an OCaml community resource.
Much-improved support for package pinning and updates. This is the\nfeature that makes OPAM work well with\nMirageOS, since we often need to do\ndevelopment work on a low-level library (such as a device\ndriver and\nrecompile all the reverse dependencies.
\nSupport for post-installation messages (e.g. to display licensing\ninformation\nor configuration hints) and better support for the external library\nmanagement issues I explained in an earlier post about OCamlot\ntesting.
\nBetter library structuring to let tools like\nOpam2web work with the\npackage metadata. For instance, my group’s OCaml\nLabs has a comprehensive list of the software\npackages that we work\non\ngenerated directly from an OPAM remote.
\nA growing set of administration tools (via the opam-admin binary)\nthat run health checks and compute statistics over package\nrepositories. For example, here’s the result of running\nopam-admin stats over the latest package repository to show\nvarious growth curves.
Paper on the OCaml Platform at the OCaml Workshop 2013, presenting the first version of integrated development tools for the OCaml ecosystem. This was a collaborative effort with Amir Chaudhry, Thomas Gazagnaire, David Sheets, Philippe Wang, Leo White, and Jeremy Yallop to define what would become the standard toolchain for OCaml development. The platform concept has evolved significantly since then, but this v0.1 established the foundational principles.
", "url": "https://anil.recoil.org/notes/2013-oud-platform-1", "title": "The OCaml Platform v0.1", "summary": "Paper presenting first version of OCaml Platform development tools and infrastructure at OCaml Workshop 2013.", "date_published": "2013-09-01T00:00:00.000000Z", "date_modified": "2013-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "tooling" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2013-oud-platform.pdf", "mime_type": "application/pdf", "title": "The OCaml Platform v0.1" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2013-ocamlot-1", "content_html": "Presented an OCaml ecosystem testing system at the OCaml Users and Developers Workshop. This was an early CI/CD system designed specifically for the OCaml package ecosystem, long before GitHub Actions and similar services became commonplace. The work with David Sheets, Amir Chaudhry and Thomas Gazagnaire laid important groundwork for what would eventually become the modern OCaml-CI infrastructure we use today across hundreds of projects.
", "url": "https://anil.recoil.org/notes/2013-ocamlot-1", "title": "Ocamlot: Online OCaml Testing", "summary": "Presentation of online testing system for the OCaml ecosystem at OCaml Users and Developers Workshop.", "date_published": "2013-09-01T00:00:00.000000Z", "date_modified": "2013-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "testing", "ci", "devops", "tooling" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2013-ocamlot.pdf", "mime_type": "application/pdf", "title": "Ocamlot: Online OCaml Testing" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2013-foci-signposts-1", "content_html": "Paper on DNSSEC-based Signpost servers for better p2p communications at USENIX FOCI. This paper expands on our SIGCOMM demo, presenting the full technical details of how Signposts uses DNSSEC to provide secure peer-to-peer communications and NAT traversal. The system enables users to establish direct connections between their personal devices even when separated by complex network middleboxes. By building on DNS infrastructure and adding cryptographic authentication, Signposts provides a practical solution to the end-to-end connectivity problems plaguing modern networks.
", "url": "https://anil.recoil.org/notes/2013-foci-signposts-1", "title": "Lost in the Edge: Finding Your Way with DNSSEC Signposts", "summary": "Paper on DNSSEC-based Signposts system for improved peer-to-peer communications and NAT traversal at USENIX FOCI.", "date_published": "2013-08-01T00:00:00.000000Z", "date_modified": "2013-08-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "dns", "signpost", "networking", "p2p", "nat" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2013-foci-signposts.pdf", "mime_type": "application/pdf", "title": "Lost in the Edge: Finding Your Way with DNSSEC Signposts" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/762795c5-9f3b-499b-a054-b2af37d1ddd2-1", "content_html": "Mirage Developer Preview 1 screencast, showing developers how to get started with MirageOS. The screencast walked through initializing and building both Unix and Xen kernels from the same OCaml codebase. This was an important milestone in making MirageOS accessible to developers, demonstrating the workflow for creating unikernels and the power of portable functional code that could target multiple backends.
", "url": "https://anil.recoil.org/notes/762795c5-9f3b-499b-a054-b2af37d1ddd2-1", "title": "Mirage Developer Preview 1 screencast", "summary": "Screencast demonstration of Mirage Developer Preview 1 release.", "date_published": "2013-07-26T00:00:00.000000Z", "date_modified": "2013-07-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "ocaml", "unikernels", "demo" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/grepping-every-known-ocaml-package-source", "content_html": "A regular question that comes up from OCaml developers is how to use\nOPAM as a hypothesis testing tool against the\nknown corpus of OCaml source code. In other words: can we quickly and\nsimply run grep over every source archive in OPAM? So that’s the topic\nof today’s 5 minute blog post:
git clone git://github.com/ocaml/opam-repository\ncd opam-repository\nopam-admin make\ncd archives\nfor i in *.tar.gz; \\\n do tar -zxOf $i | grep caml_stat_alloc_string; \\\ndone\nIn this particular example we’re looking for instances of\ncaml_stat_alloc_string, so just replace that with the regular\nexpression of your choice. The opam-admin tool repacks upstream\narchives into a straightforward tarball, so you don’t need to worry\nabout all the different archival\nformats\nthat OPAM supports (such as git or Darcs). It just adds an archive\ndirectory to a normal\nopam-repository checkout, so\nyou can reuse an existing checkout if you have one already.
$ cd opam-repository/archives\n$ du -h\n669M .\n$ ls | wc -l\n2092\nThe first paper on unikernels is published at ASPLOS 2013. This landmark paper introduced unikernels as a new approach to cloud computing, where applications are compile-time specialized into standalone kernels sealed against modification. Working with Richard Mortier, Charalampos Rotsos, Dave Scott, Thomas Gazagnaire, and others, we showed how Mirage achieves an order of magnitude reduction in code size while maintaining performance. The paper demonstrated that combining static type-safety with single address-space layouts creates secure, efficient cloud services, establishing unikernels as a viable alternative to traditional OS stacks.
Paper on dataware computing in the digital economy, examining perceived risks of personal data sharing. This interdisciplinary work with researchers from psychology, HCI, and computer science explored how people perceive and manage risks when sharing personal data. The paper contributed to understanding the human factors in personal data management, informing the design of systems like Databox that aimed to give individuals more control over their information.
", "url": "https://anil.recoil.org/notes/de13-dataware-1", "title": "Perceived risks of personal data sharing", "summary": "Paper on dataware computing in the digital economy.", "date_published": "2013-02-01T00:00:00.000000Z", "date_modified": "2013-02-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "privacy", "databox", "personal-data", "security" ], "attachments": [ { "url": "https://anil.recoil.org/papers/de13-dataware.pdf", "mime_type": "application/pdf", "title": "Perceived risks of personal data sharing" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2012-conext-pvtcp-1", "content_html": "Paper on extending TCP in a backwards compatible way at CoNeXT 2012. This work explored the practical challenges of evolving TCP to add new features while maintaining compatibility with the existing internet infrastructure. We investigated how to introduce protocol extensions in a way that degrades gracefully when communicating with legacy systems, addressing the fundamental tension between innovation and backwards compatibility in critical internet protocols. The paper provides insights into the engineering challenges of updating foundational network protocols.
I’m very excited to announce OCaml Labs, the latest project\nto hit the Cambridge Computer Lab. As anyone that hangs out near me\nprobably realises, I very much enjoy functional programming. My weapon\nof choice tends to be OCaml, as it\ncondenses decades of\nresearch\ninto a pragmatic blend of functional, imperative and object-oriented\nprogramming styles. What’s perhaps less well known are the steady\ninroads that OCaml has been\nmaking into mission-critical areas of industry. At Jane\nStreet, billions of dollars of\ntransactions are routed through a huge ML code-base that is designed to\ncatch bugs at compile-time. At\nCitrix, the Xen management\ntoolstack that powers\nmillions of\nhosts in the cloud is largely written in\nOCaml. Facebook does\nsophisticated static\nanalysis using OCaml over\ntheir vast PHP codebase to close security holes.
\nThe OCaml community is small but dedicated, but there is always more to\ndo to improve the language and ecosystem. So, thanks to a generous\nplatform grant from Jane Street, we are\nlaunching a program to help with the open-source development of OCaml\nfrom Cambridge.
\nThe OCaml Labs are\nbased in the Cambridge Computer Lab and led\nmy myself, Alan Mycroft and Ian\nLeslie. We’re closely affiliated with\nother\ngroups,\nand will be:
\ndeveloping the OCaml Platform, which will bundle the official OCaml\ncompiler from INRIA with a tested set of community libraries that\nrefreshed every six months.
\nworking with the core OCaml team at INRIA’s\nGallium group on the compiler, and with\ncommercial partners like OCamlPro on tool\ndevelopment. OCamlPro are making some very impressive progress\nalready with the OPAM packge manager and\nTypeRex IDE helper.
\nsupporting the online presence with more teaching material and\ncontent. Yaron, Jason and I are working hard on a new\nbook that will be published next year,\nand the OCaml Web team (led by Ashish\nand\nChristophe)\nhave made great progress on a brand new\nwebsite that we will move to the\nocaml.org domain soon.
Of course, it is difficult to hack on a language in a void, and we also\nuse OCaml heavily in our own research. The other half of OCaml Lab’s\ngoals are more disruptive (and riskier!):
\nSo, how can you get involved? We are initially advertising three\npositions for full-time developers and researchers\n(junior and\nsenior) to help us get started\nwith the OCaml Platform and compiler development. These aren’t\nconventional pure research jobs, and a successful candidate should enjoy\nthe open-source development cycle (you retain your own copyright for\nyour own projects). The Computer Lab offers a pretty unique environment:\na friendly, non-hierarchical group in a beautiful city, and some of the\nbest faculty and students you could hope to hang out with.
\nAnd finally, there is a longer lead time on applying for\nPhDs, but this is a great time\nto get involved. When I started at the Lab in 2002, a little project\ncalled Xen was just kicking off, and many of us had a\nwild (and oft great) time riding that wave. Get in touch with myself,\nAlan,\nIan or\nJon soon if you are interested in\napplying! There’s some more information available on the OCaml Labs\npages\nabout options.
", "url": "https://anil.recoil.org/notes/announcing-ocaml-labs", "title": "Announcing OCaml Labs", "summary": "Introducing OCaml Labs, a new project at Cambridge Computer Lab to develop and improve the OCaml programming language.", "date_published": "2012-10-19T00:00:00.000000Z", "date_modified": "2012-10-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "janestreet", "cambridge", "funding" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/breaking-up-mirageos", "content_html": "Once the main advantages of having hypervisors is that you can have strongly isolated services within a single machine. But it's really hard to actually build these specialised services; that is, until MirageOS came along. This post discusses how to build so-called "stub domains" for Xen using MirageOS.
", "url": "https://anil.recoil.org/notes/breaking-up-mirageos", "external_url": "https://mirageos.org/blog/xenstore-stub-domain", "title": "Breaking up is easy (with OPAM)", "summary": "Build isolated services with ease using MirageOS and OPAM on Xen hypervisors.", "date_published": "2012-10-17T00:00:00.000000Z", "date_modified": "2012-10-17T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "xen", "mirageos", "ocaml", "packaging" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/dd8b1f58-c43c-4422-9963-d3a980529e57-1", "content_html": "Recording of the OCaml Labs announcement at ICFP 2012, where I announced the formation of OCaml Labs together with Yaron Minsky from Jane Street. We outlined our vision for the OCaml Platform - a comprehensive set of development tools, libraries, and infrastructure to make OCaml more accessible and productive. This marked the beginning of a major effort to strengthen the OCaml ecosystem with funding and dedicated engineering resources.
", "url": "https://anil.recoil.org/notes/dd8b1f58-c43c-4422-9963-d3a980529e57-1", "title": "OUD 2012: Towards an OCaml Platform and Introducing OCaml Labs", "summary": "Recording of OCaml Labs announcement and OCaml Platform vision at OCaml Users and Developers workshop.", "date_published": "2012-09-17T00:00:00.000000Z", "date_modified": "2012-09-17T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "platform", "ocaml-labs", "community", "tooling" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/xenstore-stub-domain", "content_html": "", "url": "https://anil.recoil.org/notes/xenstore-stub-domain", "external_url": "https://mirageos.org/blog/xenstore-stub-domain", "title": "Building a Xenstore stub domain with MirageOS", "summary": "Learn how to build a Xenstore stub domain using MirageOS for improved security and performance.", "date_published": "2012-09-12T00:00:00.000000Z", "date_modified": "2012-09-12T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2012-sigcomm-signposts-1", "content_html": "Demoed the Signposts DNSSEC system at SIGCOMM. Working with Amir Chaudhry, Charalampos Rotsos, Richard Mortier, and others, we presented a system providing secure, simple communication channels between personal devices in a world dominated by NATs and middleboxes. Signposts uses DNSSEC-based naming to establish secure end-points while clients dynamically discover routes to overcome network obstacles. The demo showed devices on different networks behind various middleboxes seamlessly interconnecting to share data, with the system automatically adapting as network configurations changed.
Paper on programming the Xen cloud using OCaml at the OCaml Workshop. This work presented how we use OCaml throughout the Xen cloud stack, from the control plane tooling to the unikernel applications running as guests. We demonstrated how OCaml's strong type system and functional programming features enable building reliable cloud infrastructure, and discussed the practical experiences of deploying OCaml in production cloud environments. The paper helped establish OCaml as a viable language for systems programming in the cloud era.
", "url": "https://anil.recoil.org/notes/2012-oud-xen-1", "title": "Programming the Xen cloud using OCaml", "summary": "Paper on programming Xen cloud infrastructure using OCaml at the OCaml Workshop.", "date_published": "2012-09-01T00:00:00.000000Z", "date_modified": "2012-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "xen", "ocaml", "cloud", "virtualization" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2012-oud-xen.pdf", "mime_type": "application/pdf", "title": "Programming the Xen cloud using OCaml" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2012-ahans-soapp-1", "content_html": "Paper on control flow analysis to break up applications into compartments. This work led by Khilan Gudka introduced SOAAP (Security-Oriented Analysis of Application Programs), a tool for exploring compartmentalization hypotheses in existing C software. The system uses static and dynamic analysis driven by source code annotations to help programmers evaluate different strategies for decomposing applications into sandboxed components. This semi-automated approach addresses the difficulty of adapting legacy software for security compartmentalization while maintaining correctness and performance.
Paper on using MirageOS for better SDN infrastructure with OpenFlow. Working with Charalampos Rotsos, Richard Mortier, and others, we demonstrated how applications could express their own forwarding logic using OpenFlow to achieve optimal performance in cloud environments. We built modular OpenFlow controllers and switches as Mirage libraries that link directly into applications, providing a safe yet extensible framework for programming network control. The work showed how the unikernel approach could address the lack of traffic isolation in virtualized datacenters while giving applications direct control over their networking.
Paper on our use of data lockers within Cambridge to incentivise more green commuting patterns. This work with Chris Elsmore, Ian Leslie, and Amir Chaudhry explored building a privacy-sensitive architecture for measuring employee travel-to-work carbon footprints. Rather than centralizing location data, we built a distributed system where individuals record fine-grained location information in personal data containers they control, trading portions of data to the organization in exchange for benefits. We piloted this on Cambridge's cloud service, demonstrating how to transform private information into public good with minimal privacy loss.
Paper on a new design for reconfigurable IO that copes with heterogeneous software/hardware. Presented at the RESoLVE workshop at ASPLOS, this work proposed a reconfigurable I/O channel architecture to address the challenges of diverse computing environments combining different software stacks and hardware accelerators. We explored how to build flexible I/O abstractions that can adapt to heterogeneous systems, from CPUs to FPGAs to specialized accelerators, without sacrificing performance or composability.
", "url": "https://anil.recoil.org/notes/2012-resolve-fable-1", "title": "The case for reconfigurable I/O channels", "summary": "Workshop paper proposing reconfigurable I/O channel architecture for heterogeneous software and hardware environments at RESoLVE ASPLOS.", "date_published": "2012-03-01T00:00:00.000000Z", "date_modified": "2012-03-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "systems", "io", "architecture" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2012-resolve-fable.pdf", "mime_type": "application/pdf", "title": "The case for reconfigurable I/O channels" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/dreamplug-debian-and-ocaml", "content_html": "I’ve been meaning to play with Plug\nComputers for some time now, as I need a\nlow-power embedded system around the house. I recently bought a Soekris\nNet6501 (a pretty powerful\nIntel CPU, that even has VT support), but had annoying\nissues getting\nit working reliably. I ordered an ARM-based\nDreamplug as an\nalternative (and as a bonus, the Dreamplug is 6x cheaper than the\nSoekris!). Here are my notes on getting it to work.
\n![\"dreamplug\"](\"http://farm6.staticflickr.com/5230/5693063642_47aa7c4c99.jpg\")
Requirements:
\nThe Dreamplug arrived with a working installation, but running the\nabsolutely ancient Debian Lenny. A dist-upgrade through to Wheezy led to\nbricking it almost immediately, and so I did a fresh installation from\nscratch.
\nFor a fresh installation, place a USB stick of suitable size (greater\nthan 2GB is best) into your functional Debian installation. Then:
\nThe Marvell bootloader boots from a VFAT partition, so you will need\ntwo partitions. The first should be a small fat16 (I picked 150MB)\nand the remainder an ext3 partition for Linux itself. There are\ngood instructions available on the\nTor/Dreamplug\nwiki which show you how to do this.
I grabbed the latest kernel (at this time, 3.2.7) from\nwith-linux, and\ninstalled it with the following commands (assuming your USB stick is\n/dev/sdb).
$ sudo mount /dev/sdb1 /mnt\n$ sudo cp uImage /mnt\n$ sudo umount /mnt\nYou now need to use debootstrap to install a fresh root image.\nBecause it is ARM and your main PC is probably an x86, you will need\nto setup the QEMU CPU emulator. An extremely cool feature of QEMU is\nthat it can do transparent\nemulation of foreign\nbinaries, so you can chroot directly into the ARM filesystem and run\ncommands as if they were x86. The qemu-deboostrap command will\ntake care of this for you, if you perform the steps below (again,\nassuming your USB stick is /dev/sdb).
$ sudo apt-get install qemu-user-static debootstrap\n$ sudo mount /dev/sdb2 /mnt\n$ sudo mkdir -p /mnt/usr/bin\n$ sudo cp /usr/bin/qemu-arm-static /mnt/usr/bin/\n$ sudo qemu-debootstrap --arch=armel wheezy http://ftp.uk.debian.org/debian/\nNow grab the kernel modules from the same place as your uImage (for\n3.2.7, from\nhere).\nThen, chroot into your fresh installation and untar them.
\n$ cd /mnt\n$ sudo tar -zxvf ~/sheeva-3.2.7-Modules.tar.gz\n$ sudo chroot /mnt\n$ depmod -a\n# edit /etc/network/interfaces\n# edit /etc/resolv.conf\nThe wireless setup involves some extremely crap firmware which\nrelentlessly kernel panicked for me, so I just disabled it by adding\nthe following to /etc/modprobe.d/dpwifiap.conf, as I only want\nwired access:
blacklist libertas\nblacklist libertas_sdio\nFrom there on, put the USB stick into the Dreamplug, and follow the\nrest of the boot instructions from the Tor\nwiki\nto interact with the Marvell BIOS and boot from the USB stick. I\ncopied the contents of the USB stick onto the internal MicroSD, and\nit all boots standalone now.
\nOne of the reasons I wanted an ARM-based setup is to experiment with the\nOCaml native code generation. Benedikt\nMeurer has been doing\nsome excellent work on improving code\ngeneration\nfor embedded systems, including support for 16-bit Thumb code, exception\nbacktraces, and dynamic linking and profiling.
\nOnce Linux was up and running, compiling up the latest ocaml-trunk was\nstraightforward.
\n $ sudo apt-get install build-essential git\n $ git clone http://github.com/OCamlPro/ocp-ocaml svn-trunk\n $ cd ocp-ocaml\n $ ./configure && make world opt opt.opt install\nThis compiles the bytecode and native code compilers, and then compiles\nthem again using the native code generator. This takes a while to do on\nthe poor little ARM CPU. Once that finished, I compiled up a few simple\nmodules and they worked great! Since the trunk of OCaml is a development\nbranch, you may run into a few packaging issues (use the very latest\nOASIS to regenerate any setup.ml, and you will need a small patch\nuntil PR 5503 is\napplied).
Incidentally, if anyone is interested in working on a\nMirage port to ARM as an internship in the\nCambridge Computer Lab,\ndo get in touch with me...
", "url": "https://anil.recoil.org/notes/dreamplug-debian-and-ocaml", "title": "Dreaming of an ARM OCaml", "summary": "Getting OCaml running on an ARM-based Dreamplug device with Debian and native code generation.", "date_published": "2012-02-25T00:00:00.000000Z", "date_modified": "2012-02-25T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "unikernels", "mirageos", "ocaml" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2011-cufp-scribe-1", "content_html": "Published the scribe's report for CUFP 2011. This workshop report, co-authored with Yaron Minsky and Marius Eriksen and published in the Journal of Functional Programming, summarizes the talks delivered at CUFP 2011 in Tokyo. The Commercial Users of Functional Programming workshop brings together software developers who use FP in real-world settings, and our scribe report captures the diverse applications and experiences shared by practitioners. Videos and slides from all the talks are available online for the community.
At the OCaml Meeting 2011 speaking about MirageOS in France, presenting the project to the OCaml community. This was an early opportunity to share our vision for using OCaml to build entire operating systems and get feedback from the language developers and enthusiasts. The talk covered how we were extending OCaml with new runtime capabilities to generate Xen kernels, and the potential for functional programming in systems development.
", "url": "https://anil.recoil.org/notes/7d949597-b864-4ada-ab1a-81ff8c0463e2-1", "title": "OCaml Meeting 2011 - MirageOS", "summary": "Presentation on MirageOS at the OCaml Meeting 2011 conference.", "date_published": "2011-10-19T00:00:00.000000Z", "date_modified": "2011-10-19T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "ocaml", "unikernels", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/cufp-2011-mirage", "content_html": "We signed up to do a MirageOS tutorial at ICFP, which is a bit daunting: we had to get all the embedded ARM hardware and laptop support in shape, as well as make it work for a bunch of discerning hackers.
\n![\"%c\"](\"/images/cufp11-1.webp\")
![\"%c\"](\"/images/cufp11-2.webp\")
Published dyntype at the Workshop on Generative Technologies. This paper presents the dyntype library for ML meta-programming, which provides statically typed value persistence through code generation. The work explores how to use OCaml's type system and meta-programming capabilities to automatically derive serialization and deserialization code, enabling type-safe storage and retrieval of complex data structures. This became a foundational library for many OCaml systems that needed reliable data persistence.
Paper on statically typed value persistence for OCaml in ENTCS 2011. This work on the dyntype library explored using meta-programming to provide type-safe serialization and persistence in OCaml. The approach uses OCaml's type system to automatically generate code for converting values to and from various representations, enabling developers to persist complex data structures without sacrificing type safety. The library became an important building block for systems that needed to store and retrieve OCaml values reliably.
", "url": "https://anil.recoil.org/notes/2010-dyntype-wgt-1", "title": "Dynamics for ML using Meta-Programming", "summary": "Paper on statically typed value persistence for OCaml published in ENTCS 2011.", "date_published": "2011-07-01T00:00:00.000000Z", "date_modified": "2011-07-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "metaprogramming", "fp", "types", "persistence" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2010-dyntype-wgt.pdf", "mime_type": "application/pdf", "title": "Statically-typed value persistence for ML" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/delimited-cont-vs-lwt", "content_html": "", "url": "https://anil.recoil.org/notes/delimited-cont-vs-lwt", "external_url": "https://mirageos.org/blog/delimcc-vs-lwt", "title": "Delimited continuations vs Lwt for threads", "summary": "Delimited continuations and Lwt compared for threading purposes.", "date_published": "2011-06-18T00:00:00.000000Z", "date_modified": "2011-06-18T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "ocaml", "multicore", "mirageos" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/k1mwb-s6k33", "content_html": "Distributed programming frameworks like\nHadoop and\nDryad are popular\nfor performing computation over large amounts of data. The reason is\nprogrammer convenience: they accept a query expressed in a simple form\nsuch as MapReduce, and\nautomatically take care of distributing computation to multiple hosts,\nensuring the data is available at all nodes that need it, and dealing\nwith host failures and stragglers.
\nA major limitation of Hadoop and Dryad is that they are not well-suited\nto expressing iterative\nalgorithms or dynamic\nprogramming problems.\nThese are very commonly found patterns in many algorithms, such as\nk-means clustering,\nbinomial options\npricing or\nSmith Waterman\nfor sequence alignment.
\nOver in the SRG in Cambridge,\nwe\ndeveloped a Turing-powerful distributed execution engine called\nCIEL that addresses\nthis. The CIEL: A universal execution engine for distributed data-flow computing\npaper describes the system in detail, but here’s a shorter introduction.
\nCIEL consists of a master coordination server and workers installed on\nevery host. The engine is job-oriented: a job consists of a graph of\ntasks which results in a deterministic output. CIEL tasks can run in any\nlanguage and are started by the worker processes as needed. Data flows\naround the cluster in the form of references that are fed to tasks as\ndependencies. Tasks can publish their outputs either as concrete\nreferences if they can finish the work immediately or as a future\nreference. Additionally, tasks can dynamically spawn more tasks and\ndelegate references to them, which makes the system Turing-powerful and\nsuitable for iterative and dynamic programming problems where the task\ngraph cannot be computed statically.
\nThe first iteration of CIEL used a domain-specific language called\nSkywriting to\ncoordinate how tasks should run across a cluster. Skywriting is an\ninterpreted language that is “native” to CIEL, and when it needs to\nblock it stores its entire execution state inside CIEL as a\ncontinuation. Derek Murray has\nwritten a blog post explaining this in more\ndetail.
\nMore recently, we have been working on eliminating the need for\nSkywriting entirely, by adding direct support for CIEL into languages\nsuch as Python, Java,\nScala, and the main subject of this post –\nOCaml. It works via libraries that communicate\nwith CIEL to spawn tasks, publish references, or suspend itself into the\ncluster to be woken up when a future reference is completed.
\nRather than go into too much detail about the innards of CIEL, this post\ndescribes the OCaml API and gives some examples of how to use it. The\nsimplest interface to start with is:
\ntype 'a ref\nval deref : 'a ref -> 'a\nThe type 'a ref represents a CIEL reference. This data might not be\nimmediately present on the current node, and so must be dereferenced\nusing the deref function.
If the reference has been completed, then the OCaml value is\nunmarshalled and returned. If it is not present, then the program needs\nto wait until the computation involving the reference has completed\nelsewhere. The future reference might contain a large data structure and\nbe on another host entirely, and so we should serialise the program\nstate and spawn a task that is dependent on the future’s completion.\nThis way, CIEL can resume execution on whatever node finished that\ncomputation, avoiding the need to move data across the network.
\nLuckily, we do not need to serialise the entire heap to suspend the\nprogram. DataCaml uses the\ndelimcc\ndelimited continuations library to walk the stack and save only the\nsubset required to restart this particular task. Delimcc abstracts this\nin the form a “restartable exception” that supplies a closure which can\nbe called later to resume the execution, as if the exception had never\nhappened. Delimcc supports serialising this closure to an output\nchannel, which you can read about in Oleg’s\npaper.
\nSo how do we construct references? Lets fill in more of the interface:
\nmodule Ciel = struct\n type 'a ref\n val deref : 'a ref -> 'a\n val spawn : ('a -> 'b) -> 'a -> 'b ref\n val run : (string list -> 'a) -> ('a -> string) -> unit\nend\nThe spawn function accepts a closure and an argument, and returns a\nfuture of the result as a reference. The run function begins the\nexecution of a job, with the first parameter taking some\nstring arguments and returning an 'a value. We also supply a\npretty-printer second argument to convert the 'a into a string for\nreturning as the result of the job (this can actually be any JSON value\nin CIEL, and just simplified here).
let r1 = spawn (fun x -> x + 5) arg1 in\nlet r2 = spawn (fun x -> deref r1 + 5) arg1 in\nderef r2\nWe first spawn a function r1 which simply adds 5 to the job argument.\nA job in CIEL is lazily scheduled, so this marshals the function to\nCIEL, creates a future, and returns immediately. Next, the r2 function\nspawns a task which also adds 5, but to the dereferenced value of r1.\nAgain, it is not scheduled yet as the return reference has not been\ndereferenced.
Finally, we attempt to dereference r2, which causes it be scheduled on\na worker. While executing, it will try to dereference r1 that will\nschedule it, and all the tasks will run to completion.
Programming language boffins will recognise that this interface is very\nsimilar to AliceML’s concept of\nlazy futures.\nThe main difference is that it is implemented as a pure OCaml library,\nand uses a general-purpose distributed engine that can also work with\nother languages.
\nThe references described so far only have two states: they are either\nconcrete or futures. However, there are times when a task can\nprogressively accept input and make forward progress. For these\nsituations, references can also be typed as opaque references that are\naccessed via in_channel and out_channel, as networks are:
type opaque_ref\n\nval spawn_ref : (unit -> opaque_ref) -> opaque_ref\nval output : ?stream:bool -> ?pipe:bool -> (out_channel -> unit) -> opaque_ref\nval input : (in_channel -> 'a) -> opaque_ref -> 'a\nThis interface is a lower-level version of the previous one:
\nspawn_ref creates a lazy future as before, but the type of\nreferences here is completely opaque to the program.output is called with a closure that\naccepts an out_channel. The stream argument informs CIEL that a\ndependent task can consume the output before it is completed, and\npipe forms an even more closely coupled shared-memory connection\n(requiring the tasks to be scheduled on the same host). Piping is\nmore efficient, but will require more work to recover from a fault,\nand so using it is left to the programmer to decide.input function is used by the receiving task to parse the\ninput as a standard in_channel.The CIEL engine actually supports multiple concurrent input and output\nstreams to a task, but I’ve just bound it as a single version for now\nwhile the bindings find their feet. Here’s an example of how streaming\nreferences can be used:
\nlet x_ref = spawn_ref (fun () ->\n output ~stream:true (fun oc ->\n for i = 0 to 5 do\n Unix.sleep 1;\n fprintf oc "%d\\n%!" i;\n done\n )\n ) in\n let y_ref = spawn_ref (fun () ->\n input (fun ic ->\n output ~stream:true (fun oc ->\n for i = 0 to 5 do\n let line = input_line ic in\n fprintf oc "LINE=%s\\n%!" line\n done\n )\n ) x_ref\n ) in\nWe first spawn an x_ref which pretends to do 5 seconds of work by\nsleeping and outputing a number. This would of course be heavy number\ncrunching in a real program. The y_ref then inputs this stream, and\noutputs its own result by prepending a string to each line.
If you are interested in a more real example, then read through the\nbinomial\noptions\ncalculator that uses streaming references to parallelise a dynamic\nprogramming problem (this would be difficult to express in MapReduce).\nOn my Mac, I can run this by:
\nINSTALL file)\nand OCaml libraries\n(delimcc\nand Yojson).<repo>/src/python to your PYTHONPATH./scripts/run_master.sh./scripts/run_worker.sh -n 5 (this allocates\n5 execution slots)cd src/ocaml && make./scripts/sw-start-job -m http://localhost:8000 ./src/package/ocaml_binopt.pack10.x)The DataCaml bindings outlined here provide an easy way to write\ndistributed, fault-tolerant and cluster-scheduled jobs in OCaml. The\ncurrent implementation of the engine is aimed at cluster computation,\nbut Malte has been working on\ncondensing CIEL onto multicore\nhardware.\nThus, this could be one approach to ‘solving the OCaml multicore\nproblem’ for problems that fit nicely into the dataflow paradigm.
\nThe biggest limitation for using these bindings is that delimited\ncontinuation serialisation only works in bytecode. Native code delimcc\nsupports shift/reduce in the same program, but serialising is\nproblematic since native code continuations contain a C stack, which may\nhave unwrapped integers. One way to work around this is by switching to\na monadic approach to dereferencing, but I find delimcc programming more\nnatural (also see this\ndiscussion).
Another important point is that tasks are lazy and purely functional\n(remind you of Haskell?). This is essential for reliable fault-tolerance\nand reproducibility, while allowing individual tasks to run fast, strict\nand mutable OCaml code. The tasks must remain referentially transparent\nand idempotent, as CIEL may choose to schedule them multiple times (in\nthe case of faults or straggler correction). Derek has been working on\nintegrating non-determinism into\nCIEL,\nso this restriction may be relaxed soon.
\nFinally, these ideas are not limited to OCaml at all, but also apply to\nScala, Java, and Python. We have submitted a draft paper dubbed ‘A\nPolyglot Approach to Cloud\nProgramming’\nwith more details and the ubiquitous evaluation versus Hadoop. There is\na really interesting line to explore between low-level\nMPI coding and\nhigh-level MapReduce, and we think CIEL is a useful spot in that design\nspace.
\nIncidentally, I was recently hosted by Nokia\nResearch in Palo Alto by my friend\nPrashanth\nMundkur, where\nthey work on the Python/Erlang/OCaml Disco\nMapReduce engine. I’m looking forward to seeing more critical\ncomparisons and discussions of alternatives to Hadoop, from them and\nothers.
\nThanks are due to Derek,\nChris and\nMalte for answering my incessant CIEL\nquestions while writing this post! Remember that DataCaml is a work in\nprogress and a research prototype, and feedback is most welcome.
", "url": "https://anil.recoil.org/notes/datacaml-with-ciel", "title": "DataCaml: distributed dataflow programming in OCaml", "summary": "DataCaml brings distributed dataflow programming to OCaml using the CIEL engine.", "date_published": "2011-06-11T00:00:00.000000Z", "date_modified": "2011-06-11T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "distributed", "ocaml", "cloud", "fp" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2011-fccm-cloudfpga-1", "content_html": "Paper on what a Xen+FPGA cloud would look like at FCCM. This work with Satnam Singh explored reconfigurable data processing for clouds, addressing how FPGAs could help solve practical problems in scaling out data-centers where computation is limited by energy consumption or latency. We identified the prerequisites for making reconfigurable computing practical in cloud environments and described scenarios enabled by combining virtualization with hardware acceleration. The paper laid groundwork for thinking about heterogeneous computing in the cloud era.
We've been doing loads of OCaml programming in the Systems Research Group, and this blog post lays out some of the things going on. It ranges from OCaml hacking, over to the CIEL distributed execution engine, and even some Haskell hacking ongoing for distributed execution.
", "url": "https://anil.recoil.org/notes/srg-fp", "external_url": "https://web.archive.org/web/20151028011702/http://www.syslog.cl.cam.ac.uk/2011/04/18/functional-programming-gone-wild-in-the-srg/", "title": "Functional programming gone wild in the SRG", "summary": "Exploring functional programming projects in the SRG, including OCaml and Haskell.", "date_published": "2011-04-18T00:00:00.000000Z", "date_modified": "2011-04-18T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs", "computerlab", "ocaml", "fp" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/zb6ye-yfk38", "content_html": "I'm at the 2011 OCaml Users Group in Paris, reporting on some splendid talks this year. It looked like around 60-70 people in the room, and I had the pleasure of meeting users all the way from Russia to New York as well as all the Europeans!
\nFirst up was Pierre Chambart talking about the js_of_ocaml compiler. It compiles OCaml bytecode directly to Javascript, with few external dependencies. Since the bytecode format changes very rarely, it is simpler to maintain than alternatives (such as Jake Donham’s ocamljs) that require patching the compiler tool-chain. Javascript objects are mapped to dynamic OCaml objects via a light-weight ## operator, so you can simply write code like:
class type window = object\n method alert : js_string t -> unit meth\n method name : js_string t prop\n end\n let window : window t =\n JS.Unsafe.variable "window"\n \n let () = \n window##alert ( window##name)\n name <- Js.string "name"\nOverloading is handled similarly to PyObjC, with each parameter combination being mapped into a uniquely named function. Raphael Proust then demonstrated a cool game he wrote using via bindings to the Raphael Javascript vector graphics library. Performance of js_of_ocaml is good compared to writing it by hand, and they have have quite a few benchmarks on their website.
Overall the project looks very usable: the main omissions are Bigarray, no dynlink, no Str (replaced by native regexps), no recursive modules or weak references. None of these missing features seem very critical for the sorts of applications that js_of_ocaml is intended for.
Next up Phillipe Wang presented something completely different: running OCaml on tiny 8-bit PIC microcontrollers! These PICs have 4-128Kb of flash (to store the code), and from 256 bytes to 4 kilobytes. Not a lot of room to waste there. He demonstrated an example with a game with 24 physical push buttons that beat humans at a conference (JFLA).
\nIt works by translating OCaml bytecode through several stages: ocamlclean to eliminate dead code in the bytecode (which would be very useful for native code too!), a compression step that does run-length encoding, and then translation to PIC assembly. They have a replacement stop-and-copy GC (150 lines of assembly) and a full collection cycle runs in less than 1.5ms. Integers are 15-bits (with 1 bit reserved) and the block representation is the same as native OCaml. Very cool project!
We went onto static analysis and Julien Signoles presented Frama-C, a powerful static analysis tool for real-world C. It forks the CIL project from Berkeley and adds ocamlgraph and GUI support. He demonstrated a simple loop counter plugin to count them in C code, and the homepage has many interesting plugins maintained by the community.
\nI hadn’t realised that CIL was still maintained in the face of clang, so it’s nice to see it live on as part of Frama-C.
\nThe ever-cheerful Vincent Balat updated us about the Ocsigen web framework, including unveiling their exciting new logo! This was written using an amazing collaborative editor that lets users edit in real time.
\nOcsigen is based around services of type service: parameters -> page. Services are first-class values, and can be registered dynamically and associated with sessions. The code for the collaborative editor was about 100 lines of code.
There is a syntax extension to distinguish between client and server side code, and both can be written in the same service (invoking js_of_ocaml to compile the client code to Javascript). They have bindings to Google Closure in order to provide UI support. There is a really nice “bus” service to pass messages between the server and the client, with seamless integration of Lwt to hide the details of communication to the browser.
Ocsigen is looking like a very mature project at this point, and I’m very keen to integrate it with Mirage to specialise the into micro-kernels. A task for the hacking day tomorrow morning I think!
\nI talked about Mirage, hurrah! Good questions about why we need a block device (and not just use NFS), and I replied that everything is available as the library and the programmer can choose depending on their needs (the core goal of exokernels).
\nA highlight for me was lunch where I finally met Richard Jones, who is one of the other OCaml and cloud hackers out there. Wide ranging conversation about what the cool stuff going in KVM and Red Hat in general. Richard also gave a short talk about how they use OCaml to generate hundreds of thousands of lines of code in libguestfs. There are bindings for pretty much every major language, and it is all generated from an executable specification. He notes that “normal” programmers love the OCaml type safety without explicit annotations, and that it is a really practical language for the working programmer. The Xen Cloud Platform also has a similar generator for XenAPI bindings, so I definitely agree with him about this!
\nXavier “superstar” Leroy then gave an update of OCaml development. Major new features in 3.12.0 are first-class modules, polymorphic recursion, local module opens, and richer operations over module signatures. Version 3.12.1 is coming out soon, with bug fixes (in camlp4 and ocamlbuild mainly), and better performance on x86_64: turns out a new mov instruction change improves floating point performance on x86_64.
OCaml 3.13 has no release date, but several exciting features are in the pipeline. Firstly, more lightweight first-class modules by permitting some annotations to be inferred by the context, and it introduces patterns to match and bind first-class module values. Much more exciting is support for GADTs (Generalised Algebraic Data Types). This permits more type constraints to be enforced at compile time:
\n type _ t =\n | IntLit : int -> int t\n | Pair : 'a t * 'b t -> ('a * 'b) t\n | App : ('a -> 'b) t * 'a t -> 'b t\n | Abs : ('a -> 'b) -> ('a -> 'b) t\n \n let rec eval : type s . s t -> s = function\n | IntLit x -> x (* s = int here *)\n | Pair (x,y) -> (eval x, eval y) (* s = 'a * 'b here *)\n | App (f,a) -> (eval f) (eval a)\n | Abs f -> f\nIn this example of a typed interpreter, the eval function is annotated with a type s . s t -> s type that lets each branch of the pattern match have a constrained type for s depending on the use. This reminded me of Edwin Brady’s partial evaluation work using dependent types, but a much more restricted version suitable for OCaml.
There are some really interesting uses for GADTs:
\nThe challenges in the implementation are that principle type inference is now impossible (so some annotation is required), and pattern matching warnings are also trickier.
\nFrom the IDE perspective, the third bit of work is to have the OCaml compiler save the full abstract syntax tree annotation with source locations, scoping information, types (declared and inferred) and addition user-defined annotations. This generalises the -annot flag and can help projects like OCamlSpotter, OCamlWizard, OcaIDE, etc. It also helps code-generators driven by type-generators (such as our SQL ORM or ATDgen).
The OCaml consortium has new members; MLState and MyLife, and Esterel, OCamlPro and one unnamed new member are joining. The consortium goals are to sell permissive licensing (BSD) to members, and sound off new features with the serious users. Three companies are now doing commercial development (Gerd, OCamlCore, OCamlPro) which is growing the community nicely.
\nLuc Maranget (who looks like an archetypal mad professor!) gave a great rundown on JoCaml, a distributed programming extension to OCaml. This extends the compiler with join-definitions (a compiler patch), and a small bit of runtime support (using Thread), and significant extensions for concurrent and distributed programming in a type-safe way.
\nIt extends the syntax with three new keywords: def, spawn and reply, and new usage for or and & (you should be using || and && anyway). Binary libraries remain compatible between matching versions of JoCaml and OCaml. An example of JoCaml code is:
let create n =\n def st(rem) & tick() = st(rem-1)\n or st(0) & wait() = reply to wait in\n spawn st(n) ; { tick=tick; wait=wait; }\n \n type t = {\n tick: unit Join.chan;\n wait: unit -> unit;\n }\nAfter n messages to tick, the wait barrier function will be called.
let c = create n\n let () =\n for k = 0 to 9 do\n spawn begin printf "%i" k; c.tick ()\n done;\n c.wait ()\nHere we asynchronously print the numbers of 0 to 9, and then the wait call acts as a barrier until it finishes. JoCaml is useful for distributed fork-join parallelism tasks such as raytracing, but with the type system support of OCaml. It is a bit like MapReduce, but without the data partitioning support of Hadoop (and is more light-weight). It would be quite interesting to combine some of the JoCaml extensions with the dynamic dataflow graphs in our own CIEL distributed execution engine.
Francois Bobot talks about the problem of memoizing values so that they can be re-used (e.g. in a cache). Consider a standard memoiser:
\n let memo_f =\n let cache = H.create () in\n fun k ->\n try H.find cache k\n with Not_found ->\n let v = f k in\n H.add cache k v;\n v\n \n let v1 = memo_f k1\n let v2 = memo_f k2 in (* k2 = k1 in O(1) *)\nIf a key is not reachable from anywhere other than the heap, we want to eliminate it from the cache also. The first solution is a normal hashtable, but this results in an obvious memory leak since a key held in the cache marks it as reachable. A better solution is using OCaml weak pointers that permit references to values without holding on to them (see Weaktbl by Zheng Li who is now an OCaml hacker at Citrix). The problem with Weaktbl is that if the value points to the key, forming a cycle which will never be reclaimed.
\nFrancois solves this by using Ephemerons from Smalltalk. They use the rule that the value can be reclaimed if the key or the ephemeron itself can be reclaimed by the GC, and have a signature like:
\n module Ephemeron : sig type ('a,'b) t\n val create : 'a -> 'b -> ('a,'b) t\n val check : ('a,'b) t -> bool\n val get : ('a,'b) t -> 'b option\n val get_key : ('a,'b) t -> 'a option\n end\nThe implementation in OCaml patches the runtime to use a new tag for ephemerons, and the performance graphs in his slides look good. This is an interesting topic for me since we need efficient memoisation in Mirage I/O (see the effects on DNS performance in the Eurosys paper which used Weaktbl). When asked if the OCaml patch will be upstreamed, Damien Doligez did not like the worst-case complexity of long chains of ephemerons in the GC, and there are several approaches under consideration to alleviate this without too many changes to the runtime, but Francois believes the current complexity is not too bad in practise.
\nSylvain came on stage later to give a demonstration of OASIS, an equivalent of Cabal for Haskell or CPAN for Perl. It works with a small _oasis file that describes the project, and then the OASIS tool auto-generates ocamlbuild files from it (this reminds me of Perl’s MakeMaker). Once the files are auto-generated, it is self-contained and there is no further dependency on OASIS itself.
![\"%r\"](\"/images/ocaml-users-1.webp\" "\\"How")
\n![\"%r\"](\"/images/ocaml-users-3.webp\" "\\"Wearing")
\n![\"%r\"](\"/images/ocaml-users-2.webp\" "\\"Team")
OASIS works with either an existing build system in a project, or can be integrated more closely with ocamlbuild by advanced users. Lots of projects are already using OASIS (from Cryptokit to Lwt to the huge Jane Street Core). He is also working on a distribution mechanism on a central website, which should make for convenient OCaml packaging when it is finished and gets more adoption from the community.
Finally, Ashish Agarwal led a discussion on how OCaml can improve its web presence for beginners. Lots of good ideas here (some of which we implemented when reworking the CUFP website last year). Looking forward to seeing what happens next year in this space! I really enjoyed the day; the quality of talks was very high, and many engaging discussions from all involved!
\n![\"%c\"](\"/images/sf-ocaml.webp\")
Of course, not all of the OCaml community action is in France. The ever-social Jake Donham organised the First Ever San Francisco User Group that I attended when I was over there a few weeks ago. Ok, admittedly it was mainly French people there too, but it was excellent to meet up with Mika, Martin, Julien, Henri and of course Jake when over there.
\nWe should definitely have more of these fun local meetups, and a number of other OCaml hackers I mentioned it to want to attend next time in the Bay Area, if only to cry into their drinks about the state of multi-core... just kidding, OCamlPro is hard at work fixing that after all :-)
", "url": "https://anil.recoil.org/notes/ocaml-users-group", "title": "Camel Spotting in Paris", "summary": "Report from the 2011 OCaml Users Meeting in Paris, covering talks on js_of_ocaml, OCaml on PIC, and more.", "date_published": "2011-04-15T00:00:00.000000Z", "date_modified": "2011-04-15T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocamllabs" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2011-nsdi-ciel-1", "content_html": "Paper on CIEL, a distributed dataflow engine, at USENIX NSDI 2011. This work led by Derek G Murray introduced a universal execution engine that, unlike previous systems, enables data-dependent control-flow decisions for computing iterative and recursive algorithms. We also developed Skywriting, a Turing-complete scripting language that runs directly on CIEL with transparent fault tolerance and distribution. The system was deployed on cloud platforms and demonstrated scalable performance for both iterative and non-iterative algorithms, addressing a key limitation of earlier MapReduce-style systems.
", "url": "https://anil.recoil.org/notes/2011-nsdi-ciel-1", "title": "CIEL: A universal execution engine for distributed data-flow computing", "summary": "Paper on CIEL universal execution engine for distributed data-flow computing using dynamic task graphs at USENIX NSDI 2011.", "date_published": "2011-03-01T00:00:00.000000Z", "date_modified": "2011-03-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "distributed", "dataflow", "cloud", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2011-nsdi-ciel.pdf", "mime_type": "application/pdf", "title": "CIEL: A universal execution engine for distributed data-flow computing" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2011-icdcn-droplets-1", "content_html": "Paper on a vision for a semi-federated cloud for personal data at ICDCN. Working with Jon Crowcroft, Richard Mortier, Malte Schwarzkopf, and Theodore Hong, we proposed the Droplets architecture as a middle ground between fully centralized cloud services and completely decentralized systems. The work contrasts these two extremes and shows how Droplets enables controlled trade-offs between the costs and benefits of each approach. We demonstrated three sample applications that substantially benefit from this flexibility, providing a more nuanced vision of cloud computing than the prevailing centralized model.
", "url": "https://anil.recoil.org/notes/2011-icdcn-droplets-1", "title": "Unclouded vision", "summary": "Paper on Droplets architecture enabling controlled trade-offs between centralized cloud and fully decentralized personal data approaches at ICDCN.", "date_published": "2011-01-01T00:00:00.000000Z", "date_modified": "2011-01-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "cloud", "personal-data", "privacy", "distributed", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2011-icdcn-droplets.pdf", "mime_type": "application/pdf", "title": "Unclouded vision" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/43ab3ae0-9ffc-474f-aa02-3cc1139f54d1-1", "content_html": "Talk on building the Xen toolstack using OCaml at ICFP 2010. This was one of the early talks that helped establish OCaml as a viable language for systems programming. We shared our experiences rewriting parts of the Xen hypervisor toolstack in OCaml, demonstrating that functional programming could deliver the performance, reliability, and safety needed for critical infrastructure software. The work laid the groundwork for MirageOS and showed that you could build high-performance systems software with strong type safety. It was exciting to show the functional programming community that OCaml wasn't just for compilers and theorem provers - it could handle real systems work.
", "url": "https://anil.recoil.org/notes/43ab3ae0-9ffc-474f-aa02-3cc1139f54d1-1", "title": "Building the Xen toolstack using OCaml", "summary": "Talk on building the Xen toolstack using OCaml.", "date_published": "2010-11-05T00:00:00.000000Z", "date_modified": "2010-11-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "xen", "virtualization", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/be2f049b-174a-4e5b-b30e-0319793487c7-1", "content_html": "At LinkedIn giving tech talk about Mirage, presenting our new multi-scale operating system approach for clouds. This early talk in California introduced the concept of compiling functional code into specialized kernels for cloud deployment. I discussed how MirageOS could address the bloat and security issues of traditional OS stacks by building minimal, purpose-built systems from libraries written in a high-level language.
", "url": "https://anil.recoil.org/notes/be2f049b-174a-4e5b-b30e-0319793487c7-1", "title": "Mirage: A New Multi-Scale Operating System for Clouds and Crowds (2014)", "summary": "Tech talk at LinkedIn presenting Mirage operating system for cloud computing.", "date_published": "2010-10-25T00:00:00.000000Z", "date_modified": "2010-10-25T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "ocaml", "cloud", "unikernels", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/mirage-self-hosting", "content_html": "I managed to get early MirageOS suitably feature-complete enough that we could run the Mirage website using Mirage. This was all very satisfying after hacking on the TCP/IP stack for ages.
", "url": "https://anil.recoil.org/notes/mirage-self-hosting", "external_url": "https://mirageos.org/blog/self-hosting-mirage-website", "title": "Self-hosting MirageOS website", "summary": "Running the MirageOS website on its own self-hosted infrastructure using the TCP/IP stack.", "date_published": "2010-10-11T00:00:00.000000Z", "date_modified": "2010-10-11T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "unikernels", "ocaml", "mirageos", "selfhosting" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/de10-perscon-1", "content_html": "Paper on personal containers for data management at the UK Digital Economy meeting, exploring how individuals could better manage their personal data. Co-authored with Richard Mortier, Jon Crowcroft, and others, this work introduced the concept of personal containers - secure environments for storing and processing personal information. This research laid the groundwork for later projects like Dataware and Databox, addressing growing concerns about privacy and data ownership in ubiquitous computing.
", "url": "https://anil.recoil.org/notes/de10-perscon-1", "title": "The personal container, or your life in bits", "summary": "Paper on personal containers for managing personal data presented at UK Digital Economy meeting.", "date_published": "2010-10-01T00:00:00.000000Z", "date_modified": "2010-10-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "privacy", "personal-data", "data-management", "ubicomp" ], "attachments": [ { "url": "https://anil.recoil.org/papers/de10-perscon.pdf", "mime_type": "application/pdf", "title": "The personal container, or your life in bits" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2010-icfp-xen-1", "content_html": "Paper on our experiences with writing the Xen control stack in OCaml at ICFP 2010. Working with Dave Scott, Richard Sharp, and Thomas Gazagnaire, we presented the perspectives and perceptions from using functional programming within an industrial product group at Citrix. The paper discusses the practical realities of deploying OCaml in a commercial virtualization product, covering both the technical benefits of the functional approach and the organizational challenges of introducing FP into an established engineering team. This became an influential case study for FP adoption in industry.
At HotCloud for the first talk about MirageOS, presenting our vision for software specialization in the cloud. This was the first public outing of the MirageOS concept at USENIX HotCloud 2010 in Boston. The paper proposed running applications directly on cloud hardware without traditional OS layers, using OCaml to generate specialized binaries that execute as Xen guest operating systems. Our early prototype showed significant performance improvements for I/O and memory handling compared to the same code running under Linux/Xen.
", "url": "https://anil.recoil.org/notes/4957325f-d7f5-4a29-95b6-a1e1f61ea5cf-1", "title": "Turning Down the LAMP: Software Specialisation for the Cloud", "summary": "First HotCloud talk presenting the initial MirageOS concept for cloud software specialization.", "date_published": "2010-06-22T00:00:00.000000Z", "date_modified": "2010-06-22T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "cloud", "unikernels", "systems", "specialization" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2010-hotcloud-lamp-1", "content_html": "Workshop paper on the early MirageOS architecture and evaluation at HotCloud 2010. This paper presented our vision of "turning down the LAMP" stack by running applications directly on cloud infrastructure as specialized unikernels. Working with Richard Mortier, Ripduman Sohan, Thomas Gazagnaire, and others, we showed how Mirage could compile OCaml applications into Xen guest operating systems, achieving significant performance improvements for I/O and memory handling compared to traditional LAMP stacks. The work demonstrated that cloud computing offered an unprecedented opportunity to rethink application architecture from the ground up.
", "url": "https://anil.recoil.org/notes/2010-hotcloud-lamp-1", "title": "Turning Down the LAMP: Software Specialisation for the Cloud", "summary": "Workshop paper presenting early MirageOS architecture for running OCaml applications directly on cloud platforms as unikernels.", "date_published": "2010-06-01T00:00:00.000000Z", "date_modified": "2010-06-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mirageos", "unikernels", "cloud", "ocaml", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2010-hotcloud-lamp.pdf", "mime_type": "application/pdf", "title": "Turning Down the LAMP: Software Specialisation for the Cloud" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/anil-phd-thesis-2", "content_html": "My PhD thesis is now also published as a print book, making the work on type-safe network application development more accessible. The thesis demonstrated how functional programming languages with strong type systems could be used to build network protocols and applications with correctness guarantees. These ideas about using types for security and verification continue to influence my work on systems programming with OCaml.
", "url": "https://anil.recoil.org/notes/anil-phd-thesis-2", "title": "Creating high-performance, statically type-safe network applications", "summary": "PhD thesis on type-safe network application development now published as print book.", "date_published": "2010-05-01T00:00:00.000000Z", "date_modified": "2010-05-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "fp", "networking", "security", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/anil-phd-thesis.pdf", "mime_type": "application/pdf", "title": "Creating High-Performance, Statically Type-Safe Network Applications" } ], "_references": [] }, { "id": "https://doi.org/10.59350/4b3zw-zen8", "content_html": "The App Engine data collector for Personal Containers is coming on nicely, and is on track for an alpha preview release fairly soon. Working with AppEngine has been interesting; it’s got excellent availability and you can’t beat the price (free), but coding robust Python that doesn’t trip over the tight resource limits for individual requests, asynchronous tasks and queries is tricky. While it is good for small records such as my iPhone or Find My iPhone GPS traces traces, it doesn’t work so well with my gigabytes of photographs or decades of e-mail.
\nThis confirmed our earlier intuition that there is no one perfect solution for personal data handling; instead, we need to embrace diversity and construct an infrastructure that can cope with change over the coming decades. Mobile programming has changed beyond recognition in just a few years, and cloud providers are specialising in different ways (e.g. PiCloud for simple compute, or EC2 for fancy services like elastic load balancing).
\nSo to recognise this, we are building components that all interoperate with your personal data, keep it secure, and ensure it persists for more than a few years. Malte Schwarzkopf came up with the term "digital yurts", and it's stuck. We’ve written a draft paper about it, and would love to hear your comments and feedback on the approach.
\n![\"\"](\"/images/nomads-diagram.webp\")
There are some interesting recent trends that make doing this\nparticularly important:
\nIf you’re interested, join our group\nor contact Anil Madhavapeddy directly. At this stage, you\nneed desire and the ability to hack code, but things are settling down\nover the next few months...
", "url": "https://anil.recoil.org/notes/yurts-for-digital-nomads", "external_url": "https://web.archive.org/web/20110315011341/http://perscon.net/2010/04/29/yurts-for-digital-nomads.html", "title": "Yurts for Digital Nomads", "summary": "Digital nomads can secure their personal data with digital yurts, a system for diverse data handling and storage.", "date_published": "2010-04-29T00:00:00.000000Z", "date_modified": "2010-04-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "perscon", "ocaml", "cloud", "selfhosting" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/uiprototype", "content_html": "We’ve been hacking away on fleshing out the App Engine node for personal containers. We’re building this node first because, crucially, deploying an App Engine VM is free to anyone with a Google account. The service itself is limited since you can only respond to HTTP or XMPP requests and do HTTP fetches, and so its primary use is as an always-on data collection service with a webmail-style UI written using extjs.
\nPersonal containers gather data from a wide variety of sources, and normalise them into a format which understands people (address book entries, with a set of services such as e-mail, phone, IM and online IDs), places (GPS, WOEID), media (photos, movies) and messages (Tweets, emails, Facebook messages). I’ll post more about the data model behind personal containers in a follow-up as the format settles.
\n![\"%c\"](\"/images/perscon-extjs.webp\")
The App Engine node has a number of plugins to gather data and aggregate them into a single view (see screenshot). Plugins include:
\nI'm switching tacks briefly; we received an Amazon Research Grant recently and I’m building a node that runs as a Linux server to act as a longer-term archival and search server. This is being written in OCaml and uses Tokyo Cabinet (with Jake Donham’s excellent bindings) and so should be speedy and a useful alternative implementation of the HTTP REST interface. The plan is to automatically synchronize meta-data across all the nodes of a personal container, but store large and historical data away from expensive cloud storage such as App Engine.
\nThere are lots more plugins in development, such as Foursquare and Gowalla OAuth collectors, an Android mobile application to upload location and contacts information, and Google GData synchronization. If you’re interested in one of these or something else, please do get in touch or just fork the project and start hacking!
", "url": "https://anil.recoil.org/notes/uiprototype", "external_url": "https://web.archive.org/web/20110313101153/http://perscon.net/2010/04/15/uiprototype.html", "title": "Pulling together a user interface", "summary": "Building a user interface for personal containers on App Engine with extjs.", "date_published": "2010-04-15T00:00:00.000000Z", "date_modified": "2010-04-15T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "selfhosting", "cloud", "ui", "perscon" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2010-bcs-visions-1", "content_html": "Paper on our vision for multiscale programming at the BCS Visions 2010 conference. Working with Richard Mortier, Jon Crowcroft, and Steven Hand, we presented a clean-slate approach to heterogeneous cloud computing that sweeps away decades of system software cruft. The paper argues that the move to virtual clouds creates an opportunity to build a unified platform spanning from cloud servers through desktops to mobile smartphones - this became the foundation for the Mirage framework. We demonstrated how this multiscale approach could deliver significant benefits in security, reliability and efficiency across the entire computing spectrum.
", "url": "https://anil.recoil.org/notes/2010-bcs-visions-1", "title": "Multiscale not multicore: efficient heterogeneous cloud computing", "summary": "Paper on vision for multiscale programming presented at BCS Visions 2010 conference.", "date_published": "2010-04-01T00:00:00.000000Z", "date_modified": "2010-04-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "cloud", "heterogeneous", "systems", "programming" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2010-bcs-visions.pdf", "mime_type": "application/pdf", "title": "Multiscale not multicore: efficient heterogeneous cloud computing" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/opening-a-website", "content_html": "We've been working away at building a new type of database to help individuals\nkeep reigns on their ever-increasing personal digital information. The first\nprototypes run freely on Google App Engine to gather your data\nbehind-the-scenes, and we are working on more advanced versions that run on\nembedded devices and the cloud.
\nIf you’re interested in keeping track of your personal data, you can start off\nwith the installation instructions to clone your own version. After that, read\nup on the design of the system (which is still changing as we research new\nideas around it). When you find something you want to fix, or add a new plugin\ndata source, just clone the code and send us back fixes!
", "url": "https://anil.recoil.org/notes/opening-a-website", "external_url": "https://web.archive.org/web/20110509135538/http://perscon.net/2010/03/29/intro.html", "title": "Opening a website", "summary": "Learn about opening a personal data tracking website with a customizable database system.", "date_published": "2010-03-29T00:00:00.000000Z", "date_modified": "2010-03-29T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "perscon", "ui", "selfhosting" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2010-smarte-privacybutler-1", "content_html": "Paper on privacy butler services for more private data management. This work with Ryan Wishart, Domenico Corapi, and Morris Sloman proposed an automated Privacy Butler service that monitors a person's online presence and makes corrections based on their privacy policies. The system addresses the challenge of third-parties modifying online content - such as tagging photos or posting comments - without the consent of the person being referenced. We demonstrated how policy-driven automation could help people maintain control over their digital footprint across social networks and online communities.
PhD thesis now available as a technical report, documenting my work on creating high-performance, statically type-safe network applications. The thesis explored how ML-style type systems and model checking could be combined to build secure network services. This foundational work on using functional programming for systems development would later influence the design of MirageOS and our approach to building secure, verified network stacks.
", "url": "https://anil.recoil.org/notes/anil-phd-thesis-1", "title": "Creating high-performance, statically type-safe network applications", "summary": "PhD thesis published as technical report on building secure network applications using ML type systems and model checking.", "date_published": "2010-03-01T00:00:00.000000Z", "date_modified": "2010-03-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ocaml", "fp", "networking", "security", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/anil-phd-thesis.pdf", "mime_type": "application/pdf", "title": "Creating High-Performance, Statically Type-Safe Network Applications" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2009-icfem-spl-1", "content_html": "Paper on a DSL for specifying temporal protocol automata at ICFEM 2009. The Statecall Policy Language (SPL) provides a practical middle ground between ad-hoc coding and full formal verification for complex protocol implementations. SPL lets programmers embed automata directly in their code which can be both statically model-checked using SPIN and dynamically enforced at runtime with minimal performance overhead. I demonstrated the approach with an SSH server written entirely in OCaml/SPL, showing how the automata provide higher-level debugging capabilities while maintaining the benefits of formal verification.
Well, the big launch of XenServer 5 has gone smoothly, and with it have arrived a flood of questions about how exactly the new High Availability functionality works. I’ll use this post to explain the overall architecture of HA in XenServer 5, and also how some of the fault detection and failure planning works.
\nFundamentally, HA is about making sure important VMs are always running on a resource pool. There are two aspects to this: reliably detecting host failure, and computing a failure plan to deal with swift recovery.
\nDetecting host failure reliably is difficult since you need to remotely distinguish between a host disappearing for a while versus exploding in a ball of flames. If we mistakenly decide that a master host has broken down and elect a new master in its place, there may be unpredictable results if the original host were to make a comeback! Similarly, if there is a network issue and a resource pool splits into two equal halves, we need to ensure that only one half accesses the shared storage and not both simultaneously.
\n![\"%c\"](\"/images/ha-wizard-3b.webp\")
We solve all these problems in XenServer by having two mechanisms: a storage heartbeat and a network heartbeat. When you enable HA in a pool, you must nominate an iSCSI or FC storage repository to be the heartbeat SR. XenServer automatically creates a couple of small virtual disks in this SR. The first disk is used by every physical host in the resource pool as a shared quorum disk. Each host allocates itself a unique block in the shared disk and regularly writes to the block to indicate that it is alive.
\nI asked Dave Scott, the principal engineer behind HA about the startup process:
\n\n\nWhen HA starts up, all hosts exchange data over both network and\nstorage channels, indicating which hosts they can see over both\nchannels; i.e. which I/O paths are working and which are not. This\nliveness information is exchanged until a fixed point is reached and\nall of the hosts are satisfied that they are in agreement about what\nthey can see. When this happens, the HA functionality is ‘armed’ and\nthe pool is protected.
\n
\n\nThis HA arming process can take a few minutes to settle for larger\npools, but is only required when HA is first enabled.
\n
\n\nOnce HA is active, each host regularly writes storage updates to the\nheartbeat virtual disk, and network packets over the management\ninterface. It is vital to ensure that network adapters are\nbonded\nfor resilience, and that storage interfaces are using dynamic\nmultipathing\nwhere supported. This will ensure that any single adapter or wiring\nfailures do not result in any availability issues.
\n
![\"%c\"](\"/images/ha-wizard-5.webp\")
\n![\"%c\"](\"/images/ha-wizard-5-1.webp\")
The worst-case scenario for HA is the situation where a host is thought to be off-line but is actually still writing to the shared storage, since this can result in corruption of persistent data. In order to prevent this situation without requiring active power strip control, we implemented hypervisor-level fencing. This is a Xen modification which will hard-power the host off at a very low-level if it doesn’t hear regularly from a watchdog process running in the control domain. Since it is implemented at a very low-level, this also covers the case where the control domain becomes unresponsive for any reason.
\nHosts will self-fence (i.e. power off and restart) in the event of any heartbeat failure unless any of the following hold true:
\nThe heartbeat system gives us reliable notification of host failure, and so we move onto the second step of HA: capacity planning for failure.
\nA resource pool consists of several physical hosts (say, 16), each with potentially different amounts of host memory and a different number of running VMs. In order to ensure that no single host failure will result in the VMs on that host being unrestartable (e.g. due to insufficient memory on any other host), the XenServer pool dynamically computes a failure plan which calculates the actions that would be taken on any host failure.
\nBut there’s one more complexity... a single host failure plan does not cover more advanced cases such as network partitions which take out entire groups of hosts. It would be very useful to be able to create a plan that could tolerate more than a single host failure, so that administrators could ignore the first host failure and be safe in the knowledge that (for example) three more hosts could fail before the pool runs out of spare capacity.
\nThat’s exactly what we do in XenServer... the resource pool dynamically computes a failure plan which considers the “number of host failures to tolerate” (or nhtol). This represents the number of disposable servers in a pool for a given set of protected VMs.
\nThe planning algorithms are pretty complex, since doing a brute force search of all possible failures across all hosts across all VMs is an exponential problem. We apply heuristics to ensure we can compute a plan in a reasonably small time:
\nSince planning algorithms are designed for unexpected host failures, we only consider absolutely essential resource reservations which would prevent the VM from starting on the alternative host (e.g. storage is visible, and enough memory is present). We do not perform CPU reservation on the basis that it can be optimised at a later stage via live relocation once the VM is back up and running.
\nWe now have HA armed and a failover plan for our VMs. But what if you want to make changes to your configuration after HA is enabled? This is dealt with via overcommit protection.
\nThe XenServer pool dynamically calculates a new failover plan in response to every XenAPI call which would affect it (e.g. starting a new VM). If a new plan cannot be calculated due to insufficient resources across the pool, the XenServer will return an overcommitment error message to the client which blocks the operation.
\n![\"%c\"](\"/images/ha-wizard-4b.webp\")
This overcommit protection would be quite irritating if you have to keep trying things and seeing if a plan exists or not, and so we built in a "What If?" machine into XenServer to facilitate counter-factual reasoning.
\nWhen reconfiguring HA via XenCenter, you can supply a hypothetical series of VM priorities, and XenServer will return a number of host failures which would be tolerated under this scheme. This lets you try various combinations of VM protections depending on your business needs, and see if the number of host failures is appropriate to the level of paranoia you desire.
\nThis can even be done via the CLI, using the snappily named "xe pool-ha-compute-max-host-failures-to-tolerate" when HA is enabled.
\nThe nice thing about XenServer HA is that it is done at the XenAPI level, and so any of the standard clients (such as the xe CLI or XenCenter) or any third-party clients which use the XenAPI will all interoperate just fine. The XenServer pool dynamically recalculates plans in response to the client requests, and so no special “oracle” is required outside of the pool to figure out HA plans.
\nFinally, HA makes master election completely invisible. Any host in a pool can be a master host, and the pool database is constantly replicated across all nodes and also backed up to shared storage on the heartbeat SR for additional safety. Any XenAPI client can connect to any host, and a redirect is issued to the current master host.
\nEach VM in an HA pool can be either fully protected, best-effort or unprotected. VMs which are protected are all included in the failover planning, and if no plan exists for which they can all be reliably restarted then the pool is considered to be overcommitted. Hugh Warrington (who implemented the XenCenter HA support) explained what use protection levels are:
\n\n\nBest-effort VMs are not considered when calculating a failover plan,\nbut the pool will still try to start them as a one-off if a host that\nis running them fails. This restart is attempted after all protected\nVMs are restarted, and if the attempt to start them fails then it will\nnot be retried. This is a useful setting for test/dev VMs which\naren’t critical to keep running, but would be nice to do so in a pool\nwhich also has some important VMs which absolutely must run.
\n
There are some advanced features which are only available via the CLI. Each protected VM in an HA pool can be assigned a numeric ha-restart-priority. If a pool is well-resourced with a high nhtol, then these restart priorities are not relevant: the VMs are all guaranteed to be started.
If more hosts fail than have been planned for, then the priorities are used to determine the order in which VMs are restarted. This ensures that in over-committed pools, the most important VMs are restarted first. Although the pool will start priority 1 VMs first, they might not finish booting before the priority 2 VMs, and so this should not be used as the basis for service ordering.
\nNote that it's very important to ensure that a VM is agile when protecting it by HA. If the VM is not agile (e.g has a physical CD drive mapped in from a host), then it can only be assigned Best Effort restart since it is tied to one host.
\nThe best practice for HA is not to make configuration changes while it is enabled. Instead, it is intended to be the "2am safeguard" which will restart hosts in the event of a problem when there isn't a human administrator nearby. If you are actively making configuration changes such as applying patches, then HA should be disabled for the duration of these changes.
\nXenCenter makes some common changes under HA much more user-friendly, which I asked Ewan Mellor (the principal GUI engineer) about:
\nSo, I hope this short article has given you a taster... just kidding! This post is almost as long as my PhD thesis, but then, HA is a complex topic. Please do feel free to get back to me with comments and feedback about how we can improve it in the future releases, or if you just love it the way it is. Many thanks to Dave Scott, Richard Sharp, Ewan Mellor and Hugh Warrington for their input to this article.
", "url": "https://anil.recoil.org/notes/peeking-under-the-hood-of-high-availability", "external_url": "https://web.archive.org/web/20090126095912/http://community.citrix.com/blogs/citrite/anilma/2008/09/17/Peeking+under+the+hood+of+High+Availability", "title": "Peeking under the hood of High Availability", "summary": "Learn how XenServer's High Availability feature works, including host failure detection and automatic VM restarts.", "date_published": "2008-09-17T00:00:00.000000Z", "date_modified": "2008-09-17T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "xen", "citrix", "consensus", "distributed" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/48sxx-hjt49", "content_html": "You won’t be surprised to hear that we spend a lot of time improving\nXenApp performance when running on\nXenServer. Although there are some\ngood benchmark comparisons available (such as the Tolly\nGroup report), I still get a lot\nof customers asking about what the “secret sauce” is. I sat down with\nGeorge Dunlap, the lead XenServer performance engineer to chat about the\nvery first optimisation we did back in XenServer 4.0 last year.
\nBefore we dive in, we first need to explain how a normal operating\nsystem handles memory. George explains:
\n\n\nModern desktop and server processors don’t access memory directly\nusing its physical address. They use ‘virtual\nmemory’ to separate the\naddresses that processes use to read and write memory from the actual\nmemory itself. This allows operating systems to hide from processes\nall the dirty details of how much memory there is, where in physical\nmemory the process needs to write to, and so on.
\nHowever, the actual processor still needs to translate from a\nvirtual address to the\nphysical memory address in order to actually read and write any\nmemory. This translation is done with something called page\ntables.
\nPage tables are used to implement virtual memory by mapping virtual\naddresses to physical addresses. The operating system constructs page\ntables using physical memory addresses, and then puts the physical\naddress of the “top-level” page table into a hardware register called\nthe ‘base pointer’. Then the processor will read these page tables to\ntranslate virtual addresses to physical addresses as needed, before\nreading and writing to physical memory.
\n
Most modern processor types have some sort of paging mechanism, although\nXenServer is specifically tuned for\nx86-64 CPUs. An excellent book on\nthe general topic is Modern Operating\nSystems by\nAndrew Tanenbaum. When XenServer creates\nWindows VMs, it takes advantage of the virtualization\nextensions in modern\nCPUs, which requires special memory handling in Xen. George explains\nthis further:
\n\n\nWhen we create a virtual machine, we virtualize the memory as well;\nthat means that the guest operating system’s idea of physical memory\ndoes not match up to real physical memory on the host. Traditionally,\nwhat the guest thinks of as physical memory is called “physical\nmemory”, and what the hypervisor thinks of as physical memory is\ncalled “machine memory”. Since this terminology is a bit confusing,\nXen tends to call what the guest thinks of as physical memory as\n“guest physical” memory, just to help make things more clear.
\n
\n\nThis means that any fully-virtualized operating system, like Windows,\nwill create page tables using guest physical memory, and will point\nthe base pointer at the guest physical address of the top-level page\ntable. Unfortunately, the hardware still needs to map from virtual\nmemory address to machine addresses, not guest physical addresses.
\n
\n\nIn order to allow this to happen, the hypervisor sets up shadow\npage tables. These page tables are generated by the hypervisor are\ncopies of the guest page tables, but with the guest physical addresses\nconverted into machine physical addresses. The guest cannot access\nthem directly, and they don’t reside in the guest’s physical memory;\nthey’re generated out of a pool of memory that the hypervisor\nallocates when a VM is created, called shadow page table memory.
\n
\n\nWhat this means is that whenever the guest operating system wants to\nmap some new memory, after it writes the data into the page table but\nbefore it can actually use it, the hypervisor needs to translate the\nchange to the guest page table into changes to the shadow page table.\nSo any workload that involves a lot of this will necessarily involve\nthe hypervisor a lot, which causes overhead.
\n
So shadow page tables are our mechanism of giving a guest an interface\nwhich is identical to real hardware (so it doesn’t need to be modified),\nbut still intercepting changes before they reach the real hardware. You\ncan find more details from the XenSummit 2006\ntalk or\nfrom the 2005 NSDI\npaper.\nSo how is this all relevant to XenApp performance? Back to George…
\n\n\nThe hypervisor allocates a certain amount of memory for each VM to\nuse for shadow page tables; this is called shadow page table\nmemory. As new page tables are created and old ones aren’t used\nanymore, the hypervisor cycles through this shadow page table memory.\nWhen it needs a new page and there isn’t enough, it will ‘unshadow’\nthe guest page tables that haven’t been used for the longest time to\nreclaim shadow memory, so that it can use more.
\n
\n\nWe don’t know ahead of time how much shadow memory a given workload\nwill use, but we can estimate based on the amount of memory that the\nVM has. We allocate enough shadow memory for each page to be mapped\nonce, more or less, then add an extra 50% to have some slack. For all\nthe workloads we’ve tested, that’s been enough – except XenApp.
\n
\n\nXenApp is the one workload we’ve found that requires more shadow page\ntable memory than our standard default. Because XenApp generally\nstarts hundreds of copies of the same process, the same memory ends up\nmapped in hundreds of different processes. What happens when all of\nthose processes are active is that XenServer is continually\nunshadowing one process’ page tables in order to shadow another\nprocess’ pagetables; only to have to re-shadow the original ones a\nsecond or two later when it runs again! This is called\nthrashing,\nwhen there’s not enough of a limited resource.
\n
Once the bottleneck was discovered, the solution was simple. In\nXenServer 4.1, we created a special XenServer application template\ncalled “Citrix XenApp”, which has an increased shadow multiplier that\nreserves more shadow memory for the guest when it starts. This is also a\ngood example of how templates hide the complexities of performance\ntuning from the user, but still permitting custom modifications if they\nare required. For example, on your XenServer host with a VM called\n“XenApp”, you could view the shadow multiplier by using the CLI:
\n# xe vm-list name-label=XenApp params=HVM-shadow-multiplier\n HVM-shadow-multiplier ( RW) : 4.000\nThe same value is also available from XenCenter in the Optimization\npane, but of course do remember that the default value was chosen\nthrough extensive testing and doesn’t need to be changed. Most of the\nother templates in XenServer also have carefully tuned settings (e.g.\nthe hardware platform flags) to ensure smooth running, or in the case of\nLinux templates, to support para-virtual\ninstallation.\nThis is why it’s so important that you not use the “Other Install\nMedia” template in preference of a more specialised one!
\nI mentioned at the beginning of this post that this was the first of\nmany XenApp optimisations. We’ve just released the public\nbeta\nof the latest XenServer (“Orlando”) which is even faster. The story of\nwhat those improvements are, and the tools which George and his team\nuses to analyze the inner workings of Xen, are a topic for a future\npost. For now, get downloading XenServer and start virtualizing your\nXenApp installations! Or if you’re feeling inspired, go over to\nxen.org, check out the source, and get coding…
", "url": "https://anil.recoil.org/notes/shedding-some-light-on-xenapp-on-xenserver-performance-tuning", "external_url": "https://web.archive.org/web/20090131071235/http://community.citrix.com/blogs/citrite/anilma/2008/08/04/Shedding+some+light+on+XenApp+on+XenServer+performance+tuning", "title": "Shedding light on XenApp on XenServer performance tuning", "summary": "Optimize XenApp performance on XenServer with expert tuning tips.", "date_published": "2008-08-04T00:00:00.000000Z", "date_modified": "2008-08-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "xen", "systems", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://doi.org/10.59350/fxmfq-3b085", "content_html": "I thought I’d kick off my Citrix blog with a question I get pretty often\nfrom Linux enthusiasts: how to install unsupported Linux distributions\non XenServer 4.1.
\nThe most common solution people find is to use the "Other Install Media"\ntemplate, insert the distribution installation CD, and find that the\nmouse cursor doesn’t work when they boot into X11. The reason for this\nis that they are using the hardware-assisted emulation mode of\ninstalling Linux. In this mode (dubbed “HVM”), all input and output is\nemulated, and in particular the mouse interface uses the USB tablet\ninterface. If the distribution doesn’t include a driver for USB tablets,\nthen no mouse will appear.
\nWindows guests run at high-speed in HVM mode due to the installation of\nthe XenServer tools which install high-speed\ndrivers, but these\nare not necessary for Linux distributions since they can be run in\npara-virtualized mode\n(dubbed “PV”). This involves obtaining a Xen-enabled PV kernel from the\ndistribution, and modifying the VM record in XenServer to boot into this\nkernel instead of HVM mode. The XenServer built-in templates for popular\ndistributions such as RHEL, CentOS or SUSE Linux already automate all\nthis and are in PV mode from the installer onwards.
\nIn the remainder of this post, I’ll explain how to take a distribution\nwithout direct support (Ubuntu\n8.04), get it installed in HVM\nmode on XenServer 4.1, and convert it to PV mode with a XenCenter\ngraphical console.
\nDownload the "Alternative Installation\nCD". The main\ninstallation CD uses graphical mode, which won't install as well in\nHVM mode due to the use of esoteric 16-bit mode instructions for the\ngraphics operations. The 16-bit emulation mechanisms vary between\nprocessors (with better support on AMD chips, and a software\ninstruction emulator required on Intel VT chips). However, the\nUbuntu alternate CD uses a text-based installer which works fine.
\nCreate a new VM on the XenServer 4.1 host using the "Windows Server\n2003" template. This template is set up with a sensible set of\nhardware emulation flags and default disks, and so is a good base\nfor the HVM installation of Ubuntu as well. Attach the Ubuntu ISO\nyou just downloaded to the VM, and proceed to install Ubuntu as\nnormal. You should install it onto the first disk, to make the\nsubsequent steps in this guide easier.
\nWhen the installation is finished, reboot the VM (don't forget to\ndetach the installation ISO first). It should boot up in HVM mode\ninto the graphical login screen. The XenCenter display will show it\nas not being optimized, which is fine. At this stage, I prefer to\nwork via a remote command-line using SSH. Open up a Terminal from\nUbuntu, and run "sudo apt-get install openssh-server". Then find\nout the VM's IP address with "ifconfig eth0", and then connect to\nit remotely. Alternatively, you can continue to type in the commands\ndirectly into the terminal as well.
On the Ubuntu guest, you now need to install the latest Xen version\nof the Ubuntu kernel:
\nsudo apt-get install linux-image-xen". This is a virtual\npackage which pulls in the latest Xen kernel and modules, in my\ncase 2.6.24.19.21.update-grub doesn't update the grub configuration\nwith your newly installed Xen kernel. To fix this:\nOpen /boot/grub/menu.lst in your favourite editor.
Scroll to the bottom to the kernel list, and find the entry\nwhich looks like:
\ntitle Ubuntu 8.04, kernel 2.6.24-16-generic\nroot (hd0,0)\nkernel /boot/vmlinuz-2.6.24-16-generic root=UUID=<uuid> ro quiet splash\ninitrd /boot/initrd.img-2.6.24-16-generic\nquiet\nAdd a new entry which is similar to this, but change all\nreferences to the 2.6.24-16-generic to the Xen kernel. In\n/boot I have vmlinuz-2.6.24-19-xen, so my new entry\nlooks like:
title Ubuntu 8.04, kernel 2.6.24-19-xen\nroot (hd0,0)\nkernel /boot/vmlinuz-2.6.24-19-xen root=UUID=<uuid> ro quiet splash\ninitrd /boot/initrd.img-2.6.24-19-xen\nquiet\nAlso edit the default entry in the menu.lst to match the\nnumber of the kernel you just added. I set mine to 3, since\nit is the fourth entry in the list and the indexing starts\nfrom 0.
When this is done, shut down the guest but do not reboot it just\nyet. You first need to edit the VM record for your Ubuntu VM to\nconvert it to PV boot mode. From the control domain console of your\nXenServer:
\nxe CLI:\nxe vm-list name-label=Ubuntu params=uuid --minimal : this\nwill print out the UUID of the VM named "Ubuntu". If you are\nlogged into the control domain, pressing the <tab> key\nwill perform auto-completion of UUIDs in subsequent XE\ncommands, so you don't need to keep typing it in every time!xe vm-param-set uuid=<uuid> HVM-boot-policy= : this will\nclear the HVM boot mode from the VM.xe vm-param-set uuid=<uuid> PV-bootloader=pygrub : this\nwill switch the VM to using to the pygrub bootloader which\nstarts the guest in PV mode by examining its filesystem for\nkernel.vm vm-param-set uuid=<uuid> PV-args="console=tty0 xencons=tty"\n: this configures the kernel boot arguments to display the\nlogin console on the correct TTY, so that it shows up in the\nXenCenter console.xe vm-disk-list uuid=<uuid> and look for the UUID of the\nVBD for the disk. VBD stands for "Virtual Block Device" and\nrepresents how to map the virtual disk into the virtual\nmachine.xe vbd-param-set uuid=<vbd uuid> bootable=true will set\nthe root disk VBD to be bootable.You should be all set now! If you boot up the Ubuntu VM, it should\nstart up in text-mode with the high-speed PV kernel. If it doesn't\nwork due to an incorrect grub configuration, you can use the\nxe-edit-bootloader script in the XenServer control domain to edit\nthe grub.conf until it works.
The next step is to install the XenServer tools within the guest, so\nthat metrics such as the network interface IP addresses are recorded\nand reported from XenCenter. To do this:
\nsudo apt-get -y install bash && sudo dpkg-reconfigure dash.\nWe've actually fixed this issue in future releases of XenServer,\nbut for XenServer 4.1 you will need to use bash.sudo mount /dev/xvdd /mntsudo dpkg -i /mnt/Linux/xe-guest-utilities_4.1.0-257_i386.deb.In order to access the Ubuntu installation via the graphical\nconsole, you need to configure it to run\nVNC on the external network interface.\nXenCenter polls the guest to see if it is listening on the VNC port\n5900, and offers the option to switch to the graphical console if it\nfinds it. I followed the excellent instructions on this forum\npost.\nTo summarise them:
\nsudo apt-get install vnc4server xinetd : to install the\nrequired packages
Edit /etc/gdm/gdm.conf and uncomment the\nRemoteGreeter=/usr/lib/gdm/gdmlogin line, set the key\nEnable=true in the [xdcmp] section.
Install a new service file for xinetd into\n/etc/xinetd.d/Xvnc with the following contents:
service Xvnc\n{\n type = UNLISTED\n disable = no\n socket_type = stream\n protocol = tcp\n wait = no\n user = nobody\n server = /usr/bin/Xvnc\n server_args = -inetd -query localhost -geometry 1024x768 -depth 16 -cc 3 -once -SecurityTypes=none -extension XFIXES\n port = 5900\n}\nThe major difference from the forum poster is to run it on port\n5900, and not to restrict it to just localhost (since XenCenter\nalso needs to connect to it).
\nFinally, restart the xinetd service by running\nsudo /etc/init.d/xinetd restart.
Once you're done with this installation, you can shut down the VM and\nconvert it to a template. Any exports or clones will continue to run in\nPV mode, since the XenServer XVA export format records all of the\nmetadata required to re-create the VM records.
\nEnjoy the Ubuntu on XenServer experience! Remember to report any issues\nyou have with the in-guest packages on the Ubuntu support forums, or\njust give them positive feedback.
\nPS: many thanks to Andrew Peace and Ian Campbell for assistance. May\ntheir Linux beards remain long and uncut.
", "url": "https://anil.recoil.org/notes/installing-ubuntu-on-xenserver", "external_url": "https://web.archive.org/web/20090123155914/http://community.citrix.com/blogs/citrite/anilma/2008/07/02/Installing+Ubuntu+on+XenServer", "title": "Installing Ubuntu on XenServer", "summary": "This guide provides step-by-step instructions for installing Ubuntu as a paravirtualized (PV) guest on Citrix XenServer 4.1, including configuring the grub boot loader, installing XenServer tools, and setting up VNC for graphical console access.", "date_published": "2008-07-02T00:00:00.000000Z", "date_modified": "2008-07-02T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "xen", "linux", "ubuntu" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2008-mobisys-splittrust-1", "content_html": "Paper on splitting trust between smartphones and web browsers at MobiSys 2008. This work with Richard Sharp, Roy Want, and Trevor Pering extended our earlier crimeware research to enhance security on public terminals using mobile composition. The system splits web browsing functionality between an untrusted public terminal and a trusted personal mobile device, protecting users from keyloggers and screengrabbing attacks when using shared computers. We demonstrated how combining the convenience of public terminals with the security of personal devices could enable safer web browsing in public spaces.
Won best student paper for my PhD work on a high-performance functional packet parsing DSL at Eurosys 2007! The Melange work combined strong static typing with generative meta-programming to create MPL, a domain-specific language for describing Internet packet protocols. Our approach generated fast, zero-copy code that outperformed C implementations - we built fully-featured SSH and DNS servers that showed greater throughput and lower latency than OpenSSH and BIND. The work demonstrated that type-safe languages could eliminate the performance penalty typically associated with memory safety, opening the door to more secure protocol implementations.
Journal paper on interacting with mobile services using camera-phones. This Personal and Ubiquitous Computing article presents both qualitative user-experience studies and quantitative pointing-device experiments with visual tags. We found that novice users could aim and click on visual tags remarkably quickly (under 3 seconds on average) and accurately (meeting our 6% error-rate threshold). The work with Eleanor Toye Scott, Richard Sharp, and others revealed positive attitudes toward visual-tag applications while identifying important reservations about camera-phone technology more generally, providing concrete design lessons for future mobile interaction systems.
New paper Fighting Crimeware: An Architecture for Split-Trust Web Applications available. This Intel Research technical report presents a split-trust browsing architecture that splits web applications across a trusted personal device and an untrusted PC. Working with Richard Sharp, Roy Want, and others at Intel, we developed a system where sensitive information entered on a phone's keypad cannot be read by PC-based keyloggers, and data displayed on the phone's screen is hidden from screengrabbing malware. We implemented a prototype using a commercial cell phone and showed it could defend against a range of crimeware attacks including active injection and browser compromises.
", "url": "https://anil.recoil.org/notes/2006-fighting-crimeware-1", "title": "Fighting Crimeware: An Architecture for Split-Trust Web Applications", "summary": "Paper presenting architecture for split-trust web applications to combat crimeware attacks.", "date_published": "2006-04-01T00:00:00.000000Z", "date_modified": "2006-04-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "security", "web", "privacy", "malware" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2006-fighting-crimeware.pdf", "mime_type": "application/pdf", "title": "Fighting Crimeware: An Architecture for Split-Trust Web Applications" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2005-spin-splat-1", "content_html": "Workshop paper on temporal automata for protocol specifications at SPIN 2005. We presented SPLAT, a tool that makes model-checking more accessible by helping developers bridge the gap between complex applications and their abstract models. The key challenge we addressed was how non-experts could create accurate abstractions without inadvertently hiding critical bugs, and how to determine whether counter-examples represent genuine bugs or just modeling artifacts. SPLAT combines static model-checking with dynamic enforcement of abstractions to validate the correspondence between models and implementations.
Position paper on ubiquitous computing approaches to emerging stream media appliances. This work argued that the ubiquitous computing community needed to catch up with the rapid evolution of streaming media devices and services. We explored how ubicomp principles could be applied to the emerging landscape of media appliances, considering the challenges of integrating location-aware and context-sensitive features into streaming media systems that were becoming increasingly prevalent in everyday environments.
New paper Audio networking: the forgotten wireless technology available. This IEEE Pervasive Computing article explores how audio can be used as a wireless networking technology for mobile devices. We investigated various modulation schemes for transferring data to nearby smartphones through audio signals, covering both the technical aspects and usability considerations. The work included a case study applying audio networking techniques to solve real-world problems in telephone conferencing, demonstrating how this "forgotten" wireless technology could enable new types of proximity-based interactions.
Ubicomp paper on a study of indoor bluetooth propagation using the Active Bat system. This work with Alastair Tse used ultrasonic location tracking to conduct fine-grained Bluetooth signal strength surveys, giving us unprecedented accuracy in mapping radio propagation indoors. We discovered that Bluetooth is actually poorly suited for fine-grained location inference due to hardware and specification limitations, and that device movement speed significantly impacts available bandwidth. The study provided valuable data sets for the research community and practical insights about the limitations of using Bluetooth for ubiquitous computing applications.
Finally had some time to get back from the OpenBSD hackathon and take\nstock of what I worked on. It was pretty interesting one this year, as I\nwent without having much idea of what to work on (unlike last year, when\nI had a mad backlog to catch up on).
\nSome stuff I did during the week included:
\n-1 on error,\nwhich means a signed ssize_t is returned. However, they accept an\nunsigned value as the size of the buffer to process, which means\nthey could potentially return a value outside the range of the\nreturn value. This means you have to check if the return is -1,\nwhich indicates an error, and otherwise cast to a size_t to\ncorrectly get the buffer size back. With the new atomicio, it always\nreturns a size_t, and returns 0 to signal an error (with errno\ncontaining the error, and EPIPE being set for an EOF condition).netbt socket interface (much like\nnetinet or netinet6) to Bluetooth.kerneltrace just accepts a regular\nexpression and inserts a printf in the function prologue which\noutputs the function name and any arguments passed into it. Had this\nidea when chatting with Marius,\nand it turned out to be very useful when trying to figure out\ndataflow in the Bluetooth stack (just compile with\nmake CC="/usr/local/bin/cilly --dokerneltrace --trace-regexp='ubt|ng_blue'").\nThe second one was even simpler; randomvars assigns a non-zero\nvalue to every local variable in a function call to help track down\nuninitialized-local-variable bugs. Heres\none\nChad Loder found in\nmg(1).All in all, the hackathon re-motivated me to continue work on the\nOCaml-based daemons that Dave Scott and I have been\nhacking on. I don't want to be fixing random buffer or integer overflows\nin an OpenBSD hackathon 5 years from now; we need to move on to more\nhigh-level issues.
", "url": "https://anil.recoil.org/notes/c2k5-thoughts", "title": "OpenBSD C2K5 thoughts", "summary": "Reflections on OpenBSD C2K5 hackathon projects and progress.", "date_published": "2005-06-04T00:00:00.000000Z", "date_modified": "2005-06-04T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "melange", "openbsd", "ocaml", "security", "livenotes" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2005-hotdep-spl-1", "content_html": "Paper on temporal automata for protocol implementations at HotDep 2005. This work, done with Dave Scott, explored the challenge of building internet servers that are simultaneously high-performance, dependable, and formally verified through model checking. We investigated how to use temporal automata to specify and verify protocol implementations without sacrificing the performance needed for production systems. The key insight was finding ways to make formal verification practical for real-world network services.
", "url": "https://anil.recoil.org/notes/2005-hotdep-spl-1", "title": "On the challenge of delivering high-performance, dependable, model-checked internet servers", "summary": "Paper on using temporal automata for protocol implementations presented at HotDep 2005.", "date_published": "2005-06-01T00:00:00.000000Z", "date_modified": "2005-06-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "formal-methods", "model-checking", "protocols", "systems" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2005-hotdep-spl.pdf", "mime_type": "application/pdf", "title": "On the challenge of delivering high-performance, dependable, model-checked internet servers" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2005-mc2r-visualtags-1", "content_html": "While designing Spotcodes, we realised that visual\ntags are a much better mechanism to advertise security keys to users instead\nof the error prone and much more difficult to use Bluetooth device discovery\nprotocol. We duly implemented the direct system, and conducted a user study\nwith Eleanor Toye Scott who was down the corridor working with the HCI group in the\nComputer Lab. The resulting journal paper on our SpotCode visual tag login\nsystem is a fun blend of systems and human factors.
\n\n\nWe do not believe that printed tags are competing with RFID and NFC for the prize of "universally accepted connection establishment technology". Instead we observe that each offers complementary tradeoffs in terms of cost, data capacity, interaction distance, client-device compatibility and visibility. We predict that all three of these technologies will ultimately be integrated into mobile applications to provide consumers with the flexibility and functionality they require.
\n
(Update: and indeed, two decades on in 2025, this has played out pretty accurately. It is common now to use QRCodes to access services, and wireless scanning is a relatively rare thing to do but still available.)
New article out on using cameraphones to access site-specific services in IEEE\nPervasive Computing. This work with Eleanor Toye Scott, Richard Sharp, and Dave Scott explored\nhow smartphones could enhance location-based services like ticket machines and\ninformation kiosks while reducing deployment costs. We developed the Mobile\nService Toolkit, a complete client-server framework that lets site-specific\nservices automatically tailor their behavior using personal information stored\non users' phones. The paper presents case studies demonstrating how this\napproach could transform the way people interact with services in physical\nspaces.
Report on our hacking on the AT&T Broadband Phone. This technical report describes our experiments with context-aware telephony, exploring how broadband phone networks could be enhanced with location and presence information to provide smarter calling features. We worked with Ripduman Sohan and Alastair Tse to prototype various context-aware services that could improve the user experience of IP telephony systems.
", "url": "https://anil.recoil.org/notes/2005-bbphone-1", "title": "The Broadband Phone Network: Experiences with Context-Aware Telephony", "summary": "Report on context-aware telephony experiments with AT&T Broadband Phone network.", "date_published": "2005-01-01T00:00:00.000000Z", "date_modified": "2005-01-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "telephony", "context-aware", "pervasive-computing" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2005-bbphone.pdf", "mime_type": "application/pdf", "title": "The Broadband Phone Network: Experiences with Context-Aware Telephony" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/2004-spotcodes-1", "content_html": "A technical report is now available on our SpotCode visual tag system, and includes a user study lead by Eleanor Toye Scott which tested its benefits against conventional mobile interfaces. The work demonstrates how camera-phones can be used for "point-and-click" interactions with visual tags in the real world, measuring both pointing speed and accuracy. We built a complete client/server framework to make it easy to develop applications using this technique, and validated it with prototype applications. The results showed that even novice users could quickly and accurately interact with site-specific mobile services using just their camera phones and public displays.
In what is definitely our most exciting media coverage yet, Spotcodes are featured in the New York Times!
\n\n\nWhen you think of a public information kiosk, your mental picture might include greasy touch screens, broken trackballs and frozen monitors.\nBut researchers at an Intel-financed lab at Cambridge University have developed a way to replace displays like those with something portable, not to mention personal: a cellphone's built-in camera and screen. They and others plan to use commercially available hardware to turn the camera-equipped cellphone into a mouse, remote control, keyboard and more.\n-- New York Times
\n
Richard Sharp got cited as I wasn't in the department that day when the journalist showed up at Intel Research!
\n\n", "url": "https://anil.recoil.org/notes/spotcodes-nytimes", "external_url": "https://www.nytimes.com/2004/10/07/technology/circuits/connecting-paper-and-online-worlds-by-cellphone-camera.html", "title": "Connecting Paper and Online Worlds by Cellphone", "summary": "Researchers use cellphones to connect physical and online worlds, replacing public displays with portable technology.", "date_published": "2004-10-07T00:00:00.000000Z", "date_modified": "2004-10-07T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "spotcodes", "ubicomp", "mobile" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/2004-ubicomp-camera-1", "content_html": ""Instead of having all the hassle of putting things out in the environment that you have to maintain and that people can vandalize, you get a cheap PC, shove it in the back room of your shop and just put posters out front," said Richard Sharp, an Intel researcher here.
\n
We gave a demo at UbiComp 2004 all the way in Tokyo\non our SpotCode visual tag system. It went very well, including some time to do some\nsightseeing in Japan and visit the sumo wrestling championships!
\n", "url": "https://anil.recoil.org/notes/2004-ubicomp-camera-1", "title": "Using Camera-Phones to Enhance Human-Computer Interaction", "summary": "Demo of SpotCode visual tag system at UbiComp 2004 conference in Tokyo.", "date_published": "2004-09-01T00:00:00.000000Z", "date_modified": "2004-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "spotcodes", "ubicomp", "japan" ], "attachments": [ { "url": "https://anil.recoil.org/papers/2004-ubicomp-camera.pdf", "mime_type": "application/pdf", "title": "Using Camera-Phones to Enhance Human-Computer Interaction" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/mit-spotcodes", "content_html": "We got more coverage of SpotCodes and our startup High Energy Magic, leading to lots of interest in the technology.
\n\n", "url": "https://anil.recoil.org/notes/mit-spotcodes", "external_url": "https://cdn.technologyreview.com/s/403022/phone-it-in/", "title": "MIT Technology review covers SpotCodes", "summary": "MIT Technology Review covers SpotCodes technology by High Energy Magic.", "date_published": "2004-09-01T00:00:00.000000Z", "date_modified": "2004-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "spotcodes", "ubicomp", "mobile" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/netgames04-ctf-1", "content_html": "Public touch-screen displays such as airport check-in kiosks aren’t known for having versatile interfaces; they usually lack keyboards or pointing devices, limiting users to a few navigational buttons. But new software from High Energy Magic of Cambridge, England, turns a camera phone with a Bluetooth wireless connection into a portable mouse and keyboard that can take full command of public displays, doing away with the old touch screen. Working with Intel’s Cambridge research lab, High Energy Magic has developed a set of circular symbols, similar in concept to bar codes, that can be displayed by public terminals. Camera phones loaded with the company’s software can translate the symbols into data. Once a phone locks onto one of the symbols, it uses the Bluetooth short-range wireless protocol to send information about its size, position, and orientation to the computer running the display. The phone can then act as a mouse, manipulating on-screen controls such as scroll bars. The company plans to license the technology to businesses, such as travel agencies, that operate public kiosks.\n-- MIT Technology Review
\n
The summer of 2004 was sufficient full of procrastination that the members\nof the SN17 collective in the\nComputer Lab decided to build a computer game. But it wasn't enough to just play\nthe game on our phones -- instead, we combined all the public displays in the corridors,\nand then added in a cutting-edge 5cm-accurate ActiveBAT,\nand built a Symbian-based Capture The Flag game where we all had to run around and\ntag each other physically while tracking the flag virtually.
\nWas it mad? Yes. Was it fun? Yes. Did it get us a paper into the SIGCOMM NetGames\nworkshop? Yes!
\n\nOur novel contributions include: (i) creating a fast-paced, close quarters, location-aware game, (ii) exploring the tradeoffs between the accuracy of a location system, the I/O capabilities of current mobile hardware, and the latency of user feedback, and (iii) investigating the viability of Bluetooth as a component in a low-latency location-aware gaming infrastructure.
\n
I gave a talk at NotCon 2004 on SpotCodes, and it got covered in Wired Magazine!\nOf course, I shared the stage with a man telling the time via a prawn sandwich, so the limelight wasn't all just mine...
\n\n", "url": "https://anil.recoil.org/notes/wired-spotcode", "external_url": "https://www.wired.com/2004/06/from-the-prawn-of-time/", "title": "From the prawn of time", "summary": "Talk at NotCon 2004 on SpotCodes and camera phone technology covered in Wired Magazine.", "date_published": "2004-06-07T00:00:00.000000Z", "date_modified": "2004-06-07T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "spotcodes", "ubicomp", "mobile" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/hem", "content_html": "Anil Madhavapeddy and his colleagues at High Energy Magic think camera phones should be used for more than taking bad pictures. The company's SpotCode reader software lets camera phones recognize a circular tag and then communicate via Bluetooth with a local server.\n-- Roxanne Khamsi for Wired
\n
The SpotCode cellphone software was spun out into a startup called High Energy Magic Ltd, and was covered on Slashdot.
\n\n\nCheck this out! High Energy Magic have announced a public beta of software to let you use your camera-phone as a physical mouse by just pointing and clicking and rotating it in the air.
\n
There were also articles on DeviceForge that were picked up by quite a few outlets.
\nUpdate: You can see some of the videos under Ubiquitous Interaction Devices as well.
", "url": "https://anil.recoil.org/notes/hem", "external_url": "https://slashdot.org/story/04/05/27/1849209/cellphone-as-virtual-mouse-keyboard", "title": "Cellphone as a virtual mouse/keyboard", "summary": "Use your cellphone as a virtual mouse and keyboard with innovative software.", "date_published": "2004-05-27T00:00:00.000000Z", "date_modified": "2004-05-27T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "ubicomp", "mobile", "spotcodes", "startups" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/audio-networking-1", "content_html": "While working as an intern at Intel Research Cambridge, Dave Scott and Richard Sharp and I put together a fun system based on the emerging new class of smartphones. The project kicked off when we randomly experimented with our fancy Nokia smartphones and discovered that they didn't have anti-aliasing filters on the microphones! We argued that
\n\n\n[...] audio networking can be used as the basis for developing context-aware applications. Audio networking allows standard devices fitted with speakers and microphones (e.g. PDAs, laptops, desktop PCs and mobile phones) to exchange data and infer information about their environment. One of the key advantages of audio networking is that it enables context-aware applications to be immediately deployed on a large scale without requiring users to purchase and install additional hardware.
\n
We used the lack of antialiasing filters to create a set of inaudible location beacons that would allow laptop computers to simply listen using their microphones and discover their current location without any advanced equipment being required!
\n\nYou can read more details in the paper or in the Ubiquitous Interaction Devices project page.
After many rounds of review and helpful feedback from fellow developers,\nI merged my GCC static bounds checking extension into OpenBSD today!
\n\n\nIntroduce a simple static checker for making sure that the bounds\nlength passed to common functions such as strlcpy/strlcat match the\nreal length of the buffer. It also checks to make sure that the bound\nlength was not incorrectly derived from a sizeof(pointer) operation.
\nFunctions must be marked with the new attribute bounded, and warnings\nare turned on by -Wbounded. Specifying -Wformat also enables bounds\nchecking for scanf(3) bounds to '%s' format variables. -Wall now turns\non -Wbounded also.
\nThe checking is pretty limited right now to constant parameters, and the\nbuffers must be statically declared, and not inside a record type. This\nsimple checking still found hundreds of bugs around the ports tree though,\nand there have been no false positive warnings.
\n
You can read more details in the gcc-local(1) manual page as well.
", "url": "https://anil.recoil.org/notes/gcc-bounds", "external_url": "https://undeadly.org/cgi?action=article&sid=20030627104847", "title": "My static C bounds checker extension merged into OpenBSD", "summary": "OpenBSD merges static C bounds checker extension into its codebase.", "date_published": "2003-06-27T00:00:00.000000Z", "date_modified": "2003-06-27T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "openbsd", "compiler", "security", "opensource" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/sam03-secpol-1", "content_html": "My first ever academic paper, written with the expert guidance of Alan Mycroft and my PhD colleagues Dave Scott and Richard Sharp! We worked on a system call policy language to help constrain application access to privileged resources, and implemented this on OpenBSD using systrace. The paper describing the declarative language was presented at SAM 2003 in Las Vegas.
\n\n", "url": "https://anil.recoil.org/notes/sam03-secpol-1", "title": "The Case for Abstracting Security Policies", "summary": "First academic paper on system call policy language to constrain application access to privileged resources, implemented on OpenBSD using systrace and presented at SAM 2003.", "date_published": "2003-06-01T00:00:00.000000Z", "date_modified": "2003-06-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "security", "openbsd", "kernel" ], "attachments": [ { "url": "https://anil.recoil.org/papers/sam03-secpol.pdf", "mime_type": "application/pdf", "title": "The Case for Abstracting Security Policies" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/opening-anil-recoil-org", "content_html": ""Untrusted code" is just as much a social problem as it\nis a technical problem. Looking for a complete solution\nis unrealistic: it is analogous to looking for a solution to\ncrime in general. With this in mind, we do not claim that\nour proposed framework is a panacea. However, although\na number of security problems remain (e.g. covert channel\nleakage), we claim that our system offers the potential to\nraise the security level of existing general purpose operating systems significantly.
\n
I've taken the opportunity to redesign my homepage and switch to its hopefully-permanent\nURL on anil.recoil.org. Many thanks to Jon Parise for giving me permission to base my\nHTML upon his homepage's, saving me lots of design trouble!
The first technical report on the Xen Hypervisor hypervisor is now available. I mainly\ncontributed to the early NetBSD port (but have run into a snag with the lack of\nlinear page tables in our paravirtual page implementation). This was the\ninitial documentation of Xen at the Cambridge Computer Lab, describing the\nparavirtualization approach that would later become foundational to cloud\ncomputing.
I started my PhD at the Systems Research Group in Cambridge this week, based in\nthe Computer Laboratory and Robinson\nCollege. I'll still be working part-time at\nNetApp, but my primary focus will be on the Xen\nhypervisor and other systems research topics.
", "url": "https://anil.recoil.org/notes/starting-phd", "external_url": "https://web.archive.org/web/20030218053705/http://www.cl.cam.ac.uk/Research/SRG/netos/people.html", "title": "Started PhD at Cambridge", "summary": "Started PhD in Systems Research at Cambridge University, focusing on Xen hypervisor.", "date_published": "2002-09-01T00:00:00.000000Z", "date_modified": "2002-09-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "xen", "phd", "cambridge", "computerlab", "systems" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/php-port-layout-openbsd", "content_html": "I've committed a big improvement to the PHP port on OpenBSD, by switching from a complex set of FLAVOR tags over to a set of independently installing "multi packages".
\nThe first thing I did was to import the core PHP without any extensions.
\ncommit 15dc0f67ef5fd0cae9fb841e608b90b9f51c71ca\nAuthor: avsm <avsm@openbsd.org>\nDate: Mon Jun 24 19:23:41 2002 +0000\n\n Import php4-core-4.2.1\n \n Installs the barebones php4 with only the gettext, iconv and recode\n modules compiled in.\n \n All of the other modules have to be installed as shared modules on\n top of this.\n \n In addition to the Apache module, this package also includes a php\n command-line binary which can be used in shell scripts. The binary\n uses the same /var/www/conf/php.ini file as the Apache module.\n \n There is some non-i386 breakage at the moment (notably macppc).\nAfter that, I imported in the extensions system which has many more dependencies, and that generates the multi packages.
\ncommit a5c226010f93bd3ce70667b801d6518354f44914\nAuthor: avsm <avsm@openbsd.org>\nDate: Mon Jun 24 19:27:46 2002 +0000\n\n Import php4-4.2.1 extensions\n \n This module generates a bunch of php4 extensions as shared modules,\n and seperates them out into multiple packages.\n \n End result is that you can pkg_add individual modules now without\n getting into the mess of flavors that we've had in the past.\nThis should make the use of pkg_add for PHP much simpler for new users. Any problems, please file a bug report or let me know.
After the Mars Polar Lander crashed, I took a job at NetApp working as the\nproduct architect for NetCache. Among the hundreds\nof deployments that I help setup across the world, the most fun was figuring out how to scale\none of the biggest bands in the world at the time wanting to stream their concert live to a\nglobal audience.
\nTo pull this off, I moved to Sardinia where Tiscali was based, and worked with their team\nto integrate new support we added to NetCache for the RealPlayer RTSP protocol and extensions.\nThe concert ended up reaching a global audience of 5 million viewers (huge at the time for streaming video across the Internet!) and\neventually well over 30 million watched the archived version.
\nI wrote up a technical report at NetApp on setting up the CDN for the live streaming,\nwhich was picked up by a number of other streaming companies in the next few years.
", "url": "https://anil.recoil.org/notes/netapp-tr-3152-1", "title": "Streaming U2 live across the Internet", "summary": "Technical report on setting up CDN for U2's live concert stream from Sardinia, reaching 5 million viewers globally while working as NetApp product architect.", "date_published": "2002-04-01T00:00:00.000000Z", "date_modified": "2002-04-01T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "distributed", "internet", "streaming", "netapp", "caching", "italy" ], "attachments": [ { "url": "https://anil.recoil.org/papers/netapp-tr-3152.pdf", "mime_type": "application/pdf", "title": "Tiscali: How to build a Content Delivery Network" } ], "_references": [] }, { "id": "https://anil.recoil.org/notes/horde-cache", "content_html": "While hacking on making Chora performant enough to work as the official PHP CVS web-viewer, I added in a general caching subsystem into the Horde PHP framework. Do let me know if you end up finding a use for it in your own applications.
\n\n", "url": "https://anil.recoil.org/notes/horde-cache", "title": "Added a caching subsystem to Horde", "summary": "Horde PHP framework now includes a general caching subsystem for improved performance.", "date_published": "2001-08-20T00:00:00.000000Z", "date_modified": "2001-08-20T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "horde", "storage", "email", "php" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/chora-live-on-php", "content_html": "Add in a Cache framework for persistent storage and retrieval of cached\nobjects. Consider it experimental for now.
\nBasically works for Chora's needs ... implements a filesystem driver\nwhich tries to act sensibly (writes to a tmp file, then does an atomic\nrename to the cache object), to avoid synchronization issues.
\nIt does not cleanup the cached repository at the moment - needs to have\na garbage collection function done at some point.
\n\n
I spent a chunk of time through the year working on the Horde project. I began when I got commit to IMP webmail to fix some bugs in the MIME rendering for our Recoil deployment. You can see my code commits on the horde-cvs mailing list archive.
\nAfter getting to grips with the PHP code, I then went on to totally rewrite the Chora version control viewer so that the CVS repositories for Horde could be browsed online instead of only via the command line.
\nI'm extremely proud to report that the PHP project has now deployed Chora for production use to serve up cvs.php.net, making it our biggest user by far. Thanks for making my day, Rasmus!
\n", "url": "https://anil.recoil.org/notes/chora-live-on-php", "external_url": "https://lists.horde.org/archives/dev/Week-of-Mon-20010806/002886.html", "title": "Chora now the production CVS viewer for PHP", "summary": "Chora is now the production CVS viewer for PHP, replacing viewcvs on cvs.php.net.", "date_published": "2001-08-05T00:00:00.000000Z", "date_modified": "2001-08-05T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "email", "web", "horde", "selfhosting", "recoil", "php" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/chora-internationalised", "content_html": "I switched Chora over to be the default web cvs system behind cvs.php.net\nnow. The old viewcvs site is still available at viewcvs.php.net (dns may\nnot have updated yet)\n -- Rasmus Lerdorf, php.net
\n
One of the coolest things about hacking on the Horde framework is that it gives me lots of features for free that I can use in my web applications. The latest thing I added to the Chora CVS viewer today is the internationalisation framework, so that the frontend can be translated to multiple languages.
\nI've added in a simple German translation to start with, but please contribute your own strings if you get the opportunity.
", "url": "https://anil.recoil.org/notes/chora-internationalised", "title": "Added internationalisation to the Chora viewer", "summary": "Chora viewer now supports internationalisation with multiple language translations available.", "date_published": "2001-08-03T00:00:00.000000Z", "date_modified": "2001-08-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "horde", "php", "cvs" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/openfx", "content_html": "Slashdot covers the GPL release of OpenFX, which I worked on with Stuart Ferguson (my brother's PhD supervisor in Queen's University Belfast).
\n\n", "url": "https://anil.recoil.org/notes/openfx", "external_url": "https://tech.slashdot.org/story/01/02/10/0340210/gpled-3d-modeler-and-renderer", "title": "GPL release of OpenFX", "summary": "OpenFX is now available under the GPL with advanced features like raytracing and NURBS support.", "date_published": "2001-02-10T00:00:00.000000Z", "date_modified": "2001-02-10T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "3d", "openfx", "opensource", "graphics" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/commit-access-to-php", "content_html": "It has a renderer and raytrace engine, NURBS support, kinematics-based animation, morphing, a plugin API - and it's under the GPL. Currently only for Windows, but they're working on a Linux and FreeBSD port.
\n
I've been maintaining PHP on OpenBSD for a while now, including the core package distributed as binary packages.
\nSo as of today, the core team has decided I'm trustworthy enough to have my own commit bit to the central PHP repository, where I can commit code fixes and also maintain the OpenBSD on PHP official instructions. You can contact me on avsm@php.net if you need any help!
I've been using OpenBSD for a few years now as the primary OS for ##recoil\nand have been contributing fixes and ports when I get a chance. So I'm\nincredibly excited to report that the project leader, Theo de Raadt, has\ninvited me to become an OpenBSD developer. I've registered my keys now, and\nwill be known as avsm@openbsd.org!
My first commit is to start fixing up the PHP port, which I have been working\non in PHP-land for a while now.
\ncommit 93d5cc5ae56b22b19aa3bce34d38fa260b882d16\nAuthor: avsm <avsm@openbsd.org>\nDate: Tue Dec 26 23:35:43 2000 +0000\n\n - update to php-4.0.4\n - bump NEED_VERSION\n - no longer need extra distfile number4.tar.gz since it has\n been integrated into the main distribution\n - ltconfig, mysql socket patches are in main distribution now,\n so they are removed. Note that the ltconfig patch was only\n applied to the 4_0_4 branch by the PHP team, so we will have\n to resubmit it for the next version, unless libtool-cvs has\n been updated with our information.\n - Since php3/4 conflict with each other anyway, versioning is\n not needed.\nMany thanks to Jakob for the help with getting started, and for ok'ing my first commit.
", "url": "https://anil.recoil.org/notes/openbsd-developer", "title": "I'm now an OpenBSD developer", "summary": "I'm now an official OpenBSD developer, contributing fixes and ports.", "date_published": "2000-12-26T00:00:00.000000Z", "date_modified": "2000-12-26T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "openbsd", "opensource", "packaging", "recoil", "php" ], "attachments": [], "_references": [] }, { "id": "https://anil.recoil.org/notes/horde-developer", "content_html": "After contributing some patches, I've now got the honour of becoming a core team member of the Horde project. Many thanks to Chuck Hagenbuch and Jon Parise for their trust in me!
\nI'm planning on fixing bugs in IMP and the webmail subsystem, and am getting interested in version control and CVS as well, so I'm going to look at Whups and Chora ore. You can follow my commits on the Horde-CVS commit archives, where I am avsm@horde.org!
Although the Mars Polar Lander ended up crashing, the website itself was one of the busiest websites in the world at the time during the approach to the landing. I was the person handling the website architecture and the amazing webmaster@mars.nasa.gov account at the time. I worked closely with Sun and NetApp and wrote up a technical report on how the Mars Polar Lander website acceleration architecture worked. The report detailed our distributed web site architecture using caching and load balancing to handle massive traffic spikes, with lessons applicable to designing scalable internet services. It was an early real-world example of content distribution networks and cacheability design principles.
In my capacity as webmaster of the Mars Polar Lander, I submitted a note to Slashdot. Although our amazing distributed website took quite a beating (with some estimated 1 in 4 Internet users trying to access it simultaneously), the Lander itself sadly crashed. On the bright side, I got mentioned in a Sun press release because of the Sun Netra T1 servers they gave us to host the website!
\nYou can read more about the architecture behind the site in "Application of a Distributed Web Site Acceleration: Mars Polar Lander".
", "url": "https://anil.recoil.org/notes/mars-polar-lander", "external_url": "https://science.slashdot.org/story/99/12/03/0755200/mars-polar-lander-lands-today", "title": "Slashdot covers the Mars Polar Lander", "summary": "Mars Polar Lander's webmaster shares story of website survival despite spacecraft crash.", "date_published": "1999-12-03T00:00:00.000000Z", "date_modified": "1999-12-03T00:00:00.000000Z", "authors": [ { "name": "Anil Madhavapeddy", "url": "https://orcid.org/0000-0001-8954-2428", "avatar": "https://anil.recoil.org/images/anil-headshot.webp" } ], "tags": [ "mars", "space", "distributed", "nasa", "netapp" ], "attachments": [], "_references": [] } ] }