Steps towards an ecology of the Internet / Jun 2025
Every ten years, the city of Aarhus throws a giant conference to discuss new agendas for critical action and theory in computing. Back in 2016, Hamed Haddadi, Jon Crowcroft and I posited the idea of personal data stores, a topic that is just now becoming hot due to agentic AI. Well, time flies, and I'm pleased to report that our second dicennial thought experiment on "Steps towards an Ecology for the Internet" will appear at the 2025 edition of Aarhus this August!
This time around, we projected our imaginations forward a decade to imagine an optimistic future for the Internet, when it has exceeded a trillion nodes. After deciding in the pub that this many nodes was too many for us to handle, we turned to our newfound buddies in conservation to get inspiration from nature. We asked Sam Reynolds, Alec Christie, David Coomes and Bill Sutherland first year undergraduate questions about how natural ecosystems operate across all levels of scale: from DNA through to cells through to whole populations. We spent hours discussing the strange correspondences between the seeming chaos in the low-level interactions between cells through to the extraordinary emergent discipline through which biological development typically takes place.
Then, going back to the computer scientists in our group and more widely (like Cyrus Omar who I ran into at Bellairs), it turns out that this fosters some really wild ideas for how the Internet itself could evolve into the future. We could adopti biological process models within the heart of the end-to-end principle that has driven the Internet architecture for decades!
[…623 words]
papersSemi distributed filesystems with ZFS and Sanoid / Apr 2025
Over in my EEG group, we have a lot of primary and secondary datasets lying around: 100s of terabytes of satellite imagery, biodiversity data, academic literature, and the intermediate computations that go along with them. Our trusty central shared storage server running TrueNAS stores data in ZFS and serves it over NFSv4 to a bunch of hosts. This is rapidly becoming a bottleneck as our group and datasets grow, and Mark Elvers has been steadily adding lots more raw capacity. The question now is how to configure this raw SSD capacity into a more nimble storage setup. If anyone's seen any systems similar to the one sketched out below, I'd love to hear from you.
[…1676 words]
notesAre you still using OCaml 4.08 or earlier? If so, we need to know (via ocaml.org)/ Mar 2025
I started pushing OCaml Docker images over to the Docker Hub in around 2017, to support the burgeoning automated build infrastructure around the use of the language. Back then, OCaml 4.06 was the latest release, and so I wrote an ocaml-version library to track the release metadata. It has been a bit of a success disaster, as that library now tracks every release of OCaml in the modern era, and also backs the automatic building of a huge array of compiler versions and variants across Linux and Windows.
The problem is...we're now building the full set of images from OCaml 4.02 onwards through to the latest OCaml 5.3.0 release, which is unsustainable for obvious reasons; despite the hosting being kindly sponsored by Docker, we must also consider the carbon footprint of our infrastructure. So the question for the OCaml community: are there are any remaining users who still need images earlier than OCaml 4.08 or can we can stop pushing those now?
[…313 words]Thoughts on the National Data Library and private research data / Feb 2025
Over the past year, Sadiq Jaffer and I have been getting an object lesson in how the modern Internet handles researcher access to data, as we've been downloading tens of millions of research papers towards our Conservation Evidence project. This is legally possible via our institutional subscriptions that give us license to fulltexts, and the incredibly helpful head of electronic services at the University Library who wields encyclopedic knowledge of each of our agreements with the hundreds of publishers out there. My thoughts on this then segwayed into recent conversations I've been having about the emerging National Data Library and also with the UK Wildlife Trusts...
[…2532 words]OCaml.org: recapping 2022 and queries on the Fediverse (via OCaml.org)/ Jan 2023
I recap the OCaml community progress in 2022, which covers a number of bases ranging from the release of OCaml 5.0, the launch of a new website with integrated documentation for 20000+ packages, prototyping new developer workflows that are better integrated into editors, and the launch of ActivityPub based services such as https://watch.ocaml.org.
OCaml Multicore Monthly: post merge activites (via OCaml.org)/ Feb 2022
After we got the massive OCaml 5.0 pull request merged, we've taken some time to consolidate the trunk branch of OCaml and start down the release path towards getting OCaml 5.0 out of the door.
Decentralised Capability-based Code Collaboration using Matrix / Jan 2022
This is an idea proposed as a Cambridge Computer Science Part II project, and has been completed by Samuel Wedgwood.
In 2005, due to licensing disputes, the team behind Linux parted ways with their proprietary source management tool BitKeeper, and needed a new solution. This prompted the development of Git, an open-source decentralised version control system (DVCS), which was soon used to manage the source code of Linux. Contributions were submitted as patch files, which contained just the differences that the contribution made, to an email list, which were reviewed and applied to the central Git repository for Linux.
Git grew in popularity and other projects started using it to manage their source code. Then, in 2008, the GitHub.com platform launched, providing Git repository hosting alongside other project management tools. Notably, GitHub facilitates "pull requests", where contributors fork the repository, make changes to their fork, and then request that their changes be merged back into the central repository. As of 2023, GitHub hosts over 364 million repositories and is the most popular version control platform for both personal and professional use, followed by GitLab and BitBucket, which are all centralised version control platforms (CVCPs).
[…386 words]
ideasOCaml Multicore Monthly: code review complete with Inria (via OCaml.org)/ Dec 2021
We've been working hard on OCaml multicore support, and went over to Paris to sit down with some core developers from Inria and work through code review of our proposed patches.
What is an Operating System? (via Signals and Threads)/ Nov 2021
I am the latest person to feature on the first season of the Signals and Threads podcast hosted by Yaron Minsky (you may recognise him as my co-author on Real World OCaml).
Anil Madhavapeddy is an academic, author, engineer, entrepreneur, and OCaml aficionado. In this episode, Anil and Ron consider the evolving role of operating systems, security on the internet, and the pending arrival (at last!) of OCaml 5.0. They also discuss using Raspberry Pis to fight climate change; the programming inspiration found in British pubs and on Moroccan beaches; and the time Anil went to a party, got drunk, and woke up with a job working on the Mars Polar Lander. -- Signals and Threads
I think I might be the first non- Jane Street person to be on their podcast! Quite the honour.
OCaml Multicore Monthly: effect handling confirmed for 5.0 (via OCaml.org)/ Oct 2021
We're making steady progress on getting multicore support merged into OCaml, including some great developer meetings where we achieved consensus with the core team to include support for effect handlers in the 5.0 release.
Decentralised tech on Recoil / Sep 2021
Nick Ludlam and I have self-hosted recoil.org since around 1996, typically for email and web. These days, there are a number of interesting software stacks around decentralised communication that we deploy. This note keeps track of them.
[…458 words]State of the OCaml Platform 2020 / Aug 2020
Talk on the state of the OCaml Platform in 2020
talksOCaml Multicore Monthly: systhreads compatibility merged (via OCaml.org)/ Aug 2020
The big advance in the multicore OCaml branch is that we restored compatibility with the traditional OCaml systhreads. This in turn means that many existing software packages just work out of the box on the new runtime.
Big news this month is that the systhreads compatibility support PR has been merged, which means that Dune (and other users of the Thread module) can compile out of the box. You can now compile the multicore OCaml fork conveniently using the new opam compiler plugin (see announcement). -- me, on the discussion forum
OCaml Multicore Monthly: starting upstream to OCaml (via OCaml.org)/ Jan 2020
We started the process of upstreaming our multicore OCaml branch to mainline OCaml, and so I started posting regular updates to the community forum.
The most common question we get is how to contribute to the overall multicore effort. As I noted last year, we are now in the process of steadily upstreaming our efforts to mainline OCaml. Therefore, the best way by far to contribute is to test for regressions or opportunities for improvements in the patches that are outstanding in the main OCaml repository. -- me, on the discussion forum
New opam repository layout for large libraries (via OCaml.org)/ Nov 2018
Managing package manager constraints is getting difficult, particularly given the growth of the number of packages in the opam repository. I'm therefore laying out a new mechanism for the OCaml contributors to submit large package sets, such as those from Jane Street.
Displaying the 15 most recent news items out of 38 in total (see all the items)