CausalRPC: a traceable distributed computation framework

This is an idea proposed in 2018 as a Cambridge Computer Science Part II project, and has been completed by Craig Ferguson.

The project aims to implement an RPC framework in OCaml using the Irmin distributed database library as a network substrate. It will explore the trade-offs of a novel data-oriented approach to RPC in which race conditions between clients are resolved automatically by the middleware layer. The core deliverable is a demonstration of an RPC client remotely executing functions with Irmin-serialisable parameters on a server capable of handling concurrent client requests.

The project was completed successfully, with an implementation of CausalRPC, a distributed computation framework satisfying the above criteria. The approach of making the statefulness of RPC explicit was surprisingly effective, allowing CausalRPC to provide stronger consistency and traceability guarantees than conventional RPC systems. This broadened the scope of the project considerably, allowing for a variety of extensions to explore the inherent trade-offs of the approach. The final version of CausalRPC supported fault-tolerant worker clusters and is compatible with MirageOS.

Related reading

• Mergeable persistent data structures

Links

The project PDF writeup is publically available, and Craig Ferguson won the G-Research Prize for Best Individual Project 2018 departmental prize.

Craig Ferguson also gave a talk about CausalRPC at the 2019 OCaml Workshop. Unfortunately the videos of that year's ICFP don't seem to have made it online, but the slides are available.

Craig Ferguson followed up with a podcast where he discussed his subsequent work on Irmin in 2022: