# Language integrated LLMs as an OCaml function *2026-06-14 — note* Fable [cut out on me](https://www.theverge.com/ai-artificial-intelligence/949553/anthropic-fable-5-mythos-5-government-national-security) at 1am on Saturday while I was sweeping over the OCaml runtime looking for concurrency bugs. There have been [excellent takes](https://x.com/rosstaylor90/status/2066067747738431504) on the sovereignity implications of this, and I figured I'd roll my sleeves up and get serious about using the open weights models. DeepSeek's models have been getting more capable since their [first release](https://anil.recoil.org/notes/deepseek-r1-advances), and [v4 Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) is small enough to run on my Mac (admittedly, very high-end Macs with 128GB/512GB of RAM respectively for my laptop and desktop). The question is whether the agentic CLIs I've [been using](https://anil.recoil.org/notes/aoah-2025) can be easily replaced. The best way to learn how a system works is to build it [unikernel style](https://anil.recoil.org/projects/unikernels), and so I aimed to expose the LLM as a normal OCaml library. This avoids routing via [bloated CLIs](https://github.com/anthropics/claude-code/issues/8382), and lets the linking application drive the agentic loop according to its specific needs. What makes this practical is [Antirez](https://github.com/antirez)' [Dwarfstar](https://github.com/antirez/ds4), a self-contained native inference engine that supports [Apple Metal](https://developer.apple.com/metal/) and portable(ish) C. I bound this directly to OCaml 5 and [Eio](https://github.com/ocaml-multicore/eio) as **[ocaml-deepseek](https://tangled.org/anil.recoil.org/ocaml-deepseek)**, and now a plain function call on my laptop gets me an LLM in my application.
The Humpty OCaml deepseek agent in full (local) poetic flow
The Humpty OCaml deepseek agent in full (local) poetic flow
For example, I can now embed Deepseek inference directly into the OCaml webserver that drives this very site in order to look for suspicious bot activity, and because it's open weights and running locally, there's no dependency on external services\! ```ocaml (* A traffic-triage agent in-process in OCaml. The agent is handed two OCaml function tools and works out for itself how to combine them. *) let agent = Agent.create engine ~system:"You are a web-traffic analyst." ~tools:[ Toolbox.read ~dir:logs; (* read-only sandboxed handle to the logs dir *) query_db ~conn; (* a SELECT-only tool over the local database *) ] in Agent.send agent ~on_event "Cross-reference today's 404 spikes in the access log against the \ client IPs in the requests table. Anything coordinated indicating a bad bot?" ``` The log reader and the database query are just two OCaml functions the model is allowed to call, each scoped and sandboxed (using Eio) to exactly what it needs. The model decides when and how to combine them. ## Trying out Humpty the OCaml agent I've [submitted](https://github.com/ocaml/opam-repository/pull/30053) the package to opam, so `opam install deepseek` or `opam pin add deepseek https://tangled.org/anil.recoil.org/ocaml-deepseek.git` should work. The package also ships a binary called humpty[^1] with two variants: `humpty-metal` for Apple Silicon and a portable `humpty-cpu` that should run anywhere (slowly). There are four subcommands that we'll use to explain how to build an agent up in OCaml: first [`list`](#choose-the-right-deepseek-model) the available models and [`download`](#grab-the-deepseek-model-weights) one, then [`chat`](#an-llm-is-a-stateless-request-reply-function) with it statelessly, and then wrap that into an [`agent`](#adding-state-to-make-an-agentic-ocaml-library). ### Choose the right Deepseek model Before we can get started you'll first need the [open model weights](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) downloaded. [`humpty list`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/bin/humpty.ml#L101-134) prints a the catalogue of available weights: ```bash $ humpty-metal list Models (download dir: /Users/avsm/.local/share/ds4) TARGET ALIASES DESCRIPTION [ ] q2-imatrix q2 2-bit Flash routed experts (~81 GB); for 96-128 GB RAM. [*] q2-q4-imatrix q2q4 Mixed Flash quant (~98 GB); higher quality for 128 GB. [ ] q4-imatrix q4 4-bit Flash routed experts (~153 GB); for 256 GB+ RAM. [ ] pro-q2-imatrix pro-q2 PRO q2 single file (~430 GB); for 512 GB RAM. [*] = present, [ ] = not downloaded ``` Pick one based on how much RAM you have; I use `q2q4` on my laptop (with 128GB RAM), and the extremely beefy `pro-q2` on my Mac Studio (with 512GB RAM). There are also split files for running the model distributed across several machines, which I'll skip here for now. ### Grab the Deepseek model weights Once you've chosen, [`humpty download q4`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/bin/humpty.ml#L66-97) (or `pro-q2`, or whichever) shells out to the Hugging Face CLI to fetch the GGUF. You'll either need the [Huggingface CLI installed](https://huggingface.co/docs/huggingface_hub/en/guides/cli) or have [uvx](https://github.com/astral-sh/uv) in your path. Once this gets doing go have a cup of tea while the gigabytes of LLM weights download, and then we'll start to build an agent from the camel up\! ## Building an agent from the ground up I first want to pin down what an "agent" actually means, as the term seems to have accreted much mystique this year. The whole OCaml Deepseek stack is a small library you can read through quickly, so let me build an agent up from scratch. The code below links to the [Tangled source](https://tangled.org/anil.recoil.org/ocaml-deepseek/commit/1edcdad29a19924f988c7adef4343cb189dcb4a2). ### An LLM is a stateless request-reply function A basic LLM takes in a text prompt, performs inference on some weights, and generates a text reply back. To illustrate this in our OCaml code, we need to load the model weights and spin up an `engine` with a cache directory for the compiled Metal kernels (if using the Apple GPU version): ```ocaml let engine = Deepseek.V4.create ~cache ~model ~domain_mgr ~sw () in V4.generate engine "Explain monads in one sentence." ~on_token:print_string; - : unit => Monads are a design pattern that allows you to chain operations together while automatically handling extra behavior like error handling, state, or side effects, by wrapping values in a context and providing a way to transform and combine them. ``` [`Deepseek.V4.create`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/v4.mli#L41-47) opens the [GGUF](https://huggingface.co/docs/hub/gguf) model file and, the first time it runs, materialises the embedded Metal shaders in the `cache`. Generating a reply is then a single call to `V4.generate` that encodes the supplied prompt, runs a prefill, and samples one token at a time into the `on_token` callback until the end-of-sequence marker. All the inference is done in the Metal library in a separate OCaml domain, so we can continue to use other Eio fibres in our main application. You can try this single-shot request/response using [`humpty chat`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/bin/humpty.ml#L44-62), which keeps no memory between runs and can't take any action beyond showing the reply. ```bash $ humpty-metal chat 'Explain algebraic effects in OCaml 5 in one sentence' Algebraic effects in OCaml 5 allow functions to suspend execution and invoke user-defined handlers for operations (like state, exceptions, or generators) via a lightweight, type-safe mechanism that integrates with the language's existing type system and is used primarily for effectful programming, such as with the new `Effect` library for handling delimited continuations. ``` A "conversation" is therefore just a list of role-tagged messages (e.g. system, user, assistant, tool) that we concatenate in our library into a prompt string for the LLM. ## How the stateless LLM asks for effects to the external world The single-step text-to-text function from earlier emits text in an agreed "shape" so we can figure out what to do next based on its output. DeepSeek has trained their model to understand a little markup language called [DSML](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/encoding/encoding_dsv4.py), which looks something like this: ```xml <｜DSML｜tool_calls> <｜DSML｜invoke name="edit"> <｜DSML｜parameter name="path" string="true">/tmp/x.c <｜DSML｜parameter name="line" string="false">42 ``` DSML is a pseudo-XML language that's interspersed in the text responses from the agent. Those bars are actually the full-width vertical line `｜` (U+FF5C) and not an ASCII `|`. DeepSeek reserves the rarer codepoint for DSML's control tokens, so they can't be produced by ordinary text or code the model emits. We've got a [DSML implementation in OCaml](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/main/dsml/lib/dsml.mli), which [parses the XML into an OCaml record type](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/dsml/lib/dsml.mli#L83-98): ``` type thinking_mode = Chat | Thinking type reasoning_effort = High | Max type task = Action | Query | Authority | Domain | Title | Read_url type tool_call = { id : string option; name : string; arguments : string } type parsed_message = { content : string; reasoning_content : string; tool_calls : tool_call list; } ``` Parsing a raw text reply splits it into the visible content the user will see, the model's hidden [reasoning content](https://en.wikipedia.org/wiki/Reasoning_model), and any tool calls it emitted along the way. A `tool_call` is just a string name plus its arguments as a JSON string, with an optional identifier to pair the result back up. The other three types are knobs on *how* the model replies rather than what it returns: - a `thinking_mode` of `Chat` answers directly while `Thinking` reasons in a `` block first - `reasoning_effort` turns that reasoning up to maximum at the cost of slower inference time - `task` is a quick-instruction hint into DeepSeek's internal pipeline as to whether this turn is an `Action`, a `Query`, etc. ### Making custom tool functions in OCaml We now need to define specific tools that the model knows about. We do this by defining a bidirectional codec that decodes the model's JSON/DSML arguments into a typed OCaml value, and renders that value back. Here's a simple [touch](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/tool.mli#L11-20) tool: ``` let touch = let open Dsml.Codec in Invoke.map "touch" (fun path -> path) |> Invoke.param ~enc:Fun.id "path" string ~description:"file to create" |> Invoke.seal in Tool.v ~description:"Create an empty file." touch (fun path -> Out_channel.with_open_text path ignore; "created " ^ path) ``` We just wrap the effect we are doing (in this case, just writing an empty file) with the JSON metadata to let the model know _when_ and _how_ to invoke the tool as it works its way through the prompt. Unlike most policy languages, we describe these in plain text since we're dealing with an LLM, and it decides when the description of the tool should be applied in the conversation. There's still no state here, but we can now use the DSML library to build up a full prompt string by keeping track of all the messages: ```ocaml val encode_messages : ?context:message list -> ?drop_thinking:bool -> ?add_default_bos_token:bool -> ?reasoning_effort:reasoning_effort -> thinking_mode -> message list -> string (** [encode_messages mode messages] renders the conversation to the prompt string. [context] prepends an already-encoded prefix and suppresses the leading token; [drop_thinking] (default true) drops reasoning from turns before the last user message; [reasoning_effort] [Max] maximises reasoning. *) ``` ## Adding state to make an agentic OCaml library We're now ready to add state to this! An "agent" is just a wrapper around the LLM that does three things: - remember the conversation so far for `encode_messages` - keep the engine's KV-cache warm via [a persistent session](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/v4.mli#L95-108) - run the tool callbacks as they arrive from the LLM The loop is expressed as a simple [OCaml event datatype](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/agent.mli#L27-33): ```ocaml type event = | Reasoning of string (* the model's text *) | Content of string (* a chunk of the reply *) | Tool_call of Dsml.tool_call | Tool_result of string * string | Done val send : t -> on_event:(event -> unit) -> string -> unit ``` When the LLM responds each turn, plain text means the turn is `Done`. Tool calls get looked up by their name, then run, and the results folded back into the conversation as `tool` messages. All the agent function does is just run this to a fixed point until the model answers with text alone. ### Writing custom tools using OS sandboxing and Eio capabilities Here's where the unikernel-style magic shows up though. Given that a tool is just something we define ourselves, then we can start to take advantage of OCaml itself! And in particular, I want better security and more fine-grained tool calls that are tailored to my applications and not generic shell scripts that are difficult to sandbox. [Eio is a library](https://anil.recoil.org/papers/2023-ocaml-eio) built over [OCaml 5's effects](https://anil.recoil.org/papers/2021-pldi-retroeff) that follows an [object-capability](https://en.wikipedia.org/wiki/Object-capability_model) discipline to eliminate ambient authority where possible. In our [Toolbox](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/toolbox.mli#L4-22) module, we define some example Eio tools: ```ocaml val read : dir:_ Eio.Path.t -> Tool.t val write : dir:_ Eio.Path.t -> Tool.t val dns : net:_ Eio.Net.t -> Tool.t val bash : proc:_ Eio.Process.mgr -> Tool.t ``` Notice that each OCaml signature here takes in a capability for that particular tool: - `read` and `write` can only access the directory you pass as `~dir` and nothing above it, since Eio uses *[openat(2)](https://linux.die.net/man/2/openat)* to sandbox the call. - `dns` reaches the network only because it holds a net capability - `bash` spawns processes only because it holds a [process manager](https://github.com/ocaml-multicore/eio/blob/main/README.md#running-processes). When using these from applications, we can select the exact sandboxing we need, or just write our own tool functions with application-specific logic. This is exactly what [`humpty agent`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/bin/humpty.ml#L154-201) does, confining the file and shell tools to a workspace directory you pass with `--dir`: ``` (* Sandbox the filesystem tools to [workspace] so that tools only have access there *) Eio.Path.with_open_dir Eio.Path.(fs / workspace) @@ fun ws -> let agent = Agent.create engine ~system ~thinking:think ~tools: [ Toolbox.read ~dir:ws; Toolbox.write ~dir:ws; Toolbox.dns ~net; Toolbox.bash ~proc; ] in ... ``` The `fs`, `net` and `proc` values all come from Eio's stanard environment, and its now up to the programmer to decide how to dole them out. If you want a read-only agent, then just drop `write` and `bash`. Since a tool is just a [`Tool.v`](https://tangled.org/anil.recoil.org/ocaml-deepseek/blob/1edcdad29a19924f988c7adef4343cb189dcb4a2/lib/tool.mli#L26) function over a codec and a handler, callers can add their own depending on the business logic of the app. For example, I've now got some tools in this webserver that query the size of the connection pool, another that can search the in-memory logs, and so on. ### Managing deterministic and reproducibility One of the advantages of running a local model is that we have better control over the determinism of the inference. Aside from obvious factors like the weights and the inference code being the same, we normally still get different results from GPU based inference due to the parallelism. However, if we don't mind taking a slowdown, the CPU backend here supports saving and passing in the same seed: ``` $ humpty-cpu chat 'tell me a joke about camels in one sentence' -v humpty-cpu: [INFO] model: DeepSeek V4 Flash (vocab 129280) humpty-cpu: [INFO] seed: 1690400691090126 (random; pass --seed N to reproduce) Why did the camel cross the desert? To get to the other hump-side! $ humpty-cpu chat 'tell me a joke about camels in one sentence' -v humpty-cpu: [INFO] model: DeepSeek V4 Flash (vocab 129280) humpty-cpu: [INFO] seed: 1690392111533869 (random; pass --seed N to reproduce) Why did the camel cross the desert? Because it was sick of the hump-drum of the same old sand dunes. $ humpty-cpu chat 'tell me a joke about camels in one sentence' --seed 1690400691090126 Why did the camel cross the desert? To get to the other hump-side! ``` I was surprised, however, to find that the deterministic seed also worked in the Metal backend\! ``` $ humpty-metal chat 'tell me a joke about camels in one sentence' -v humpty-metal: [INFO] model: DeepSeek V4 Flash (vocab 129280) humpty-metal: [INFO] seed: 2042575750328474 (random; pass --seed N to reproduce) Why did the camel cross the desert? Because it was tired of the hump-drum of its daily routine. $ humpty-metal chat 'tell me a joke about camels in one sentence' -v humpty-metal: [INFO] model: DeepSeek V4 Flash (vocab 129280) humpty-metal: [INFO] seed: 2043271538873965 (random; pass --seed N to reproduce) Why do camels never get stuck in traffic? Because they have built-in "hump" day passes. $ humpty-metal chat 'tell me a joke about camels in one sentence' -v --seed 2043271538873965 humpty-metal: [INFO] model: DeepSeek V4 Flash (vocab 129280) humpty-metal: [INFO] seed: 2043271538873965 Why do camels never get stuck in traffic? Because they have built-in "hump" day passes. ``` ## Implications of using LLMs as a library We've seen so far that the LLM model can be exposed as a mostly pure function; and that a tool is a function whose type says what it may touch; and that an agent is just a loop over them. Crucially, there's no need for serialisation protocols, REST APIs, MCP authentication and many of the other layers built over them unless the application wants it. One of the big advantages of unikernel-style libraries is that necessary functionality can be specialised at compile-time much more easily. ### Building a safe bastion using Dikjstra monads or refinements This library approach is orthogonal to the idea of "language integrated" LLMs, which involve discharging verification conditions by having LLMs attempt to synthesise proofs of statements embedded within the source code. Ranjit Jhala [observed](https://x.com/RanjitJhala/status/2065280989367357725) that: > "Language integrated" is a drum I've been beating on for a while (e.g. with > [refinement types](https://youtube.com/watch?v=F2tYCxb30WU)), but in the age of > LLMs I wonder if it *really* matters, if the AIs are going to also be generating > the proofs? > -- [Ranjit Jhala, June 2026](https://x.com/RanjitJhala/status/2065280989367357725) [Yaron Minsky](https://github.com/yminsky) [noted that](https://x.com/yminsky/status/2065403677309915570) *"language-integrated assertions and modular specification look like a form of intelligence amplification for whatever is doing the proofs"*. I also found [Shriram Krishnamurthi](https://cs.brown.edu/~sk/) and Flatt's ["Type-Error Ablation and AI Coding Agents"](https://arxiv.org/abs/2606.01522) paper that found richer type errors help an agent fix code, and that a type system helps it more than test failures do. All this means is that we needn't stop at just using types to encode our safety properties. Since each tool is an ordinary OCaml function, nothing stops them being synthesised or checked by a proof assistant like [Fstar](https://www.fstar-lang.org/), so that a tool can ship with a machine-checked guarantee that it stays inside the policy its signature advertises.
The design of our Bastion agentic system in F* and OCaml
The design of our Bastion agentic system in F* and OCaml
 We explored this last year in a [HOPE abstract](https://anil.recoil.org/papers/2024-hope-bastion) via [Dijkstra monads](https://arxiv.org/abs/1903.01237), since they let you reason precisely about the effects a computation is allowed to have. [Patrick Ferris](https://patrick.sirref.org) [Cyrus Omar](https://web.eecs.umich.edu/~comar/) and I sketched how to modularise some more dependently typed policy reasoning in [Bastion](https://anil.recoil.org/papers/2024-hope-bastion). Eio's capabilities are really just a lightweight version of the sorts of things you can express in a full proof framework, and as Ranjit notes above, LLMs make it easier than ever to just pick the right specification language for the problem at hand. Another obvious direction to take my library-agentic-harness is to link it in with OCaml's [compiler-libs](https://ocaml.org/manual/5.4/api/compilerlibref/Compiler_libs.html) to build a much more tightly integrated agent debugger that doesn't need to go via CLI tooling\! ### Letting a service watch...and mutate...itself What else could we do with an LLM we can call as cheaply as any function, on hardware you own? (I'm assuming we have spare CPU/GPU here!) The experiment I'm mid-way through is wiring this into my [zero-allocation OxCaml webserver](https://anil.recoil.org/notes/oxcaml-httpz), which already emits custom [runtime events](https://ocaml.org/manual/runtime-events.html) alongside OCaml's native GC and scheduler ones. Normally they pile up in a ring buffer nobody reads until something's already broken. The idea is to spend idle CPU on them, so when the server isn't busy, I hand a window of recent events to my agent function in a separate domain and check to see if the latency is drifting, or if that event is firing far more than yesterday (based on summary stats), and so on. Whether a local model is any good at this is something I don't know yet, but I'm enjoying experimenting. As we discussed in the [rewilding the web](https://anil.recoil.org/notes/rewilding-the-web-report) workshop and our [Internet ecology](https://anil.recoil.org/papers/2025-internet-ecology) paper, self-hosted services unfortunately don't just [run themselves](https://anil.recoil.org/notes/recoil-self-hosting-2026) and need tending to. I could therefore imagine building enough local tooling so that the agent harness would recompile and re-exec the binary autonomously in response [to external stimuli](https://anil.recoil.org/notes/internet-immune-system)... The [code's here](https://tangled.org/anil.recoil.org/ocaml-deepseek) if you'd like to pull it apart. I'd love to hear more about any improbable stunts or agentic harnesses you build yourself! In the future, I'll investigate expanding this out to CUDA and also beyond just Deepseek. [^1]: `humpty` is the OCaml binary, and named due to the [Dwarfstar](https://github.com/antirez/ds4) needing a camel connection. Synopsis: Using a local DeepSeek model as an ordinary OCaml library and building sandboxed agents from simple primitives Words: 3204 DOI: 10.59350/61cdd-r5a25 Discussion: - Bluesky: - LinkedIn: - Mastodon: - Twitter: ## Related - [Self-hosting email the hard way from your own routable IPv4 block up](https://anil.recoil.org/notes/recoil-self-hosting-2026) (note, 2026-06-06) - [Rewilding the Web: my workshop report from Edinburgh](https://anil.recoil.org/notes/rewilding-the-web-report) (note, 2026-05-30) - [The Internet needs an antibotty immune system, stat](https://anil.recoil.org/notes/internet-immune-system) (note, 2026-04-08) - [My (very) fast zero-allocation webserver using OxCaml](https://anil.recoil.org/notes/oxcaml-httpz) (note, 2026-02-01) - [2025 Advent of Agentic Humps: Building a useful O(x)Caml library every day](https://anil.recoil.org/notes/aoah-2025) (note, 2025-12-26) - [Steps towards an Ecology for the Internet](https://anil.recoil.org/papers/2025-internet-ecology) (paper, 2025-08-01) - [Deepdive into Deepseek advances](https://anil.recoil.org/notes/deepseek-r1-advances) (note, 2025-02-01) - [Modularizing Reasoning about AI Capabilities via Abstract Dijkstra Monads](https://anil.recoil.org/papers/2024-hope-bastion) (paper, 2024-09-01) - [Eio 1.0 – Effects-based IO for OCaml 5](https://anil.recoil.org/papers/2023-ocaml-eio) (paper, 2023-09-01) - [Retrofitting effect handlers onto OCaml](https://anil.recoil.org/papers/2021-pldi-retroeff) (paper, 2021-06-01) - [Unikernels](https://anil.recoil.org/projects/unikernels) (project, 2010-01-01) --- Canonical: https://anil.recoil.org/notes/language-integrated-llms Type: note Tags: ocaml, ai, llm, eio, agents