# Local AI agents can now pass the IUCN Red List assessor exam *2026-11-03 — note* [Shane Weisz](https://shaneweisz.com) become a certified IUCN Red List assessor last November, and then wondered whether [Claude Code could pass the same exam](https://www.shaneweisz.com/blog/can-claude-code-pass-the-red-list-exam) he had just sat. It turned out that it could, with a similar mark to what Shane himself achieved! That was a big discovery at the time, but we kind of got used to frontier models being really good. The advance of _local_ models since then has been so rapid that they can now pass the exam as well alongside frontier models, and by running on a desktop-class machine! This is detailed in our new preprint on [evaluating AI agents on the IUCN Red List assessor exam](https://anil.recoil.org/papers/2026-ai-red-list-exam).

## Why bother testing on an exam? Conducting accurate species assessments is painstaking work. The [IUCN Red List](https://www.iucnredlist.org) is the world's most widely used system for assessing the extinction risk of species, and it [underpins conservation policy](https://www.iucnredlist.org/resources/guidelines/iucn-red-list-report-60-years-success) and action everywhere. But the assessments are severely resource constrained; \<20% of plant species have been assessed at all, and over 30% of the assessments that do exist haven't been revisited in 10+ years. One bottleneck is the number of trained assessors available, which is why Shane's [PhD](https://anil.recoil.org/ideas/living-iucn-redlist) and his [Dash of Life](https://anil.recoil.org/projects/enki) project is looking at how to inject modern tooling to the Red List to help them out. Modern AI can help assessors increase both the throughput and the quality of assessments, but it raises very reasonable concerns about trust and reliability, data security, cost, and the environmental impact of the datacentres doing the inference. The official assessor training exam is a convenient starting point for examinging these tradeoffs. The exam covers species categories and criteria, and mapping standards for extent and area of occupancy, and also regional assessment protocols. The questions are hard enough that Shane found them tough, and I didn't know the answers to most things\! ## What we found We built a benchmark of exam questions and executed a spread of agents against it, each with the same official IUCN guidance documents that a human candidate has access to. We found that frontier models from Anthropic and OpenAI reliably clear the 75% pass mark, with the best of them (Claude Opus 5) reaching 96% mean accuracy, rising to 99% with a majority vote over five attempts. The breaking result is that open-weight models running on local hardware now get a pass as well. [Qwen3.8 Flash](https://qwen.ai/blog?id=qwen3.8-omni-flash), self-hosted on a desktop Mac Studio on Shane's desk, attains 81% mean accuracy (87% with majority voting). That is a comfortable pass mark, which means assessment agents could plausibly be deployed on offline and mobile devices in the field. For many Red List partners, the location of sensitive species data (think the coordinates of the last few individuals of a [poached orchid](https://www.theguardian.com/lifeandstyle/2023/feb/24/orchid-show-new-york-botanical-garden-black-market)) is a blocker to using AI, so having a usable local alternative could be a game changer for conservation practitioners. The exam is of course only a start. Passing it doesn't make a model a "species expert" any more than it makes a newly certified human one, and we're keen to encourage the responsible use of AI assistants _within_ assessors' workflows (such as quality and consistency checks on draft assessments) rather than as a replacement for the humans. ## Assessors in the loop Our hope is that AI responsibly deployed, with assessors-in-the-loop and firmly in control, could be a powerful tool to meet the scale of the resourcing challenge in global biodiversity. But we need to collectively work through issues of [transparency, traceability and auditability](https://anil.recoil.org/notes/principles-for-collective-knowledge) of anything an agent produces. Just as importantly, the human communities of assessors and specialist groups who do this work need to end up stronger and better connected as a result of these technologies, not more fragmented. The paper is a collaboration between [Shane Weisz](https://shaneweisz.com), [Michael Dales](https://mynameismwd.org), [Sadiq Jaffer](https://toao.com), [Steve Bachman](https://orcid.org/0000-0003-1085-6075) at Kew. You can read the [preprint on EcoEvoRxiv](https://doi.org/10.32942/X2870H), and comments and thoughts are extremely welcome, to any of us. ## References 1. Weisz et al (2026). Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments. EcoEvoRxiv. 2. Madhavapeddy (2025). Four Ps for Building Massive Collective Knowledge Systems. Synopsis: New preprint evaluating frontier and local AI agents on the IUCN Red List assessor exam, and finding that open-weight models on a desktop now clear the pass mark. Words: 655 ## Related - [Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments](https://anil.recoil.org/papers/2026-ai-red-list-exam) (paper, 2026-11-01) Shane Weisz, Michael Winston Dales et al. - [Enki, a Dashboard of Life on Earth](https://anil.recoil.org/projects/enki) (project, 2026-01-01) - [Four Ps for Building Massive Collective Knowledge Systems](https://anil.recoil.org/notes/principles-for-collective-knowledge) (note, 2025-11-23) Design principles for collective knowledge systems—permanence, provenance, permission, and placement—that enable robust networks for evidence-based decision making. - [Can Claude Code pass the IUCN Red List Assessor Exam?](https://www.shaneweisz.com/blog/can-claude-code-pass-the-red-list-exam) (feed, 2025-11-18) Shane Weisz. Documents an experiment testing whether Claude Code could pass the IUCN Red List Assessor certification exam, achieving an 80% average across five exam attempts and passing four of them. - [A Living IUCN Red List of the World's Species](https://anil.recoil.org/ideas/living-iucn-redlist) (idea, 2025-09-01) Ongoing · PhD --- Canonical: https://anil.recoil.org/notes/ai-red-list-exam Type: note License: CC BY 4.0 Tags: conservation, biodiversity, iucn, ai