Accurate summarisation of threats for conservation evidence literature

This is an idea proposed in 2024 as a Cambridge Computer Science Part III or MPhil project, and is currently being worked on by Kittson Hamill. It is co-supervised with Sadiq Jaffer.

At the Conservation Evidence Copilots project, we are interested in constructing a taxonomy of threats to wildlife from the literature. This involves scanning the body of conservation literature and gathering/synthesising evidence for conservation interventions from a threats perspective. Once the text has been retrieved, it needs to be summarised in a way that is accurate, concise and relevant and verified with human experts. This is particularly important for conservation evidence, where the key findings need to be communicated clearly to inform policy and practice.

This project therefore investigates how to generate threats, and to verify their accuracy as generated by LLMs and RAG pipelines from the CE literature. Our goal is to develop a pipeline that can reliably go from extracting relevant information from text to a summary that is verifiably (by a human) correct.

As of June 2025, the project has been successfully completed and submitted for Kittson's MPhil. A test version of the avian threats dataset is online for browsing, and we're spending the summer working on widening the evaluation with the wider CE team.

Related Reading

• The Ragas framework for RAG evaluation
• CheckEmbed: Effective Verification of LLM Solutions to Open Ended Tasks, arxiv:2406.02524v2, June 2024
• Calibrating Sequence Likelihood Improves Conditional Language Generation, arxiv:2210.00045, September 2000