Evaluating RAG pipelines for conservation evidence

This is an idea proposed in 2024 as a good starter project, and has been completed by Radhika Iyer. It was supervised by Sadiq Jaffer and Anil Madhavapeddy.

At the Conservation Evidence Copilots project, we are interested in finding and synthesising evidence for conservation interventions. Once we have this evidence, it needs to be synthesised into a form that can be used to inform policy and practice via natural language query interfaces. One way to do this is to use a RAG (Retrieval Augmented Generation) pipeline, which can automatically retrieve relevant information from a large corpus of documents, analyse it to extract key information relevant to CE, and generate a summary of the key findings.

This project involved involve evaluating the performance of RAG pipelines for conservation evidence, comparing different models, configurations and benchmark sets, and identifying areas for improvement. The goal is to develop a set of best practices for using RAG pipelines in the context of conservation evidence, and to make recommendations for future work in this area.

A first preprint on this work titled "Careful design of Large Language Model pipelines enables expert-level retrieval of evidence-based information from conservation syntheses" is now available.

Anil Madhavapeddy, Professor of Planetary Computing

Evaluating RAG pipelines for conservation evidence

Related Reading

Related News