# Generating chunk-free embeddings for LLMs *2024-01-01 — idea* This project aims to explore the development of a chunk-free approach for generating embeddings in Retrieval-Augmented Generation (RAG) models. Traditional RAG workflows often involve manual or predefined chunking of documents, and we seek to bypass this requirement. Instead, our approach involves generating multiple embeddings for unchunked text using a synthetic dataset created by (e.g.) a 7b parameter LLM. This dataset would feature structured, point-by-point summaries of each paragraph. An off-the-shelf embedding model could then be modified by removing its mean pooling layer and incorporating cross-attention layers. These layers, inspired by T5's encoder-decoder architecture, would enable a frozen set of embeddings to interact with summary-based embeddings via cross-attention, creating a more nuanced chunk-free representation. Additionally, the research aims to explore adaptive chunking driven by a trained model, allowing context-aware embedding generation end-to-end. This method promises a more integrated and efficient approach, eliminating the need for separate summarization and embedding processes. This project has been completed with great success. The resulting system, `dynachunk`, is written up in Mark Jacobsen's thesis, which is available on request until it is published. Status: Completed Level: MPhil Year: 2024 Project: Conservation Evidence Copilots Supervisors: Sadiq Jaffer, Anil Madhavapeddy Students: Mark Jacobsen --- Canonical: https://anil.recoil.org/ideas/chunk-free-embeddings Type: idea License: CC BY 4.0 Tags: ai, llms, :2024-ce-llm