RAG Pipeline Builder

---
name: rag-builder
description: |
  Trigger when the user asks to build a RAG system, add document retrieval,
  add vector search, or ground LLM output in their docs. Phrases: "RAG",
  "retrieval augmented", "vector search", "semantic search", "ground the LLM".
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash(pnpm add *)
---

# RAG Builder

Stand up a retrieval-augmented generation pipeline. Covers ingestion,
chunking, embedding, storage, retrieval, and reranking.

## Prerequisites

- Documents to retrieve over (markdown, PDFs, transcripts)
- Vector store (Turbopuffer, Pinecone, pgvector, or local Lance)
- Embedding model (OpenAI text-embedding-3-small is a reasonable default)

## Steps

1. **Ingest.** Parse source documents into text. Preserve metadata (title,
   url, section) — retrieval without metadata is worse than keyword search.

2. **Chunk.** Not every piece of content is a chunk. Rules:
   - Split by semantic boundary (heading, paragraph) first
   - Hard cap around 512-1024 tokens per chunk
   - Overlap 10-20% between adjacent chunks to preserve context
   - Keep the section heading as a prefix on every chunk from that section

3. **Embed.** Pass each chunk through the embedding model in batches:
   ```ts
   const { data } = await openai.embeddings.create({
     model: "text-embedding-3-small",
     input: chunks,
   });
   ```

4. **Store.** Vector store with metadata. Always store:
   - The chunk text (for synthesis)
   - The embedding vector
   - The source URL or doc ID
   - The heading path

5. **Retrieve.** Query-time pipeline:
   - Embed the query with the same model
   - Top-k similarity search (k=20, not 5; you'll rerank)
   - Rerank with a cross-encoder (Cohere Rerank, or a small LLM)
   - Return top 5-8 to the synthesis prompt

6. **Synthesize.** Pass retrieved chunks into the LLM with:
   - Explicit instruction to cite chunks
   - Instruction to say "I don't know" if the answer isn't in the chunks
   - The user's original query
   - The chunks with their metadata

7. **Evaluate.** A handful of canonical questions with known correct
   answers. Track retrieval recall (was the right chunk in top-k?) and
   answer quality (did the LLM use it well?). Both can degrade independently.

## Common RAG failures

- **Chunks too large** - retrieval works, but the LLM drowns in context
- **Chunks too small** - no context, the LLM can't synthesize
- **No reranking** - top-k similarity is noisier than people assume
- **No citations** - users can't verify; trust drops
- **Stale index** - data changed but the vectors didn't

## Output

- Ingest script that produces chunks with metadata
- Embedding + store pipeline
- Retrieval endpoint with reranking
- Synthesis template with citation requirement
- Evaluation harness with 10+ canonical Q/A pairs