Retrieval-Augmented Generation in EtherBranch Lab

Large language models are trained on the open web. They are brilliant at fluency and fragile at fidelity. Ask a general model about Stewart's Calculus and you may get a correct-sounding explanation that quietly invents a theorem, misnumbers a section, or blends two editions. For academic study that is unacceptable: a student's trust is the product.

Retrieval-Augmented Generation (RAG) fixes the grounding problem by inserting an explicit retrieve step before generate. Instead of asking the model to remember your textbook, Lab first fetches the passages that matter, then asks the model to answer using those passages. Hallucinations drop; citations become first-class UI.

EtherBranch Lab is intentionally narrow. Every session is tied to an isbn13 the user owns (via checkout → BookOwned). The retriever never leaves that book's corpus. That constraint is pedagogical and commercial: you study what you purchased; the model cannot wander into pirated PDFs or unrelated blogs.

When the user selects a mode—Ask, Quiz Me, Explain Simpler, Give an Example, Test My Recall—the retrieval query is rewritten. “Explain Simpler” prefers definitional passages and worked examples; “Quiz Me” prefers exercise stems and answer keys; “Test My Recall” prefers objective titles the SM-2 scheduler flagged as due.

Naive fixed-size windows destroy textbooks. Lab's curriculum seeder already mirrors a hierarchical syllabus: Book → Chapter → Section → Objective. Chunking should follow that grain. Prefer section-aligned chunks of roughly 400–800 tokens with overlapping edges so definitions that straddle a page break still retrieve. Page and section labels travel with every chunk so the UI can render Ch. 4, p. 112 → citation chips the student can click.

Worked examples and theorem boxes get their own chunk type. When the mode is “Give an Example,” the retriever boosts example-typed chunks. When the mode is “Ask,” narrative exposition ranks higher. This is still retrieval—just retrieval with intent.

Production Lab will embed with a model chosen for STEM + humanities textbooks (dense bi-encoders such as a fine-tuned E5 / BGE variant, or a proprietary academic embedder). Similarity is cosine over L2-normalized vectors. Hybrid retrieval—dense scores plus BM25 keyword over the same chunk store—covers rare symbols and proper nouns that dense models under-weight.

Re-ranking is the quiet hero. After top-k (say 24) candidates return, a cross-encoder re-ranks to 4–8 passages that actually enter the prompt. That keeps context windows honest and costs predictable—especially once CAG warms the static syllabus layer.

The generator is instructed to answer only from provided excerpts, to refuse when the book does not cover the question, and to emit structured citation markers the client turns into chips. Lab's session UI already expects referencedPageOrSection on assistant messages—so the wire format is ready before the GPU arrives.

Feedback buttons (thumbs, “still confused,” Easy/Hard/Again) do not change the model weights online. They write RecallEvent rows that feed SM-2 and, later, offline preference datasets for retrieval tuning.

Today, lib/lab/mockInference.ts streams book-title-aware placeholder prose with mode branching and artificial latency. That is deliberate: the session layout, mode selector, citation chips, and message persistence are production-shaped. Grep TODO(AI-INTEGRATION) to find every swap point. The RAG service should expose the same function signatures: generateChatResponse, generateStudyMaterial, gradeQuizAnswer.

Vectors and chunks are tenant-isolated per user ownership. We do not build a global textbook index that would recreate a pirate library. Ingest happens for titles the user owns or uploads under license. Rate limits and max-context caps protect both the student experience and the inference budget.