Cache-Augmented Generation in EtherBranch Lab

Cache-Augmented Generation (CAG) is the practice of preloading and reusing stable context across turns—often via KV-cache reuse on the model side, or via application-level caches of structured knowledge—so generation starts warmer and cheaper. It complements RAG rather than replacing it. RAG answers “which pages matter for this question?” CAG answers “what durable context should already be in working memory?”

In Lab, durable context includes: book metadata, the learning-objective tree, the student's live mastery vector, the current session's touched objectives, and recent assistant/user turns. Those change slowly relative to a single user message. Caching them avoids re-tokenizing the same syllabus on every keystroke.

1. Curriculum cache. ensureCurriculum(bookId) materializes chapters, sections, objectives, and milestones once per book. Subsequent sessions read the tree rather than regenerating it. Bloom tags travel with objectives so the tutor can bias difficulty without another model call.

2. Session context cache. The right-hand “Session Context” panel is not decoration—it is the human-visible face of CAG. Objectives touched in this conversation stay pinned; mastery bars update from recall events; “Generate Study Material From This Session” snapshots the warm context into a GeneratedMaterial row.

3. Model KV / prompt prefix cache (production). When the real inference service arrives, the static system prompt + book primer + objective digest should be a reusable prefix. Only the retrieved RAG excerpts and the latest user turn change. That is classic CAG at the transformer layer: pay once to fill the cache, then stream cheaply.

A textbook is a closed world with a stable table of contents. Unlike open-domain chat, Lab's “system of record” for a book barely changes after ingest. Caching the syllabus and the student's progress profile is therefore high-ROI: you save tokens, reduce latency, and keep the tutor persona consistent across a two-hour study block.

CAG also makes offline-friendly study materials possible. Flashcard decks and summaries generated mid-session are stored as JSON artifacts. Reviewing them later does not require a fresh RAG pass unless the student asks a new free-form question.

Caches are only safe with clear invalidation. Lab invalidates curriculum cache when a book is re-ingested; mastery digest when a RecallEvent writes; session prefix when the user switches books or starts a New Session. Ownership checks remain authoritative: if BookOwned disappears, Lab redirects to the picker—no stale cache can resurrect access.

Production CAG should log cache hit rates, prefix token counts, and time-to-first-token. Those metrics tell us whether to widen the cached primer or lean harder on RAG. The UI already surfaces the human metrics that matter: mastery rings, needs-review lists, and study streaks.