Graph RAG: engineering answers you can trust

Ask Google or a bare language model about leading change and you get the crowd's answer: popular, recent, easy to find. I wanted a system that answers from the proven and the deep instead. So I built Ask-A2A, a question-answering engine over my handbook Ambition to Action and the research library around it. This report dissects what it took. The pattern is called graph RAG, but the lesson is broader than the acronym: a good answer is engineered along the whole path, and no single model can hand it to you.

The corpus behind the system holds the enduring research on transformative change, the field-tested concepts, and thirty years of my own led and observed experience. The ambition was a specific kind of outcome: an answer a practitioner could act on, grounded in sources they could inspect, delivered in a form that feels considered rather than generated. Sophistication in working with AI is exactly this. It is not a clever prompt. It is a chain of deliberate choices, each one measured.

The input

The first version worked the way most RAG systems are built. A script cut the source material into 1,521 chunks of equal size and embedded them. Retrieval was technically fine. Answers were mediocre. The reason took a while to see: the chunks did not correspond to ideas. A concept would be sliced across three chunks, or fused with its neighbor into one. The retriever was returning shreds of meaning, and the answer model was left to reassemble thought from confetti.

So I rebuilt the corpus as an authored artifact. One idea, one card, one chunk. Each concept became a single card with a defined shape: a summary, the essence of the idea, its aliases, and its links to researchers and book passages. Cards that sprawled across several ideas were split. Duplicates were merged. Twenty-seven missing concepts were written. The chunker stopped cutting text and started reading units the corpus already declared.

Each card gained two faces. The embedding text carries the title, aliases, and searchable handles: it is written to be found. The answer text is clean prose: it is written to be read. The searcher sees one face and the reader sees the other, and conflating the two is one of the quietest, commonest mistakes in RAG.

1,521 machine-cut chunks became 475

authored passages: one idea per chunk, each with a searchable face and a readable face.

Graph neighbors per concept, 3.1 before 5.8

after the rebuild, with zero concepts left isolated. Before, 26 concepts had no connections at all.

Book sections unreachable by the graph 8 of 64

after tagging, down from nearly all of them. The manuscript joined the graph instead of floating beside it.

The two engines

A RAG system runs on two kinds of intelligence, and they should not be asked to do each other's jobs. I had learned this once before, building Antilibrary: a frontier model asked to group thousands of claims drew the groups differently every run, and an embedding model gave the search a fixed frame. Ask-A2A applies the same division of labor deliberately.

The embedding engine Geometry

Turns every passage, once, into a fixed position in meaning-space. Finding candidates is then arithmetic: stable, instant, the same every run.

It knows what is near. It does not know what is right.

The language model engine Judgment

Reads the question against the candidates and reorders them by mechanism rather than shared vocabulary. Then a larger model composes the answer.

It knows what is right. It is too slow and too variable to search with.

Keyword search runs beside the vector search and the two lists are fused by rank. A knowledge graph of 1,940 nodes sits underneath, not to make search smarter, but to carry provenance: how each idea connects to its evidence, its author, and its place in the book.

The hinge between the engines is the reranker. Fused retrieval puts the right passages in the room, but nothing has yet read the question against them. A small language model does exactly that, scoring each candidate on whether it addresses the mechanism the question is about, whether a practitioner could act on it, and whether it is a primary source. A passage that repeats the question's words without explaining anything ranks low. A passage that never uses those words but names the mechanism precisely ranks high. That judgment is what vocabulary matching, however well fused, cannot make.

The proof

Every stage had to justify itself against an evaluation set of 39 questions with known expected sources. The reranker is the clearest example, in both directions.

First, the direction you would hope for. Measured over three trials, the reranker raised mean reciprocal rank from 0.707 to 0.875, and the right source landed first on 82 percent of queries, up from 51. The model is stochastic and rejects a temperature of zero, so one run is a sample, not a result. Three trials put the run-to-run spread under one point, which made the lift trustworthy.

Then the direction nobody warned me about. When I held a stopwatch to the pipeline, search took 0.4 seconds. The reranker took 53. It is a reasoning model, and it was thinking hard about a judgment that does not need depth. The model exposes a dial for reasoning effort, and because the evaluation harness already existed, each setting took minutes to score.

Reasoning effort	Seconds per call	Mean reciprocal rank	Right source first
No reranker	0	0.707	51%
Minimal	~8	0.813	72%
Low (chosen)	~14	0.842	77%
Default	~53	0.875	82%

Quality degrades smoothly as reasoning is dialed down, so the trade can be chosen instead of suffered. Low effort keeps most of the lift at a quarter of the wait. With the retrievals also parallelized, the first words of an answer arrive in about 15 seconds instead of 80.

The measurement discipline paid a second dividend. The reranker's few consistent regressions turned out to be defects in the evaluation questions themselves, not in the ranking. And a proposed rule that capped how many book passages retrieval could return failed its sweep: the ordering barely moved across every value tested, so the cap came out. An evaluation that can fail, and that you then investigate, teaches you twice.

The industry has started naming these disciplines. Instructing a model is prompt engineering. Wrapping the work in a test that must pass, and rerunning until it does, is loop engineering. Wiring the stages into a system with routes, gates, and fallbacks is graph engineering. This build practices all three. The answer contract is a prompt. The evaluation harness is a loop: 39 queries, scored, fixed, and rerun until they pass. The pipeline itself is a small, fixed process graph: the reranker fails open to the fused order, a timed-out model routes to a fallback answer, and the privacy gate sits at the exit. The knowledge graph holds the ideas. This second, quieter graph runs the operation.

The experience

Reliability the user cannot see might as well not exist. The last layer of the system is the experience of the answer, and it was engineered with the same care as the retrieval underneath it.

The answer has a deliberate shape. Up to three passages from the handbook frame it; up to seven research cards support it. That rule lives in assembly, not retrieval. Retrieval answers one question: what is relevant? Assembly answers a different one: what should this answer look like? For weeks a quota hid inside the retrieval stage, and both questions were answered badly. Splitting them made each rule simple, explicit, and testable.

The answer arrives as it is written. Streaming turned a 30-second silence into words flowing at reading speed, and it also exposed every remaining second of upstream latency to the stopwatch. Beside the answer sits the source trail: the exact passages used, labeled by role, so a skeptical reader can check the answer against its evidence. Below it, the graph trail shows the researchers and neighboring ideas the answer drew near.

The page itself carries the same argument. It follows a design law I maintain across projects: a grayscale palette, hairline rules, serif for reading and sans for structure, one reserved accent, no decoration. A reader should feel the same austerity in the form that the method practices underneath. Content and form make one claim: this answer was considered.

The learnings

1 · Quality is decided before retrieval. The largest gain in the project came from rewriting the corpus, and it required no new model and no new algorithm. Knowledge authored as single retrievable ideas beat clever search over machine-cut text. If your answers disappoint, read your chunks before you tune your retriever.

2 · Give each engine the job it is built for. The embedding engine supplies stable geometry; the language model supplies judgment; the graph supplies provenance. Most RAG failures I now recognize as one engine doing another's job: a language model asked to be a search index, or vector similarity asked to make a relevance judgment.

3 · Every stage earns its place by measurement. The reranker earned its place with a 24 percent lift measured over trials. The book-passage cap failed its sweep and was removed. Nothing survives on plausibility. The discipline is cheap: the whole harness is a few hundred lines and a few cents per run.

4 · Hold the stopwatch next to the scoreboard. I measured the reranker's quality carefully and its wall-clock not at all, and it quietly cost 53 seconds per call. Measure quality and latency together, because the stage that helps the most can also be the one that costs the most.

Ask-A2A runs in private beta. Behind it sits a written design: a 21-step method covering build time, question time, and the discipline around both. The appendix below is that design document, annotated with this build's numbers and telemetry, so every step carries its evidence.

Appendix: the method, with this build's numbers

Build time
1	Source collection	136 concept cards · 146 researcher cards · 27 practitioner cards · 38 research dossiers · 18 evidence packs · 7 chapter syntheses · 64 book sections.
2	Authoring atomic units	One idea per card. 27 missing concepts written; sprawling cards split; duplicates merged; 87 cards enriched with structured frontmatter.
3	Chunking	1,521 machine cuts became 475 authored passages. Each carries two faces: text written to be found, text written to be read.
4	Metadata tagging	Aliases, anchors, source types, and authority roles in every chunk; the manuscript tagged against the concept vocabulary.
5	Embedding	text-embedding-3-large at 3,072 dimensions. Re-embedding the entire corpus costs $0.053.
6	Index building	The lexical index is built in the same run as the chunks, so index and corpus cannot drift apart.
7	Graph construction	1,940 nodes. Concept cards average 5.8 neighbors, up from 3.1; zero concepts isolated, down from 26.
8	Graph validation	8 of 64 book sections unreached by any edge, down from nearly all of them before tagging.
Question time
9	Query understanding	The question is routed through the method's lenses, and an anchor query runs beside it.
10	Lexical retrieval	Keyword postings over all 475 passages.
11	Dense retrieval	Cosine over the 3,072-dimension vectors. About 0.4 seconds, query embedding included.
12	Fusion	Reciprocal rank fusion, k = 60, merging the lexical and dense lists by position.
13	Filtering and diversity	Per-source cap of 2, kept after a sweep: uncapped, one source took 6 of 8 slots with zero quality gain.
14	Reranking	Listwise over 24 candidates. MRR 0.707 → 0.842 at low reasoning effort (~14 s); 0.875 at default (~53 s). About a third of a cent per call.
15	Graph expansion	Retrieved passages pull in their connected researchers and neighboring ideas; up to 16 additions shown per question.
16	Context assembly	Up to 3 book passages frame the answer; up to 7 research cards support it. The rule lives here, not in retrieval.
17	Prompt construction	An answer contract with a licensed refusal: if the handbook does not fit the question, say so rather than force the frame.
18	Generation	gpt-5.1, streamed as written. First words in about 15 seconds; about 3 cents per question, all stages included.
19	Post-processing	Source trail with roles, graph trail, and follow-on questions extracted from the answer.
Around both
20	Evaluation	39 queries with expected sources, three trials, run-to-run spread under one point. Its consistent regressions exposed two defective eval questions.
21	Privacy gating	Every payload stamped private; nothing is public-ready until it is deliberately released.

Numbers from the build of 2026-08-06 and 07: chunk and graph counts from the build reports, retrieval and reranker figures from the 39-query evaluation harness, latency from stopwatch runs against the live pipeline, costs from per-call token telemetry.