Retrieval Is Not Verification
In this essay
A financial research assistant returns a polished answer with three citations. Clicking them reveals a related annual report, a quarterly presentation, and a transcript. The links are real. The conclusion can still be wrong.
A citation may lead to the right company but the wrong period, or to a management target instead of a reported result. Retrieval solves part of the access problem. Verification is the additional work of checking whether a specific statement follows from a specific source.
What RAG actually contributes
The original retrieval-augmented generation paper combines a language model with retrieved external passages and evaluates the approach on knowledge-intensive tasks. Its results support retrieval as a useful mechanism; they do not certify the accuracy of an arbitrary financial chatbot.
In a research workflow, retrieval can provide fresher documents, a smaller relevant context, and a path back to the material used. It cannot repair a source that is itself misleading, or force a model to preserve every qualifier.
A good system therefore stores both the answer and the evidence relationship. “Revenue increased” is a claim. “The consolidated revenue table for fiscal year X compared with fiscal year Y” is evidence with a scope. The connection between them must survive inspection.
Tables need their surroundings
Suppose a filing contains the number 1,250. Detached from its heading, that number has very little meaning. It might represent millions of dollars, thousands of shares, a year-end balance, or an average over a reporting period.
Chunking a document into equal blocks of text can separate a table from its unit, date, column headings, or footnotes. A retriever can then return the correct row with insufficient context to interpret it.
For table-heavy documents, retain the document identifier, reporting period, page or section, table title, column labels, unit, and relevant footnote together. Preserve the original page as a reference when extracting a machine-readable table. A successful text extraction is not evidence that the layout was interpreted correctly.
This is an engineering design recommendation: it is meant to prevent a common class of ambiguity, not to prescribe one universal chunk size.
Test the claim, not the presence of a citation
Consider an invented source sentence: “Management expects adjusted operating margin to reach 20% next year.” An answer saying “The company reported a 20% operating margin” changes the time, the status of the statement, and potentially the accounting definition.
An evidence check should ask whether the source supports each of those dimensions. At minimum, track the entity, metric, period, unit, accounting basis, and whether the statement is reported, forecast, or inferred.
The model can draft this mapping, but material conclusions deserve independent review. A second model agreeing with the first is useful feedback, not a substitute for reading the cited table or passage. Both may share the same mistaken interpretation.
Evaluate retrieval and answers separately
If the necessary evidence never enters the context, improving the final prompt may not solve the problem. If the evidence is present but misread, a better embedding model may also miss the actual bottleneck.
Build a modest test set of representative questions with expected documents and manually checked answers. Include cases with conflicting versions, ambiguous periods, missing disclosures, and no valid answer. Useful measures include retrieval recall for the required evidence, support for individual claims, numerical correctness, and the handling of unanswerable questions.
The Lost in the Middle study found position-sensitive performance in the models and tasks it tested. That is a reason to test the placement of decisive evidence in your own workflow, rather than assuming a large context window guarantees dependable use of every page. It is not a universal performance estimate for all later models.
Make abstention a normal output
A useful answer might say: “The supplied filing reports total revenue but does not isolate the AI-related portion.” That leaves a research gap visible. Replacing the gap with an invented estimate makes the document look complete while making the decision worse.
Define when the assistant should abstain: missing units, an unresolved conflict between filings, absent segment disclosure, or inadequate evidence for a causal explanation. Give the reader a next step, such as checking a specific note or obtaining a later filing.
A research interface can show three simple fields: what the source states, what the system calculates, and what remains uncertain. Keeping those fields separate reduces the chance that a plausible inference will be mistaken for a disclosure.
A useful acceptance test
Take one important paragraph in the answer and try to reproduce it from the cited material without the model. If you cannot locate the input, identify its scope, and repeat the calculation or inference, the paragraph is not yet ready to carry weight in a research note.
The goal is not a chatbot with more decorative footnotes. It is an evidence trail short enough that a reader will actually follow it.