5 min
Zach Chmael
In This Article
A 1,280-condition RAG study found heading-based retrieval improved answer quality in eight structured documents. Use its limits before changing content.
Updated
TL;DR
📚 The preprint reports 1,280 conditions across four experiments, using eight documents and 40 unique queries.
🧭 Heading retrieval produced a reported Cohen's d of +0.41 over the chunk-based baseline in Experiment 1.
📄 The heading channel added an average of 2.9 pages per query when enabled.
🔗 Those pages supplied 20.4% of answer citations in the study's ToC-on conditions.
⚠️ The length trend was not significant across eight documents, with p = .52.
Do Headings Help AI Find Better Answers In Long Documents?
Yes, descriptive headings can help an AI retrieval system recover evidence that ordinary chunks miss. One June 2026 arXiv preprint tested a heading index across 1,280 experimental conditions in a production enterprise RAG system. Heading-based retrieval raised the study's composite answer score from 3.72 to 4.04 on a five-point scale.
That result belongs to eight structured documents, one retrieval pipeline, 40 unique designed queries, and model-judged answers. It does not prove headings improve public search rankings or AI citations.
The useful mechanism is narrower: a heading can give a retriever a second address for a section when local chunks do not carry its full meaning.
For a content team, that is enough to justify a structural audit. It is not enough to justify a ranking claim.
What did the heading-retrieval study actually test?
The study tested whether a second index built from document headings could improve answers over a RAG pipeline that already used chunks, keyword matching, embeddings, and reranking.
The author evaluated eight enterprise documents covering HR, procurement, ERP, IT, and travel policy. They ranged from 5 to 195 pages. Three were public documents; five were internal institutional guides.
Every document had detectable bookmarks or visual heading formatting.
The baseline split documents into 512-token chunks with 64-token overlap. It used 768-dimensional nomic-embed-text vectors, cosine similarity, BM25, pgvector, and cross-encoder reranking. That is already a more capable baseline than plain vector search.
The added channel extracted headings without an LLM. At query time, it matched the query against heading embeddings, selected up to three headings above a 0.3 similarity threshold, and loaded up to five full pages beside the ordinary chunks.
Experiments 1 and 2 each used 24 queries across four query types and two answer models. Each produced 384 conditions. A length test added 32 conditions, and a 60-configuration parameter sweep added 480.
I downloaded the 28-page PDF and checked the appendix because the condition count is easy to overread. There were 1,280 runs across configurations, not 1,280 independent buyer questions.
Why can headings recover evidence that chunks miss?
Headings can recover missed evidence because they describe the job of a section at a broader level than any one paragraph inside it.
A chunk may contain a limit without the definition that scopes it. Another may contain an exception without naming the policy it modifies. Their embedding vectors reflect local language. A heading such as "Data breach notification requirements" can point to the whole section even when the buyer asks about the deadline, required recipients, and disclosure contents in one question.
The paper calls out two failures. A cousin-section gap appears when evidence sits under several neighboring branches, such as annual leave, carry-forward, and home leave. A coarse-heading gap appears when one long section is split into pieces that are semantically distant even though they belong to the same topic.
The heading index creates a parallel route. Chunks still provide precise passage matches. Headings retrieve a structurally coherent section, including nearby definitions, conditions, and exceptions.
That architecture matters more than the phrase "use headings." The study did not count H2 tags on web pages. It embedded inferred PDF headings, matched them separately, and loaded full pages. A marketing team cannot inherit the result by changing typography alone.
How much did heading retrieval improve the answers?
Heading retrieval improved the paper's model-judged answer scores, but the effect was small and tied to one experimental system.
In Experiment 1, the baseline composite score was 3.72 out of 5. The heading-only configuration reached 4.04. Answer-side completeness checking reached 4.10, while headings plus completeness checking reached the highest displayed mean at 4.22.
Source: Three Sides of Retrieval.
The ANOVA treated heading retrieval as a significant main effect with F = 4.69, p = .031, and partial eta squared = .012. The last number is the brake: the heading feature explained a small share of score variance. Query and document difficulty did much more of the work.
The planned heading-versus-baseline comparison reports d = +0.41 with a 95% bootstrap interval from +0.00 to +0.87. The paper gives that pairwise test an adjusted p-value below .001. The ANOVA and pairwise test ask related but different questions, so the p-values need not match. The paper does not explain the gap in much depth.
At the score-component level, enabling the heading channel added 0.20 completeness points and 0.11 accuracy points. That is encouraging. It is still a five-point judgment from Claude Opus 4.6 against system-generated, human-checked references, not a human buyer outcome.
Did the heading channel supply evidence the model used?
Yes. In this experiment, the heading-retrieved pages contributed citations inside generated answers rather than sitting unused in the context window.
Across 192 ToC-on conditions, the channel fired on 92.7% of queries and added 2.9 pages on average. It supplied 250 citations, or 20.4% of the 1,226 citations reported for those configurations. Ordinary chunks supplied the remaining 976.
Source: Three Sides of Retrieval.
The contribution varied by query type. The chart places the heading share around 17% for cousin-gap and multi-section questions, about 19% for simple questions, and roughly 27% for coarse-gap questions. Those values describe this generated-answer system. They are not public-web citation rates.
There is a useful content principle inside the result. Local relevance and structural context do different jobs. A precise chunk can answer one clause. The surrounding section can preserve the condition that keeps the clause honest.
I checked the full chart before cropping it because "20% of citations" is easy to turn into a fake GEO benchmark. The denominator is citations generated in a private RAG experiment when the heading feature was already on. It says nothing about what percentage of ChatGPT or Google citations come from headings.
Where does the evidence stop?
The evidence stops well before public AI search, websites, rankings, or buying behavior.
This is a version-one arXiv preprint, not a peer-reviewed paper. It tests one internal RAG system and eight documents. Five documents are private, and the paper links no raw run-level data, evaluation code, or independent replication. The 40 queries were designed to include cases where structure might help.
The evaluator is also part of the instrument. Reference answers came from the all-features system and were manually checked. A separate model scored accuracy, completeness, and usefulness. Human review improves the setup, but model judging can still reward the style or coverage patterns produced by a related pipeline.
The document-length result shows why the caveats matter. Heading retrieval improved six of eight documents and produced its largest gain, 1.50 points, on a 118-page UN staff document. Yet the regression across eight documents was not significant: r = .27 and p = .52. One 48-page guide fell by 0.33 points.
The parameter sweep also used only eight queries and one answer model. None of its three parameter main effects reached significance, with all p-values above .38. That supports stability inside the test grid. It does not establish universal defaults.
The safe conclusion is simple. Heading-aware retrieval deserves testing on structured documents. The paper does not support "add more headings to rank in AI search."
What should a content team change after reading this?
A content team should make section structure inspectable, then test whether the intended evidence survives retrieval and synthesis.
Start with the buyer condition each section resolves. "Security" is a topic label. "Which data regions are available on the enterprise plan?" is a retrieval address with a job.
The section below it should keep the answer, scope, source, exclusions, and update owner close enough that a fragment does not invent certainty.
Use this audit:
Structural check | Weak pattern | Better content contract |
|---|---|---|
Section job | Broad label such as "Benefits" | Buyer condition or decision the section resolves |
Evidence boundary | Proof scattered across unrelated sections | Claim, source, scope, and exception stay connected |
Heading fidelity | Clever line that hides the subject | Plain heading that names the actual topic |
Ownership | Long page with no review boundary | Section owner, effective date, and update trigger |
Retrieval test | Assume formatting worked | Run the buyer question and inspect answer plus sources |
Do not turn every paragraph into a heading. The paper's own default selected three headings and loaded up to five pages. Structure has a cost when it becomes noisy, repetitive, or detached from real evidence.
The production choice may be a clearer section, a documentation repair, a consolidated comparison, or no new asset. If the proof does not exist, a better heading merely makes the absence easier to find.
How can Trovance connect document structure to an evidence gap?
Trovance can connect an observed AI answer to the exact claim, source, and section that failed to survive retrieval or synthesis.
A team can use Trovance to observe how AI systems explain, cite, compare, and recommend the company for a defined buyer question. That answer trail shows whether the system found the right page, attached the wrong source, omitted a decision condition, or repeated a claim without its limit.
A visibility score alone cannot tell whether the problem is the heading, the buried proof, a stale product fact, inaccessible documentation, or legitimate bad fit. Trovance helps diagnose that evidence gap before production begins.
When a public asset is justified, Trovance helps the team produce and publish the proof-backed section, page, comparison, FAQ, or source-backed article that should exist. A person keeps the final claim and publication decision. A later rerun can check whether the explanation changed without promising a citation, recommendation, or sale. Use Trovance to connect an observed answer to the evidence your team should repair.
FAQs
Do headings improve AI search rankings?
This study does not answer that question. It tested a private enterprise RAG pipeline over eight structured documents, not Google, ChatGPT, Perplexity, Gemini, websites, or search rankings. It supports testing headings as a second retrieval address inside similar document systems. It does not support a public ranking claim or a guaranteed citation outcome.
What is heading-based retrieval?
Heading-based retrieval creates an index from section titles or inferred document headings, then matches a query against that index. In this paper, matched headings caused full pages to load beside ordinary retrieved chunks. The method preserved broader section context without replacing passage retrieval, and it required no LLM calls for heading extraction.
Why can chunking lose document structure?
Fixed-size chunks divide text by token count rather than by the document's argument. A definition may land in one chunk while its exception lands in another. Headings and section boundaries preserve a wider topical relationship. That does not make chunks useless; it explains why a second structural retrieval path can sometimes recover missing context.
Does this mean every page needs more H2 headings?
No. The experiment did not test HTML heading count, and extra headings can create noise when they do not map to real sections. Use a heading when it names a distinct buyer condition or evidence job. Keep related proof and limits together, then test the rendered page or document against the questions it must answer.
Are the study's 1,280 conditions independent queries?
No. The paper reports 1,280 conditions across feature configurations, models, queries, and parameter settings. Its appendix lists 40 unique queries across four experiments.
Calling every condition an independent question would inflate the sample. The repeated configurations are useful for comparison, but they do not provide the breadth of 1,280 different information needs.
When should a team consolidate evidence into one section?
Consolidate when several facts answer one decision condition and their separation causes incomplete or misleading retrieval. Keep them separate when different owners, update cycles, legal boundaries, or product surfaces require distinct sources of truth. The goal is a clear evidence trail, not a single giant page. Test the answer before choosing either structure.
How should teams test whether structure helped?
Define the buyer question, model, date, and repeated-run policy before changing the asset. Preserve each raw answer and source. After the update, rerun the same protocol and report mentions, citations, conditions, and recommendations separately. A changed answer is an observation, not proof that the heading caused a business outcome.
Related Resources
Connect a missing answer condition to the evidence action with Trovance
