The Index You Forgot to Rebuild
Your RAG pipeline retrieved the right answer six months ago. The source doc changed. Nobody re-indexed it.
The document got updated in February. The policy changed, the pricing changed, the procedure changed. Someone edited the source, pushed it to Confluence, closed the ticket. Done.
Except your vector index still has the February version. And it’s May.
This is the most underestimated failure mode in RAG production.
Not retrieval quality. Not chunking strategy. Not embedding model choice. Those all get discussed at design time, benchmarked in evals, iterated on.
Stale index content gets discovered by users. Quietly. When the answer they got from your AI assistant contradicts the email they just received from legal, or the pricing page they’re looking at right now.
The gap between “source of truth” and “what’s in the index” is your liability. Most teams don’t measure it.
How it happens.
You build the pipeline. You ingest the documents. Everything looks great. You ship.
The index is a snapshot. But documents are not static — they’re edited, archived, replaced, restructured. New pages get added. Old ones get deprecated but not deleted. The index doesn’t know any of this unless you tell it.
If your ingestion pipeline runs once at launch, or runs on a cron that nobody monitors, or re-indexes the full corpus weekly but doesn’t handle deletions — the index drifts. Slowly at first. Then suddenly you’re retrieving confidently from documentation that’s been wrong for three months.
The problems that compound it:
No deletion handling. A document gets removed from the source. The chunks stay in the index forever. The retriever still surfaces them. They still look relevant. The content is just… no longer true.
No change detection. Ingestion pipelines that don’t diff against the current index re-ingest everything or skip everything, depending on how they’re written. Either you’re wasting compute or you’re missing updates.
No freshness metadata. Vectors in the index with no last_updated timestamp, no source URL, no version marker. You can’t audit what’s stale because you don’t track when anything was indexed.
Eval sets built at launch. You benchmarked retrieval quality on documents as they existed when you shipped. Your evals don’t know the docs changed. They still pass. The answers are still wrong.
What solid RAG index management actually looks like.
Every chunk carries metadata: source URL, ingestion timestamp, document hash. When you re-ingest, you diff against stored hashes — only updated or new documents get re-embedded. Deleted documents trigger chunk removal, not chunk orphaning.
Change detection runs continuously or on a tight schedule, not weekly. Not “whenever we remember.” Your pipeline watches the source, not a calendar.
You have a freshness dashboard. You know the oldest chunk in your index, the stalest source, the documents that haven’t been touched in 90 days. That’s not a reporting exercise — it’s ops hygiene.
And your evals get updated when docs change. If a document revision would break an existing eval, that’s a signal, not a maintenance chore.
The retriever will always find something. The question is whether that something is still true.
Treat your index like a cache with a TTL. Evict aggressively. Validate constantly. The pipeline isn’t done when ingestion ships — it’s done when staleness is impossible to ignore.
Right now, something in your index is wrong. The only variable is whether you’ll find it before your users do.