Azure AI Search: Why Reindexing Can Leave Stale Runbook Chunks

Short answer: sometimes an indexer replaces the old content; sometimes stale search documents remain. Azure AI Search's Markdown oneToMany mode can leave documents for sections removed from a file. With oneToOne, reindexing replaces the document for the same file, but changing the blob path or URI can create a new document beside the old one. Resetting and rerunning an indexer does not remove orphaned documents that no longer exist in the source. (Markdown indexing behavior, reset and run behavior)

For a support copilot that retrieves incident runbooks, a successful indexing run is not proof that an old instruction disappeared. Check the Markdown parsing mode, source identity, and delete policy, then query for a synthetic phrase that existed only in the retired section. This is an AI integration lifecycle problem: the source, index, and retrieval adapter must agree about which runbook revision is current.

Quick take: Keep the native indexer when each runbook is one search document, its source path stays stable, and the configured change detection sees edits. If section-level parsing is required, or a rename leaves an old path behind, add a tested cleanup step that deletes the retired document keys. A reset is a reprocessing control, not an orphan cleanup operation.

On narrow screens, scroll each table horizontally to see every decision and release check.

The table describes Azure's documented Markdown and indexer paths. A separate skillset, index projection, or application-owned chunker can add another document-generation layer, so inspect the actual IDs returned by the production retrieval adapter before choosing a cleanup design.

Why reindexing can leave an old instruction searchable

Azure AI Search's Markdown parser can turn a blob into one search document or one document per content section. In one-to-many mode, an edit that removes sections can leave old entries behind when the new file produces fewer sections than the old file. In one-to-one mode, the entire Markdown file is represented by one search document and the new file replaces its prior contents when the source identity remains the same. Microsoft also documents that changing the file path or blob URI can create a second document alongside the old one. (Index Markdown blobs and files)

Deletion is a separate signal from change detection. For Azure Storage blobs, the indexer can detect updates through LastModified; it does not automatically infer every deletion. Microsoft documents soft-delete policies that let an indexer remove a search document before the source file is physically removed. The policy must be established before the first indexer run, and the documented soft-delete policies do not apply to one-to-many indexing. (Change and delete detection for Azure Storage)

Resetting is useful when you need to reprocess indexed source documents, but it is not a substitute for deletion. A reset followed by a run overwrites documents that still exist in the source and adds new ones. It cannot identify and remove an index document whose source file has already disappeared. (Run or reset indexers)

When the native indexer is enough—and when custom cleanup is justified

Start with the existing Azure path. A native indexer is a reasonable fit when source identity stays stable, one search document per runbook is sufficient, edits update the source timestamp the indexer tracks, and a supported delete policy covers removals. In this case, test the configured schedule or run trigger and the index's retrieved content before building another synchronization service.

One-to-many Markdown parsing has a documented deletion gap when sections disappear. If chunk-sized retrieval is important, a narrow custom integration can keep a source-to-index-key ledger, compare the keys for the current runbook revision with the previously indexed keys, and issue delete actions for keys that are no longer present. Azure's delete API requires each document key; generated keys for chunked documents can be difficult to identify after the fact. Record those keys during ingestion or build a reliable lookup before relying on a cleanup job. (Delete documents in a search index)

Avoid custom work when the native one-to-one mode and supported source change/delete controls already satisfy the workflow. Add the smallest reconciliation step only for an observed mismatch—such as a removed section, changed path, unsupported delete policy, or generated key that the indexer cannot clean up on its own. For a broader production assessment, the Agent Reliability Audit reviews failure evidence and release gates; the production-ready agent checklist covers the surrounding operational criteria.

Release test: retire a synthetic runbook section

Use a test index and synthetic Markdown. Put a unique canary sentence such as LEGACY-QUORUM-CANARY-7318 in a section that will be removed; do not use a real incident, customer name, or credential. Before editing, record the blob path, the indexed document keys, and the current version marker. Query through the same retrieval adapter the agent will use and inspect the returned document IDs and text—not only the final answer.

An Azure indexer run can have Success status while individual documents have errors below the configured failure threshold, and warnings do not necessarily stop indexing. Inspect the execution details and warnings for the specific run, then verify the document by key or query. A delete can also take time to appear in index statistics, so use a document lookup and retrieval query as the release evidence rather than treating a count change as the only check. (Monitor indexer status and results, Indexer errors and warnings, verify document deletion)

Keep the release record small: parser mode, source path and revision, index keys before and after, indexer run outcome, and the canary query result. This checklist is an engineering recommendation for a runbook workflow, not an Azure-required schema or a guarantee that every downstream chunking step follows the same behavior.

The neighboring RAG tenant-isolation release test addresses which customer's content may be retrieved. This article addresses whether a superseded version of an authorized runbook is still present. Both checks matter, but they test different failure paths. Teams building the retrieval and agent workflow can use the production agent build hub after the platform's documented controls and cleanup gap are clear.

Claim boundaries

The platform-specific findings here are limited to Microsoft's current Azure AI Search documentation for Markdown parsing, blob change and delete detection, indexer reset behavior, document deletion, and status reporting, reviewed on October 15, 2026. A custom parser, skillset projection, separate chunker, or different source connector can behave differently. Recheck the documentation for the API version and data source you run, and test the actual retrieved documents. This is a release-test pattern, not a guarantee that an agent will always cite only the latest content.

Sources

• Microsoft Learn: Index Markdown blobs and files in Azure AI Search
• Microsoft Learn: Change and delete detection using indexers for Azure Storage
• Microsoft Learn: Run or reset indexers, skills, or documents
• Microsoft Learn: Delete documents in a search index
• Microsoft Learn: Monitor indexer status and results
• Microsoft Learn: Troubleshooting common indexer errors and warnings