Microsoft Explains Azure Content Understanding Agentic Mode: When Document Extraction Requires Multi-Step Tool Loops
Joe Filcik, Principal Product Manager at Microsoft, explains the newly launched agentic mode in Azure Content Understanding: when answers are scattered across multi-page tables, require exclusion rules, involve calculations, or need visual evidence, a single-pass extraction is insufficient. The agentic mode wraps the language model in a reasoning loop equipped with tools for document navigation, visual inspection, calculation, and validation, at the cost of more tokens and longer latency.
Most document extraction tasks can be completed in a single pass: read the content, locate the corresponding values according to the schema, and return structured results. From the outset, the Microsoft Azure Content Understanding team designed this as the default path. However, in a blog post on November 9, Principal Product Manager Joe Filcik pointed out that there is a category of tasks that cannot be solved with a single inference—answers are scattered across different pages, require exclusion before normalization, involve cross-row calculations, or need a different perspective to re-examine after the initial read.
He used data from the U.S. Department of the Treasury to illustrate this boundary. The schema requires extracting the total agency outlays for January 2003, but the relevant data is distributed in tables spanning two pages within a 137-page bulletin. Not every number in the row for that month should be included—receipt items and reported totals need to be excluded, revision marks need to be removed, and negative signs need to be retained. The final calculator yields $180,681 million. Filcik emphasized that a single-pass model must simultaneously handle page relationships, exclusion rules, symbol annotations, and arithmetic; this is precisely where the agentic mode intervenes.
The mechanism of agentic mode is not simply increasing the number of conversation turns. Filcik described it as wrapping the language model in a document-native harness: a reasoning loop equipped with tools for navigating document text and tables, inspecting visual evidence, performing calculations, and validating results. The model first plans the steps, then calls the corresponding tools, and returns to the source document to re-check if necessary. The output is still defined by the schema, and the returned result remains structured fields.