In November 2025, the U.S. House Committee on Oversight and Accountability released documents related to the Epstein case. The JMail team—built by Riley Walls and Luke Igel within five hours on the night of the release—created an interface allowing users to browse these files like an email inbox, later expanding into applications such as JFlights, JDrive, and JPhotos. Reducto was responsible for parsing over three million pages of documents involved, including redacted PDFs, scans, complex tables, and handwritten records from numerous individuals.
Omar Alhait, a founding engineer at Reducto, described an unexpected issue encountered while processing flight logs at the 2026 AI Engineer World Expo: some PDFs contained black boxes drawn over text, but because Reducto parses PDF metadata rather than just rendered images, the underlying text data remained fully readable despite being visually obscured. Alhait stated that you cannot simply place a box on a PDF and expect it to be truly redacted. He further explained that while some files were correctly redacted, others were not, depending on whether the original document creators understood the layer structure of PDFs.
For the JMail team, this meant they had to implement an additional layer of redaction to ensure information intended to be hidden remained invisible in the final product. Alhait summarized this dilemma with the phrase: "Our tool was working a little too well."
The stronger the extraction capability, the easier it is to violate the document's intended disclosure limits.
This failure mode is easily masked in demonstrations because small samples often do not reveal problems. Alhait recalled that on the night the first batch of files was released, he pushed throughput hard to complete parsing quickly, repeatedly triggering infrastructure alerts. On-call engineer Alvin messaged him late at night saying, "Hey bro, can you slow down?" Small-scale Studio tests can verify if extraction results for a single document are reasonable, but they cannot expose the true performance of a three-million-page corpus under concurrency, diverse formats, and consistency of redaction.
Alhait outlined a two-step evaluation path: first, perform intuitive checks (which he calls "vibe evals") on a small number of documents in Studio to confirm outputs look reasonable; then expand to one hundred or one thousand relevant documents for systematic evaluation using appropriate configurations. He emphasized that configuration itself affects evaluation results—customers might test a vendor using unsuitable API settings, yielding numbers that do not represent the vendor's performance under correct configurations. Therefore, meaningful comparisons must include the actual processing settings used, not just the vendor name.
Regarding the relationship between parsing and extraction, Alhait explained that Reducto’s pipeline first parses (restoring document structure) and then extracts (requesting specific fields according to a schema). The parse output becomes the input for extraction, enabling intermediate operations: for example, with a math textbook, one can first parse the content, split it by chapter boundaries, and then execute extraction on each chapter separately to generate chapter summaries. The extraction schema is like a paintbrush on the canvas—it determines what the system returns, but correctness still depends on the ability to actually extract the requested information from the document.