ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

Snowflake: Two Organizations with the Same Number of Prompts May Have AI Do Completely Different Things

Three authors from Snowflake propose a four-dimensional framework that separates AI adoption rates from AI fluency, using internal practices from their own data science and marketing teams to illustrate that only by tracking the level of work actually undertaken by AI can one determine if it is truly changing output.

2026-10-24 01:07 GMT+8

A company logs one thousand AI interactions per week, and another also logs one thousand. In the first, most interactions involve asking AI to explain a piece of code or look up a concept; in the second, people are using AI to build data pipelines, investigate anomalies, and run multi-step workflows. Most enterprise AI dashboards would score these two companies identically.

Siddharth Dwivedi, Product Marketing Manager at Snowflake, data scientist Zachary Blackwood, and product manager Swati Humbad proposed a four-dimensional framework in a blog post on October 23 to answer a key question: Is AI truly driving meaningful work output?

The first step of the framework is to categorize the level of work AI is actually undertaking: the foundational layer involves explaining code and answering questions; the application development layer involves writing code, troubleshooting, and building reports; the agent layer involves multi-step processes where AI reasons across contexts, calls tools, and completes an outcome rather than just answering a single question. The authors point out that if AI is truly creating capacity, the time saved should be reflected in the agent layer.

The second step is to separate adoption rate from fluency. Adoption measures how broadly AI has spread among the target population, while fluency measures how complex the work people delegate to AI has become. These two dimensions cross to create four positions: high adoption/low fluency, low adoption/high fluency, both high, and both low. The third step uses industry benchmarks to provide a reference frame—software companies and regulated insurance companies cannot share the same baseline. The fourth step looks at the gap between advanced users and everyone else, treating the effective workflows of a few as a roadmap.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The authors offer four specific methods for diffusing individual experience to the team: have those who have achieved results lead practical workshops rather than just doing demos, with the validation criterion being whether weekly usage holds steady a month later; write successful workflows into recipes and templates that colleagues can run directly, with the validation criterion being that the next person executes it without needing to ask the author questions; hold regular sharing sessions that include failure cases, because failures tell others more about which problems are worth attempting than successes do; make the workflow library searchable, otherwise discovery cost replaces creation cost as the bottleneck.

The framework was applied to their own two teams. The data science team runs thousands of data pipelines. In Q1 of FY27, there were 960 pipeline incidents requiring attention, each taking a skilled engineer approximately 30 minutes to diagnose, amounting to roughly 160 hours of expert time per month. The team had previously accumulated a diagnostic manual refined through hundreds of incidents, sufficient for new hires to solve problems step-by-step. They handed this manual to CoCo, triggering it via alerts rather than human keystrokes. With the same steps and standards, the runtime shifted from team scheduling to incident rhythm. Over these two quarters, CoCo improved over 500 pipelines, generated over 100 merged pull requests (PRs), and compressed the time from incident to reviewable fix to under 16 minutes.

The marketing team provided evidence in another direction. Snowflake's global marketing organization has nearly 600 people. An internal survey in February 2025 showed that only 11.6% of North American marketing staff felt very confident in their AI usage capabilities. By May 2026, weekly CoCo usage across the global marketing team reached 70%, and 93% self-reported daily use of AI tools. The authors specifically noted that the former is observed behavior while the latter is self-reported, so they prioritize citing the former. What marketers built included: a motion graphics export process that compressed a full week of Summit preparation work into 20 minutes; a win/loss reporting app that automated approximately 90% of the monthly three-hour candidate identification plus one week of quarterly reporting material work; and an inbound automation engine that handles 2,000 to 3,000 global marketing email replies daily without human intervention, replacing software originally planned for purchase.

The authors acknowledge boundaries: all numbers come from Snowflake's internal usage records of its own product, CoCo; the benchmark test data-eng-bench is a vendor result built, tested, and self-selected by Snowflake AI Research; the ratio of proposals accepted to proposals raised has not yet been published because the team is currently instrumenting this metric; and they are still exploring which scenarios this model applies to. The premise of the framework is that the organization already has a documented method, not a specific technology.

Returning to the comparison of the two companies at the beginning: the difference was never in the interaction count, but in one having pushed a documented method into the agent layer while the other had not—only measuring by work level reveals this.

The authors finally suggest: pick a repeatable, documented process your team is already running, measure what it costs in money and time today, and then decide whether to hand it over to an agent.

Sources

  • snowflake.com↗
Continue researching →