The Short Version
Lean Six Sigma AI means running DMAIC before and around an agent build: Define the decision the agent owns, Measure the current process, Analyze where it really breaks, Improve it with the agent, and Control it with monitoring after launch. Each phase is a gate. If a gate fails, you fix the process before you write another prompt.
- Cost: the gates add a few weeks up front and remove the far larger cost of rebuilding an agent nobody trusts.
- Effort: most of the work is measurement and process mapping, not model tuning.
- Risk: skipping Measure means you can never prove the agent made anything better.
- When it fits: any agent that touches a repeatable business process with a measurable outcome.
I have spent more than 30 years implementing enterprise systems inside IBM, Ernst & Young, Johnson & Johnson and McKesson, and I hold a Lean Six Sigma Master Black Belt. The pattern I see with agents is the same one I saw with ERP rollouts twenty years ago. A team picks a painful process, automates it as it stands, and is surprised when the automated version is faster at producing the same defects.
The standards bodies are starting to say the same thing in their own language. NIST’s AI Risk Management Framework, released on 26 January 2023 and voluntary, organizes AI risk work around measuring and managing systems over their whole life, not just at launch. And the OWASP Top 10 for Agentic Applications, published on 9 December 2025, names cascading failures and tool misuse among the ten risks specific to agents, according to OWASP’s own summaries of the list. Both are process failures before they are model failures.
The hard part for a mid-market team is not the model. It is knowing which steps of the current process are worth automating, what “better” means in numbers, and who owns the agent once it is live. DMAIC answers all three if you use it as a set of gates rather than a slide.
Why Do Agent Builds Stall Before the Model Is the Problem?
Most stalled agent projects I review share one root cause: nobody wrote down how the process performs today. The team knows the work is slow and error-prone, so they build an agent to make it fast and accurate. Six weeks later the demo looks good, but the business owner asks a simple question, “is it better than what we had?”, and nobody can answer, because there is no baseline to compare against. The project drifts into a permanent pilot.
The second cause is that the process itself has variation the agent inherits. If three teams handle the same request three different ways, the agent learns all three, or picks one at random, and its output looks inconsistent. That is not hallucination in the model sense. It is the process teaching the agent that inconsistency is normal. The third cause is data. An agent that retrieves from tables nobody has profiled will repeat whatever duplicates, stale rows and conflicting definitions those tables hold, and it will do so with confidence. Our data quality framework covers how to measure that before an agent ever reads it. None of these three problems is solved by a better prompt, and all three are caught by a DMAIC gate.
How Does Each DMAIC Phase Map to an Agent Build?
Here is the mapping I use. Each row is a gate: the build does not move to the next stage until the exit criterion is met and signed by the process owner, not just the engineer.
| DMAIC phase | Agent build stage | Gate question | Evidence on the lakehouse | Exit criterion |
|---|---|---|---|---|
| Define | Use case and charter | Which single decision or task will the agent own, and for whom? | Charter with scope, owner, customer and the metric that defines success | One named process owner and one primary metric |
| Measure | Baseline | How does the process perform today, in numbers? | Cycle time, error rate, rework and volume from source system tables | A baseline the owner agrees is true |
| Analyze | Process redesign | Where does the defect come from, and which steps should the agent do? | Process map, root-cause evidence, data profiling of every source the agent reads | A redesigned process the agent will run, not the old one |
| Improve | Build and evaluate | Does the agent beat the baseline on the agreed metric? | Evaluation runs against a labelled dataset, compared with the baseline | Measured improvement on a pilot, with failure cases reviewed |
| Control | Production and monitoring | How will we know when it drifts, and who acts? | Scored production traces, spend and rate limits, an owner and a reaction plan | A control plan in use, not in a document |
What Belongs in the Define Gate?
Define is where most agent projects go wide when they should go narrow. “An agent for customer service” is not a charter. “An agent that drafts the first response to warranty claims under a set value, reviewed by a person before it is sent” is a charter. It names the task, the customer, the boundary and the human in the loop.
Write the charter in one page: problem statement, scope in and out, process owner, the customer of the output, and one primary metric. If the team cannot agree on the metric, stop. An agent with no agreed definition of success will be judged on anecdotes, and anecdotes always favour whoever is loudest in the review.
What Should You Measure Before You Automate Anything?
Measure the human process as it runs today. For most agent candidates that means four numbers: how long one unit of work takes end to end, how often it is wrong, how often it is reworked, and how many units arrive per week. Pull them from the systems of record, not from a survey. If the work lives in an ERP, a ticketing tool and a shared mailbox, land all three in one place first, which in our builds means bronze tables in the lakehouse with the timestamps intact.
Then measure the data the agent will read. Profile every table it will retrieve from for completeness, duplicates and freshness. Unity Catalog captures lineage automatically, down to the column level, for queries run on the platform, so you can see exactly which upstream sources feed each table the agent touches. That lineage view becomes part of the Measure evidence.
How Do Analyze and Improve Change the Agent Design?
Analyze is where Lean earns its place. Map the process step by step and mark each one as value-adding, required but not value-adding, or waste. Most processes I map carry handoffs and approval loops that exist only because of an old system limit. Remove those before the agent arrives. An agent should run the redesigned process, because automating a waste step just makes the waste cheaper to repeat.
Analyze also decides the split between agent and person. Steps with high volume, clear rules and a checkable output suit an agent. Steps that need judgement on rare cases stay with a person, and the agent hands them over with context. Write that split into the design, so nobody discovers it in production.
Improve is the build, but it is judged against the baseline from Measure. On Databricks, MLflow 3 GenAI evaluation runs mlflow.genai.evaluate() over a labelled dataset, with built-in LLM judges for correctness and completeness plus your own code-based scorers. Use the cases your human team already got wrong as part of the dataset. The gate passes when the agent beats the baseline on the primary metric in a pilot, and when someone has read the failures, not just the average score.
What Does a Control Plan for an Agent Look Like?
Control is the phase teams skip, and it is the one that decides whether the agent is still useful in six months. A control plan names what is measured, how often, the limit that triggers action, who acts and what they do. For an agent, that means three layers.
- Quality: score a sample of production traces with the same scorers you used in Improve, and plot the results over time. A run of scores outside the limit is a signal to investigate, the same way a control chart works on a production line.
- Spend and access: Unity AI Gateway, which Databricks made generally available on 4 August 2026, governs models, agents, MCP servers and tools with identity-aware access, rate limits, spend caps and guardrails. Set the limits from the volumes you measured, not from a guess.
- Ownership: one process owner who reviews the numbers weekly, and a written reaction plan for each limit. If no one owns the signal, the signal is noise.
This is the same discipline we apply to data pipelines in our Lakeflow and Lean Six Sigma piece, moved one layer up to the agent that reads the governed lakehouse. Because the traces, scores and usage all land in tables, the control plan is a query and a dashboard, not a spreadsheet someone forgets to update.
Which Red Flags Mean You Should Stop and Fix the Process First?
These are the signals I treat as a hard stop. Any one of them means the agent will automate a problem rather than solve it.
- Nobody can state today’s error rate. If the current performance is a feeling, you cannot prove the agent improved it. Go back to Measure.
- Three teams do the same task three ways. Standardize the work first. An agent trained on variation produces variation.
- The source data has no owner. If no one is accountable for the tables the agent reads, defects in them will never be fixed, and the agent will repeat them.
- The business case is “it will be faster”. Faster at what, measured how? Speed without an accuracy baseline is how defects scale.
- There is no plan for what happens after launch. If the project ends at go-live, the agent starts drifting on day one and nobody is watching.
When one of these shows up, the cheapest move is a short process fix before any agent code is written. We make the same argument at length in Don’t Automate Chaos. If you want a Master Black Belt to run the Define and Measure gates with your team, our process improvement practice does exactly that, using our AIM-IT method: Assess, Innovate, Model, Implement, Track.
Frequently Asked Questions (FAQs)
Is Lean Six Sigma still relevant for AI projects?
Yes. An AI agent is a new way of executing a process, and DMAIC is a method for improving processes. The gates force a baseline, a root-cause view and a control plan, which are exactly the pieces most agent projects lack.
What is the difference between DMAIC and an AI project lifecycle?
A typical AI lifecycle runs from data preparation to training, deployment and monitoring. DMAIC adds what comes before and around it: a defined business problem, a measured baseline and a redesigned process. Used together, the agent is built on a process worth automating.
How long should the Define and Measure gates take?
For a single, well-scoped process, a few weeks is typical in our experience, most of it spent pulling and validating baseline data. If Measure drags on, it usually means the data is spread across systems and needs to be landed in one place first.
Can AI help with Lean Six Sigma itself?
Yes, within limits. Agents can draft process maps from logs, classify defect types and summarize root-cause evidence. A person with process ownership still makes the call on what to change and signs off each gate.
Do I need a Master Black Belt to run DMAIC on an agent build?
No, but you need someone trained in the method who is independent of the build team. The value comes from a person whose job is to ask whether the process is ready, not whether the agent works.