Use the Minimum Necessary Autonomy

Minimum necessary autonomy

There is a strong temptation in AI product development to turn everything into an agent: give a model tools, give it a goal, let it reason, let it decide what to do next and repeat until the task is complete.

That architecture is powerful for some problems and introduces needless uncertainty in others. Use the minimum amount of AI autonomy needed to build a reliable system.

AI and deterministic software solve different problems

Traditional software is excellent when the rules are explicit. If account status equals suspended, prevent the transaction. There is little reason to ask a language model whether a suspended account should be permitted to transact. The rule is already known.

AI becomes valuable when the input is ambiguous. A customer says they were charged twice yesterday but are not sure which payment is the duplicate. Now the system needs to interpret language, retrieve transaction history, compare records, and understand what the customer means.

The mistake is assuming that because one part of the workflow benefits from AI, the entire workflow should become probabilistic.

Production systems should be hybrid systems

Consider an AI system monitoring patients remotely. A patient submits blood pressure of 175/110 with the message: feeling a little strange and my head hurts.

Several kinds of computation may be required. AI interprets what the patient means and extracts headache, abnormal sensation, and possible severity. Deterministic logic compares the measurement against an agreed clinical threshold. Retrieval pulls previous readings, medications, history, and the current care plan. AI prepares a concise summary for the care team. Workflow creates an urgent review case. A qualified clinician determines the clinical action.

That architecture is safer than handing everything to an autonomous healthcare agent. The intelligence remains useful. Autonomy stays constrained.

Think in levels of autonomy

Treat autonomy as a spectrum, and allow different parts of the same product to operate at different levels.

• Level 1 — AI generates: summarise a consultation, extract contract clauses, draft an email. Human control remains high.
• Level 2 — AI recommends: interpret information and propose routing or priority. A human still decides.
• Level 3 — AI prepares actions: prepare a refund for review. The employee clicks Approve.
• Level 4 — AI acts within defined boundaries: categorise a ticket, update a CRM field, request missing information, schedule a standard follow-up.
• Level 5 — AI manages the workflow: plan and execute multiple steps, adapting to new information, with human intervention mainly for exceptions.

When agents fit

Anthropic distinguishes between workflows, where models and tools follow predefined paths, and agents, where models dynamically determine their own processes and tool use. Its experience building agents suggests that simple, composable architectures frequently outperform unnecessarily complicated frameworks.

OpenAI similarly recommends agents for workflows involving complex judgement, difficult-to-maintain rules, or significant unstructured information, while noting that deterministic approaches may be enough elsewhere.

The design question is whether you can reliably specify what should happen next. If yes, encode it. If no, AI reasoning may be appropriate.

Healthcare shows why boundaries matter

A 2026 JAMA multisite study examined 8,581 clinicians across five academic healthcare institutions. Among the 1,809 clinicians who adopted AI scribes, adoption was associated with 16 fewer minutes of documentation time per eight scheduled patient hours and 0.49 additional visits per week. The study did not find a significant overall reduction in after-hours EHR work.

The AI improved one specific part of the workflow. Serious evaluation asks exactly which part improved and which sources of administrative burden remain.

Design the human loop

Human judgement is a component of many AI systems.

Ask where that judgement creates the most value. AI can retrieve, summarise, classify, handle repetitive documentation, compare options and prepare workflow steps. Humans focus on exceptions, accountability, ambiguous decisions, sensitive communication and high-consequence actions.

Capability alone should not set autonomy. Risk, reliability, observability, reversibility, failure detection and the cost of being wrong should. The best AI product gives AI exactly as much work as it can handle safely.