Imagine Monday at 9:12. The AI agent writes: “Lead processed, task created, manager assigned.” You open the CRM. The lead is there, but with no owner. Tasks - empty. And there’s a draft email sitting in the outbox that nobody asked it to send.
The agent’s response sounds confident. It might even politely list each step: read, matched, created, verified. But what matters to the business is not how elegantly the work was described. What matters is what actually happened in the working system after it ran.
That’s exactly why early tests of agents often feel strange. You hand off the routine, then manually check the CRM, tasks, files, and recipients. That’s not delegation. That’s the old manual work with a mandatory reading of an optimistic report tacked on. The process broke in such a polite way that it’s even awkward to complain about it.
The answer isn’t to demand a longer report from the agent. You need a short acceptance contract: what state should exist afterward, what proves it, what must not happen, what the agent is allowed to write, and when it must stop.
1. Beautiful output, activity and outcome - these are three different things
Before you set up any controls, separate three things that are easy to confuse. They can be parts of the same job, but they don’t substitute for each other.
Output: what the agent said or showed
Output is a text, a table, a list of contacts, a draft email, or a message saying “all done.” It’s useful for understanding what the agent saw and how it explains its reasoning. But by itself, it doesn’t prove anything changed.
The phrase “created a task for the manager” doesn’t answer the basic questions: exactly where is that task, who is the assignee, which fields are filled in, did a duplicate appear. Confident language doesn’t become a system record just because there’s a period at the end.
Activity: what happened along the way
Activity is a list of actions: the agent opened an email, compared fields, found a card, prepared the data, tried to create a task. That kind of log can help when you need to untangle an exception or an error. But a large number of steps doesn’t make the final state correct.
A printer that diligently prints forty pages of the wrong document also shows impressive activity. Unfortunately, that doesn’t add any usefulness.
Outcome: what was verifiably changed
Outcome is the target state in the working system. The right object was created or updated, the key fields are correct, the right person sees it where they expect it, and the forbidden action did not happen.
If the agent was supposed to create a task, the outcome is not the message “task created.” It’s one specific task in the right project, with the correct assignee, the data needed for the next step, and no duplicate. If the agent was supposed to respond to a request, the outcome at the start might only be a saved draft. A sent email is a different, riskier action.
Diagnosis 1: the report substitutes for verification
The symptom is simple: the team accepts the work because the report says “done,” but can’t quickly point to a specific created or changed object. A few days later you find missed tasks, duplicates, or records in the wrong queue. And it’s already hard to tell where the problem started.
The remedy is equally simple: for every action taken, require a trace that can be checked. That might be an ID or a link to the object, a check on mandatory field values, a saved file, or a log entry. A person doesn’t need to click every link every day. But the process should allow that kind of check without an archaeological expedition through open tabs.
2. Five parts of a short acceptance contract
An acceptance contract is not a legal document and not a fifty-page technical specification. For a first process, one card is often enough. Its job is to remove different interpretations of the word “done” before you launch.
1. Target state
Describe what must be true after the agent has run. Not an action - a state.
Bad: “the agent qualifies applications.” Better: “for an application that meets the rules, one record exists in the defined pipeline; source and topic are filled in; the manager sees the next step.” That state can be seen and verified without guessing.
At the start, don’t try to cover an entire sales flow or an entire support queue in one phrase. A narrow state doesn’t limit the value of the test. It makes that value measurable.
2. Proof
Now name what confirms the target state. Proof is not the agent’s retelling of what it claims to have done. It’s a trace in the system or a repeatable check.
For example, a check might find one record by key, confirm the values of mandatory fields, show where a file was saved, or compare the recipient against an approved list. Audit trails and deterministic checks reduce the risk of a silent error, though they don’t make the process error-free.
3. Forbidden side effects
Name not only the desired result, but also what must not happen. For example: do not create duplicates, do not change the owner of an existing card, do not send emails, do not delete files, do not edit records outside the defined queue.
This part is critical, because side effects are often more painful than a missed action. One missed record can usually be created later. An unsanctioned message or a corrupted existing record can require significantly more manual recovery.
4. Write permission
Separately define what the agent can read, what it can prepare, and what it can change on its own. This is not distrust of the tool - it’s a normal boundary of responsibility.
For a first test, the agent can read incoming requests, classify them, create drafts, and add data to a separate queue. Sending an email, changing an important status, or deleting a record can remain with a human. That’s exactly how AI systems with controlled actions are built: through defined boundaries, not through unlimited access in the hope that things will go well.
5. Stop condition
The last part defines when the agent must not guess. It must stop, flag the case, and hand it to a person.
A stop condition might be a missing mandatory field, conflicting data, an unfamiliar request type, a similar existing record, or an action outside its write permission. This is not the agent failing. This is the right behavior in places where the cost of a mistake is higher than the benefit of moving automatically.
3. Diagnosis 2: the agent got a task but no boundaries
Requests like “sort through all the emails,” “clean up the CRM,” or “prepare responses for clients” sound clear to a person who knows the context. For an agent, that’s too wide a space for assumptions. It may try to be helpful, but the team will get decisions nobody approved.
A conditional example: similar data in a request
Suppose the agent is moving requests from email into CRM. A letter arrives from a company whose name looks similar to an existing client, but the contact person is different. Without a stop condition, the agent might create a new lead, merge incompatible data, or overwrite the old card.
The contract changes the behavior. The agent creates a record only under unambiguous rules. If a similar record is detected, it edits neither of them, leaves a note for the manager, and adds proof of why it stopped. The person resolves a specific exception, not a weekend data-recovery exercise.
A conditional example: the response is ready but sending was not approved
Another situation: the agent analyzes a request and prepares a reply. The text might be appropriate, the recipient might be correct, but the contract says: “external sending only after approval.” In that case, the accepted outcome is a draft in the right inbox, linked to the request. Not a sent email.
That division doesn’t mean you’re giving up on automation. You’re testing classification, routing, and data preparation without adding an irreversible communication risk to your first test.
A boundary makes the error visible
When boundaries are described, it’s easier to find the actual cause of a failure. Is it a problem with the input data, the routing rule, access permissions, or the acceptance criteria? Without a contract, all those causes blur into a vague “the agent misfired somewhere.”
That’s why a person needs to set the intent, the boundaries of risky actions, and the acceptance criterion - even when the agent is walking through routine steps on its own. The agent follows the route. Responsibility for which route is acceptable stays with the process owner.
4. Canary, QA and human approval: each proves its own thing
Don’t test the agent on the full operation right away. Choose a narrow route where the inputs are clear, the required result is known, and there’s a way to stop an error without damage. This kind of controlled run is called a canary.
What a canary proves
A successful canary proves only that the agent completed a specific route under specific conditions. For example, it took a request of a certain type, created a draft, and left a verifiable trace.
It does not prove readiness for all languages, all non-standard emails, any future changes to the process, or every new integration. One successful run is a reason to run the next careful test - not a reason to hand the agent keys to the office, the warehouse, and the coffee machine.
What QA checks
QA checks the contract: was the target state reached, is there proof, and did no forbidden side effects occur. Some checks can be deterministic: one record exists, the required fields are filled in, the status is correct, the draft didn’t turn into a sent message.
Those checks are especially useful where a person previously repeated the same manual verification. They don’t override good judgment in a complex case, but they remove the need to take the agent’s word for it every single time.
When approval must stay with a person
Human approval should remain for external, sensitive, or hard-to-reverse actions: sending a message, transferring data, changing an important status, editing a record where a mistake is financially or reputationally costly. It’s also needed when data is contradictory or doesn’t fit the rules.
See how a narrow route can combine machine processing with human oversight in the Klar mail triage case. Its value isn’t a promise of full autonomy - it’s a clearly scoped process that can be checked.
5. Diagnosis 3: they check for success but not for the absence of harm
The team sees a created task and checks the box. But at the same time, the agent might have created a second identical one, assigned the wrong person, or touched someone else’s queue. If you only look at the positive part of the outcome, a silent error easily slips through.
Add a negative check
For every expected result, name at least one check that says “this did not happen.” For a CRM, that might be no duplicate by key. For email, no sent message. For files, no overwrite of the original. For tasks, no changes in someone else’s project.
A negative check doesn’t need to be complicated. It just acknowledges: the success of one step doesn’t excuse an unwanted consequence somewhere else.
Don’t confuse control with distrust
The contract doesn’t make the agent a suspect. It makes expectations visible to the owner, the operations team, and the person who configures the process. When the rules are clear, it’s easier to understand what to fix: the data, the route, the permissions, or the acceptance condition.
Control also gives an honest answer to the question “can we expand the test.” Not because the report became more convincing, but because a specific route left a verified result with no forbidden consequences.
6. The minimum result card and your first test
For a first test, you don’t need a sophisticated control panel. Take one repeating process where the result is visible in a specific system and an error can be stopped without damage. For example, preparing response drafts or creating tasks from clearly structured requests.
A card with five fields
Write on one page: target state, proof, forbidden side effects, write permission, and stop condition. Add one scenario with normal data and one with ambiguous data. That way you test not only whether the agent can move forward, but also whether it knows not to do too much.
Don’t expect the first version to cover every exception. Its job is more modest - and more useful: to show whether you can accept a specific result without manually hunting through different systems.
One first step
Pick one action the agent is already supposedly “completing,” and today write down one target state and one proof in the working system. That’s where verifiable testing begins.