Regulated organizations are often told to create an AI policy and then train staff to follow it. That is necessary, but it is not sufficient once agents begin performing multi-step work.
An agent can retrieve information, call tools, pass work to another agent, and create an output that becomes part of a real decision. The control model has to exist at runtime.
From policy to authority
Runtime governance should answer:
- Which agent is acting?
- On whose behalf?
- What data can it access?
- Which tools can it call?
- What decisions can it make?
- What requires human approval?
- What evidence must be retained?
- How can the action be stopped or reversed?
These are operational controls, not just policy language.
Evidence needs to be designed in
For regulated work, an output without provenance is difficult to defend. The system should preserve enough evidence to explain:
- the source material used;
- the instructions and authority context;
- the policy decision;
- the agent and tools involved;
- the human review, if any;
- the final action and resulting state.
Evidence does not mean storing everything forever. It means defining what must be retained, what can be redacted, and how an auditor or operator can reconstruct the decision.
Fail closed where the risk requires it
An agent should not proceed when a required mandate is missing, expired, revoked, or outside scope. It should return a clear reason and route the work to the appropriate review path.
This is better than relying on a general instruction such as “be careful.” Deterministic boundaries make the system easier to test and easier to explain.
Start with a bounded reference workflow
Regulated teams should begin with a workflow where the data, decision, reviewer, and rollback path are clear.
Examples include:
- intake and triage;
- risk snapshot preparation;
- evidence collection;
- policy comparison;
- renewal preparation;
- client-service research.
The first version should be local or pilot-scoped, with human approval and clear records of what the agent did.
The insurance opportunity
As organizations deploy more agentic workflows, they will need to describe their exposures more precisely. “We use AI” is too broad to support useful underwriting or operational risk management.
The relevant questions become:
- Is the agent internal or customer-facing?
- Does it access personal or regulated data?
- Can it make or trigger decisions?
- Can it communicate externally?
- Can it move money or change records?
- Is there human review and evidence?
Better runtime controls make those answers more concrete.