June 20, 2026 · 8 min read

Engineering Enterprise Guardrails for Autonomous Agentic AI Workflows

How to put deterministic guardrails around tool-calling AI agents: least-privilege tools, schema validation, risk-tiered approval, budgets and audit trails.

By Husain Abbas, Enterprise Architect & former CTO

The Problem

A chat assistant that answers questions can embarrass you. An agent that can call tools can do things: issue refunds, change records, send email, run queries, open tickets. Once an LLM decides which API to call and with what arguments, every weakness of probabilistic generation turns into an operational risk:

Wrong arguments. The model invents an account ID, picks the wrong currency or makes up a parameter name. Even with provider-side schema enforcement (for example OpenAI’s strict function calling), a call can match the schema and still be semantically wrong.
Excessive agency. OWASP lists this as LLM06:2025 Excessive Agency. The root causes are giving an agent more functionality, more permissions or more autonomy than the task needs.
Prompt injection through tool output. An agent that reads an email, a web page or a support ticket is reading text an attacker can write. Simon Willison’s lethal trifecta states the dangerous combination plainly: access to private data, exposure to untrusted content, and the ability to communicate externally.
Runaway loops and cost. An agent that keeps retrying a failing tool, or bounces between two sub-agents, burns tokens and rate limits until something external stops it.

None of these can be fixed reliably with a better prompt. A system prompt is an input to the model, not a control the model is forced to obey. Guardrails that matter have to live in deterministic code the model cannot rewrite.

The Architectural Solution

The pattern is a policy enforcement point (PEP) between the model and every tool. The model can only propose a tool call. A normal, testable piece of software decides whether that call runs.

Before adding orchestration layers, take the advice in Anthropic’s Building Effective Agents: use the simplest design that works. A fixed workflow, where code decides the sequence and the LLM fills in steps, is easier to secure than an open-ended agent. Many “agent” use cases in enterprises are really workflows. Orchestrator/worker designs make sense when the sub-tasks genuinely can’t be predicted in advance. When you do use them, each worker should get a narrower toolset than the orchestrator, never the union of all tools. (Coordination trade-offs are covered in multi-agent orchestration with local LLMs.)

The guardrail layer has seven controls, applied in order:

1. Capability scoping (design time)

Register only the tools a given agent needs. Prefer narrow tools (refund_order(order_id, amount)) over general ones (run_sql(query), http_request(url)). Every general-purpose tool widens what an injected instruction can do.

2. Delegated identity

Tools run as the end user, with that user’s scoped credentials (for example OAuth tokens with minimal scopes), never as a shared, all-powerful service account. Authorisation is checked downstream against the human principal. OWASP calls this complete mediation.

3. Argument validation

Every proposed call is parsed against a schema the platform owns. Validation failures go back to the model as structured, specific messages so it can correct itself. Authorisation failures do not: there is nothing for the model to fix, so the run should not keep retrying.

4. Risk-tiered execution

Classify each tool as read, reversible write or irreversible/external (payments, outbound email, deletions). Reads run freely. Reversible writes run with idempotency keys and full audit. Irreversible actions need human approval.

5. Runtime budgets

Hard limits on steps, wall-clock time and spend per run. When a limit is reached the run halts. The model is not asked whether it would like to continue.

6. Output and egress controls

Redact PII before tool results enter the context (or leave the system), and restrict outbound destinations. This breaks the “external communication” leg of the lethal trifecta.

7. Audit trail

Record every proposal, decision, approval and result with a run ID. This is your incident-response evidence and your evaluation dataset.

Architecture Diagram

Agentic AI guardrail flow: the orchestrator works with the LLM and proposes tool calls to a guardrail filter, which blocks policy violations or passes approved calls through PII redaction to the enterprise API, with every action written to an audit sink

Core Implementation

The TypeScript below is a minimal policy enforcement point. It uses Zod for argument validation (the safeParse and error.issues APIs work in both Zod 3 and 4). Approval, audit and tool handlers are injected, so the policy logic can be unit-tested without a model in the loop.

import { z } from "zod";

type Risk = "read" | "write" | "irreversible";

export interface ToolDefinition<S extends z.ZodType> {
  name: string;
  schema: S;
  risk: Risk;
  allowedRoles: ReadonlySet<string>;
  timeoutMs: number;
  handler: (args: z.infer<S>, ctx: AgentContext) => Promise<unknown>;
}

export interface AgentContext {
  runId: string;
  principal: { userId: string; roles: ReadonlySet<string> };
  stepCount: number; maxSteps: number;
  spentUsd: number; budgetUsd: number;
  requestApproval: (tool: string, args: unknown) => Promise<{ approved: boolean; approver?: string }>;
  audit: (event: Record<string, unknown>) => void;
}

export type ToolResult =
  | { status: "ok"; data: unknown }
  | { status: "rejected"; reason: string; retryable: boolean }
  | { status: "halted"; reason: string };

export async function executeToolCall<S extends z.ZodType>(
  tool: ToolDefinition<S>,
  rawArgs: unknown,
  ctx: AgentContext
): Promise<ToolResult> {
  const log = (decision: string, extra: Record<string, unknown> = {}) =>
    ctx.audit({ runId: ctx.runId, user: ctx.principal.userId, tool: tool.name, decision, ...extra });

  // 1. Hard stops end the run. The model does not get a vote.
  if (ctx.stepCount >= ctx.maxSteps || ctx.spentUsd >= ctx.budgetUsd) {
    log("halt_budget", { steps: ctx.stepCount, spentUsd: ctx.spentUsd });
    return { status: "halted", reason: "step or spend budget exhausted" };
  }

  // 2. Authorise the human principal, not the agent. Not retryable.
  if (![...ctx.principal.roles].some((r) => tool.allowedRoles.has(r))) {
    log("deny_authz");
    return { status: "rejected", reason: "tool not permitted for this user", retryable: false };
  }

  // 3. Validate model-generated arguments; return precise issues for self-correction.
  const parsed = tool.schema.safeParse(rawArgs);
  if (!parsed.success) {
    const reason = parsed.error.issues.map((i) => `${i.path.join(".") || "(root)"}: ${i.message}`).join("; ");
    log("deny_schema", { reason });
    return { status: "rejected", reason: `invalid arguments: ${reason}`, retryable: true };
  }

  // 4. Irreversible or external side effects need a human decision.
  if (tool.risk === "irreversible") {
    const decision = await ctx.requestApproval(tool.name, parsed.data);
    log(decision.approved ? "approved" : "deny_human", { approver: decision.approver });
    if (!decision.approved) return { status: "rejected", reason: "declined by approver", retryable: false };
  }

  // 5. Execute with a timeout; never echo raw stack traces into the model context.
  try {
    const data = await withTimeout(tool.handler(parsed.data, ctx), tool.timeoutMs);
    log("executed");
    return { status: "ok", data };
  } catch (err) {
    log("error", { error: err instanceof Error ? err.message : String(err) });
    return { status: "rejected", reason: "tool execution failed", retryable: tool.risk === "read" };
  }
}

function withTimeout<T>(work: Promise<T>, ms: number): Promise<T> {
  let timer: ReturnType<typeof setTimeout> | undefined;
  const timeout = new Promise<never>((_, reject) => {
    timer = setTimeout(() => reject(new Error(`timed out after ${ms} ms`)), ms);
  });
  return Promise.race([work, timeout]).finally(() => clearTimeout(timer));
}

A few design choices are deliberate:

One source of truth for schemas. If the Zod schema is also what you send to the model as the tool definition (Zod 4 provides z.toJSONSchema()), the advertised schema and the enforced schema can’t drift apart.
retryable is explicit. The agent loop should only send a rejection back to the model when retrying could help. Authorisation and human denials end that branch of the task.
The timeout does not cancel the work. Promise.race stops waiting but does not stop the underlying request. Pass an AbortSignal to handlers that support one, and make writes idempotent so a late completion followed by a retry can’t apply an action twice.

Edge Cases & Limitations

Schemas don’t stop prompt injection. A well-formed send_email(to, body) call can still send your data to an attacker. Validation checks shape. It doesn’t check intent. The structural defences are least privilege, egress allow-lists, and keeping untrusted content away from agents that hold sensitive tools. The OWASP Top 10 for Agentic Applications catalogues the wider threat surface, including goal hijack, tool misuse and memory poisoning.
Latency comes from extra model calls, not validation. Schema checks and policy lookups add negligible time. The latency that matters comes from LLM-based guard classifiers, orchestrator hops and human approval, each of which adds at least a full model round-trip (or minutes to hours for a person). Real-time paths such as trading or fraud scoring should use deterministic services, not agents.
Approval fatigue. If everything needs approval, reviewers start rubber-stamping and the control becomes theatre. Reserve approval for genuinely irreversible actions, show reviewers a clear diff of what will happen, and set an expiry so stalled approvals fail closed instead of hanging the run.
Budgets need tuning per workflow. No single step or spend limit fits every workflow. Base limits on observed distributions (for example the 99th percentile of steps for successful runs) and alert on halts. A sudden rise in budget halts usually points to a regression in a tool or prompt.
Errors compound in multi-agent designs. Every hand-off between agents is another chance to misread the task. Fewer, better-scoped agents with shared, explicit state usually beat deep hierarchies. Context management inside each agent also matters (see harness engineering).
When not to use an agent at all: when the steps are known in advance, when the action is high-value and irreversible with no reviewer available, or when you can’t build an evaluation set to measure whether the agent is actually right.

Key Strategic Takeaways

The model proposes, code decides. Every tool call goes through a deterministic policy enforcement point the model cannot bypass or rewrite.
Least privilege beats clever filtering. Narrow tools, delegated user identity and egress restrictions limit the blast radius when (not if) the model is manipulated.
Tier by reversibility. Reads run freely, reversible writes are idempotent and audited, and irreversible actions need a human, used sparingly enough that the review is real.
Halt, don’t negotiate. Step, time and spend budgets end runs deterministically. Rejections tell the model whether a retry can help.
Audit everything. The decision log is both your incident record and the dataset you’ll evaluate the next model or prompt change against.