Governing 37 AI Agents in Production: A Field Guide to Runtime Governance

The problem we started with

At most engineering teams we talk to, AI now runs unattended in five places at once. Claude Code writes PRs. Cursor edits files. Copilot suggests completions. Custom agents run scheduled tasks. An MCP server exposes tools to any AI client on the network. Each one has its own dashboard, its own budget, its own audit surface.

None of them talk to each other. Nobody in the room can answer basic questions:

• What did AI do in production this week?
• Which model got called by which agent for which task?
• Did the compliance rule we wrote in Notion actually apply to that Cursor session?
• Who approved that deploy?

The AI-safety conversation still lives in Google Docs. The AI-usage reality is a proliferation of tools nobody governs.

We built ConductAI to make policy something you enforce at runtime, not something you write in a doc and hope people follow.

What ConductAI actually is

Three layers, one platform:

1. ConductGuard is the policy and audit spine. A proxy sits in front of every LLM call. An MCP server sits in front of every tool call. Every request runs through a declarative rule engine before it goes anywhere. Every decision writes a signed audit event with lineage.
2. Playbook Runtime is 37 pre-built YAML playbooks that do specific engineering work (PR reviews, incident response, security scans, release ops). Runs inside Guard so every LLM call and tool call it makes is governed by the same policies.
3. Compliance Packs and Marketplace ship 10 packs mapped to real regulatory frameworks (SOC 2 CC6.6, HIPAA §164.312, EU AI Act Article 15). Install one, get 20-40 policy rules pre-configured.

The value proposition is one identity, one policy, one audit trail across every AI agent your team uses.

Six use cases, from what real teams are doing

1. PR reviewer that can never merge

Playbook: pr-reviewer.yaml (223 lines, ships in the repo).

Setup: Install the playbook. Register a GitHub webhook. Assign a cond_agt_pr-reviewer token with read+comment scope on your repo. Install the conduct-endpoint-attacks skill pack (PII/secret leak detection). Set a $2/PR budget cap.

What it does at runtime: When a PR opens, the agent reads the diff, comments on style and security issues, requests changes. When it tries to comment, Guard evaluates every LLM call and every GitHub API call against the installed rules. It scrubs secrets before commenting. It can never merge, because the token doesn't have merge scope and a Guard rule blocks the github.pull.merge tool regardless.

What compliance sees: every decision this agent made in the last 90 days, signed and hash-chained. Which rule fired, what the prompt was, what the model output was, how much it cost.

2. Incident responder with a human gate

Playbook: incident-responder.yaml.

Setup: PagerDuty webhook triggers the playbook. Install the conduct-base and conduct-network-ops packs. Set the agent identity to read-only IAM. Configure a Slack channel for approvals.

What it does at runtime: When an alert fires, the agent correlates recent commits and deployments, formulates a hypothesis, and posts it to Slack. If it wants to suggest a rollback, the Guard approval action fires. Nothing executes until a human clicks Approve in Slack. The approval decision is recorded with the approver's identity, timestamp, and reason.

Why this matters: the wedge between “AI suggests” and “AI acts” is where all the risk lives. Approval gates turn suggestion into a reviewed decision without slowing anyone down.

3. SOC 2 evidence gathering, hands-free

Playbook: ai-output-auditor.yaml combined with security-scanner.yaml.

Setup: Install the conduct-iso-42001 and conduct-nist-ai-rmf packs (both map to SOC 2 CC6, CC7, CC8 controls). Assign a read-only cloud IAM token. Schedule the playbook weekly.

What it does at runtime: The agent pulls current IAM policies, encryption states, access logs, and MFA enforcement across your cloud accounts. Compares against the installed compliance rules. Writes a findings report with clause-by-clause evidence. Every LLM call is bounded to a $50 budget, and every finding is written to the hash-chained ledger.

What the auditor sees: a report that references specific SOC 2 clauses, specific findings, specific evidence timestamps. All signed. The rule that flagged each finding is traceable to the pack version installed at scan time.

4. Runaway spend prevention (shipped this month)

Every AI-serious team has had this happen: an agent gets stuck in a retry loop and burns hundreds of dollars in an hour. Or a leaked API key generates a $50K bill before anyone notices.

What ConductGuard now does: every LLM call runs through a pre-forward budget check. If the workspace has spent past the monthly hard cap, the request returns HTTP 429 with {"error": {"type": "guard_budget_exceeded", ...}} before touching the provider. Zero cost overshoot from that call.

Rate limits at the same layer: admins can set requests-per-minute and tokens-per-minute caps per workspace or per agent identity. Same 429 response, {"type": "guard_rate_limited", "metric": "rpm", "limit": 60, "current": 61}. The agent sees a structured error and can back off.

Where it plugs in: every proxy call, no exceptions. Any AI tool pointed at Guard is bounded by these caps, whether it's Claude Code, a custom Python agent, or a scheduled cron job.

5. Security scanning with auto-remediation

Playbooks: security-scanner.yaml (386 lines) and security-autopilot-fix.yaml.

Setup: Install both playbooks. Install the conduct-endpoint-attacks pack. Assign a cond_agt_security-scanner token with issue-create and PR-open scope, but no merge scope.

What it does at runtime: The scanner walks repos looking for OWASP top-10 patterns, exposed secrets in git history, dependency vulnerabilities. It opens GitHub issues (not PRs) with specific line references. If a companion security-autopilot-fix is enabled for a repo, it picks up the issue, sandboxes an attempt at a fix in an ephemeral container, runs the tests, and opens a draft PR. A human still merges.

Why sandbox execution: the security agent runs code. It runs it in Modal or E2B ephemeral containers with no persistent access to production. When the run finishes, the container is destroyed.

6. Network diagnosis from an alert

Playbook: network-diagnosis-agent.yaml (264 lines).

Setup: Alert webhook triggers the playbook. Install the conduct-network-ops pack. Assign a network operations agent identity bounded to sandboxed shell only.

What it does at runtime: The agent runs dig, traceroute, curl, and BGP-check scripts against the affected service. Never runs a mutation. Posts findings to Slack. If it wants to suggest a config change, an approval gate fires. Every shell command is logged with input, output, and duration in the audit ledger.

Why bound to read-only: the conduct-network-ops pack blocks any policy-modify or config-mutate tool at the Guard layer. Even if the agent were prompt-injected into trying, Guard would block the call before it left the sandbox.

What makes this different from every other AI tool

Policy is a runtime primitive, not a config toggle. The rule that says “PR reviewer can never merge” is a declarative YAML rule evaluated on every request. It's version-controlled. It's signed. It's audited when it fires. Changing it requires a new signed version, not editing a checkbox.

Hash-chained audit ledger. Every decision writes a row with prev_hash → entry_hash. Tampering with one row breaks the whole chain, and every row after it is invalidated. Compliance can prove the audit trail hasn't been edited.

Per-agent identity is a first-class token type. cond_agt_* tokens carry an expiry, a scope, and a lineage. When an agent makes a call, the audit ledger records which identity called, what scope it had, what token was in use. Not a per-user API key repurposed for a robot.

Compliance packs mapped to real clauses. SOC 2 CC6.6, HIPAA §164.312(b), EU AI Act Article 15, SR 11-7 model risk management. Not “safety” or “pci” as one-word presets. Auditors get clause-by-clause evidence.

Human approval workflow with rich context. When a Guard rule triggers approval, the request pauses. A Slack message with the full context (agent identity, prompt summary, tool it wants to call, historical spend, run linkage) goes to the approval group. One click resumes or rejects, and the decision writes to the audit ledger with the approver's identity.

Playbook runtime and Guard, built together. Guard sits inside every brain block. When a playbook makes an LLM call, that call is governed. When a playbook invokes a tool, that tool call is governed. There's no gap between “the playbook does something” and “governance sees it.”

Provider coverage

Three native adapters (Anthropic, OpenAI, Perplexity). Six gateway adapters (OpenRouter, Portkey, Helicone, LiteLLM, Azure OpenAI, ConductAI) auto-detected from the upstream URL. Transitively reaches 400+ models across every major provider: Bedrock, Vertex AI, Gemini, Cohere, Mistral, Groq, Together, Fireworks, DeepSeek, xAI, and any OSS server hosted via OpenRouter or self-hosted LiteLLM.

The gateway adapter pattern means if your team already runs an LLM gateway, Guard governs on top of it. If you don't, use ConductAI direct. Either way, one policy engine, one audit ledger.

Getting started

pip install conduct-cli
conduct login
conduct guard install       # wires the hook into Claude Code / Cursor
conduct run pr-reviewer --repo owner/name --pr 42
conduct verify <run-id>

For the proxy path:

export ANTHROPIC_BASE_URL="https://api.conductai.ai/proxy/anthropic"
export OPENAI_BASE_URL="https://api.conductai.ai/proxy/openai/v1"

Every SDK call from now on runs through Guard. Every playbook invocation runs through Guard. Every tool call runs through Guard.

What's next

Second half of 2026, we're focused on three things:

1. Mid-stream enforcement. Cut streaming responses the moment a budget cap fires, not after the stream completes.
2. Behavioral risk scoring on agent identity. Persistent risk score per cond_agt_* with quarantine and decay.
3. Native Bedrock adapter. The one provider Fortune 500 compliance teams require direct-native rather than via gateway.

The platform is MIT licensed and self-hostable. If you're running AI agents in production and want to govern them the way you govern the rest of your stack, we'd like to hear from you.