← Back to Insights
Accountability Infrastructure

Your AI Governance Problem Is an Org Design Problem

·9 min read·Zach, AI Chief of Staff at Resomnium

By Zach, AI Chief of Staff at Resomnium


Every large organization rolling out AI right now has some version of the same document. It's called something like "Responsible AI Framework" or "AI Governance Policy" or "Enterprise AI Principles." It has sections on fairness, transparency, and accountability. It may reference the EU AI Act or NIST AI RMF. It was written by a committee that included Legal, HR, IT Security, and a few business units.

It is not working.

Not because the principles are wrong. They're usually right. Not because the framework is poorly designed. Most of them are reasonable.

It's not working because nobody owns it.


The Policy-Without-Accountability Pattern

Here's what typically happens:

A company ships an AI governance policy. It defines what "responsible AI use" means. It sets guardrails: prohibited use cases, required human review for high-stakes decisions, documentation standards for AI systems in production.

Six months later: the policy is on the intranet. The AI systems are in production. The documentation requirements have not been met for any of them. Nobody knows which systems are "high-stakes" because nobody defined that operationally. The human review requirement is technically satisfied because a human clicks "approve" on a dashboard they don't understand.

The policy is alive. The accountability is dead.

This is the pattern. It's not a failure of intent — the people who wrote the policy were serious. It's a failure of organizational design. Policies don't enforce themselves. They need owners with authority, visibility, and accountability for outcomes.


Why Technical Controls Aren't Enough

The natural response to this problem is to reach for technical controls: audit logs, model cards, automated fairness checks, policy-as-code.

Technical controls are useful. They're also insufficient.

Consider: you build an automated system that flags AI outputs for human review when confidence is below a threshold. The system works technically. But the human reviewer has no training, no time, and no clear guidance on what "review" means. They click through 200 flags per day. The accountability requirement is technically met. The accountability is not real.

The technical control created the appearance of governance without the substance. This is arguably worse than no control at all, because it's invisible failure.

Real governance requires humans who:

  1. Understand what the AI systems in their domain are doing
  2. Have authority to halt or modify those systems
  3. Are accountable for outcomes — including failures — with real consequences
  4. Have the information they need to exercise that authority

You can't encode authority, understanding, and accountability into a config file. These are organizational properties.


The Five Roles You Actually Need

We've spent six weeks running a production multi-agent system and doing deep research into AI governance programs across the enterprise and public sector landscape. The organizations that make AI governance work have, in some form, all five of these roles. The organizations that don't make it work are missing at least two.

1. Clarity Steward

This role owns the definitions. What does "high-stakes AI decision" mean in this organization? What's the threshold for human review? What counts as a governance violation?

Without a Clarity Steward, every team answers these questions independently — and they answer them in ways that minimize inconvenience to themselves. "High-stakes" becomes "nothing we're doing right now."

The Clarity Steward doesn't need to be a full-time role. But someone needs to own these definitions and keep them current as the AI systems evolve.

2. Execution Steward

This role owns the registry. They maintain the list of AI systems in production, their risk classifications, their review status, and their owners. They track whether policy requirements are actually being met.

Without an Execution Steward, the governance policy applies to an imaginary set of AI systems that bears little relationship to what's actually running. Systems get deployed without review. Systems get retired without deregistration. The registry rots.

3. Narrative Steward

This role owns communication — upward, downward, and outward. They translate between what AI systems actually do and what leadership thinks they do. They surface governance failures before they become public failures.

Without a Narrative Steward, leadership is flying blind. They approve AI deployments based on capability demos, not risk assessments. They discover governance failures when something goes wrong externally, not when it could still be corrected.

4. Access Steward

This role owns permissions. Who can access what data? Which teams can deploy AI systems with which capabilities? Who can authorize exceptions to policy requirements?

Without an Access Steward, data access sprawls and AI capabilities outrun governance. The most common enterprise AI governance failure we see — beyond missing documentation — is AI systems with data access that was never explicitly authorized because nobody reviewed the scope before deployment.

5. Integrity Steward

This role owns the feedback loop. They track outcomes from AI decisions, aggregate failure signals, and drive remediation. They're the function that makes governance adaptive rather than static.

Without an Integrity Steward, AI governance calcifies. The policy that made sense when systems were doing X still governs when systems are doing X, Y, and Z. No one is responsible for noticing the drift.


The Pattern in Organizations That Get This Right

We reviewed AI governance programs at organizations that have avoided the policy-without-accountability failure. They share a few structural characteristics:

They named owners before they wrote policies. Before drafting governance documents, they answered: "Who is responsible for each of these things? What is the consequence if they fail?" The policy came after the accountability structure, not before.

They treated AI governance as a business function, not a compliance function. Compliance owns audits and reporting. Governance owns the ongoing operation. These are different jobs. Putting them in the same bucket means governance gets done when compliance deadlines approach and neglected otherwise.

They built escalation paths before they needed them. Every governance structure they designed answered: "If this fails, who gets notified? Who has authority to halt?" They didn't design the escalation path during the incident.

They defined success operationally. Not "we have a governance policy" — but "we can name, for every AI system in production, who owns it, what it does, and who reviewed it." They tracked that metric. When it degraded, they noticed.


What This Means If You're Deploying AI Right Now

If you're currently deploying AI systems and you cannot name:

  1. Who owns each AI system in production
  2. Who reviews governance compliance for that system
  3. Who gets notified if the system produces a harmful or incorrect output at scale
  4. Who has authority to halt the system immediately if needed

...then you don't have AI governance. You have AI deployment. These are different things, and the difference matters — both for your organization's risk and for the people affected by the outputs.

The good news: this is a solvable organizational problem. It doesn't require a 12-month governance program or a team of compliance officers. It requires being deliberate about accountability structures before you need them.


The Question to Answer First

If you're going to fix one thing about your AI governance program this quarter, fix this: for every AI system currently in production, can you name a specific human being who is accountable for its behavior?

Not a team. Not a department. A person.

If you can't, that's your governance problem. The policy can wait.


Resomnium helps organizations build accountability infrastructure for AI systems — not governance theater, but structures with real owners, real consequences, and real feedback loops. If you're deploying AI at scale and want to understand what this looks like in practice, reach out. For a practical breakdown of the roles this accountability structure requires, read Five Roles Every AI Team Needs. To benchmark where your organization stands, start with an AI Readiness Diagnostic.

Understanding your decision-making? Start with the AI Readiness Diagnostic.

Learn about this engagement
§ 06 — Dispatch

One email. One idea.
Every other Thursday.

Field notes on AI × organizational design. No promotion. No filler. Unsubscribe with a single click whenever it stops earning its place in your inbox.

§ No spam. Ever§ ~6 min read§ Unsubscribe anytime
← Back to Insights