A prospect's security questionnaire arrives with a new line: "Please provide your AI governance framework." You ship two LLM-powered features and your team uses four AI tools nobody formally approved. What you have is a Slack thread and good intentions.
An AI governance framework is the set of roles, inventory, risk rules, policies, review gates and monitoring that decides how a company builds, buys and uses AI. You build it in seven steps, in a fixed order, because each step produces the input the next one needs.
By the end of this guide, you will be able to build that framework yourself and hand a customer, an investor or an auditor a clear answer.
Here's what I'll cover:
- What an AI governance framework is, and how it differs from a policy and from ISO 42001
- Why building one before a customer asks costs less
- The seven steps in build order: roles, AI inventory, risk tiers, policies, a review gate, monitoring and a review cycle
- How the result maps to ISO 42001, the NIST AI RMF and the EU AI Act
- The mistakes that quietly break these programs
What an AI Governance Framework Is, and What It Is Not
An AI governance framework is the operating system for AI inside your company: it names who is accountable, lists every AI system, sorts them by risk, sets the rules, checks new AI before launch and watches it afterward. It covers AI you build, AI you buy and AI your employees pick up on their own.
That last group matters most for early-stage SaaS teams. Most of the AI risk in a ten-person company sits in the tools and vendor models it uses, not in a model it trained.
A framework is not a single document. It is a program with a few moving parts, and a policy is one of them.
AI Governance Framework vs AI Governance Policy vs ISO 42001
These three get mixed up constantly, and they answer different questions.
| AI governance framework | AI governance policy | ISO 42001 | |
|---|---|---|---|
| What it is | The whole program: roles, inventory, tiers, gates, monitoring | One document stating the rules | A certifiable management system standard |
| Question it answers | How do we run AI responsibly day to day? | What are the rules, and who must follow them? | Can an outside auditor confirm our system works? |
| Proof you hold it | Records, reviews, evidence | The signed, versioned document | A certificate from an accredited body |
| Where to go deeper | This guide | ComplyJet's AI governance policy guide | What is ISO 42001 |
So when this guide says "write the policy," it points you to the policy guide for the document itself and spends its words on the program around it. If a buyer eventually asks for a certificate, you can build this framework toward ISO 42001 without starting over.
Why Build an AI Governance Framework Before a Customer Asks
The cheapest time to build an AI governance framework is before anyone asks for it. After a questionnaire lands, you are reconstructing history under a deadline. Before, you are writing down decisions you are already making.
Three pressures are converging on SaaS teams at once:
- Customer diligence. Enterprise buyers now add AI questions beside the SOC 2 ones: which models you use, where prompts go, who approved it.
- A gap your audit report leaves open. SOC 2 was not built to test fairness, safety or model behavior, as ComplyJet's guide to whether SOC 2 covers AI explains.
- Regulation with dates attached. The EU AI Act applies to any company whose AI is placed on the EU market or whose output is used there, wherever it is based.
On dates, be careful which ones you quote. Under Regulation (EU) 2026/1744, the Digital Omnibus on AI, in force since 27 July 2026, high-risk obligations for Annex III systems now start on 2 December 2027, and for AI embedded in regulated products on 2 August 2028.
Prohibited practices have applied since February 2025, and transparency duties since 2 August 2026. The full timeline is in our EU AI Act compliance guide, and the European Commission's AI Act Service Desk timeline tracks the official dates.
None of this requires a large program. It requires an order of operations.
How to Build an AI Governance Framework in Seven Steps
Here is how to build an AI governance framework without stalling: follow these AI governance framework steps in order, and let each one produce a concrete artifact you can show someone.
- Assign roles. One accountable owner and clear backups.
- Build the AI inventory. Every system, vendor model and AI feature, with an owner.
- Set risk tiers. A rule that sorts the inventory into three levels.
- Write the policies. A short document set tied to the tiers.
- Add a review gate. A checklist new AI passes before it ships.
- Set up monitoring. Watch quality, drift and incidents per tier.
- Review and keep evidence. A quarterly cycle with records.
AI Governance Framework Steps, in Build Order and Why That Order
The order is not arbitrary. You cannot tier systems you have not listed, and you cannot list them without someone accountable for the list. Policies written before tiers end up generic, and monitoring written before a review gate watches things nobody approved.
| Step | What you produce | Typical owner | Rough effort on a small team |
|---|---|---|---|
| 1. Roles | One-page responsibility table | Founder or CTO | Half a day |
| 2. AI inventory | Spreadsheet of AI systems | AI owner | 1 to 3 days |
| 3. Risk tiers | Tiering rule plus tiered inventory | AI owner with security | 1 day |
| 4. Policies | Five short documents | AI owner with legal | 3 to 5 days |
| 5. Review gate | Intake form and approval path | Product and security | 1 day |
| 6. Monitoring | Checks per tier, incident path | Engineering | 2 to 5 days |
| 7. Review cycle | Calendar, minutes, evidence folder | AI owner | 1 day, then quarterly |
The effort column is my own estimate for a team of 50 or fewer, not a benchmark. Your first inventory will take longer if people have used AI tools you did not know about.
Step 1: Assign AI Governance Roles
AI governance roles come first because every later step needs an owner, and a missing owner is the usual reason these programs die. Name one accountable person, not a committee. A committee can come later, once you have enough AI systems to justify one.
The owner does not need to be an AI expert. They need authority to say no to a launch and the time to run a quarterly review. At a startup that is usually the CTO, head of security or a technical co-founder.
AI Governance Roles for a Team of Fifty or Fewer
Small teams double up. The point is that every row has a name next to it, even if one person fills three rows.
| Role | Accountable for | Typical holder |
|---|---|---|
| Executive sponsor | Approves the framework and its budget; signs off on top-tier AI | CEO or CTO |
| AI governance owner | Runs the program, the inventory and the quarterly review | CTO, head of security |
| Security lead | Vendor and data-handling review; incident response | Head of security or IT |
| Legal and privacy | Regulatory mapping, customer terms, data processing terms | Counsel or outsourced DPO |
| System owners | Intended use, testing and monitoring of one AI system each | Product or engineering lead |
| Everyone else | Follow the acceptable-use rules; report problems | All staff |
Verdict: one name beats a five-person committee. If you cannot say who would be called when an AI feature misbehaves at 2 a.m., you do not have a role structure yet.
Step 2: Build the AI Inventory Your Framework Runs On
An AI inventory is a living list of every AI system your company builds, buys or allows. Everything downstream, from risk tiers to monitoring, runs off it. If a system is not on the list, it is not governed.
Include three kinds of entries:
- AI you ship: features in your product that call a model or run your own
- AI you embed from vendors: the AI features switched on inside tools you already pay for
- AI staff use on their own: chat assistants, browser extensions, note-takers
The third kind is the one teams miss. A Hacker News commenter wrote in May 2026 that the model card "quickly became a standard component of AI governance frameworks," which is a reminder that documentation habits, not just policies, are part of what an inventory feeds.
AI Inventory Fields That Customers and Auditors Ask For
Keep the schema small enough that people will actually fill it in. These fields cover what a customer or auditor is likely to ask:
| Field | Why it exists |
|---|---|
| System name and owner | Someone is accountable by name |
| Purpose and intended use | The basis for judging whether it is used as designed |
| Out-of-scope uses | What it must never be used for |
| Vendor and underlying model | You inherit your model provider's risks |
| Data it touches | Customer data, personal data, secrets, public data |
| Who it affects | Employees, customers, applicants, the public |
| Risk tier | Set in step 3 |
| Approval date and approver | Evidence the review gate ran |
| Last review date | Evidence the quarterly cycle runs |
The NIST AI RMF names this artifact directly: its GOVERN function includes keeping an inventory of AI systems. We cover the framework in our NIST AI Risk Management Framework guide.
Step 3: Set AI Risk Tiers for Your AI Governance Framework
AI risk tiers let you spend effort where it matters. A summarizer for internal meeting notes and a model that screens job applicants should not face the same review, and a framework that treats them alike will be either too heavy to survive or too light to defend.
Three tiers are enough for most small teams. Use a rule anyone can apply, not a judgment call:
| Tier | Test (any one is enough) | Minimum controls |
|---|---|---|
| High | Influences decisions about people (hiring, credit, access, pricing), or handles sensitive personal data, or acts without human review in a customer-facing flow | Executive sign-off, documented evaluation, human oversight, ongoing monitoring, quarterly review |
| Medium | Customer-facing output with a human in the loop, or touches customer data, or depends on a third-party model | Owner sign-off, vendor review, data-handling check, periodic spot checks |
| Low | Internal use, no customer or personal data, easy to reverse | Acceptable-use rules and an inventory entry |
Your internal tiers do not need to match the EU AI Act's legal categories. They need to be at least as cautious. The legal classification of a feature, including which Annex III areas count as high-risk, is covered in our EU AI Act risk classification guide. Run that test on any system you rate High, and record the result in the inventory.
Verdict: when in doubt, tier up. Moving a system down later with a written reason is easy. Explaining an under-tiered one after an incident is not.
Step 4: Write the Policies Your AI Governance Framework Needs
Policies turn the tiers into rules. Resist the urge to write one 40-page document. A short set, each answering one question, is easier to keep current and easier for staff to read.
AI Governance Framework Template: The Minimum Document Set
Treat this AI governance framework template as the paper layer of your program. Five documents cover most small-team needs:
| Document | The question it answers | Typical length |
|---|---|---|
| AI policy | What is our position, scope and the principles we hold to? | 1 to 2 pages |
| Acceptable use rules | What may staff do with AI tools, and with which data? | 1 page |
| Third-party AI rule | How do we approve and review vendor AI and model providers? | 1 page |
| AI change and release procedure | What review applies when a model, prompt or data source changes? | 1 to 2 pages |
| AI incident path | Who is called, and what happens, when an AI system fails? | 1 page, or a section of your existing incident plan |
For the policy itself, ComplyJet's AI governance policy guide includes a full template and sample text. Use it rather than starting from a blank page.
Step 5: Add a Review Gate for New AI Before It Ships
A review gate is the checklist every new AI system or major change passes before it goes live. It is where the framework stops being paperwork and starts changing decisions. Without it, the inventory is a record of what already happened.
Keep the intake to questions the owner can answer in ten minutes:
- What is the system for, and what must it never be used for?
- Which model and vendor sit underneath it, and under what data terms?
- What data goes in, and what comes out to whom?
- Who is affected if it is wrong, and how badly?
- How was it tested, and what would you rerun after a model or prompt change?
- Can a human override it, and can you switch it off in minutes?
- What tier does the rule in step 3 assign?
Then route by tier: Low tier is approved by the system owner and logged, Medium by the AI governance owner, and High by the executive sponsor. A decision is one of three outcomes: approve, approve with conditions, or reject. Log which, with a date.
Step 6: Set Up AI Governance Monitoring and Incident Response
AI governance monitoring is how you notice that an approved system has started behaving differently. A review gate checks a system once. Monitoring checks it for as long as it runs, which is where most real failures show up.
Scale the effort to the tier:
| What to watch | High | Medium | Low |
|---|---|---|---|
| Output quality against your test set | Every release plus scheduled reruns | After model or prompt changes | Not required |
| Vendor or model changes | Tracked and reviewed | Tracked | Noted at quarterly review |
| Unusual usage or data flows | Alerting | Periodic review | Not required |
| User complaints and overrides | Logged and reviewed | Logged | Logged |
| Kill switch | Required, rehearsed | Required | Not required |
Two parts deserve more attention than they usually get. First, a kill switch: a feature flag or config that turns the AI behavior off in minutes without a deploy. Second, an incident path. You likely already have one for security; extend it to cover an AI failure, such as a leaked prompt, a harmful output or a vendor outage. ComplyJet's incident management policy guide covers building that foundation.
Step 7: Review the AI Governance Framework and Keep the Evidence
A framework that is never reviewed decays into a document. Put a quarterly review on the calendar, chaired by the owner, with three standing questions: what AI did we add or change, what did monitoring show, and what should we tier up or down?
Also define triggers for an off-cycle review: a new high-tier system, a vendor switching its model or terms, an AI incident, or a change in law that touches you. Write down the decisions. Minutes are evidence.
What you keep is what a customer or auditor will eventually ask for. A short evidence folder covers it:
- The current role table and the dated policy set
- The inventory with tiers, owners and last-review dates
- Approved review-gate forms for Medium and High systems
- Test results and monitoring summaries for High systems
- Quarterly review minutes and any incident records
- Staff training records on acceptable use
Verdict: if you can produce those six items in an afternoon, you can answer almost any AI governance questionnaire honestly.
Which AI Governance Frameworks to Build On: ISO 42001, NIST AI RMF and the EU AI Act
You do not need to invent your structure. The established AI governance frameworks give you vocabulary, and your seven steps line up with all three. The table maps each building block to where it shows up.
| Building block | ISO 42001 | NIST AI RMF | EU AI Act |
|---|---|---|---|
| Roles | Clause 5 Leadership; Annex A.3 Internal organization | GOVERN | Provider and deployer role duties; AI literacy expectations |
| AI inventory | Clause 4 context and scope; Annex A.4 Resources | GOVERN (inventory of AI systems); MAP | Needed to apply any classification |
| Risk tiers | Clause 6 Planning; Clause 8 risk assessment | MAP, MEASURE | Four-tier classification; Annex III |
| Policies | Annex A.2 Policies related to AI | GOVERN | Documentation duties for high-risk systems |
| Review gate | Clause 8 impact assessment; Annex A.5 | MAP, MEASURE | Risk management system for high-risk systems |
| Monitoring | Annex A.6 and A.9; Clause 9 | MANAGE | Post-market monitoring, logging, human oversight |
| Review cycle | Clauses 9 and 10 | GOVERN | Ongoing for high-risk systems |
The mapping is my own reading, not an official crosswalk, and a certification auditor will test against the standard's exact text.
Which to choose comes down to what you need to prove:
- Build toward ISO 42001 if buyers ask for a certificate. What is ISO 42001 covers the standard, and the ISO 42001 checklist lists the controls. Already running ISO 27001? ISO 42001 vs ISO 27001 shows what carries over, and ISO 27001 and AI compliance covers the security side.
- Align to the NIST AI RMF if customers want a vocabulary and no certificate is needed.
- Add EU AI Act checks if EU customers or users are in scope.
- Otherwise, finish steps 1 to 3 this month. They cost almost nothing.
Mistakes That Break an AI Governance Framework
These are the failure modes I would watch for first:
- Starting with the policy. A policy written before the inventory describes AI you imagine, not the AI you run.
- Governing only what you build. Vendor AI and staff-adopted tools carry most small-team risk.
- A committee with no owner. Shared accountability means nobody gets called.
- One tier for everything. Uniform heavy review gets bypassed; uniform light review proves nothing.
- Approving once and never again. Model and prompt changes alter behavior after sign-off.
- No evidence trail. A program you cannot show is a program you cannot defend.
Each of these shows up as a weak answer to a customer question. Fixing them early is cheaper than fixing them in front of a buyer.
FAQs
What Is an AI Governance Framework?
It is the set of roles, inventory, risk rules, policies, review gates and monitoring that decides how a company builds, buys and uses AI. A policy is one document inside it. The framework is the program that makes the policy real.
How Do You Build an AI Governance Framework?
In seven steps, in order: assign roles, build the AI inventory, set risk tiers, write the policies, add a review gate, set up monitoring and run a quarterly review with evidence. Each step produces the input for the next.
What Are the Components of an AI Governance Framework?
Accountable roles, an AI inventory, risk tiers, a short policy set, a pre-launch review gate, monitoring with an incident path, and a review cycle with records. Larger organizations add committees and tooling on top of these.
Who Is Responsible for AI Governance in a Company?
One named owner, backed by an executive sponsor. At a startup that is typically the CTO or head of security. System owners are responsible for individual AI systems, and legal and privacy advise on regulation and contracts.
What Is the Difference Between an AI Governance Framework and a Policy?
The policy states the rules. The framework is everything that applies them: owners, inventory, risk tiers, reviews and monitoring. You need both, and the framework is what produces the evidence behind the policy.
Do Small Companies Need an AI Governance Framework?
Not as a legal matter in most cases, but any company shipping AI or using vendor AI on customer data will meet customer questions about it. A small framework answers them honestly and is quick to build.
How Long Does It Take to Build an AI Governance Framework?
My estimate for a team of 50 or fewer is a few weeks of part-time effort for the first working version, assuming no surprises in the inventory. Treat the quarterly review as the ongoing cost.
Which Framework Should You Base Yours On?
Base it on what you need to prove. ISO 42001 gives you a certificate, the NIST AI RMF gives you a shared vocabulary, and the EU AI Act sets legal duties if EU customers or users are in scope. Many teams use the NIST language and certify to ISO 42001 when asked.
Related Reading
- AI governance policy: the policy document and template inside this framework
- What is ISO 42001: the certifiable standard to build toward
- ISO 42001 vs ISO 27001: what carries over if you already run an ISMS
- EU AI Act compliance guide: requirements, deadlines and who it covers
- NIST AI Risk Management Framework: the US voluntary framework, explained
- Does SOC 2 cover AI?: where your SOC 2 report stops on AI risk





