Skip to main content
AI governance is the set of accountability structures, decision rights, and oversight mechanisms that control how AI is selected, deployed, monitored, and retired in a regulated organisation. Guardrails are the operational limits — input filters, output thresholds, human review triggers — that prevent AI systems from producing harmful, inaccurate, or non-compliant outputs. In regulated markets — banking, insurance, healthcare, government — governance must be designed before deployment, not bolted on after. Regulators will ask who approved the use case, who monitors the outputs, and who is accountable when the system produces an incorrect result. If those answers do not exist before the system goes live, the organisation has a compliance gap.

​
Why must governance come before deployment?

Most organisations treat governance as a compliance gate at the end of a deployment process. The team builds the system, integrates it, runs a pilot, and then asks the compliance team to approve it. This sequence fails for three reasons:
  1. Compliance teams raise blockers that require redesign. If the governance framework is not considered during design, the system may collect data it should not, make decisions it cannot explain, or lack the audit trail regulators expect.
  2. Retroactive governance is superficial. A governance layer added after deployment typically covers what the system does, not why it was chosen, how alternatives were evaluated, or what limits apply. Regulators expect the full decision chain.
  3. Accountability gaps become visible under scrutiny. When a regulator or auditor asks “who approved this AI application and on what basis?”, the organisation needs a documented answer — not a retroactive memo.
The Scotland AI Age whitepaper proposed a Scottish AI Quality Mark aligned with ISO 42001 (the international standard for AI management systems). This reflects the direction of travel: regulated AI deployment will require structured, certifiable governance, not ad hoc review.

​
What does an AI governance framework include?

A governance framework for AI in regulated organisations has five components:

​
1. Use case approval

Before any AI system is built or bought, the use case must be approved through a structured process. This means:
  • Risk tiering. Classify the use case by risk level. A chatbot answering FAQ questions is low risk. An AI system making credit decisions is high risk. The governance burden should be proportionate to the risk tier.
  • Purpose definition. Document what the system will do, what data it will use, what decisions it will make or support, and who will be affected.
  • Alternative assessment. Document why AI is the right approach and what alternatives were considered.
The AI use case selection framework provides a structured scoring method for this stage.

​
2. Decision rights

Define who can approve, modify, override, and retire AI systems.
RoleResponsibility
Business ownerOwns the use case, defines requirements, accountable for outcomes
AI/ML teamBuilds or configures the system, responsible for technical performance
Risk and complianceEvaluates regulatory implications, approves risk classification
Data protection officerAssesses data handling, consent, and GDPR compliance
Executive sponsorApproves high-risk deployments, escalation point for governance disputes
In most regulated organisations, no single person can approve an AI deployment. The approval chain crosses business, technology, risk, and legal functions. Governance must define this chain explicitly.

​
3. Guardrail design

Guardrails are the operational limits that constrain AI behaviour. They fall into four categories:
  • Input guardrails. Filter or validate inputs before the AI processes them. Examples: reject prompts containing sensitive data categories, validate input data against expected schemas, rate-limit API calls.
  • Output guardrails. Review or constrain outputs before they reach the user or downstream system. Examples: confidence thresholds below which the output is routed to human review, banned content filters, output format validation.
  • Human-in-the-loop triggers. Define scenarios where the AI output must be reviewed by a human before action is taken. Examples: all decisions above a value threshold, all outputs where the model confidence is below a defined level, any output affecting a vulnerable customer.
  • Drift monitors. Track whether the AI system’s behaviour is changing over time. Examples: statistical tests comparing current output distributions against baseline, alert thresholds for accuracy degradation, scheduled model revalidation.

​
4. Audit trail

Every AI decision in a regulated environment must be traceable. The audit trail should document:
  • What input the system received
  • What output it produced
  • What model version produced it
  • Whether a human reviewed or overrode the output
  • What guardrails were active at the time
  • When the decision was made
This trail must be stored for the period required by the relevant regulator (typically 5 to 7 years for financial services) and accessible for audit or investigation.

​
5. Review and retirement

Governance does not end at deployment. Active AI systems require:
  • Scheduled reviews. Quarterly or bi-annual assessments of performance, fairness, and continued relevance.
  • Incident response. A defined process for handling AI failures, including root cause analysis, remediation, and regulatory notification where required.
  • Retirement criteria. Conditions under which the system should be switched off. Examples: performance drops below an accuracy threshold, the regulatory environment changes, the use case is superseded.

​
How should governance scale with risk?

Not every AI application requires the same governance intensity. A risk-tiered approach matches governance effort to potential impact.
Risk tierCharacteristicsGovernance requirements
LowNo customer-facing decisions, no sensitive data, advisory onlyDocumented purpose, basic guardrails, annual review
MediumCustomer-facing outputs, uses personal data, supports (not makes) decisionsFull use case approval, human-in-the-loop, quarterly review, audit trail
HighMakes or materially influences regulated decisions (credit, insurance, clinical)Executive approval, comprehensive guardrails, continuous monitoring, external audit, regulatory notification
The AI Maturity Roadmap framework stages this progression. Organisations at early stages (awareness, shadow AI) need basic governance. Organisations at later stages (supervised autonomy, role-based AI teammates) need comprehensive governance with continuous monitoring.

​
Common mistakes

  • Selecting high-risk use cases before governance exists. Starting with the highest-impact use case is tempting but dangerous. If the governance framework does not exist, the first failure will set back the entire AI programme.
  • Treating compliance as a final gate. Compliance teams need to be involved from use case selection, not invited at the end. Late involvement means late blockers.
  • Building governance on paper only. A governance policy document is necessary but not sufficient. Governance must be operationalised — embedded in workflows, automated where possible, and regularly tested.
  • Ignoring model drift. AI systems degrade. Input data changes, user behaviour shifts, and model accuracy declines. Without drift monitoring, governance is a snapshot, not a system.
  • Failing to document the decision path. Regulators do not just ask “what did the system do?” They ask “why was this system chosen, who approved it, and how was it monitored?” The full chain must be documented.
  • Applying the same governance to every use case. Over-governing low-risk applications slows adoption. Under-governing high-risk applications creates compliance gaps. Risk tiering is essential.

​
Key takeaways

  • AI governance must be designed before deployment, not added after. Regulators expect a documented decision chain from use case selection through to monitoring.
  • Five components: use case approval, decision rights, guardrail design, audit trail, and review/retirement.
  • Guardrails come in four types: input, output, human-in-the-loop, and drift monitoring. Match intensity to risk tier.
  • Risk tiering ensures governance effort is proportionate. Low-risk applications need basic guardrails. High-risk applications need comprehensive monitoring and external audit.
  • Governance is operational, not documentary. A policy without workflow integration is a compliance gap.