Skip to content

GRF · AI and Digital Governance

AI oversight and accountability

When an AI system causes harm, someone must be able to find out what happened and answer for it. Oversight relies on documentation, logging, audit, incident reporting and authorities with the powers and skills to use them. Accountability is spread across developers, deployers, boards and regulators, and can dissolve between them. This page sets out how AI oversight holds or fails, the records that show it, and who holds each decision.

02The question

AI oversight and accountability: how the risk develops.

Roles, validation and human review. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

Mechanism

Responsibility diffuses along the chain

A model developer, a fine-tuner, an integrator and a deploying organisation each shape a system’s behaviour. When obligations attach to only one of them, the others can point elsewhere after harm.

Mechanism

Records decide what can be reviewed

Without technical documentation, logs and change records, auditors and regulators cannot reconstruct how a system reached an output. Record-keeping duties make later oversight possible.

Mechanism

Authorities lack capacity

Market surveillance, data protection and sector regulators need technical staff, access to systems and powers to test them. Thin capacity turns rules into self-assessment.

Mechanism

Incidents go unreported

Failures surface through complaints, journalism and litigation rather than reporting. Mandatory incident reporting and shared incident databases let patterns be seen across deployments.

03Monitor

Public algorithm register.

The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.

Public algorithm register

AI oversight and accountability · Algorithm register · Quarter 2

Two high-risk systems without assessmentGRF · AI and Digital Governance

Registered systems

Benefit fraud flaggingSocial agencyJuuMissingPartial
Permit triagePlanningLimitedDoneYes
School placementEducationJuuDoneYes
Chat assistantTax officeMinimalDonen/a

Oversight action

Mfumo
Benefit fraud flagging
Required
impact assessment, appeal route
Deadline
73 days
Authority
Oversight body

Complaints about automated decisions

Public algorithm register: the screen’s structure, terms and states for AI oversight and accountability.Example names and illustrative values; a design, not a record or a live system.
04Viashiria

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

IndicatorUnitCadenceSource typeLabel
Designated AI supervisory or market surveillance authorityyes or no, with legal basison changeNational legislation and notificationsPublished statistics
Technical staff in AI-relevant authoritiesfull-time equivalentsannualAuthority annual reportsIMERIPOTIWA
Serious incidents reported under AI rulescount by sectorannualPublic recordsPublished statistics
Publicly recorded AI incidentscount by harm typemonthlyIncident databasesIMERIPOTIWA
Enforcement actions concerning AI systemscount by authority and typeannualRegulator enforcement registersPublished statistics
ISO/IEC 42001 certificates issuedcountannualCertification body recordsIMERIPOTIWA
05Policy scenarios

Compare instruments: rules, incentives and information.

Three families of instruments side by side against the baseline, each with its own lag and adoption speed, under the coverage and compliance you set; effect sizes are teaching values to replace with evidence. Scenario: named assumptions about the future, not a forecast.

Controls

Baseline trend of the outcome
2.0% a year
Coverage of the instrument
70%
Uzingatiaji wa kanuni
75%
Effect size, against the teaching value
100%
Years ahead
20 years

Readouts

Largest reduction at the horizon
Regulation
Its reduction against the baseline
21%
Numbers, methods and sources
YearHali ya msingiRegulationEconomic incentiveInformation and capacity
0100.099.598.899.1
1102.0101.2100.199.8
2104.0102.5101.199.9
3106.1103.5101.6100.0
4108.2103.7101.8100.7
5110.4103.2101.7102.1
6112.6102.0101.6103.9
7114.9100.4101.6105.9
8117.299.2102.1108.0
9119.598.6102.9110.1
10121.998.9104.2112.3
11124.399.7105.7114.5
12126.8101.1107.4116.8
13129.4102.7109.4119.2
14131.9104.5111.4121.6
15134.6106.5113.5124.0
16137.3108.5115.7126.5
17140.0110.7118.0129.0
18142.8112.9120.4131.6
19145.7115.1122.8134.2
20148.6117.4125.2136.9

Mbinu

  1. Tools of government Hood (1983), The Tools of Government, Macmillan
  2. Carrots, sticks and sermons Bemelmans-Videc, Rist and Vedung (1998), Transaction
  3. Diffusion of innovations Rogers (2003), Diffusion of Innovations, Free Press
  4. Uncertainty fan (parameter Monte Carlo) Spiegelhalter, Pearson and Short (2011), Visualising uncertainty about the future, Science 333
  5. One-at-a-time sensitivity (tornado) Saltelli and others (2008), Global Sensitivity Analysis: The Primer, Wiley

Data

The model runs on the teaching values set by the controls. To calibrate it for a place, read it against these admitted layers and series, which this page’s map can draw:

  • Borders Natural Earth
  • Cities Natural Earth
Compare instruments: rules, incentives and information. Illustrative teaching model: it runs on the values set by the controls, not on measured data. Methods: Tools of government (Hood (1983), The Tools of Government, Macmillan); Carrots, sticks and sermons (Bemelmans-Videc, Rist and Vedung (1998), Transaction); Diffusion of innovations (Rogers (2003), Diffusion of Innovations, Free Press).
06Vyanzo

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

ChanzoWhat it givesLicence class
OECD.AI Policy ObservatoryNational AI policies, governance bodies and the AI Incidents MonitorFree, OECD terms
AI Incident DatabaseCrowd-sourced records of harms involving AI systemsOpen, licence to check
European Data Protection Board and national authority decisionsDecisions on automated processing and AI-related complaintsPublic, EU and national reuse terms
Stanford AI Index ReportData on AI development, policy and incidentsFree to read, licence to check
Regulator annual reportsStaffing, priorities and enforcement of authorities supervising AIPublic records, terms per authority
07Utekelezaji

From mandate to evaluation, by pillar.

The steps an instrument takes from mandate to evaluation, each with the pillar that runs it, and the decisions that stay with the authority. Teaching model with stated illustrative values, not a prediction.

  1. Mandate and problem framing Competent authority (outside the programme)
    A competent authority frames the problem and the mandate under its own rules. Howlett, Ramesh and Perl (2009)
  2. Evidence and system model Foundry
    The system is modelled with its feedbacks, data and stated limits. OECD (2014), principle 2
  3. Options appraised with scenarios Foundry
    Instruments are compared under scenarios, with uncertainty shown. Hood (1983)
  4. Exercise with stakeholders Universe
    The options are tested in an exercise with those who would implement and bear them. OECD (2014), principle 3
  5. Standard or guidance drafted Nexus Standards
    What works is codified as a standard or guidance, open to comment. Nexus Standards process
  6. Decision by the authority Competent authority (outside the programme)
    The authority decides, outside the programme, with the evidence on record. Howlett, Ramesh and Perl (2009)
  7. Capacity and training Risk Academy
    Those who implement are trained on the decided instrument and its data. Sendai Framework (2015), priority 2
  8. Monitoring and public report Reports
    Indicators are published with sources, limits and corrections. OECD (2014), principle 5
  9. Evaluation and revision Nexus Standards
    Results return to the standard and the model: the cycle closes. Howlett, Ramesh and Perl (2009)

Mbinu

  1. Policy cycle Howlett, Ramesh and Perl (2009), Studying Public Policy, Oxford University Press
  2. Governance of critical risks OECD (2014), Recommendation of the Council on the Governance of Critical Risks
  3. Layered drawing of directed graphs Sugiyama, Tagawa and Toda (1981), IEEE SMC 11
From mandate to evaluation, by pillar: the steps as recorded, each with its pillar; sources as cited on each step.
08Decisions

Who decides, and what the evidence serves.

AI and Digital Governance prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

Nani anaamuaThe questionEvidence this page serves
LegislaturesWhich obligations attach to developers, deployers and providers of general-purpose modelsComparative regimes and evidence from incidents
Market surveillance and sector regulatorsWhich systems to inspect and what powers to useIncident records, risk categories and enforcement practice
Boards of deploying organisationsWho owns AI risk and what assurance they requireGovernance practice and management system standards
CourtsWho is responsible for harm caused with an AI systemRecords and expert evidence presented by parties; the page serves context only

Outputs this topic produces

  • Maps of accountability across the AI value chain for a given use
  • Comparative notes on AI supervisory authorities and their powers
  • Guide to AI incident reporting regimes and databases
  • Questions for boards and regulators on AI oversight
09Instruments and commitments

How AI oversight and accountability is governed: instruments, mandates and commitments.

The instruments, mandates, actors and accountability arrangements a policymaker, diplomat or development agency works with, citing the agreements that set them. Structures follow the frameworks named. Any state or progress drawn is illustrative, not a record.

Accountability cycle

How commitments on AI oversight and accountability are held to account, from commitment to correction.

  1. CommitGovernmentA commitment with a mandate and a legal basis.
  2. PlanLead agencyA national plan with owners and milestones.
  3. ResourceMinistry of financeA budget line or financing on record.
  4. ImplementAgencies and local governmentDelivery against the plan.
  5. ReportStatistics officeIndicators reported in agreed definitions.
  6. ReviewLegislature, audit, civil societyIndependent review of results and gaps.
  7. CorrectGovernmentChanges to the plan, published with reasons.
A generic accountability cycle; the owner of each step follows the country’s own constitution and law.
Sendai Framework for Disaster Risk Reduction · UNDRRParis Agreement · UNFCCC2030 Agenda and the Sustainable Development Goals · United NationsInternational Health Regulations · WHO
10Core questions

What institutions ask about oversight.

  • Who in the institution owns the decision to deploy this system, and who answers if it causes harm?
  • Has the system been validated for this specific use and population, by someone independent of its developers?
  • Can the people reviewing its outputs genuinely disagree with it, or do they simply approve what it suggests?
  • How will performance be monitored once the system is in use, and what triggers suspension?
11Mechanisms

Why accountability erodes.

Accountability tends to dissolve gradually. Each actor in a long chain of developers, integrators, operators and managers can reasonably point to another as responsible, until no one is. Why now: human oversight duties are being written into law, including in the European Union AI Act (2024), and public bodies must show who can understand and override an automated output. The platform is designed to convene those bodies and to exercise oversight in Universe with a scenario in which a system is confidently wrong.

Automation bias

Rubber-stamp review

People tend to defer to automated outputs, especially under time pressure, so human review can become a formality.

Diffusion

Many hands

Developers, vendors, integrators and users each own part of the system and none owns the outcome.

Drift

Changing performance

Models degrade as the world changes, and without monitoring the degradation goes unnoticed.

12Methods and limits

Instruments of oversight.

  1. 01

    Impact assessments

    Structured analysis of risks to rights and outcomes before deployment. Limits: can become paperwork if not tied to decisions.

  2. 02

    Independent validation

    Testing by people independent of developers, using AI evaluation and assurance engineering. Limits: tests show performance on test data, not in every real situation.

  3. 03

    Meaningful human review

    Reviewers with the authority, time, information and training to override. Limits: costly, and hard to sustain at scale.

  4. 04

    Audit trails and monitoring

    Logs of inputs, outputs and decisions, with performance tracked over time. Limits: only useful if someone reviews them.

13Mbinu

How it is studied, and what the methods cannot say.

  1. AI evaluation and assurance engineeringThird-party audit scope · Fairness testing in public systems · Post-deployment monitoringApplied here
  2. Policy, institutional and decision scienceComparing AI regimes · Value chain accountability mapping · Forum and mandate mappingApplied here
  3. Artificial intelligence and machine learningHow models fail in deployment · Reading model documentation · Agent architectures explainedApplied here
14Maswali

Questions and answers.

What is the difference between an AI provider and a deployer?
In the EU AI Act, a provider develops an AI system or model and places it on the market; a deployer uses it under its authority. Each carries different obligations.
Why does logging matter for AI oversight?
Logs and documentation let auditors, regulators and courts reconstruct how a system behaved. Without them, accountability depends on what the operator chooses to disclose.
Does ISO/IEC 42001 certify that an AI system is safe?
No. It certifies that an organisation runs a management system for AI meeting the standard’s requirements. The safety of a particular system needs its own evaluation.
Is having a human in the loop enough?
Not by itself. Review is meaningful only if the reviewer has the authority, time, information and competence to disagree with the system and does so when warranted.
Who should validate an AI system?
Someone independent of those who developed or sold it, with access to relevant data and the expertise to test it for the intended use.
Does the platform audit AI systems?
No. It compares governance practices and does not test or certify any system.
What should trigger suspending a system?
Institutions set their own triggers, typically including evidence of error rates above an agreed level, unexplained shifts in outputs or credible complaints of harm.
RipotiChangia

Changia ushahidi kutoka ukurasa huu.

Reports ni chapisho la kila robo mwaka la Muungano. Unachojua kuhusu somo hili kinaweza kuwa ushahidi, kisa au sahihisho katika toleo lijalo, kikitajwa kwa jina lako na kuchapishwa chini ya CC BY 4.0 pale ambapo ni kazi yako mwenyewe.

Global Risks ForumAI and Digital GovernanceExponential TechnologyEvidence Brief
Akili Bandia na Utawala wa Kidijitali

Bring a governance question to the Forum.

Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.