Internal audit of your AI management system: ISO 42001 clause 9.2 in practice

Certification bodies often start an ISO 42001 audit at chapter 9, because that is where you find out whether an organisation checks itself seriously. Clause 9.2 requires internal audits at planned intervals, plus an audit programme that steers them. A gap here means you will not pass the certification audit. This article covers how to set up that programme, what an auditor tests, and the nonconformities we run into most often.

What clause 9.2 asks of you

The standard sets two requirements that are easy to confuse. The first (9.2.1) is that you conduct internal audits establishing whether the AI management system meets the requirements of ISO/IEC 42001 and your own requirements: your AI policy, your risk treatment plan, your own procedures. The second (9.2.2) is that you maintain an audit programme covering frequency, methods, responsibilities, planning requirements and reporting. A single audit without a programme is not enough. A tidy programme with no audits performed is not either.

There is one word in 9.2.1 that separates a good internal audit from a weak one: effective. You have to show that the AIMS is set up in conformity, but also that it works. That distinction is not academic. An organisation can have an excellent AI risk assessment procedure and at the same time run three production models that were never assessed. Testing only the first produces a clean report about nothing.

The audit programme: cycle, frequency and coverage

An audit programme records how you cover every requirement across a cycle. In practice organisations work with a cycle of one to three years, within which all HLS clauses (4 through 10) and all applicable Annex A controls come up at least once. If your cycle runs longer than a year, you need to show which part is scheduled when, otherwise blind spots appear that the certification body will find immediately.

Frequency should be risk-driven. AI systems with a high risk profile, systems that directly affect people, and processes where nonconformities were raised last year get audited more often than an internal tool that suggests text. A programme that treats every process with the same frequency usually means the risk trade-off was never made.

The programme also has to move. New AI systems going live, a reorganisation of the governance team, an AI incident, a regulatory change: all of these are reasons to adjust the planning mid-cycle. A programme that stays unchanged for three years while the AI portfolio doubles is a nonconformity in itself.

Auditor independence and competence

Two requirements determine whether the audit is worth anything. The auditor may not assess their own work, and the auditor has to understand what they are looking at. The first is an organisational problem: in smaller companies the person who wrote the AI policy is often the one who knows it best. Rotating auditors between departments helps up to a point, outsourcing solves it outright. Record the trade-off, and record an independence or conflict-of-interest declaration for each audit.

Competence weighs heavier for ISO 42001 than for ISO 9001 or ISO 27001. An auditor who knows the standard but does not know what drift is, what a validation set does or how a production model is monitored will not get past the document cabinet. We find that a pair works well: one person with audit methodology, one with knowledge of the models. Record the competence requirements in a matrix and back them with training evidence.

What the auditor tests

An internal AIMS audit follows the standard, but attention goes to a number of points that are specific to AI.

For chapter 4 it is about role determination. Are you a provider, developer or user of an AI system, and possibly all three at once for different systems? That question drives the scope, and many organisations have never answered it explicitly.

For chapter 6 it is about the two assessments that sit side by side: the AI risk assessment (6.1.2), which looks at risks to the organisation and its objectives, and the AI system impact assessment (6.1.4), which looks at consequences for individuals and groups. They are often merged into one document in which the second perspective disappears.

For chapters 8 and 9 it is about operations. Have risk assessments actually been carried out for every system in scope, and are they current? Is measurement broader than accuracy and availability, covering things like fairness, how often a human overrides an outcome, and drift signals?

For the Annex A controls, attention usually goes to the AI system life cycle, the provenance and quality of data, the information you provide to users and affected individuals, and the arrangements with suppliers of models and APIs.

Questions an internal auditor asks

A few examples from our own internal audit module that you can use to practise. Can you show me the most recent AI risk assessment, with the date and the name of the person who approved it? Which AI systems fall outside the AIMS scope, and on what basis was that decided? How do you know that a model validated last year still does what it should?

And further: who is allowed to decide that an AI system goes out of production, and has that decision ever been made? What training did the people who work with these systems daily receive, and what is the content of that training based on? How was a concern or complaint about an AI outcome handled over the past year, and where is that recorded?

Evidence that goes beyond documents

Reviewing documentation is the easy half. The other half is establishing that the described process was actually followed. That means sampling: take three AI systems from the register and follow them through the whole chain. Is there an impact assessment? Was it updated after the last major change? Is there monitoring data behind it? Were out-of-range measurements followed up? Is the supplier in the contract register with the right arrangements?

For ISO 42001 there is a technical component on top. Ask for an export of monitoring dashboards, for logs of human interventions, for the test report of the last model update. An auditor who only interviews and reads documents cannot say anything about effectiveness.

Classifying findings and following up

Findings get a classification: a major nonconformity when a requirement is structurally not met, a minor nonconformity for an isolated shortcoming, plus observations and improvement opportunities. Set the criteria for that classification in advance, otherwise it becomes a negotiation.

Follow-up falls under clause 10.2. That is not only about repairing the situation, but about root cause analysis and preventing recurrence. We often see the containment happen (the missing document gets written after all) and the root cause analysis not. Twelve months later the same finding is back in the report. Also verify that the action worked: a corrective action without an effectiveness check is not closed.

The nonconformities we see most often on 9.2 itself

Audits that review documentation only and never test whether processes are executed. An audit programme that does not exist or was never documented, leaving audits to happen ad hoc. Annex A controls left out of the audit cycle because all attention went to the HLS chapters. Auditors without demonstrable AI knowledge. Findings that are not classified, so nobody can see whether things are improving. And audits that test conformity but never ask whether the AIMS achieves its purpose.

The link with the EU AI Act

For providers of high-risk AI systems a quality management system is not optional. Article 17 of Regulation (EU) 2024/1689 requires one, including procedures for design, verification, data governance, risk management, post-market monitoring and incident reporting. A working AIMS with a serious internal audit programme delivers a large part of that evidence. If you first want to know which risk category your systems fall into, use the self-assessment at /ai-act-check.

Source: Regulation (EU) 2024/1689 (AI Act), Article 17 (quality management system), via EUR-Lex: https://eur-lex.europa.eu/eli/reg/2024/1689/oj

How we approach it

Secure Audit performs internal audits for ISO 42001, ISO 27001, NEN 7510, ISO 27701 and the other management system standards. Our audit platform holds, per norm element, the criterion, a set of interview questions, a description of what a well-implemented situation looks like, the common nonconformities and the documents you can expect. That makes it possible to plan the audit programme across several standards and test overlapping clauses once. Get in touch if you want to discuss what your AIMS audit cycle could look like.