Continuous auditing: what it is, what it delivers and when it makes sense

Continuous auditing is an approach in which the auditor tests controls at a fixed frequency throughout the year, instead of once a year after the fact. The term is used loosely, covering everything from fully automated data analysis to a quarterly rhythm in which the organization delivers evidence and the auditor reviews it. Both variants have their place, but they deliver different things. This article explains what continuous auditing is and is not, how it relates to an ISAE 3402 or SOC 2 report, and when it is worth the effort.

Three terms that get mixed up

Three terms are used interchangeably in practice. Continuous monitoring is a management task: the first or second line checks on an ongoing basis whether controls work, for example with daily reports on failed backups or a weekly check for new administrator accounts. COSO 2013 calls this the monitoring activities component, with ongoing and separate evaluations.

Continuous auditing is a task for the audit function, internal or external. The auditor decides which controls to test, how often, and what evidence to expect. Continuous assurance is the combination of the two: the auditor relies on management's monitoring where that is justified and focuses his own work on what it does not cover. The IIA guidance GTAG 3 describes this relationship in detail.

The distinction is more than terminology. If management does not monitor, the auditor has to do everything himself and continuous auditing becomes expensive. If management does monitor, the auditor's first question is whether that monitoring is reliable enough to rely on.

How it works in practice

There are two variants. In the data-driven variant, the auditor or management runs rules over the full population. Every change in access rights is compared with the authorization matrix, every production change with an approved change request. Exceptions go on a list that someone has to follow up. This requires access to source data, stable systems and rules tested beforehand on historical data. Without that preparation it mainly produces false alerts.

In the periodic variant, the auditor selects the key controls and sets a frequency for each, with a description of the evidence expected per period and a deadline. Think of a quarterly review of user access rights, a monthly reconciliation of the change log with the change requests, or a half-yearly backup restore test. The organization delivers per period, and the auditor reviews and approves or rejects with an explanation. Whether this works depends on a clear description of the evidence. How to set that up is covered in the article on evidence management in IT audits.

For most organizations the periodic variant is the realistic starting point. The data-driven variant is an extension for a limited number of controls where the data is easy to access, such as access management and change management.

What it does and does not deliver

The main benefit is that a control that is not performed stands out within one period, not at the annual audit. A missed access review in the second quarter is then known in July and can still be remediated, instead of appearing in January as an exception in the report. It also shifts work from one busy audit period to a rhythm across the year, for both the organization and the auditor.

What it does not do is replace the annual opinion. For an ISAE 3402 type II or SOC 2 type II, the service auditor tests operating effectiveness over the entire reporting period and must gather sufficient evidence himself. Evidence delivered and reviewed per period helps, because it is already available for the whole period, but the auditor still has to assess whether it is complete and accurate. How that period works is covered in the article on the ISAE 3402 type II observation period.

Full population analysis does not automatically give more assurance at lower cost either. The rules have to be designed, validated and maintained, and every exception has to be investigated. It does produce a different kind of evidence than a sample, and for large populations with simple, unambiguous rules that evidence is often stronger.

When it makes sense

Continuous auditing suits organizations with many recurring controls at a fixed frequency, organizations that need a type II report or certificate every year, and organizations that received findings in earlier audits along the lines of "control not performed for part of the period". It also suits organizations that have to meet several standards at once, because one piece of evidence delivered per period can then serve more than one standard.

It makes little sense for a small organization with a handful of controls, for a one-off assessment with no follow-up, or for processes that are still changing so much that the controls are set up differently every quarter. In that case, settle the processes first and start monitoring afterwards.

How the Secure Audit platform supports this

The Secure Audit audit platform supports the periodic variant. For each control, the auditor sets up a monitoring control with a frequency from weekly to annual, a work instruction and a description of the expected evidence. The platform turns this into periods with a deadline. The client delivers evidence per period, the auditor approves or rejects it with an explanation, and at the next audit or surveillance visit the evidence for the whole period is ready. The platform does not read the client's systems in real time; it is periodic monitoring with evidence delivered by the client.

Getting started

Pick three to five key controls, preferably those where earlier audits found exceptions. For each control, record what the evidence per period is, who delivers it and when. After the first quarter, assess whether the evidence was usable and whether the frequency is right, and only then expand. A monitoring programme that runs well with five controls is worth more than one with fifty controls, half of which are delivered late or incomplete.