How to read and assess a SOC 2 or ISAE report as a customer

More and more organizations ask their vendors for a SOC 2 or ISAE 3402 report, and rightly so: it is one of the few ways to obtain independently tested information about the controls at an outsourced service. But then a report of eighty or more pages lands in your mailbox, and the real work begins. In practice we see many customers file the report as a piece of evidence ("they have one, box ticked") without reading it. That way you miss exactly the information the report was made for, and you also run the risk that your own responsibilities go unnoticed. In this article we offer a practical reading guide, based on our daily work as IT auditors.

How is an assurance report structured?

SOC 2 and ISAE reports largely follow the same structure, in four to five sections. Section one is the auditor's assurance report, containing the opinion. Section two is the management assertion, in which the service organization itself declares that the system description is accurate and the controls were effective. Section three is the system description: the service, the infrastructure, the processes, the risks and the controls, written by the service organization itself. Section four contains the controls, the auditor's tests and the test results. Some reports have a section five with additional information from management; that section falls outside the auditor's opinion and therefore carries no assurance value.

Important to realize: the system description is written by the vendor, not by the auditor. The auditor tests whether that description is accurate and whether the controls operate, but the tone and completeness of section three are the organization's own work.

Step 1: start with the opinion

First turn to the auditor's assurance report and read the opinion. An unqualified opinion means that the description is accurate and the controls were suitably designed, and for a Type II also operated effectively, in all material respects. A qualified opinion means the auditor reached a different conclusion on specific, explicitly named elements. In that case, read exactly which controls or criteria are affected and whether they are relevant to the service you use. An adverse opinion or a disclaimer of opinion is rare and a direct signal to talk to your vendor.

Also note the standard under which the report was issued (SOC 2 under the AICPA standards, ISAE 3402 for processes relevant to financial reporting, ISAE 3000 for broader subject matter) and who issued it: a qualified, independent audit firm.

Step 2: check scope, period and type

The best report is worthless if it does not cover what you actually use. Check three things. First, the service scope: is the service your organization uses actually included in the system description? Large vendors often have multiple reports for different products or data centers. Second, the criteria: in SOC 2, Security is always covered, but Availability, Confidentiality or Privacy only if the vendor chose to include those categories. If you process sensitive personal data with this vendor, check whether Confidentiality and Privacy are in scope. Third, the type and the period: a Type I only addresses design and implementation at a single date; a Type II tests operating effectiveness over a period. Check whether the observation period matches your period of use. If there is a gap between the end of the period and today, ask for a bridge letter, in which management states whether there have been relevant changes or incidents since.

Step 3: read the exceptions, not just the summary

The heart of a Type II report is section four: for each control it describes which tests the auditor performed and what the outcome was. Wherever something deviated, an exception is described, for example: for three of the twenty-five changes tested, approval prior to implementation was missing.

An exception is not automatically a disqualification. Auditors work with samples and materiality; a single deviation on a control with compensating measures can perfectly well coexist with an unqualified opinion. What to look at as a customer: how many exceptions are there, are they clustered in one domain (access management, for example), do they affect controls that are critical to your service, and what does the management response say? A mature vendor acknowledges the deviation, names the cause and describes the remediation. A response that argues the finding away often says more than the finding itself. In an annual cycle, also compare with the previous report: recurring exceptions on the same subject point to a structural problem.

Step 4: take the CUECs seriously

Every report contains a list of complementary user entity controls (CUECs): measures the service organization assigns to you as the customer and that the auditor assumed to be in place when forming the opinion. Classics: the customer is responsible for managing its own user accounts and permissions in the service, for promptly reporting departed employees and for the accuracy of the data it supplies.

This is the most skipped part of the report, and at the same time the part with the most direct consequences. If you do not implement the CUECs, the auditor's opinion rests on an assumption that does not hold at your end, and the control of the chain leaks anyway. Our advice: walk through the CUEC list, assign an owner within your own organization for each CUEC and periodically verify that the measure actually operates. For your own auditor (for example in your ISO 27001 or DigiD audit) this also serves as demonstrable vendor oversight.

Step 5: check what was carved out

Almost every service organization outsources as well, usually at least the hosting. For those subservice organizations there are two methods. Under the inclusive method, the subservice organization's controls are included in the report and therefore tested. Under the far more common carve-out method, they are left out of scope: the report describes which controls are expected at the subservice organization (complementary subservice organization controls), but the auditor tested nothing there.

For you as a customer, a carve-out means: the controls at, say, the cloud provider were not tested in this report. Check which parties were carved out and whether the vendor demonstrably monitors them, for example by reviewing those parties' own SOC reports. For critical chains it may be necessary to request the subservice organization's report yourself.

Document your assessment

Assessing an assurance report is a control in its own right, and one you want to be able to demonstrate. Record briefly per vendor: which report was received, which period it covers, what the opinion was, which exceptions are relevant to your service and what follow-up was agreed, how the CUECs have been assigned, and when the next report is expected. That turns the report into what it should be: not a checkbox in a file, but an annual measuring point in your vendor management.

Conclusion

A SOC 2 or ISAE report is a rich source of information, but only for those who actually read it. Start with the opinion, check scope and period, weigh the exceptions in the context of your service, assign the CUECs and know what was carved out. Anyone who walks through these steps annually gets more assurance out of the chain than many an internal audit delivers. Need help assessing your vendors' reports, or do you want to obtain a report for your own service organization? Feel free to contact us.