Resources / Practitioner guide
How to prove AI controls to an auditor
Prepare AI control evidence with approval records, action logs, boundary tests and signed-release records. Includes an illustrative NIST AI RMF and SOC 2 mapping.
An evidence-preparation guide, not an audit opinion. The worked example is synthetic and is not a Vantage product test or customer result. Agree scope and sufficiency with your auditor.
Agree what the control must do.
Start with the control objective, its owner, the system boundary and the review period. State the risk in operational terms: an AI-generated payment request must not reach execution without the designated person's approval of that exact request.
Ask the auditor which population, sample and exceptions they need to examine. A successful demonstration on one build can support a narrow observation. It cannot establish that a control operated consistently across an entire period or all integrations.
Prepare evidence that lets another reviewer reconstruct the decision. A screenshot helps with orientation, but the reviewer also needs the source records and a way to check their completeness and integrity.
Assemble a traceable evidence set.
| Artefact | What to retain | What it cannot prove alone |
|---|---|---|
| Approval record | Request identifier, proposed action and target, payload hash, reviewer identity, authority, decision time and expiry. | That execution used the same payload, or that approval could not be bypassed. |
| Action log | Correlation identifier, actor, source, policy version, tool request, execution result and relevant timestamps. | That every event was captured, or that the log cannot be altered. |
| Boundary test | Test input, expected refusal, observed decision and independent check of the target state. | That every wording, role, integration or future build is covered. |
| Signed-release record | Build digest, signer identity, approved trust root and signature verification result, linked to deployed configuration. | That the code is correct, that the signer was authorised, or that a runtime control operated. |
| Evidence manifest | Artefact paths and hashes, collection method, scope, retention, exceptions and responsible reviewer. | That the underlying records are complete or truthful. Verify their provenance and collection path. |
Work through an approval boundary.
Synthetic example: an AI Worker prepares a payment instruction in a test system. No real payment is submitted. The following is a test protocol with expected outcomes, not a report of a completed run.
- Record the test identity, source, proposed payload and build. Keep the source state before execution as a comparison point.
- Attempt execution without approval. Expect denial and no matching target-side action. Retain both observations under the same request identifier.
- Have an authorised reviewer approve a specific proposal. In the test system, execute only that proposal and compare the execution payload with the approved hash.
- Change the target or amount after approval. Expect a fresh approval requirement or denial. A successful original run does not excuse a failed substitution test.
- Revoke the worker's permission and repeat the previously allowed operation. Check both the denial record and target state. Include already queued work and the documented revocation window.
- Give a separate reviewer the evidence manifest. Ask them to reconstruct the sequence and record gaps, failed checks and any unverified assumptions.
Map evidence to the review question.
This is an illustrative mapping to NIST AI RMF 1.0 and the AICPA Trust Services Criteria. It is not an official crosswalk, an assertion of compliance or a substitute for control design. The auditor determines which criteria and evidence are applicable.
| Evidence from the example | NIST AI RMF reference | SOC 2 family to discuss |
|---|---|---|
| Named control owner and reviewer authority | GOVERN 2.1: documented roles and responsibilities. | CC6: logical and physical access controls. |
| Approval-bound payload and rejected unauthorised execution | MEASURE 2.1: documented tests and methods; MEASURE 2.7: security evaluation. | CC6: access controls; CC7: system operations. |
| Revocation test, denied retry and target-state check | MANAGE 2.4: mechanisms to disengage or deactivate systems. | CC6: access controls; CC7: system operations. |
| Build verification, approved change and regression test | MANAGE 4.1: post-deployment monitoring and change management. | CC8: change management. |
| Missing event investigation and corrective action | MANAGE 4.3: incident and error response records. | CC7: system operations. |
Sources: NIST AI RMF Core; AICPA Trust Services Criteria.
Deliver the limitations with the result.
The handoff should name the environment, build, test date and evidence owner; show pass, fail or not tested for each case; and link to the supporting records. Include the population and period actually examined. Do not present a fresh document date as fresh operating evidence.
Record retention gaps, clock uncertainty, missing target-side observations and substitutions explicitly. Distinguish a request refused by the interface from a denied operation at the enforcement point. If you did not inspect downstream state, say so.
Treat the evidence package as sensitive. Redact unnecessary personal data and secrets, restrict access, and keep a controlled link back to the original records. A redacted handoff should still identify the scope and limitations of the test.
Questions and answers.
Is a screenshot enough to prove an AI control?
It can show what a reviewer saw. Pair it with correlated source records, configuration and target-state checks, and agree evidence completeness and sufficiency with the auditor.
Does a signed release prove that an approval control worked?
No. Signature verification identifies the signed artefact relative to a trust root. Runtime testing and operating records are still needed to establish the behaviour of a control.
Does mapping evidence to NIST AI RMF establish SOC 2 compliance?
No. The frameworks serve different purposes. A mapping organises review questions; it is not an audit opinion, a certification or proof that all applicable controls operated.
What should an evidence handoff contain?
Include scope, environment, build, date, responsible owner, test inputs, outcomes, artefact references and limitations. State failed or untested cases plainly and protect sensitive records.