Resources / Practitioner guide
How to prove AI controls to an auditor
Prepare AI control evidence with approval records, action logs, boundary tests and signed-release records. Includes an illustrative NIST AI RMF and SOC 2 mapping.
An evidence-preparation guide, not an audit opinion. The worked example uses test data with no real customer information. It is not a Vantage product test or customer result. Agree scope and sufficiency with your auditor.
Agree what the control must do.
Start with what the control must do, who owns it, which systems it covers and the review period. State the risk in terms of the action: an AI-generated payment request must not execute without the designated person's approval of that exact request.
Ask the auditor which population, sample and exceptions they need to examine. A successful demonstration on one build can support a narrow observation. It cannot establish that a control operated consistently across an entire period or all integrations.
Prepare evidence that lets another reviewer reconstruct the decision. A screenshot helps with orientation, but the reviewer also needs the source records and a way to check their completeness and integrity.
Assemble a traceable evidence set.
| Record | What to retain | What it cannot prove alone |
|---|---|---|
| Approval record | Keep the request ID, proposed action and target. Include a digital fingerprint of the request contents to check for changes. Record the reviewer's identity, authority, decision time and expiry. | That execution used the same request contents, or that approval could not be bypassed. |
| Action log | Shared request ID, actor, source, policy version, tool request, execution result and relevant timestamps. | That every event was captured, or that the log cannot be altered. |
| Boundary test | Test input, expected refusal, observed decision and independent check of the receiving system. | That every wording, role, integration or future build is covered. |
| Signed-release record | Keep the software version's digital fingerprint and signer identity. Identify the approved key or certificate used to trust the signature. Link the verification result to the installed settings. | That the code is correct, that the signer was authorised, or that a control operated in the running system. |
| Evidence index | Record locations, digital fingerprints, collection method, scope, retention, exceptions and responsible reviewer. | That the underlying records are complete or truthful. Verify where they came from and how they were collected. |
Work through an approval boundary.
Test example: an AI Worker prepares a payment instruction using data with no real customer information. No real payment is submitted. The following is a test protocol with expected outcomes, not a report of a completed run.
- Record the test identity, source, proposed request contents and software version. Keep the source records from before execution as a comparison point.
- Attempt execution without approval. Expect refusal and no matching action in the receiving system. Retain both observations under the same request identifier.
- Have an authorised reviewer approve a specific proposal. In the test system, execute only that proposal. Check that the executed request has the same digital fingerprint as the approved request.
- Change the target or amount after approval. Expect a fresh approval requirement or denial. A successful original run does not excuse a failed substitution test.
- Remove the worker's permission and repeat the previously allowed operation. Check both the refusal record and the receiving system. Include already queued work and the documented time for permission removal to take effect.
- Give a separate reviewer the evidence index. Ask them to reconstruct the sequence. Have them record gaps, failed checks and any unverified assumptions.
Map evidence to the review question.
This example links evidence to NIST AI RMF 1.0 and the AICPA Trust Services Criteria. The links are not an official comparison between the frameworks. They do not assert compliance or replace control design. The auditor determines which criteria and evidence are applicable.
| Evidence from the example | NIST AI RMF reference | SOC 2 family to discuss |
|---|---|---|
| Named control owner and reviewer authority | GOVERN 2.1: documented roles and responsibilities. | CC6: logical and physical access controls. |
| Request contents tied to approval and refused unauthorised execution | MEASURE 2.1: documented tests and methods; MEASURE 2.7: security evaluation. | CC6: access controls; CC7: system operations. |
| Permission removal test, refused retry and check of the receiving system | MANAGE 2.4: mechanisms to disengage or deactivate systems. | CC6: access controls; CC7: system operations. |
| Build verification, approved change and regression test | MANAGE 4.1: post-deployment monitoring and change management. | CC8: change management. |
| Missing event investigation and corrective action | MANAGE 4.3: incident and error response records. | CC7: system operations. |
Sources: NIST AI RMF Core; AICPA Trust Services Criteria.
Deliver the limitations with the result.
The handoff should name the environment, software version, test date and evidence owner. Show pass, fail or not tested for each case. Link to the supporting records. Include the population and period actually examined. Do not present a fresh document date as fresh operating evidence.
Record retention gaps, uncertain timestamps, missing observations from the receiving system and substitutions explicitly. Distinguish an interface refusing a request from the system blocking the operation itself. If you did not inspect the receiving system, say so.
Treat the evidence package as sensitive. Redact unnecessary personal data and secrets, restrict access, and keep a controlled link back to the original records. A redacted handoff should still identify the scope and limitations of the test.
Questions and answers.
Is a screenshot enough to prove an AI control?
It can show what a reviewer saw. Pair it with linked source records, system settings and checks of the receiving system. Agree with the auditor whether the evidence is complete and sufficient.
Does a signed release prove that an approval control worked?
No. Signature verification checks the signed file against an approved key or certificate. Tests of the running system and operating records are still needed to establish how a control behaved.
Does mapping evidence to NIST AI RMF establish SOC 2 compliance?
No. The frameworks serve different purposes. A mapping organises review questions. It is not an audit opinion, a certification or proof that all applicable controls operated.
What should an evidence handoff contain?
Include scope, environment, software version, date, responsible owner, test inputs, outcomes, record references and limitations. State failed or untested cases plainly. Protect sensitive records.