Automating SOC 2 Evidence Collection: The Continuous-Compliance Playbook

Every SOC 2 audit cycle, the same thing happens. The audit window opens, the request list arrives, and your most senior engineers spend two to six weeks screenshotting access lists, exporting change logs, and hunting for the Q2 backup-restore test that somebody definitely ran. The controls were operating all year. Proving it becomes a quarter's worth of interruptions.

The fix isn't working harder during the window. It's collecting the evidence continuously, automatically, all year, so the audit window opens onto a binder that already exists. Here's the playbook we build for clients, including the honest part about where compliance platforms like Vanta and Drata already solve this and where they don't.

Why evidence collection eats so much time

SOC 2 evidence has three awkward properties. It's scattered (identity provider, cloud console, ticketing, code host, HR system, monitoring). It's point-in-time (the auditor wants proof the control operated throughout the period, not just today). And it's owned by nobody (the engineer who can export the access review isn't the person who tracks which control it maps to).

There is a fourth property that only becomes obvious during your first Type II, and it changes what you should build. An auditor does not ask for evidence, they ask for a population and then sample from it. "Give me every access change in the period" comes first; "now show me the approval for these twelve" comes second. A tidy folder of twelve approvals is worth nothing if you cannot produce the population it was drawn from, because then nobody can tell whether those twelve were the twelve that went well.

That is the argument for collectors over screenshots, and it is the whole argument. A scheduled job that appends every event to a table produces a defensible population as a side effect. A person exporting artifacts at audit time produces a sample they chose themselves.

So the default process is archaeology: reconstruct a year of control operation from whatever artifacts survive. The expensive part isn't the exporting. It's that reconstruction fails in a specific way. You go looking for proof the Q2 access review happened, find a Slack thread where three people agreed it looked fine, and now have to decide whether that's evidence. Sometimes you rebuild the review from scratch against six-month-old data, which is worse than useless as proof and still takes a day. The hours land in the hundreds per cycle for a mid-size engineering org, and they land on the senior people, because they're the ones who know which console holds what.

What's automatable (most of it)

Map your controls against this list and you'll usually find 70 to 80% of your evidence requests can be collected by scheduled jobs:

• Access reviews (CC6.1, CC6.2). Weekly pull of users, roles, and MFA status from your identity provider (Okta, Google Workspace, Entra), diffed against last week, filed with a timestamp. Quarterly review sign-offs get requested and chased automatically.
• Change management (CC8.1). Every production deploy captured from GitHub or GitLab with its PR, approver, and CI status. The "was every change reviewed?" question becomes a query, not an investigation.
• Backup and recovery (A1.2). Nightly backup-completion confirmations logged; restore tests scheduled quarterly with the result filed against the control.
• Monitoring and incident response (CC7.2). Alert-config snapshots, uptime reports, and incident tickets with their timelines, pulled from your monitoring stack and tracker.
• Vendor management. Subprocessor SOC 2 reports and certificates tracked with expiry dates, renewal chases sent automatically. (Same engine as the vendor due-diligence pipeline in our financial services work.)
• Policy attestations. Annual acknowledgment campaigns with escalating reminders and a completion ledger.
• Offboarding (CC6.3). The control auditors probe hardest, because it has a clock on it. An HR termination event triggers access-revocation checks across every system, with the completed checklist filed as evidence. This one doubles as an actual security control, not just proof of one.

What stays manual is the judgment layer: risk assessments, policy content, control design decisions, and the conversations with your auditor. That's maybe 20% of the effort, and it's the 20% that deserves human time.

The architecture

Every build follows the same shape:

1. Scheduled collectors hit each system's API on a control-appropriate cadence (weekly for access, per-event for changes, nightly for backups).
2. An evidence store (usually a structured bucket or database your team already runs) files each artifact against its control ID, timestamped and immutable.
3. Attestation workflows handle the human-signature evidence: request, remind, escalate, file.
4. A gap monitor checks for controls with stale evidence and alerts before the staleness becomes an audit finding. This is the piece that turns collection into continuous compliance: you find out in March that the restore test didn't run, not in November from your auditor.

When the request list arrives, fulfillment is an export grouped by control rather than a fortnight of screenshotting.

The leverage is in the audit log doing double duty. If every PHI touch already writes a structured entry with the user, the action, the element touched and a timestamp, then the access-review evidence and the chain-of-custody report are two queries over the same table, not two separate collection exercises. Our prior-auth worked example shows that logging layer in context.

"Doesn't Vanta already do this?"

Partly, and if you have Vanta or Drata, keep it. The platforms handle the standard integrations well: mainstream identity providers, big-three clouds, GitHub, common HR tools. For a SaaS company running an entirely conventional stack, they cover most of the surface.

The gaps show up where your stack stops being conventional, and that's where we get called in:

• Self-hosted and internal systems. The platforms can't see your on-prem database, your internal admin tool, or your self-hosted automation instance. Custom collectors close that.
• Evidence the integration collects wrong. Platform integrations pull what's available via API, which isn't always what your auditor asked for. A custom collector pulls exactly the artifact, in exactly the format.
• Attestation and process evidence. Restore tests, tabletop exercises, access-review sign-offs: things a human did that need documenting. Platforms nag about these; workflow automation actually runs them.
• The multi-framework problem. If you're carrying SOC 2 plus HIPAA plus FINRA obligations, one evidence store feeding all three beats three parallel scrambles. The overlap is larger than it looks: access reviews and change management satisfy controls in every framework you'll carry, they just get asked for in different formats. That mapping layer is most of what compliance automation in regulated industries turns out to be.

The pattern: platform for the commodity 70%, custom automation for the 30% that's specific to your stack. Fighting the platform to cover the last 30% wastes more time than building collectors for it.

The math

A typical build runs ~50 hours at ~$7,500 fixed-fee, covering collectors for the big evidence families plus the attestation engine and gap monitor. What we've found drives that number is not the count of controls but the count of systems: two evidence families out of one API is a fraction of the work of two out of two, and any system without a usable API is where the hours actually go. Set that against the senior-engineer hours your last cycle consumed and payback is one audit, then it repeats every year. Firms carrying multiple frameworks or annual Type II reports get there sooner. The IT and security catalogue prices the individual pieces if you want to phase it.

The return nobody counts is calendar time. Audit windows stop displacing roadmap work, because the evidence was never not-collected.

Start with two, not twelve

Don't try to cover the framework. Take the two evidence families that hurt most in your last audit, which for nearly everyone means access reviews and change management, automate those collectors, and let the next cycle prove the model before you expand.

There's a reason to start there beyond the pain. Those two are the families where the evidence is already structured and already timestamped by somebody else's system, so the collector is a scheduled API call and a file write rather than a design problem. You'll have them running in days, which matters, because the first collector is the one that has to convince a skeptical engineering lead that this isn't another compliance project that generates work.

Then wire the gap monitor before you add the third family. It's tempting to skip, since with two collectors running you can eyeball whether they're working. The problem is that collectors fail silently in exactly the way the original evidence did: a rotated API token, a permission scope changed during an access review, an endpoint deprecated in a version bump. The job keeps running, writes nothing, and reports no error. Nine months of confidence, one empty folder, and you find out from your auditor.