Automated Compliance Evidence: From Collection to Audit Pack

Automated compliance evidence collection turns recorded infrastructure activity into reviewable control evidence for a defined period. A useful collection preserves the source records, population, sample, exceptions, and scope so a reviewer can trace a conclusion back to what happened. It reduces repeated export and screenshot work while keeping the control owner responsible for explaining gaps.

This guide explains the ops0 Evidence workflow: collect records, review the results, and generate a signed audit pack. It walks through an illustrative change-management example and the configuration each stage requires. A pack supports an audit; it does not issue a certification or establish that every control operated effectively.

What Makes Infrastructure Evidence Useful?

A security check answers a narrow question about configuration or a finding. An audit request can ask a different question: did the required review happen before each production change during the period? That needs change records, approvals, timestamps, and a defensible population, alongside any policy or scan results.

For each control, establish the requirement, the systems in scope, the period, the source of evidence, and the owner who will review it. An export with no stated coverage can look complete while leaving out changes made through another deployment system or directly in a cloud console.

AWS makes the same distinction in its Audit Manager documentation: collecting relevant evidence does not itself assess overall compliance. Its collection guide also distinguishes configuration snapshots, check results, and user activity. The source and observation time matter when deciding what a record can support.

Step 1: Define the Period and Cloud Scope

In ops0, open Operations Center, Compliance, then Evidence on the IaC projects page. Choose the framework and period, review the connected cloud scope, and select Run collection. Period presets include the last 30 or 90 days, this or last quarter, the last 12 months, and a custom range. Changing the period selects the next collection; it does not change an already collected run.

Choose a period that matches the evidence request. A run over the last 30 days cannot establish that a control operated throughout a 12-month review. Current configuration and screenshots also need their own observation dates; selecting a historical period does not turn a current view into a historical snapshot.

Cloud scope can narrow a run to selected connected clouds. Environment scope currently covers all environments. Record these boundaries before interpreting the result, and confirm that the relevant accounts and source records are available.

The current built-in collection runs 16 tests mapped to selected controls across SOC 2, ISO 27001:2022, NIST 800-53, PCI DSS, HIPAA, CIS Controls, and FedRAMP Moderate. These mappings organize infrastructure evidence; they do not cover every requirement in those frameworks. Further tests defined for a later catalog version are not part of the current run.

Step 2: Review the Population and Exceptions

The Controls view connects each control to its tests. Test details show the population, sampled records, exceptions, observations, evidence strength, and limitations. Read those details before using the readiness summary as a planning signal.

A result can be passing, exception, no activity, or not evidenced. No activity means there was no relevant recorded activity for that test and scope. Not evidenced means the available records do not support the test. Neither is a reason to assume that the wider control passed.

The collection reads what ops0 records. Change evidence covers changes applied through ops0; it does not automatically include every console change or another pipeline. Retrieved populations have a 50,000-row cap, so review completeness notices and reconcile the population against the other systems your team uses. Scanner findings, current-state observations, and recorded workflow decisions support different conclusions.

Worked Example: Was a Deployment Approved Before Apply?

Illustrative example, not a customer result or an ops0 export. A platform team is reviewing recorded deployments for its audit period. It checks two separate requirements: approval happened before apply, and the approver was a different person from the requester.

The first row supports those two narrow checks within the recorded workflow. The second needs an explanation of why approval followed apply. The third needs review against the organization's separation-of-duties policy. A successful deployment alone does not answer either control question. With a population of three, the current sampling rule includes all three; the built-in test evaluates the retrieved population rather than only the sample.

Keep the original exception visible. Record the owner, cause, corrective action, and any accepted exception in the team's review process. A later improvement does not rewrite an earlier record into a passing event. Check the next collection for new activity and retain the explanation for the original period.

These tests evaluate recorded activity. Their presence should not be interpreted as a guarantee that every deployment path enforced the corresponding requirement. For configuration checks before deployment, see the separate compliance-as-code workflow.

Step 3: Make the Sample Reproducible

A sample is useful only if the reviewer can understand the population it came from and how it was selected. ops0 records the population and uses deterministic sampling. The same population and sample seed produce the same selection.

When generating a pack, the reviewer can supply a sample seed to redraw the samples. Earlier packs from that run and their seeds are listed so repeat selections are visible. This makes the selection easier to examine; it does not make an incomplete source population complete.

Use the sample alongside the population and exception lists. A sampled set does not erase the exceptions outside it, and a newly selected item may lack a drill-down screenshot. The underlying records and stated limitations still matter.

Step 4: Generate and Check the Audit Pack

After a collection completes, a user with the Download permission can generate a sealed pack for the selected framework. The background job assembles a ZIP with a report PDF, population CSVs, available screenshots, ledger records, a manifest, a signature, and verification instructions. An external timestamp token is included when timestamping succeeds.

For built-in tests, sealed evidence keeps every retrieved population ID, but detailed rows are limited to 5,000 and may be reduced further to fit an 8 MB record limit. Population CSVs still identify each item and its sample membership and exception associations; some detailed fields can be absent. Review the detail-truncation notices separately from the collection's retrieval limit.

The manifest records hashes for the listed evidence files and is signed with an organization key. Evidence results are also recorded in a hash chain. These checks help detect changes to the sealed evidence; they do not prove that the source data was complete, that an observation was true, or that a control passed.

A public verification page lets a recipient select the pack PDF without an ops0 login. The browser hashes the PDF locally and sends the hash to check it against a sealed pack. This check covers the report PDF and its signature, not every file in the ZIP. Confirm the signing-key fingerprint with one obtained independently from the organization, then follow the supplied verification instructions for the wider bundle.

Check the timestamp status before describing a pack as externally timestamped. A pack can complete without an external timestamp if the timestamp service is unavailable. Full validation of the timestamp authority's signature uses the offline OpenSSL steps in the pack README. Signing also depends on a configured signing key and, in production, a public verification URL.

Storage retention is a separate decision. S3 Object Lock is optional and requires configuration; database storage has no retention lock. Signing a pack does not automatically make it immutable or establish a retention schedule.

Where Custom Controls and AI Fit

Some internal requirements need evidence beyond the built-in tests. ops0 supports custom controls with evidence plans that a named person approves before collection. A plan can be written manually or proposed with AI using the organization's own Claude API key. Custom controls collect evidence and findings; they do not contribute a pass or fail score to built-in readiness.

The Evidence assistant can explain one test result and suggest next steps when the organization enables evidence access and supplies its own Claude key. It receives evidence scoped to the caller's permissions. It cannot apply a fix, change settings, or start a collection. Treat its explanation as assistance and verify it against the cited records.

Some custom sources represent current information, such as the newest IaC files available when collection runs. Keep that observation time visible when using them for a historical period. Uploaded documents and a human-approved plan also need substantive review; approval does not establish the truth of the evidence.

Start With One Real Evidence Request

Pick one request, such as approval before apply, and agree on its period and source population. Run the collection, compare the population with your deployment records, inspect an exception, and review a reproducible sample. Then generate a pack and have another reviewer check the PDF and the bundle instructions.

Scheduled collection can help preserve records between reviews. The first completed run creates an enabled weekly schedule; a user with Run permission can switch it off or adjust its cloud scope. The first scheduled collection covers the previous seven days; later runs use the previous scheduled run's timestamp, with a maximum lookback of 400 days. Review each run's period and completion status against the evidence you intend to retain.

Expand only after the team can explain what the first collection proves and what remains outside scope. Connect the work to your infrastructure audit process and compliance automation workflow. The useful outcome is a traceable answer to a real evidence request, with the unresolved gaps still visible.