Validation Results¶

When a validation run completes, three types of output are produced:

• Findings -- individual issues, warnings, and info messages
• Artifacts -- files produced by validators (reports, logs, transformed data)
• Summaries -- aggregated counts for dashboards and long-term reporting

These all hang off the ValidationRun and ValidationStepRun models:

ValidationRun
├── findings (many ValidationFinding rows)
├── artifacts (many Artifact files)
├── step_runs (many ValidationStepRun)
│   └── findings (many ValidationFinding rows)
└── summary_record (one ValidationRunSummary)
    └── step_summaries (many ValidationStepRunSummary)

Findings¶

A ValidationFinding is a single normalized issue discovered during validation. This is the primary unit of feedback that users see in the results table.

Fields¶

Findings are ordered by severity (ERROR first, then WARNING, INFO, SUCCESS) and then by creation date descending.

How findings are created¶

Validators don't write to the database directly. Instead, they emit ValidationIssue dataclass objects during execution. After a step finishes, the FindingsPersistence service normalizes these issues and bulk-creates ValidationFinding rows.

The flow:

Validator
  → emits ValidationIssue objects
    → collected in ValidationResult.issues
      → FindingsPersistence.persist_findings()
        → bulk-creates ValidationFinding rows

For advanced (async) validators like EnergyPlus, findings are produced in two stages. Input-stage findings (from assertions evaluated before the container runs) are persisted immediately. Output-stage findings (from the container's results and post-execution assertions) are appended when the callback arrives.

Severity levels¶

SUCCESS findings are only created when an assertion passes and either the assertion has a custom success_message or the step has show_success_messages=True. See the assertions documentation for details.

Path field¶

The path field tells the user where in their input data the issue was found. The format depends on the validator:

• JSON Schema validators produce JSON Pointers (e.g. /building/zones/0/name)
• XML Schema validators produce XPaths
• CEL assertions produce dotted paths

For JSON submissions, a synthetic payload prefix is automatically stripped before storage so users see clean paths.

Artifacts¶

An Artifact is a file produced during a validation run. Common examples include EnergyPlus simulation output, transformed data files, and debug logs.

Fields¶

Artifact files are stored at artifacts/org-{org_id}/runs/{run_id}/{uuid}/{filename}.

Not all validators produce artifacts. Built-in validators (JSON Schema, XML Schema, CEL) typically produce only findings. Advanced validators running in containers (EnergyPlus, FMU) produce artifacts alongside their findings.

Summaries¶

Summaries are lightweight aggregate snapshots that persist even after findings and artifacts are purged by the retention policy. They power dashboards and long-term reporting.

ValidationRunSummary¶

One record per run with rolled-up counts:

ValidationStepRunSummary¶

One record per step within a run summary:

Step names and order are denormalized into the summary so the data remains meaningful even if the workflow definition changes later.

Retention and cleanup¶

Each workflow defines two retention policies that control how long results are kept:

• input_retention -- how long to keep user-submitted files after all consumers finish.
• output_retention -- how long to keep detailed output, including findings, artifacts, step input/output values, detailed errors, output envelopes/hashes, callback result locations, and evidence files.

Both policies default to DO_NOT_STORE; longer storage is an explicit author choice. Output deadlines start at ValidationRun.ended_at, not launch time. Finite output access is denied at the deadline even if physical deletion is temporarily retrying.

purge_expired_outputs runs every five minutes. It deletes the complete execution bundle and detailed output projections, but preserves the run's terminal status, timing, aggregate summary counts, and purge timestamp so dashboards and historical pass-rate charts continue to work. Unknown policies fail closed to no retention, active runs are never eligible, missed terminal signals are repaired by the sweep, and deletion failures remain eligible for the next run.