Service Layer Architecture¶

This document describes how the validation run service layer is decomposed into focused modules, the rationale behind the structure, and how to extend it.

Why We Split the Service¶

ValidationRunService started as a single class handling everything: run lifecycle, step iteration, finding persistence, summary aggregation, step dispatch, and cross-step value handling. At over 1200 lines it was becoming difficult to reason about, test in isolation, and modify safely.

We decomposed it into four modules, each with a single responsibility:

ValidationRunService (facade)
├── launch()                        — create run, dispatch to worker
├── cancel_run()                    — cancel a pending/running run
├── execute_workflow_steps()        → StepOrchestrator
└── rebuild_run_summary_record()    → SummaryBuilder

StepOrchestrator
├── execute_workflow_steps()        — iterate steps sequentially
├── execute_workflow_step()         — dispatch single step to handler
├── _start_step_run()               — idempotent step creation
├── _finalize_step_run()            — persist status, duration, output
├── _record_step_result()           — persist findings (action steps only)
├── _execute_validator_step()       — call step processor
└── _resolve_run_actor()            — resolve the initiating user

RunContextBuilder
├── build()                         — compose the context for one step
├── build_upstream_steps()          — read canonical completed step values
└── _resolve_workflow_signals()     — resolve workflow-scoped signals

SummaryBuilder
├── rebuild_run_summary_record()    — idempotent public entry point
├── build_run_summary_record()      — aggregate from DB findings
└── extract_assertion_total()       — extract assertion count from stats

FindingsPersistence
├── normalize_issue()               — coerce raw issues to ValidationIssue
├── coerce_severity()               — map arbitrary severity to enum
├── severity_value()                — resolve string for DB storage
└── persist_findings()              — bulk-create ValidationFinding rows

Design Principles¶

Facade pattern preserves the public API¶

ValidationRunService remains the single import point for all callers. Views, API endpoints, task queues, and callback handlers all continue to import from validations.services.validation_run. The facade delegates to internal modules, so no call sites need updating.

Pure functions where possible¶

FindingsPersistence and SummaryBuilder expose standalone functions rather than classes. They have no state, no side effects beyond database writes, and are easy to test in isolation. The StepOrchestrator is a class because its methods call each other and share the same execution context.

Incremental extraction¶

Each module was extracted as a standalone step and tested before proceeding. This minimises risk when refactoring a critical code path.

One execution architecture¶

Validibot is pre-release, so the service does not carry runtime modes or legacy readers. Every run uses durable attempt identity and canonical step-run values. When an execution contract changes before public adoption, the application and database migration move forward together instead of maintaining parallel engines.

Execution attempts identify concrete provider work¶

ExecutionAttempt is the durable identity for one concrete container or cloud job launched for a ValidationStepRun. It records provider identity, attempt number, runner and contract policy, exact bundle/envelope locations, image identity, timeout, bounded diagnostics, and provider observations.

The database permits at most one non-terminal attempt per step and prevents two rows from claiming the same provider execution within a runner/job namespace. State changes go through execution_attempts.py, whose small monotonic graph makes same-state delivery idempotent and prevents terminal attempts from reopening. Every callback receipt has a required attempt reference.

Module Responsibilities¶

ValidationRunService (validation_run.py)¶

The public facade. Owns two web-layer operations:

• launch() — validates preconditions (permissions, limits), creates the ValidationRun record, and dispatches execution to the appropriate backend (Celery, Cloud Tasks, inline for tests).
• cancel_run() — transitions a pending/running run to CANCELED.

Everything else is delegated.

StepOrchestrator (step_orchestrator.py)¶

The worker-side execution orchestrator. Handles:

• State transitions: PENDING to RUNNING (atomic, idempotent).
• Step iteration: Processes steps sequentially, stopping on failure or when an async validator returns pending.
• Step dispatch: Routes validator steps to the processor pattern and action steps to the handler registry.
• Step lifecycle: Creates step runs (idempotent via get_or_create), finalises them with status, duration, and diagnostics.
• Result recording: Persists findings, extracts signals for downstream assertions.
• Run finalisation: Sets terminal status, builds summary, logs tracking events, stamps the run output_hash, and triggers submission purge.

SummaryBuilder (summary_builder.py)¶

Aggregates run-level and step-level summary records from persisted findings. Queries the database rather than relying on in-memory metrics, making it safe to call in resume scenarios (async callbacks, retries).

FindingsPersistence (findings_persistence.py)¶

Handles the conversion of raw validation issues into normalised ValidationFinding database records. Pure functions with no dependencies on other service methods.

Output Hash (output_hash.py)¶

Owns the tamper-evident run digest that is stamped after a run reaches a terminal state. Community Validibot provides a built-in fallback contract, and commercial packages can register one explicit provider when they need a different canonical hash shape. This keeps the host application generic while making the active Layer 3 hash contract explicit.

Type-Safe Step Config¶

Each validator and action type stores different keys in WorkflowStep.config (a JSONField). The workflows/step_configs.py module provides Pydantic models that give these configs type safety:

from validibot.workflows.step_configs import get_step_config

typed = get_step_config(step)
if isinstance(typed, EnergyPlusStepConfig):
    checks = typed.idf_checks  # list[str], type-checked

Semantic config models use extra="forbid"; runtime-injected and cosmetic keys live in the separate permissive display_settings bucket. A container-specific semantic base defines the provider-neutral execution_profile used only by container-based workflow steps.

The WorkflowStep.typed_config property provides convenient access:

config = step.typed_config  # Returns the appropriate Pydantic model

Config validation also runs during WorkflowStep.clean(), catching type mismatches at save time rather than at runtime.

Available Config Models¶

Adding a New Validator Type¶

1. Add the type to ValidationType in validations/constants.py.
2. Create a Pydantic model in workflows/step_configs.py extending BaseStepConfig.
3. Register it in the STEP_CONFIG_MODELS dict.
4. Create the validator class implementing BaseValidator.
5. Create a ValidatorConfig with validator_class pointing to your class (in the validator's config.py module).
6. Add catalog_entries to define the validator's step inputs and outputs. These appear automatically in the unified "Inputs and Outputs" card on the step detail page.
7. Run python manage.py sync_validators to sync to the database.

File Structure¶

validations/services/
├── run_context.py             # Canonical runtime namespace construction
├── execution_logging.py       # Structured run/step/attempt log correlation
├── execution_attempts.py      # Attempt selectors and monotonic transitions
├── validation_run.py          # Public facade (launch, cancel, delegation)
├── step_orchestrator.py       # Worker-side step execution loop
├── summary_builder.py         # Run/step summary aggregation
├── findings_persistence.py    # Issue normalization and finding creation
├── output_hash.py             # Run output-hash service + provider registry
├── validation_callback.py     # Async validator callback handling
├── models.py                  # ValidationRunTaskResult dataclass
└── step_processor/            # Step processor pattern (see step_processor.md)
    ├── base.py
    ├── simple.py
    ├── advanced.py
    ├── factory.py
    └── result.py

workflows/
├── models.py                  # WorkflowStep.typed_config property
└── step_configs.py            # Pydantic models for step config

Related Documentation¶

• How Validibot Works — end-to-end system overview
• Step Processor Architecture — processor pattern details
• Workflow Engine — higher-level orchestration
• Validator Architecture — execution backends
• Plugin Architecture — how validators and actions register with the host app