Skip to content

SCN-010F: add live validation gate for ACES-driven TechVault deployment #323

@Brad-Edwards

Description

@Brad-Edwards

ACES Methodology And Capture Tooling

Before starting inventory or encoding work on this issue, use the ACES asset inventory methodology and capture tooling:

Summary

Add the live validation gate proving that a fresh TechVault lab can be instantiated from ACES SDL through APTL's ACES backend path, with concrete realization derived from the ACES model/plan, and produces the expected operational evidence.

Parent tracker: #317
Depends on: SCN-010D, SCN-010E, #324
Requirement: SCN-010

Gate requirements

From a clean local environment or documented CI-capable runner, the gate must exercise:

  • aptl lab stop -v cleanup followed by ACES-driven TechVault start.
  • Docker Compose profiles/services generated or selected from interpretation of ACES runtime/backend content, not legacy scenario definitions and not a hardcoded preset selected by scenario name.
  • Health/readiness checks for core targets, Kali SSH/MCP access, Wazuh stack, Suricata/MISP sync, TheHive/Cortex/Shuffle, OTEL/Tempo/Grafana, and required backing stores.
  • Kali reachability to DMZ/internal TechVault hosts via declared DNS/host mappings and network attachments.
  • Representative SOC telemetry/evidence path: at least one expected alert/log/evidence artifact traverses the defensive stack and is reflected in run archive/snapshot surfaces.
  • Run archive manifest/snapshot captures scenario identity, ACES runtime model/provenance, objective/scoring surfaces, validation evidence, and enough model-derived realization details to audit that ACES was interpreted rather than used as a preset trigger.
  • Evidence from SCN-010G: generalize ACES-to-APTL backend interpretation for equivalent scenario expressivity #324 or equivalent live diagnostics showing the same backend interpreter path can realize scenario variation within APTL's supported expressivity class.

Acceptance

  • The live gate can be run by maintainers with documented prerequisites and cleanup behavior.
  • Gate output proves the public handoff path is ACES parser/runtime/backend driven.
  • Gate output includes evidence tying realized services/configs/networks to ACES model/plan content.
  • Gate output does not merely prove TechVault works; it also confirms TechVault is realized through the generic ACES-to-APTL interpreter contract.
  • Failures identify whether the problem is ACES specification, backend interpretation, backend instantiation, defensive stack readiness, Kali reachability, or evidence/run archive capture.
  • The final SCN-010 cutover issue cannot proceed until this live gate passes and SCN-010G: generalize ACES-to-APTL backend interpretation for equivalent scenario expressivity #324 is satisfied.
  • pytest, targeted smoke/integration checks, and pre-commit run --all-files pass for the implementation PR.

Absolute Observable-Parity Gate

This issue is not complete until the TechVault ACES SDL expresses every fact that a participant, adversary, defender, autonomous agent, tool, evaluator, or harness could observe from inside the realized range for this issue's in-scope surface.

Parity means full observable parity. It is not relevance-filtered. It is not limited to facts needed by APTL's current implementation, not limited to load-bearing behavior, and not satisfied by storing evidence outside the SDL.

Evidence bundles, checksums, source trees, Docker/Compose files, screenshots, logs, inventories, comments, and mapping ledgers are proof inputs only. They are not substitutes for SDL expression. If an observable file, config, package, version, environment value, route, user, permission, process, service, vulnerability, relationship, log path, filesystem entry, credential fixture, or scanner finding is within this issue's scope, then the SDL-backed spec must encode it directly using ACES.

If ACES can express it, encode it in scenarios/techvault.sdl.yaml and add validation that proves it is present. If ACES cannot express it, file one or more blocking issues in Brad-Edwards/aces, link them here, and do not mark this issue complete. A filed ACES issue is a blocker, not a waiver. After ACES merges/releases the needed expressivity, this issue remains incomplete until the SDL is updated to use the new ACES surface and validation passes.

APTL runtime consumption is separate. Lack of APTL support for consuming an SDL field may require a linked APTL issue, but it never excuses missing SDL parity and must not be used to close this issue.

Forbidden completion claims:

  • "Captured in evidence" unless also encoded in SDL or blocked by a specific ACES expressivity issue.
  • "Representative subset" for packages, files, vulnerabilities, processes, configs, env, services, routes, users, permissions, logs, filesystem state, or scanner findings.
  • "Relevant fields encoded", "load-bearing fields encoded", or "implementation-important fields encoded".
  • "APTL cannot consume it yet" as a reason to omit SDL content.
  • "Out of scope" for any in-scope fact observable from inside the range unless a separate linked issue owns that observable surface and blocks final SCN-010 parity.

Completion checklist:

  • Enumerated every participant/agent-observable fact for this issue's surface.
  • Encoded every ACES-expressible fact in scenarios/techvault.sdl.yaml.
  • Filed and linked ACES blocker issue(s) for every observable fact ACES cannot express.
  • Verified no evidence-only observable facts remain without SDL encoding or an ACES blocker.
  • Updated the mapping ledger so every row is encoded or blocked by a specific ACES issue.
  • Did not mark this issue complete based on evidence capture, representative samples, summaries, relevance judgments, or APTL consumption status.

For already-closed predecessor issues, this gate controls whether their artifacts may be credited toward final SCN-010 cutover; closure does not waive observable parity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions