Skip to content

SCN-010E: add static validation gate for ACES TechVault parity and conformance #322

@Brad-Edwards

Description

@Brad-Edwards

ACES Methodology And Capture Tooling

Before starting inventory or encoding work on this issue, use the ACES asset inventory methodology and capture tooling:

Summary

Add the static validation gate that blocks SCN-010 cutover unless ACES TechVault is parseable, semantically valid, conformance-aligned, parity-checked against the legacy inventory, proven not to be a preset trigger, and backed by generic equivalent-expressivity interpretation tests.

Parent tracker: #317
Depends on: SCN-010A, SCN-010B, SCN-010C, #324
Requirement: SCN-010

Backend Manifest Clarification

This issue explicitly owns replacing any provisional APTL-local manifest-shaped adapter introduced by the RuntimeManager handoff work with the canonical ACES backend manifest surface. The provisioning-only backend gate must validate against ACES backend-manifest-v2 semantics and the real ACES conformance tooling/corpus, not only the subset of manifest attributes currently consumed by the installed planner/runtime path.

PR #501 proved the public aptl start path can cross the ACES parser/runtime/backend boundary. It did not complete backend manifest conformance. If ACES contract assets, manifest loaders, or conformance commands are missing from the local/CI install, this issue must fix that installation/tooling gap or fail with an explicit diagnostic; it must not preserve a local shim as the accepted manifest contract.

Gate requirements

The gate must run locally and in CI/pre-commit as appropriate, and must verify:

  • ACES reference parser accepts scenarios/techvault.sdl.yaml.
  • ACES semantic validation passes.
  • ACES import/dependency resolution and lock/verification commands pass when imports/modules are used.
  • APTL backend manifest is loaded through the canonical ACES backend-manifest-v2 surface; provisional APTL-local manifest/dataclass shims are rejected as completion evidence.
  • APTL backend manifest passes aces conformance backend --profile provisioning-only using the real ACES conformance tooling and contract corpus in local and CI/pre-commit environments.
  • Missing ACES manifest loaders, contract corpus files, or conformance CLI wiring are treated as gate failures with actionable diagnostics, not as reasons to keep a local manifest approximation.
  • Parity manifest/checker confirms all required nodes, services, vulnerabilities, features, injects, workflows, objectives, scoring/run archive surfaces, Kali apparatus, defensive stack configs, and health surfaces are either represented or explicitly deferred with a linked issue.
  • APTL backend interpretation tests cover model/plan-derived realization decisions for services, networks, volumes, configs, health checks, telemetry, objectives, and run evidence.
  • Generic interpretation tests from SCN-010G: generalize ACES-to-APTL backend interpretation for equivalent scenario expressivity #324 prove at least two scenarios/fixtures with different declared content produce distinct realization outputs through the same interpreter path.
  • Negative tests fail if the implementation can pass by dispatching on techvault / scenario id / preset metadata alone.
  • CI makes failures visible before cutover; advisory vs blocking behavior is clearly documented for Phase A versus final cutover.

Acceptance

  • New tests/check scripts fail on missing required TechVault surfaces.
  • New tests/check scripts fail when declared ACES content is removed but the scenario name remains.
  • New tests/check scripts fail when different ACES declarations collapse to the same hardcoded realization without justified equivalence.
  • New tests/check scripts fail if APTL uses a provisional local manifest shape instead of canonical ACES backend-manifest-v2 loading/conformance.
  • New tests/check scripts fail if the ACES conformance command cannot run because required ACES contract assets or CLI/runtime wiring are missing.
  • CI/pre-commit integration is documented and follows repo plan rules.
  • Validation output is actionable enough to identify the missing surface or shortcut.
  • pytest and pre-commit run --all-files pass.

Absolute Observable-Parity Gate

This issue is not complete until the TechVault ACES SDL expresses every fact that a participant, adversary, defender, autonomous agent, tool, evaluator, or harness could observe from inside the realized range for this issue's in-scope surface.

Parity means full observable parity. It is not relevance-filtered. It is not limited to facts needed by APTL's current implementation, not limited to load-bearing behavior, and not satisfied by storing evidence outside the SDL.

Evidence bundles, checksums, source trees, Docker/Compose files, screenshots, logs, inventories, comments, and mapping ledgers are proof inputs only. They are not substitutes for SDL expression. If an observable file, config, package, version, environment value, route, user, permission, process, service, vulnerability, relationship, log path, filesystem entry, credential fixture, or scanner finding is within this issue's scope, then the SDL-backed spec must encode it directly using ACES.

If ACES can express it, encode it in scenarios/techvault.sdl.yaml and add validation that proves it is present. If ACES cannot express it, file one or more blocking issues in Brad-Edwards/aces, link them here, and do not mark this issue complete. A filed ACES issue is a blocker, not a waiver. After ACES merges/releases the needed expressivity, this issue remains incomplete until the SDL is updated to use the new ACES surface and validation passes.

APTL runtime consumption is separate. Lack of APTL support for consuming an SDL field may require a linked APTL issue, but it never excuses missing SDL parity and must not be used to close this issue.

Forbidden completion claims:

  • "Captured in evidence" unless also encoded in SDL or blocked by a specific ACES expressivity issue.
  • "Representative subset" for packages, files, vulnerabilities, processes, configs, env, services, routes, users, permissions, logs, filesystem state, or scanner findings.
  • "Relevant fields encoded", "load-bearing fields encoded", or "implementation-important fields encoded".
  • "APTL cannot consume it yet" as a reason to omit SDL content.
  • "Out of scope" for any in-scope fact observable from inside the range unless a separate linked issue owns that observable surface and blocks final SCN-010 parity.

Completion checklist:

  • Enumerated every participant/agent-observable fact for this issue's surface.
  • Encoded every ACES-expressible fact in scenarios/techvault.sdl.yaml.
  • Filed and linked ACES blocker issue(s) for every observable fact ACES cannot express.
  • Verified no evidence-only observable facts remain without SDL encoding or an ACES blocker.
  • Updated the mapping ledger so every row is encoded or blocked by a specific ACES issue.
  • Did not mark this issue complete based on evidence capture, representative samples, summaries, relevance judgments, or APTL consumption status.

For already-closed predecessor issues, this gate controls whether their artifacts may be credited toward final SCN-010 cutover; closure does not waive observable parity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions