Skip to content

feat(v0.2): RampartAttackRunner for XPIA coverage ([agentic] extra) #5

@pardamike

Description

@pardamike

Problem

The AttackRunner Protocol was shipped in v0.1 specifically to host an XPIA orchestrator. Microsoft's RAMPART provides one, and its AgentAdapter / Session Protocol is a clean bridge to our ChatbotAdapter. The blocker is dependency weight: RAMPART v0.1.0 hard-pins pyrit==0.13.0, which transitively drags in ~30 heavy packages (openai, transformers, fastapi, azure-*) that will conflict with many users' stacks.

Status — building in public

Tracking and designing this in the open. The constraint to be honest about: RAMPART v0.1.0 (released 2026-05-19, alpha) hard-pins pyrit==0.13.0 — which conflicts with many stacks — and returns UNDETERMINED for pure (non-tool) chatbots. So the shippable runner is gated on RAMPART 0.1.1 (re-check whether the pyrit pin loosens, e.g. pyrit>=0.13,<0.14). What can start now: the design, the ChatbotAdapter → RAMPART AgentAdapter/Session bridge, and the [agentic] extra scaffolding.

Proposed approach

  • RampartAttackRunner(AttackRunner) in runners/rampart.py; lazy import + rampart_available() probe; pytest.skip with install hint when absent.
  • Bridge ChatbotAdapter → RAMPART's AgentAdapter + Session; declare ObservabilityLevel.RESPONSE_ONLY (our ChatbotResponse carries no tool-call data).
  • [agentic] extra: RAMPART>=0.1,<0.2.
  • Document the pyrit==0.13.0 conflict prominently; recommend a dedicated venv for [agentic]. (Subprocess / separate-venv isolation is a possible v0.3 hardening — out of scope here.)
  • Document that XPIA verdicts are UNDETERMINED (not SAFE) for pure chatbots without tool-call observability — correct RAMPART behavior, not a bug.
  • License note: RAMPART is MIT (compatible with our Apache-2.0).

Acceptance criteria

  • pytest-wardenbot[agentic] installs in a clean venv (3.11+).
  • rampart_available() reflects install state.
  • RampartAttackRunner satisfies the AttackRunner Protocol.
  • Run against a mock chatbot (no tool calls) → AttackResult with details["status"] == "UNDETERMINED" (does not raise).
  • Skips cleanly without [agentic].
  • Docs explain the pyrit conflict + recommend isolated install.
  • One CI job with [agentic] installed to catch import regressions.

References

Complexity: needs-maintainer; blocked on RAMPART 0.1.1.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-maintainerNeeds maintainer design; not a first issuev0.2v0.2 roadmap

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions