Detector composition strategies

[patriciapampanelli]

Garak probes can declare multiple detectors, but today each detector runs independently and reports its own pass/fail. There's no way to combine signals across detectors into a richer verdict.

For example, in malwaregen we can verify whether the LLM generated code, but we can't assess whether that code is actually malware vs. just contains a vulnerability. If we could combine detectors, we'd get much richer insights: "code was generated" AND "code contains known vulnerability patterns."

This connects to #107 (detector aggregation). I've been discussing this with @jmartin-tech recently.

Why this matters now
As garak grows (more detectors, T&I mapping, agent-based probes), the gap between "run detectors independently" and "combine detector signals meaningfully" becomes more visible. Scenarios where I think composition would help:

• Corroboration: multiple detectors agreeing increases confidence in a result
• Complementary signals: a keyword detector catches surface patterns, an ML classifier catches semantic meaning
• Cost management: run cheap detectors first, escalate to expensive ones only when needed

Possible approaches
I think we can iterate on this incrementally, starting with low complexity and building from there:

• Boolean (ANY/ALL): combine detectors with OR (flag if any triggers) or AND (flag only if all agree). This is roughly what eval - any vs. all #107 proposed and would be a natural first step.
• Cascade: run detectors in sequence, e.g. fast string matching first, ML classifier if inconclusive, LLM-as-judge as a last resort. This introduces cost-aware detection.
• Weighted voting: run N detectors in parallel and combine by majority vote or weighted score (where weights reflect known detector reliability). This connects to the detector quality work.
• Learned fusion (future): train a meta-classifier on detector outputs. Most powerful, most complex.

Open questions

• What should combined output look like?
• How should users interact with composition rules? What would a good configuration experience look like for defining composition rules?

[leondz]

What should combined output look like?

I think the examples in #107 are OK , but the question remains of where this goes. Another field in the attempt makes sense. Perhaps under detector_results, with key _aggregate ?

How should users interact with composition rules? What would a good configuration experience look like for defining composition rules?

We could go wild here, which is a risk.

I'm inclined to move to a pattern where we have, for a given probe/intent/payload, something like:

• a list of n detector class names

  • the zeroth is the default one
  • an optional field "aggregation"

• iff extended_detectors is not enabled:

  • run zeroth detector only
  • copy results into _aggregate

• else

  • check out aggregation value
  • for some values (any, all) that use all detectors' output
    • run all detectors
    • apply aggregation function per response, store in _aggregate

Majority voting could fit into this pattern too, with a definition of what to do for tie-breaking somewhere (inclined to say 50% hit is a hit. build better targets)

I like the efficient cascade idea. Two options could be:

• cascade_on_hit - if detectors have propensity for false hits, keep moving to the next one in line; only store hit if all say hit
• cascade_on_pass - if detectors have propensity for false passes (e.g. phrase-based mitigation), keep moving to the next one in line; only store pass if all say pass

These involve extra orchestration, perhaps in the harness, to run detection on subsets of the original response set. One might like to use Nones in the resulting detector_result list for results that were not analysed. E.g. for cascade_on_pass:

1. detector_1 [pass hit  hit  pass pass]
2. detector_2 [pass None None hit  pass] (run only on items 0,3,4)
3. detector_3 [pass None None None hit ] (run only on item 0,4)
4. _aggregate [pass hit  hit  hit  hit ]

That extra orchestration might be a pain - on the other hand, it must be cheaper than doing more inference.

[drohrerNV]

Feels like you are "effectively" speaking to an abstract "policy" application based on "evidence" provided. Any reason the "detectors" are not just evidence producers and your "policy" of YES/NO isn't just that -- a policy that can be written against common tools like OPA REGO?

It feels like all the approaches are just one "Variation" on a POLICY (some of which I agree are likely reasonable defaults to have pre-baked). Virtue of using an established framework for that eval as you get to "import" vs "write from scratch" and hopefully alot of the pain of those constructions have been litigated in someone elses OSS history.

For extra points -- could those policies/detectors be LIFTED directly into a Guardrail? context for deployment?

[PhinehasNarh]

Both approaches here are interesting. One thing I'd add before getting to the composition layer: signal quality at the individual detector level matters a lot for how useful any aggregation ends up being.

I ran into this recently working on StringDetector (PR #1884) - Unicode normalization wasn't being applied before string matching, so a target response with decomposed characters or fullwidth lookalikes would slip through detection even when the underlying pattern clearly matched. Composition logic on top of unreliable individual signals can actually make things worse if some detectors are systematically more brittle than others - you end up with one detector's noise dominating the aggregate.

The OPA/REGO approach that @drohrerNV suggested is interesting for exactly this reason. If each detector emits structured evidence (match confidence, normalization applied, character class) rather than just a boolean, the policy layer can weight them appropriately. A detector that normalized before matching is more reliable evidence than one that didn't.

For the _aggregate key idea from @leondz - does the intent there include per-detector provenance, or just the rolled-up verdict? Being able to reconstruct which individual detector contributed to a positive finding is useful for debugging probe coverage gaps.