feat(rewrite): revisit HIGH sensitivity classification and any_high_leaked trigger for needs_human_review

[lipikaramaswamy]

Background

needs_human_review is triggered by two conditions: leakage_mass > flag_leakage_mass_above (default 2.0) or any_high_leaked=True. Investigation of TAB runs showed all flagged rows were triggered by any_high_leaked, not leakage mass.

The entities flagged as HIGH sensitivity were:

• court_name (e.g. 'District Court', 'Budapest District Court') — fixed by PR feat: Improve Sensitivity Disposition supplements #94 domain supplements
• legal_role (e.g. 'defendant_in_recovery_action')
• former_senator (occupation type)

These are quasi-identifiers at most. Flagging them as HIGH causes needs_human_review=True even when leakage mass is low and utility is good (e.g. utility=0.97, leakage=0.96).

Questions to resolve

• What is the right criteria for HIGH vs. MEDIUM sensitivity? Should quasi-identifrs ever be HIGH?
• Should any_high_leaked alone be sufficient to trigger needs_human_review, or should it require a minimum leakage mass too?
• Should sensitivity level be configurable per domain (e.g. LEGAL may have different thresholds than MEDICAL)?

Related

• PR feat: Improve Sensitivity Disposition supplements #94 partially addressed this for court_name via LEGAL domain supplements
• Ties into the domain-specific risk tolerance work removed before release (see domain supplement follow-up issue)

[lipikaramaswamy]

from @asteier2026:
We can break it into two. First immediately change needs_human_review to not use any_high_leaked (I think it's better that way and it's in sync with our docs). Second, we can revisit high sens classification, but it's no longer as important if it doesn't trigger needs_human_review (just continues to trigger repair).