Skip to content

governance_model.md

Latest commit

 

History

History
56 lines (40 loc) · 1.78 KB

governance_model.md

File metadata and controls

56 lines (40 loc) · 1.78 KB
Raw

Governance Model

Policy schema

Governance policies are defined in governance_policies.yaml. All keys are optional — only the rules present in the file are evaluated.

required_metadata:        # List of IngestionResult field names that must be non-empty
  - dataset_id
  - source_name
  - row_count
  - ingested_at

sensitive_fields:         # Column name substrings that trigger a PII/sensitivity flag
  - ssn
  - credit_card
  - password
  - account_number

allowed_domains:          # Source name must contain at least one of these keywords
  - sales
  - product
  - transaction

minimum_quality_score:    # Float — quality score must meet or exceed this value
  60.0

max_null_percentage:      # Float — null rate must not exceed this value
  20.0

max_duplicate_percentage: # Float — duplicate row rate must not exceed this value
  10.0

Decision output

Each rule produces a GovernanceDecision:

Field Type Description
rule str Rule name matching the YAML key
passed bool Whether the rule passed for this dataset
reason str Human-readable explanation of the outcome

GovernanceResult.passed is True only when all decisions pass.

Sensitive field detection

The engine checks whether any column name in the dataset contains a substring matching a declared sensitive field. This is a name-based heuristic — it does not scan cell values. Detected fields are flagged with a masking recommendation but the rule is marked as failed to prevent the dataset from proceeding to AI use without review.

Extending the policy

To add a new governance rule:

  1. Add the key to governance_policies.yaml
  2. Add a corresponding branch in governance.check_governance
  3. Add a test in tests/test_governance.py