Problem
.github/codeql/config.yml ships a query-filters: [{exclude: {id: py/unsafe-cyclic-import}}] block whose schema correctness is currently validated only out-of-band: the next CodeQL run on main either auto-closes the targeted alerts (#85, #86, #87, #253, #254, #255) or it doesn't. A typo at the schema level -- excludes: for exclude:, id: becoming a list, or the rule id misspelled -- silently no-ops the suppression with no signal at PR time.
The regression contract added in #216 (tests/simulation/test_no_cyclic_imports.py + the existing test_no_import_cycle.py) pins runtime safety, not CodeQL behaviour. A schema-level typo in config.yml would not trip either pin -- it only manifests as alerts re-appearing in the Security tab on the next scheduled scan.
This is the same shape as the dependabot.yml wrong-path bug (#235): a config file that looks right, parses as YAML, but is silently ignored by the consumer. PR #216 R5 review surfaced this concern explicitly.
Proposed fix
Add a small unit test (tests/test_codeql_config_schema.py or similar) that:
- Loads
.github/codeql/config.yml via yaml.safe_load.
- Asserts the
query-filters block matches the expected shape:
import yaml
from pathlib import Path
def test_codeql_config_query_filter_shape():
config = yaml.safe_load(Path('.github/codeql/config.yml').read_text())
filters = config.get('query-filters', [])
assert len(filters) >= 1, 'config.yml: query-filters block missing'
exclude = filters[0].get('exclude')
assert exclude is not None, "config.yml: first query-filter must be an 'exclude' (not 'excludes' or another key)"
assert exclude.get('id') == 'py/unsafe-cyclic-import', (
f"config.yml: exclude.id must be 'py/unsafe-cyclic-import', got {exclude.get('id')!r}"
)
- (Optional) Validates the rest of the config's top-level keys against a known allowlist (
name, disable-default-queries, queries, paths-ignore, paths, query-filters) so an accidental typo at the top level (querie-filters:) also fails loud.
The test should fail with a diagnostic that points at the config file and names the actual key found, so the failure mode is debuggable without reading the test source.
Acceptance criteria
Why this issue (not bundled into #216)
Context
Filed by autonomous agent. Strands Agents.
Problem
.github/codeql/config.ymlships aquery-filters: [{exclude: {id: py/unsafe-cyclic-import}}]block whose schema correctness is currently validated only out-of-band: the next CodeQL run onmaineither auto-closes the targeted alerts (#85, #86, #87, #253, #254, #255) or it doesn't. A typo at the schema level --excludes:forexclude:,id:becoming a list, or the rule id misspelled -- silently no-ops the suppression with no signal at PR time.The regression contract added in #216 (
tests/simulation/test_no_cyclic_imports.py+ the existingtest_no_import_cycle.py) pins runtime safety, not CodeQL behaviour. A schema-level typo inconfig.ymlwould not trip either pin -- it only manifests as alerts re-appearing in the Security tab on the next scheduled scan.This is the same shape as the
dependabot.ymlwrong-path bug (#235): a config file that looks right, parses as YAML, but is silently ignored by the consumer. PR #216 R5 review surfaced this concern explicitly.Proposed fix
Add a small unit test (
tests/test_codeql_config_schema.pyor similar) that:.github/codeql/config.ymlviayaml.safe_load.query-filtersblock matches the expected shape:name,disable-default-queries,queries,paths-ignore,paths,query-filters) so an accidental typo at the top level (querie-filters:) also fails loud.The test should fail with a diagnostic that points at the config file and names the actual key found, so the failure mode is debuggable without reading the test source.
Acceptance criteria
exclude.id == 'py/unsafe-cyclic-import').excludesinstead ofexcludeidvalue misspelled (e.g.py/cyclic-unsafe-import)idbecomes a list instead of a stringquery-filtersbecomesqueries-filter(top-level typo)tests/simulation/test_no_cyclic_imports.py(or wherever the CodeQL pin tests live)..github/codeql/README.mdso future readers know the schema is pin-tested.Why this issue (not bundled into #216)
config.yml:53).Context
dependabot.ymlat wrong path): same failure shape -- config that parses but is silently ignored. A pin test on the schema is the same defensive posture.Filed by autonomous agent. Strands Agents.