What has been hardened, what is deliberately left as-is, and what risk remains. Written to be read alongside a static-analysis report, because several scanner findings on this repo are structural rather than exploitable, and a couple of real risks are not things a scanner flags.
Every Hugging Face fetch takes a revision and forwards it explicitly:
| Call site | Pin source |
|---|---|
src/cmb_indic/data.py (hf_hub_download) |
--dataset-revision |
src/cmb_indic/backends/hf_local.py (AutoTokenizer, AutoModelForCausalLM) |
hf_revision in the model config |
src/cmb_indic/backends/vllm_offline.py |
hf_revision |
scripts/check_compat.py (config.json + tokenizer probe) |
hf_revision |
scripts/build_conditions.py, build_gsm8k_conditions.py, verify_item_coverage.py, check_gsm8k_contamination.py |
--revision |
utils.py (vendored, unused by this pipeline) |
revision argument |
revision is passed as a literal keyword argument, never inside a **kwargs
dict. That is not cosmetic: a value hidden in **kwargs is invisible to static
analysis, so it reads as an unpinned download in every audit.
The defaults are None, which resolves the repo's default branch. Pinning is
therefore available rather than mandatory -- pass a commit sha to get a
byte-reproducible fetch. run_meta.json records the sha256 of every dataset CSV
actually used, so a run remains auditable even when it was not pinned.
eval() is not called anywhere in the repository. The three vendored call sites
that parsed stringified Python literals now use ast.literal_eval(); see
docs/DEVIATIONS.md item 11. exec, pickle.load, yaml.load without a safe
loader, os.system, and subprocess(..., shell=True) are likewise absent.
scripts/check_compat.py and cmb-indic validate --check-hub fetch over
httpx, which speaks only http/https. A repo id or revision read out of a
config file therefore cannot redirect a fetch to file:/ or a custom scheme.
This replaced urllib.request.urlopen, where the same guarantee needed a
hand-written scheme check that a scanner cannot see.
src/cmb_indic/runner.py resolves git through shutil.which() before
invoking it, so provenance capture cannot pick up a git planted earlier on
PATH. The call passes an argument list, never a shell string.
API keys and tokens are read from the environment (CMB_API_KEY by default,
HF_TOKEN for gated repos); .env is gitignored and .env.example carries
names only. ModelConfig.api_key_env holds the name of the variable to read,
not a value -- scanners regularly misreport that field as a hardcoded
credential. The fallback string "EMPTY" is the vLLM/SGLang convention for "no
auth required" (their clients reject an empty string); it is not a credential.
Fixed by raising a minimum version:
| Package | Floor | Advisory |
|---|---|---|
| scikit-learn | 1.5.0 | CVE-2024-5206 -- TfidfVectorizer leaked training tokens via stop_words_ |
| datasets | 5.0.1 | CVE-2026-66007 -- path traversal via file_name in folder-based builders |
| tqdm | 4.66.3 | CVE-2024-34062 -- argument injection in the tqdm CLI |
| transformers | 4.53.0 | ReDoS: CVE-2025-3933, -3777, -5197, -6051, -6638, -6921 |
| sentencepiece | 0.2.1 | CVE-2026-1260 -- heap overflow |
| pytest | 9.0.3 | CVE-2025-71176 -- tmpdir handling |
| black | 26.3.1 | CVE-2024-21503, CVE-2026-31900, CVE-2026-32274 |
vllm moved from 0.10.1 to 0.10.1.1, the patch release for CVE-2025-9141
(RCE in the Qwen3-Coder tool-call parser) and CVE-2025-48956 (DoS).
The vendored requirements.txt -- upstream's install manifest, kept for
provenance -- received the same treatment for scikit_learn, datasets and
tqdm.
The vllm extra is pinned to a version with open advisories. vLLM 0.10.1.1
is the newest release that still pins torch==2.7.1+cu126, i.e. the newest one
that runs on a CUDA 12.x driver. Advisories fixed only in 0.11 and later stay
open on this pin. Most of them are in the OpenAI-compatible server and its
media handling; this pipeline drives vLLM in-process (vllm_offline.py) and
starts no server, which removes the network-facing ones from the picture. The
model-loading RCEs (CVE-2026-22807 auto_map, CVE-2025-66448 get_config,
CVE-2026-27893) are reachable and are the reason trust_remote_code defaults to
false. On a driver >= 580, relax the vllm/transformers pins as the comment
in pyproject.toml describes and this whole class goes away.
The torch floor stays at 2.3. vLLM 0.10.1.x pins torch==2.7.1, so a
higher floor makes pip install -e ".[all]" unresolvable. pip install -e ".[hf]" on its own resolves torch to the current release, which is patched. If
you install from a lockfile, pin torch >= 2.10.0 yourself (CVE-2025-32434 and
CVE-2026-24747, both weights_only unpickler RCEs).
transformers RCEs fixed only in 5.x (CVE-2026-1839, -4372, -5241, -9856)
remain open on the vllm extra, which pins transformers==4.55.2 because 5.x
breaks vLLM 0.10.1 with rope_scaling should have a 'rope_type' key. The hf
extra carries no such pin and resolves to current transformers, but its floor of
4.53.0 still permits a 4.x install for the same [all] reason as torch --
raising it to 5.x would make [all] unresolvable. If you install from a
lockfile and do not need the in-process vLLM backend, pin transformers >= 5.10.0
and torch >= 2.10.0 yourself.
trust_remote_code. Defaults to false everywhere and is opt-in per model
config. Turning it on executes code from the model repository by design; only
enable it for a repo you pinned to a reviewed revision.
Model weights are untrusted input. Nothing here verifies a checkpoint
beyond what transformers/vllm do. Pin a revision and prefer safetensors.
Local endpoints are unauthenticated by default. cmb-indic run --backend openai talks to whatever base_url says. Those servers are meant to be
loopback or a trusted LAN; do not expose one to an untrusted network.
A static scan of this repo reports ~267 LOW findings. All of them fall into these classes:
assertin tests (bandit B101, ~252). pytest's assertion rewriting is how the suite works.assertis not used for control flow or validation insrc/.try/except/passandtry/except/continue(B110/B112, 8). Optional imports, best-effort provenance capture, and per-row error isolation, where a failure is either recorded in the run metadata or intentionally skipped. Each carries a comment saying why.- Non-cryptographic
random(B311, 2).backends/openai_compat.py:91scales exponential backoff with full jitter;prompts.py:181seedsrandom.Randomto draw few-shot exemplars reproducibly (docs/DEVIATIONS.mditem 6). Both need determinism or fairness, not unpredictability, and neither feeds a security decision -- a CSPRNG would make the second one worse. - "Hardcoded password" (B105, 3).
registry.py:445compares against the string"derived", a keyword that selects a split group.utils.py:484and:487assignmax_token_key = 'max_tokens'/'max_new_tokens'-- flagged only because the variable name ends in_key. No credential in any of them. Separately, an InfoSec scan reportedconfig.py'sapi_key_envfield as a hardcoded credential (CWE-798); as described above it holds a variable name, and"EMPTY"is the local-server "no auth" convention. subprocessimport and call (B404/B603, 2). Thegitprovenance call inrunner.py, with an absolute path and an argument list.
This is a research pipeline, not a service. Open an issue for anything found here.