Skip to content

About

Causally verified extraction of knowledge from trained neural network weights

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Universal Decoder

Inspect what a trained neural network contains — its tensors, its structure, and (separately, below) what it will say if you ask it.

Two projects live in this repository and they have different evidentiary standards. Keeping them apart is the point of this README.

  • universal_decoder/ — a model-file inspection library for GGUF, Ollama and PyTorch state dictionaries. It reads tensors, metadata and structure. The "books" it emits are structural and parameter statistics. One committed sample says so explicitly: it does not prove full model knowledge extraction.
  • genuine_decoder/ — the research pipeline behind the headline numbers. It is a closed-world elicitation and injection pipeline, and this README describes exactly what it does, including the control that bounds what its headline number means.

What the pipeline actually does

Being blunt about the mechanism, because the difference matters:

The facts are not discovered; they are checked against a registry. A human authored relation_registry_v1.json: 8 relations × 16 subjects = 128 candidate facts, each with a gold answer. Models are then prompted, not read. Each model gets 5 templates per fact, its generated text is checked against the registry, and rows that agree across models survive. The 17 facts that reached the injection stage are 6 country capitals, 10 atomic numbers and 1 element symbol.

The "read out of the weights" claim holds for the activation-line work, not for this chain. The causal battery in Why this is hard — patching, dose-response, position selectivity — is real and is the strongest thing here. It is what produces the concept-level pages. The 99.3% number does not come from that battery; it comes from the registry-checked elicitation path described above.

Injection is ordinary fine-tuning. The 17 facts expand into 68 template sentences and distilgpt2 (82M parameters) is trained on them with AdamW for 8 epochs. This does train a model, and this repository contains the code that does it (genuine_decoder/brain_injection.py).

The causal tier is real but currently blocked. It passes 8 of 8 eligible rows, but that relation has 8 eligible rows against a sampling floor of 10. The repo's own status document is right that the published union rests on the agreement tier.

Why the causal work is hard

Most "model introspection" stops at correlation. You find a neuron that fires on Python syntax, note that it fires a lot, and call it a concept. That is a guess.

This project refuses to publish a guess. Every candidate concept has to survive a causal battery:

Stage Question
Unsupervised discovery Does a structure exist at all, before anyone names it?
Competing hypotheses Which of several meanings does it actually encode?
Hard negatives Does it predict anything a trivial confound also predicts?
Causal battery Does patching it move the model's output, monotonically, by dose?
Position selectivity Is the effect about content, or merely about position?
Fresh replication Does it survive in a new process with a different seed?
Sealed held-out Does it hold on data consumed exactly once, at the end?

Cross-model agreement is used as a second, independent trust signal. Relations come from cross-intervention; procedures come from causal ordering across layers.

A real negative result

On TinyLlama, the frozen patch layer is a dead zone. Subject-token patches flip the answer at layers 0–16 and do nothing at 17–21. This matches earlier causal tracing work on factual recall, so it is confirmation rather than a new discovery — but it was found by running the battery rather than by assuming, and it is kept.

What is actually proven

Numbers below are recomputed from the committed artifacts in this repository. Where something did not work, it is recorded as a failure rather than quietly dropped.

Three open models (SmolLM2-360M, TinyLlama-1.1B, Qwen2.5-1.5B) were run through a frozen fact protocol — the thresholds were committed to git before any evaluation existed:

3 source models  ->  frozen fact protocol  ->  70 validated beliefs
      ->  cross-model union with voting  ->  17 corroborated facts
      ->  expansion into training text   ->  fine-tuning of a base model
      ->  99.3% closed-book accuracy on phrasings never trained on
  • Gold precision 1.000 on every published belief. The extractor under-reads its sources rather than over-claiming: the frozen boundary rule accepts answers only as a leading prefix, so city of Paris scores as a miss. Every funnel number here is a lower bound on what the source model knows.
  • No two models ever contradicted each other across 384 elicitations and 51 believed facts. Agreement is a usable trust signal.
  • Rehearsal is a Pareto win. Mixing general text into the injection corpus improved knowledge and reduced forgetting at all six grid points.
  • Hyperparameters were pre-registered, and the selected configuration was re-validated on seeds it was not chosen on.

The control that was missing, and what it says

The 99.3% is the mean of four seeds (97.1%, then 100% three times) on 34 eval prompts. It was published without a control: nobody trained distilgpt2 on the registry's gold answers directly. That control has now been run (genuine_decoder/gold_facts_control.py, result in genuine_decoder/results/gold_control_v1/control_report.json).

Parity was enforced before the control was scored — same base model and pinned revision, same pre-registered optimizer settings, same four seeds, same 34 eval prompts read from the committed receipt, same forgetting probe — and the training text was verified byte-identical to the corpus behind the published number:

Published arm Registry-gold control
Closed-book accuracy, mean of 4 seeds 0.993 1.000
Forgetting, mean +1.928 nats +1.893 nats
Training text 68 sentences from the union the same 68 sentences, byte for byte

The control scores at least as high as the published arm. The 99.3% measures the 17 gold answers that were already sitting in the human-authored registry. The model-extraction pipeline contributes nothing measurable to that number. Given the byte-identical corpus this is the only outcome available — the control cannot differ — but it is worth stating plainly rather than leaving the number to imply more than it carries.

Full write-up: docs/gold_facts_control.md.

A second, weaker probe was added at the same time: 17 prompts from phrasings that appear in no corpus in this repository, authored after the fact and therefore not pre-registered. The injected model scores 11.8% on them, against a 0.0% baseline. Generalization to genuinely new phrasings is near chance; the original eval was measuring rewording of the training templates, as its own caveat said.

What the control does not show, stated in the report itself:

  • It does not show the extraction pipeline is useless. The pipeline still decides which 17 of 128 registry facts get injected, and a hand-picked control cannot audit that choice.
  • It does not test knowledge beyond 17 easy facts. Registry v2 (288 facts) and the causal tier remain blocked on registry size, not on method.
  • It does not clear the 34-prompt eval. Only a genuinely held-out eval would.

Honest rejections

These are kept, not hidden:

  • py_block_colon_due fails dict-key, slice and complete-statement hard negatives on every model. The feature encodes "is Python code", not "a colon is due".
  • sentence_period_due fails the open-quotation stratum. The feature marks "a clause is complete", which is also true inside an unterminated quotation.
  • arith_answer_due shows non-monotone dose response on one model — necessity fails — despite a sealed held-out AUC of 1.000.

The remaining publication blocker is registry size, not method: the causal tier passes 8 of 8 eligible rows, but the relation has 8 eligible rows against a sampling floor of 10. That floor is a threshold the protocol-amendment path correctly refuses to touch, so those rows stay diagnostic and unpublished. Widening the registry is authoring work, not science.

Known evidence-chain defects

Reported here rather than papered over:

  • fact_union_v1 self-hashes do not verify. All 8 relation tables carry a relation_union_sha256 that does not match a recomputation over the committed bytes, and union_manifest.json's fact_union_sha256 does not match either. The union artifacts were written by an earlier code revision whose dict layout differed at hash time. The stored content is consistent and the eval set derived from it reproduces exactly, but the hashes are not verifiable receipts.
  • injection_corpus_v1 cannot be rebuilt to the same digest, because it binds fact_union_sha256 (above) and the union binds absolute source paths in source_tables[*].path. The 34 eval prompts and the 68 training sentences do regenerate identically; only the enclosing digest is not portable.
  • train.jsonl and eval.jsonl are not committed, because .gitignore excludes *.jsonl. The corpus itself is regenerable from the committed extraction artifacts, and the control commits byte-identical copies under genuine_decoder/results/gold_control_v1/, but the original pair is not in the tree.
  • universal_decoder/__init__.py reports __version__ = "0.1.2" while pyproject.toml declares 1.0.0.

Install

Requires Python 3.11+. The default install gets you the library, the CLI, and structural GGUF/Ollama/PyTorch inspection. numpy is the only hard dependency.

pip install -e .

The activation-level attribution backends, everything under genuine_decoder/, and the tests that train a real model need the internal extra:

pip install -e ".[internal]"   # torch, transformers, safetensors, huggingface_hub, ...
pip install -e ".[test]"       # pytest + torch + transformers, for the full suite
pip install -e ".[all]"        # everything

Point the decoder at your models with an environment variable — no path is hardcoded:

export UD_MODEL_ROOT=/path/to/your/models

Use

# what can this machine actually read?
universal-decoder list-local-sources

# decode an installed Ollama model into verified book chapters
universal-decoder run-installed-gguf --output universal_decoder/results/installed_models_v1

# verify a published Brain Pack's checksums
universal-decoder verify-brain-pack --input <pack-dir>

The package is also importable:

from universal_decoder.v1.library_api import detect_model, get_capabilities, plan_extraction

info = detect_model("/path/to/model")
plan = plan_extraction("/path/to/model")

What it does not do

Stated plainly, because the boundary is the point:

  • No downloads, no ollama pull, no network access to model sources.
  • The structural library runs no inference. universal_decoder/v1 reads tensors and metadata; it never runs a forward pass to answer a question.
  • The research pipeline does run inference, and it does train a model. genuine_decoder/ prompts source models to elicit facts, and genuine_decoder/brain_injection.py fine-tunes distilgpt2 on the decoded facts. No trained weights are committed, and the injection weights are not persisted — the run is reproducible from its manifest. Earlier versions of this README claimed "no inference, no chat, no generation" and "no model training" for the whole repository; that was true of universal_decoder/v1 and false of genuine_decoder/, and the blanket claim has been withdrawn.
  • Raw activation captures are not committed. Note this is narrower than it sounds: ~330 .npz and .npy arrays are tracked, including SAE directions and statistics. What is excluded is capture_*.npz activation dumps; each run's artifact_manifest.json carries their SHA-256.
  • The pipeline is generation-based. The README once said the fact chain "reads parameters directly, with no generation". It does not: it prompts models and checks their text against a registry. That correction is the same one that makes the control above necessary.

Layout

universal_decoder/          the package
  v1/                      the shipping library and CLI (structural inspection)
  internal_attribution/     activation-level extraction backends
  internal_lab/             the research platform
  research_v2/              the atlas90 experimental line
  results/                  committed evidence from real runs
genuine_decoder/            fact protocol, cross-model union, injection, and the
                            gold-facts-only control
sandbox_universal_decoder/  standalone parameter probes
tests/                      the library test suite
docs/                       design notes and measurement reports

universal_decoder/results/ holds ~5,800 committed artifacts across 851 directories, and many carry the absolute paths of the machine that produced them (C:/ud-home/...). The evidence is kept, but expect noise when navigating a fresh clone.

Honesty guarantees

A few properties this codebase holds to, because they are what make the rest trustworthy:

  • register_amended_fact_protocol refuses threshold weakening. It permits changes to declared measurement methods only. Tests prove it rejects ten kinds of threshold relaxation.
  • Coverage is not inflation. Semantic knowledge coverage is reported as not_measurable rather than as a percentage, wherever it genuinely is not measurable.
  • Held-out data is consumed once. The sealed split is read exactly once, at the end, and re-reading it is a test failure.
  • Results directories are pruned from releases. Model files, databases, secrets and raw tensor dumps are excluded from packaged artifacts.
  • Controls are run even when they are unflattering. The gold-facts-only control above was the most informative experiment in the repository and it does not support the number it was run to protect.

Tests

python -m pytest tests genuine_decoder -q

Two suites with different dependency floors:

  • tests/ — the library. Runs on a bare install.
  • genuine_decoder/ — the research pipeline. Needs torch.

A test that cannot run on this machine skips; it does not fail. Guards live in universal_decoder/testing.py:

from universal_decoder.testing import requires_torch, requires_cached_model
  • Missing torch is a skip, never a collection error. Fifteen test modules import activation-level backends at module scope — all 11 in genuine_decoder/ plus 4 under tests/ — and produced fifteen collection errors on a clone without torch before pytest.importorskip("torch") was added above their imports.
  • Missing model files are a skip. Real-model tests are guarded on the local HuggingFace cache and on UD_MODEL_ROOT; none of them touches the network.
  • No test hardcodes a machine's home directory. One test (test_iterative_discovery.py) had a path relative to genuine_decoder/ that only resolved when pytest was invoked from inside that directory; it is now anchored to the test file.

Verified from a clean environment:

Environment tests genuine_decoder
Full (torch + cached models) 231 passed, 12 skipped 152 passed
torch present, empty model cache 231 passed, 12 skipped 78 passed, 74 skipped
No torch 195 passed, 9 skipped all 11 modules skipped

Counts are for the state of the tree at the time of writing and will drift; the claim worth checking is the shape: no failures and no collection errors in any of the three. pytest tests genuine_decoder -q also works as one command.

A number of tests verify committed historical evidence, which is why some result artifacts are large — they are the receipts. Those receipts have the defects listed under Known evidence-chain defects; the control exists partly because the original corpus files are not in the tree.

License

MIT — see LICENSE.

About

Causally verified extraction of knowledge from trained neural network weights

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages