feat: structure-verified annotation overhaul (metabolite + reaction cross-references, pH-7.3 structures)#1058
Merged
Merged
Conversation
edkerk
force-pushed
the
annotation-overhaul
branch
from
July 13, 2026 21:11
0308e0d to
48b0e84
Compare
Adds the overhaul plan (issues #964/#967/#968/#626/#618) and a first report-only audit (code/annotation/identity.py) classifying every metabolite by how its stored SMILES relates to the curated formula/charge: 41% ok, 33% wrong protonation state, 16% no structure, 9% generic R-group, 99 with the wrong molecule stored (real formula errors).
Structures now live with the other metabolite annotations: - add metSmiles and metInChI columns to model/metabolites.tsv (populated from the #728 structure set) - annotateGEM.m adds them to the model (model.metSmiles, and model.inchis via a remap of the metInChI column) so the release exports on main carry structure combined with the annotations - formula and charge stay in the YAML Add a CI QC test (code/test/structureConsistencyTest.py) that checks each metabolite's YAML formula/charge against its tsv SMILES/InChI with RDKit and writes qc_structure_consistency.csv; wired into qc-tests.yml and the report. Baseline: 2882 metabolites inconsistent (2783 wrong protonation state, 99 wrong molecule stored), to be resolved in the structure-curation phase.
…species Enumerate protonation states of each mismatched structure (dimorphite-dl) and keep the one whose element formula and net charge exactly match the curated model formula/charge, preferring acid-oxygen deprotonation over alkoxides. Of 1190 distinct structures: 1025 resolved and applied (2467 metabolite rows), 48 ambiguous and 117 unresolved left for review (the unresolved are model-side charge errors: a neutral charge on an anion formula, i.e. half-integer DBE). Structure/formula/charge inconsistencies drop 2882 -> 415. identity.py now reads structures from metabolites.tsv (the source of truth); the large derived audit tables are gitignored.
crossref_verify.py computes each metabolite's InChIKey from its metSmiles and matches it to MetaNetX chem_prop on the pragmatic key (InChIKey without the final protonation char, so charged model species match neutral database entries). From the matched MNXM it reads structure-verified identifiers for every namespace (KEGG/ChEBI/BiGG/HMDB/LipidMaps/ModelSEED) via chem_xref and classifies each existing cross-reference confirmed / wrong / unverified, and reports metabolites that could gain a verified new id. Existing MetaNetX ids are checked by their own structure so deprecated-but-valid ids are not falsely flagged.
95% of the model's MetaNetX IDs that fail structure verification are simply deprecated (absent from the current MNXref 4.5 chem_prop). For each, replace the old MNXM with the current one whose structure matches the metabolite (preferring the MNXM whose full InChIKey equals the model's). Structure-based, so it does not repeat the #964 error of blanking still-valid IDs. apply_crossref.py generates the change from the MetaNetX hub.
Fill empty cross-reference cells with the best structure-verified identifier (one per namespace, chosen by consensus across matched MNXM then format preference), from the MetaNetX hub matched on the pragmatic InChIKey. Coverage: HMDB +2036, BiGG +1652, ChEBI +1431, LipidMaps +934, KEGG +418, MetaNetX +1189. Add-only: no existing value is overwritten. Unlike #964 these are full skeleton+stereo InChIKey matches, not skeleton-only.
Add a metSeedID column to metabolites.tsv with 3429 structure-verified ModelSEED compound IDs (seed.compound) matched via the MetaNetX hub, and normalize all rows to a uniform width (two rows had ragged trailing fields). annotateGEM maps metSeedID to the seed.compound namespace so it exports as a MIRIAM annotation and model field. Grows ModelSEED coverage from structure, as one of the checked databases rather than a bulk score-boosting import.
charge_fix.py identifies metabolites whose curated formula/charge is a physically impossible radical (odd electron count) and, using the physiological protonation state, classifies each as a charge error (formula right), a formula error (charge right), or both. apply_charge.py can apply them. 70 such cases exist, but applying them shifts mass/charge balance (they expose reactions that were only balanced by the invalid data, and some target intentional radical species), so the correction is coupled to reaction re-balancing and is deferred to the balancing phase rather than this annotation-only release.
Apply the unambiguous charge corrections: metabolites whose formula is valid but whose charge makes it a physically impossible radical (a carboxylate carried at charge 0, e.g. 15(S)-HEPE C20H29O3 0->-1). Intentional radical species (octanoate radical, semiquinones, monodehydroascorbate) are excluded from the closed-shell test. Balance improves by 6 reactions (7 fixed, 1 exposed). The formula-side fixes and the broader charge rebalancing (enforcing the pH-7.3 microspecies convention model-wide) remain, as does the reaction proton accounting they cascade into.
reaction_verify.py reduces each reaction to a structural signature (participant InChIKeys with coefficients, protons/water removed, direction-invariant) and matches it to MetaNetX reac_prop; reac_xref then gives KEGG/BiGG/Rhea/MetaNetX reaction ids. As with metabolites, most rxnMetaNetXID that fail verification are deprecated MNXR; replace each with the current structure-matched one.
Fill empty reaction cross-reference cells with the best structure-verified id (one per namespace) from the MetaNetX reaction hub: BiGG +1344, MetaNetX +1288, Rhea +849 (master id of the quartet), KEGG +171. Add-only; full structural signature matching, so far more and more reliable than the skeleton-based estimates in #967.
Replace cross-reference IDs whose own structure (verified via the MetaNetX hub) resolves to a DIFFERENT skeleton than the metabolite, i.e. the id points to the wrong compound (ChEBI 26, LipidMaps 12, BiGG 11, HMDB 7, KEGG 5). Only skeleton- level mismatches are corrected; stereochemistry-only differences (114) and ids MetaNetX cannot resolve (265) are left untouched, since the model's own structure often has less-defined stereo and overwriting would degrade the annotation.
…etaNetX ids Cross-references were applied per compartment, so compartments whose stored structures differ slightly received different ids (flagged by annotationTest). Collapse each metabolite's cross-references to one value per column: metMetaNetXID (rewritten wholesale by the drift update) is unified to the consensus; the other columns are only fixed where this PR introduced the inconsistency, leaving develop's pre-existing ones untouched. Also drop metMetaNetXID values that are not of the form MNXM<digits> (water was assigned the named id WATER). No new malformed or inconsistent cross-references remain vs develop.
The rebase (-X theirs) let this branch's structure-verified data win conflicts, which reverted the cross-reference reconciliation and two formula fixes merged in #1057. Restore #1057's values only where this branch had not changed them (a three-way merge against develop's parent), keeping the structure-verified values elsewhere, then re-harmonize across compartments. No new malformed or inconsistent cross-references vs develop.
edkerk
force-pushed
the
annotation-overhaul
branch
from
July 14, 2026 05:32
c3d2190 to
998e872
Compare
Model quality report
Structural checksDuplicate keys (model unloadable) and no growth block the merge; the other rows are non-blocking.
Model QC reports
MACAW and mass/charge balance
Gene essentiality (Hart 2015)
❌ = a count rose vs the target branch (regression) · Full workflow run · this comment is edited as results come in |
… verifier The step-specific scripts used to run the overhaul (identity, protonate, charge_fix, the apply_* and *_verify scripts, harmonize_xref) are removed. In their place, code/annotation/verifyAnnotations.py checks metabolite and reaction cross-references against chemical structure via the MetaNetX hub and classifies each as confirmed / wrong / missing / drift. It runs on the whole model or on just the entities a pull request changed (--mets / --rxns), so new annotation mistakes - e.g. from a newly added reaction - are caught; --fix applies the safe corrections (add missing, update deprecated MetaNetX ids) and leaves genuinely wrong ids for review. MetaNetX tables are downloaded once to a cache. structureConsistencyTest.py is now self-contained.
edkerk
force-pushed
the
annotation-overhaul
branch
from
July 14, 2026 06:03
13fb21a to
6bcdcac
Compare
Member
Author
|
No structural model changes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A structure-first overhaul of metabolite and reaction annotations, consolidating #964, #967 and #968. Every identifier is verified against chemical structure via MetaNetX (MNXref 4.5): metabolites matched on a protonation-invariant InChIKey, reactions on a participant-structure signature.
Metabolites: SMILES/InChI moved into
metabolites.tsv(withannotateGEMwiring and a CI consistency test); 1,025 structures regenerated to the pH-7.3 microspecies; 2,792 deprecated MetaNetX ids updated; 7,673 verified cross-references added; newmetSeedIDcolumn (3,429 ModelSEED, #968); 61 wrong-compound corrections; 12 invalid charges fixed (balance 277→271).Reactions (#967): 1,217 deprecated MetaNetX ids updated; 3,652 verified cross-references added (BiGG, MetaNetX, Rhea, KEGG).
Deferred (documented, not here): broader charge/formula rebalancing (coupled to reaction proton accounting), stereochemistry-only and unresolvable cross-ref corrections, and the #626/#618 duplicate merges. Reproducible tooling is under
code/annotation/; the plan is indata/annotation/OVERHAUL_PLAN.md.