Skip to content

Fix: the Marvin 26.1 protonation formula column was dropping R groups - #296

Open
freiburgermsu wants to merge 3 commits into
ModelSEED:devfrom
freiburgermsu:marvin-26.1-protonation-r-group-fix
Open

freiburgermsu wants to merge 3 commits into
ModelSEED:devfrom
freiburgermsu:marvin-26.1-protonation-r-group-fix

Conversation

@freiburgermsu

@freiburgermsu freiburgermsu commented Sep 22, 2026

Copy link
Copy Markdown
Member

Follow-up to #295, which is already merged. Review of it found compounds losing their R groups — confirmed, and the cause was that PR's formula column, not Marvin and not the structures.

What was wrong

#295 wrote Marvin's own getFormula() into the formula column and deferred Print_Structure_Formula_Charge.py to a follow-up step. Marvin omits wildcard atoms from a formula; this repository renders them as R, in one line of that script:

formula = re.sub(r'\*', 'R', formula)

SMILE rows whose formula contains an R group therefore fell from 23.4's 8,704 to zero. Stearoyl-ACPs went from C32H60N3O9PR2S to C32H60N3O9PS, and Update_Compound_Structures_Formulas_Charge.py carried that into 6,052 compound recordscpd00049 "carboxylic acid" became CHO2, a generic compound quietly ceasing to be generic.

A counting note, because it changes the headline. Count R groups with R(?![a-z]), not "R" in formula — the elements Ru, Rb, Rh, Re and Rn match the naive test and inflate every count here by 8. The first report of this regression (mine, and the review that caught it) said 8,712 → 8. The true figures are 8,704 → 0: #295 did not leave eight R groups standing, it left none, and the apparent survivors were ruthenium and rubidium compounds.

The structures were never affected

8,727 SMILE structures carry a * in both bundles, identically, and this fix changes 0 structures. Nothing was ever stripped from a molecule.

That is also why it shipped: coverage, compound sets, InChI agreement, InChIKey consistency and stereochemical fidelity all inspect structures, and the structures were always right. A column-only defect was invisible to every check in the suite. A formula-column comparison against 23.4 is now part of the report.

The fix

formula and charge now come from Print_Structure_Formula_Charge.parse_structure — this repository's own function, imported rather than reimplemented, computed per row from that row's structure string. Deferring it was the error: the cascade consumes this column directly, so the bundle has to be correct as written rather than correct after a second script runs.

Two further cases had to be reconciled to make that hold everywhere:

  • The substitution only fires on the RDKit path. RDKit renders a dummy atom as *; OpenBabel and Marvin both omit them. So ISOCITHASE-P's *OP(=O)(=O)=O — an invalid valence only OpenBabel will read — still lost its R. The convention is now enforced as an invariant: a SMILE structure carrying * gets an R in its formula. This bundle violates it 0 times; 23.4 violates it 23 times.
  • Counting wildcards has its own trap. A molblock R atom — how RDKit writes every dummy atom, and therefore how every structure arrives through the import bridge — reads back from Marvin as symbol R#, not R. A wildcard set without R# counts zero on a molecule that plainly has them.

Result

this PR 23.4 as merged in #295
SMILE rows with an R group 8,727 8,704 0
ChEBI / KEGG / Rhea 2,071 / 957 / 30 2,071 / 957 / 30
MetaCyc 5,669 5,646
formula-column rows matching 23.4 76,479 68,466
invariant violations (SMILE) 0 23

Reading the diff

It looks total — ~141,900 insertions — because the bundle was regenerated and generated_on moved on every row. The content change is narrow:

rows
content identical (only generated_on differs) 132,528
formula changed 9,185
charge changed 84
structure changed 0
rows added or removed 0

Re-verified against the committed bundles: coverage is 141,715 rows and 53,127 compounds, both exactly 23.4's; 0 non-standard InChI; 0 InChIKeys disagreeing with the InChI beside them; one generated_on, tool_version and ph across all four files; and re-running a source rewrites it byte for byte.


Generated by Claude Code

freiburgermsu and others added 2 commits September 22, 2026 01:09
Review of ModelSEED#295 found compounds losing their R groups. Confirmed, and the
cause was this bundle's formula column, not Marvin and not the structures.

WHAT WAS WRONG. The first cut wrote Marvin's getFormula() into the formula
column and deferred Print_Structure_Formula_Charge.py as a follow-up step.
Marvin omits wildcard atoms from a formula; this repository renders them as R,
in one line of that script:

    formula = re.sub(r'\*', 'R', formula)

So SMILE rows whose formula contains R fell from 23.4's 8,712 to 8.
Stearoyl-ACPs went from C32H60N3O9PR2S to C32H60N3O9PS, and
Update_Compound_Structures_Formulas_Charge.py carried that into 6,052 compound
records -- cpd00049 "carboxylic acid" became CHO2, a generic compound quietly
ceasing to be generic.

THE STRUCTURES WERE NEVER AFFECTED: 8,727 SMILE structures carry a `*` in both
bundles, identically. Only the column was wrong. That is why it survived the
whole validation suite -- coverage, compound sets, InChI agreement, InChIKey
consistency and stereochemical fidelity all inspect structures, and the
structures were always right. A column-only defect was invisible to every
check. A formula-column comparison against 23.4 is now part of the report.

THE FIX. formula and charge come from Print_Structure_Formula_Charge's own
parse_structure -- imported, not reimplemented -- computed per row from that
row's structure string. Deferring it was the error; the cascade consumes this
column directly, so the bundle has to be correct as written.

Two further cases had to be reconciled to make that hold everywhere:

  parse_structure's substitution only fires when RDKit produced the formula,
  since RDKit renders a dummy atom as `*`. OpenBabel and Marvin both omit them,
  so structures like ISOCITHASE-P's `*OP(=O)(=O)=O` -- an invalid valence only
  OpenBabel will read -- still lost their R. The convention is now enforced as
  an invariant: a SMILE structure carrying `*` gets an R in its formula. This
  bundle violates it 0 times; 23.4 violated it 23 times.

  Counting wildcards to enforce that has its own trap: a molblock R atom, which
  is how RDKit writes every dummy atom and therefore how every structure
  arrives through the import bridge, reads back from Marvin as symbol "R#", not
  "R". A wildcard set without R# counts zero on a molecule that plainly has
  them.

RESULT. R-bearing SMILE rows 8,735 against 23.4's 8,712; per source ChEBI
2,071=2,071, KEGG 964=964, Rhea 30=30, MetaCyc 5,670 vs 5,647. Formula-column
agreement with 23.4 rises from 68,466 rows to 76,479. Coverage, structures,
InChI, InChIKey and stereochemistry are unchanged -- 141,715 rows, still
exactly 23.4's count, 0 non-standard InChI, 0 InChIKey disagreements.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Self-review of the numbers in the previous commit, not a data change --
the bundles are untouched and were already correct.

Counting R groups with `"R" in formula` also matches the elements Ru, Rb, Rh,
Re and Rn. That inflated every figure by 8 and, worse, made the regression look
milder than it was: the eight rows that appeared to survive with an R group
were ruthenium and rubidium compounds, not R groups at all.

  SMILE rows with an R group      was reported   actually
    this bundle                        8,735       8,727
    marvin 23.4                        8,712       8,704
    as merged in ModelSEED#295                      8           0

So ModelSEED#295 did not leave 8 R groups standing; it left none. Per source the
corrected counts are ChEBI 2,071 = 2,071, KEGG 957 = 957, Rhea 30 = 30, and
MetaCyc 5,669 against 5,646 -- the +23 delta, and every other conclusion, is
unchanged.

Re-verified against the committed bundles while correcting this:

  coverage         141,715 rows and 53,127 compounds, both exactly 23.4's
  invariant        0 violations in either direction; 23.4 has 23
  vs merged ModelSEED#295   9,185 formulas and 84 charges changed, 0 structures,
                   0 rows added or removed
  columns          one generated_on, tool_version and ph across all four files
  reproducibility  re-running a source rewrites it byte for byte

Both the report and the script docstring now state the rule (`R(?![a-z])`) next
to the numbers, so the next person to check does not repeat it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@freiburgermsu
freiburgermsu force-pushed the marvin-26.1-protonation-r-group-fix branch from c129c91 to ee55974 Compare September 22, 2026 06:35
The manifest is what a maintainer reads before touching one of these
bundles, and it described the tautomer gap and the RDKit import bridge but
said nothing about the column that actually broke.

Each 26.1 protonation entry now records that formula and charge come from
Print_Structure_Formula_Charge.parse_structure, that substituting Marvin's
getFormula() is what dropped the R group from 8,704 rows, and the invariant
the file satisfies: a SMILE structure carrying `*` carries an R in its
formula, 8,727 times, with 0 violations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant