Skip to content

Network-backend API refactor: Phase 5 #135

Closed

Description

@tonywu1999

Phase 5: merging, evidence sources, provenance, and PTM integration

Summary

Phase 5 of the refactor toward a generic network-backend API
(tracked in MSstats-ai/todos/active/TODO-MSBio-20261001_refactor_api.md).
It adds the tools for working with a network after get_network()
returns it: combining networks, filtering by evidence source, recording
where a network came from, and interpreting PTM networks with a global
profiling experiment. Phase 5 shipped as four PRs, all merged to devel.

Scope

Technical Details

5a. merge_networks() (#130)

merge_networks(...) combines networks from one or more get_network()
calls (different backends, query types, or cutoffs) into one list that
passes validate_network().

  • Edges are deduplicated on source + target + backend_database +
    statement_id. CoGEx lists an undirected Complex in both directions
    under one hash, and this key keeps both rows. When an edge appears in
    several networks, their query_type values are combined (;-joined, in
    order of first appearance) and the copy with the larger evidence_count
    is kept (the first one on ties; an NA count loses to any count). A
    message is shown when the copies differ in evidence_count or
    confidence. The same relation from two backends stays two rows.
    Columns that only one network has are filled with NA.
  • Nodes are keyed on id + site, since nodes$id repeats for PTM
    site rows. node_role values are combined (;-joined), and
    validate_network() now splits node_role on ;. included_in_query
    is any() across the networks, because it describes the query that was
    sent. has_measured_sites is set per protein id after merging.
  • Without entities, measured, logFC, and adj.pvalue must agree
    for each node key. If they don't, the merge stops, naming the nodes and
    the columns that differ. With entities, those columns are
    recomputed from the table; nodes not in it become measured = FALSE
    with NA statistics.
  • regulators and provenance elements are row-bound with column fill;
    provenance is not deduplicated, so two retrievals stay two rows. Other
    list elements are dropped with a message. Each input is validated
    first, so errors name the network (network 2, or its argument name).
  • Test helper tests/testthat/helper-contract.R:
    expect_backend_contract(backend, entities, ...) checks the shape of
    backend_capabilities(). For each query type it lists, the helper runs
    get_network(), validates the result, and checks backend_database,
    query_type, and that the interactions are within the capabilities. It
    also checks that merge_networks(network, network) gives the network
    back. It runs against the INDRA mock in test-backend-indra.R, so every
    future backend gets the same checks.

5b. INDRA evidence-source constants (#131)

  • Exported INDRA_DATABASE_SOURCES (22: acsn, bel, bel_lc,
    biogrid, cbn, conib, creeds, crog, ctd, dgi, drugbank,
    hprd, minerva, omnipath, pc, pe, psp, signor, tas,
    trrust, ubibrowser, vhn) and INDRA_TEXT_MINED_SOURCES (11:
    eidos, geneways, gnbr, isi, medscan, reach, rlimsp,
    semrep, sparser, tees, trips), documented on one help page,
    ?indra_evidence_sources.
    Usage: get_network(..., evidence_sources = INDRA_DATABASE_SOURCES).
  • The lists come from indra.util.statement_presentation (INDRA 1.23.0),
    the module CoGEx's own app uses. These are INDRA's internal names
    (psp, pc, bel_lc, pe, vhn). A live CoGEx check on 1926
    statements confirmed these are the names in source_counts. The
    SOURCE_INFO names (phosphosite, biopax, ...) would have silently
    filtered out PhosphoSitePlus, Pathway Commons, and BEL.
  • An unknown source name warns rather than errors, because INDRA adds
    sources between releases. For a known alias, the warning names the
    internal name to use ("phosphosite" → "psp").
    getSubnetworkFromIndra(sources_filter =) warns too.
  • backend_capabilities(indra_backend())$evidence_sources is now
    list(database =, text_mined =).

5c. Provenance, save_network(), load_network() (#132)

  • Every get_network() result has network$provenance: a data.frame with
    one row per call and the columns backend_database, query_type,
    retrieved_at (UTC, taken when the response arrives),
    backend_version, backend_url, organism, parameters (JSON of the
    shared and backend arguments), and package_version. It joins to edges
    on backend_database + query_type. INDRA CoGEx has no version
    endpoint, so its backend_version is NA and retrieved_at is the
    only record of when the network was retrieved.
  • It's a list element rather than an edge column because a timestamp
    describes a retrieval, not a statement. As an edge column,
    merge_networks() would have to pick one of two timestamps per edge.
  • save_network(network, file) validates the network and writes an .rds
    file. It warns when provenance is missing (a sign the list was rebuilt
    by hand). load_network(file) reads the file and validates it, so a
    file saved under an older contract gives a clear error.
  • filter_by_curation() keeps provenance. deleteEdgeFromNetwork() and
    filterSubnetworkByContext() take the tables separately and can't keep
    it; that limitation was accepted when the network was kept a plain list.

5d. PTM integration with a global profiling experiment (#134)

MSstatsPTM's groupComparisonPTM() returns site changes adjusted for
protein abundance (ADJUSTED.Model) and the global protein changes
(PROTEIN.Model). Showing both on one network used to need one stacked
input table, which also put every significant global protein into the
query. 5d separates the two steps: build the network from the PTM table,
then copy the global values onto its nodes.

add_entity_values(network, entities, overwrite = FALSE) copies an
entity table's logFC / adj.pvalue onto matching nodes. It sends no
query.

  • entities must come from convert_ids(); at least one row has to be
    grounded.
  • Nodes are matched on id + site first. Nodes left unmatched are then
    matched on grounding (namespace + entity_id). The grounding step
    covers latent nodes, which have the backend's name as their id: a
    kinase added with include_entities comes back from INDRA as MAPK1,
    while the global table has P28482. A node that matches several rows by
    grounding (isoforms) gets no values, with a message.
  • Matched rows get measured = TRUE. A protein that has only site rows
    gets a new protein-level row (site = NA) holding the global values, so
    cytoscapeNetwork() colors the protein node with no display change.
  • Nodes that already have values cause an error unless overwrite = TRUE.
    Rows that match no node are not added. Edges, regulators, and
    provenance are unchanged. The source of the values is not recorded.

filter_by_ptm_site(network) takes a network and returns a network,
like filter_by_curation(); it is not a get_network() argument. It
keeps an edge when the edge's site is one of the measured sites of its
target. Compared with the internal .filterByPtmSite() that
getSubnetworkFromIndra() uses:

  • Edges are checked one at a time, not pooled per source + target +
    interaction. Before, an edge for an unmeasured site survived whenever
    another statement for the same pair hit a measured site. Checking per
    edge also makes the result independent of whether this filter or
    filter_by_curation() runs first.
  • Node sites are split on _ and ;; before, only on _.
  • A node is dropped only when it had edges and lost all of them, so
    isolated nodes stay, and provenance is kept.
  • Edges with site = NA are dropped. That includes every non-modification
    statement and every STRING edge, which is documented. The function stops
    when no edges remain.
  • getSubnetworkFromIndra(filter_by_ptm_site = TRUE) deliberately keeps
    the old helper until Phase 7, so existing scripts keep their results.

Other changes in #134:

  • UniProt FASTA headers. prepare_entities() / convert_ids() accept
    them (sp|P32004|L1CAM_HUMAN, plus a site suffix for PTMs), as FragPipe
    writes them. id and parent_id keep the header; the conversion uses
    the accession or mnemonic (UNIPROT_HEADER_PATTERN,
    .parse_uniprot_headers(), .get_grounding_inputs()).
  • Bug fix: GlobalProtein = "missing". MSstatsPTM writes "missing"
    for sites whose protein has no global result. prepare_entities() took
    it as the parent, leaving those rows ungrounded (1143 of 3113 CPTAC
    sites). annotateProteinInfoFromIndra() had the same bug (3875 of
    16356 IFNG rows). Both now parse the parent from the site ID.
  • Bug fix: sites on several group members. .stripPtmSite() on a
    protein group dropped every member after the first site
    ("P13747_S148;P23132_T20" → "P13747"). The new
    .stripPtmSiteFromGroups() strips each member and joins the unique
    parents, both for the "missing" fallback and when there is no
    GlobalProtein column.
  • Vignette. vignettes/PTM-Analysis.Rmd is rewritten as "PTM Analysis
    with Global Profiling", on CPTAC CCRCC (tumor vs normal, TMT, FragPipe;
    Li et al., Cancer Cell 2023, doi:10.1016/j.ccell.2023.06.009). It covers
    236 sites at adj.p < 0.05. The sites-only subnetwork has 30 edges, 1
    after filter_by_ptm_site(). With ten common kinases added, it has 599
    edges, 19 after filtering. add_entity_values() then colors PML, LMNA,
    BRD4, IQGAP1, CDK18, and the kinases MAPK1, MAPK3, AKT1, and PRKACA. The
    INDRA chunks don't run at build time; their results load from
    inst/extdata/cptac-ccrcc-network.rds, made by
    inst/script/cptac-ccrcc-network.R. garrido-2024.csv was removed.

Affected Files

  • R/network-postprocess.R: merge_networks(), add_entity_values(),
    filter_by_ptm_site(), and helpers
  • R/network-io.R (new): provenance, save_network(), load_network()
  • R/backend-indra.R, R/AllGenerics.R: evidence-source constants,
    provenance in get_network(), capabilities, FASTA-header grounding
    inputs
  • R/contract.R: node_role split on ;
  • R/entities.R: FASTA headers, GlobalProtein = "missing" fix
  • R/annotateProteinInfoFromIndra.R: "missing" fix,
    .stripPtmSiteFromGroups()
  • R/getSubnetworkFromIndra.R: source-name warning
  • vignettes/PTM-Analysis.Rmd, inst/extdata/cptac-ccrcc-*.csv / .rds,
    inst/script/cptac-ccrcc-network.R
  • Tests: helper-contract.R, test-merge-networks.R,
    test-network-io.R, test-ptm-integration.R, test-backend-indra.R,
    test-entities.R, test-annotateProteinInfoFromIndra.R
  • NAMESPACE, NEWS.md, man/ for each new export

Activity

  1. changed the title [-]Network-backend API refactor[/-] [+]Network-backend API refactor: Phase 5[/+] on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions