Skip to content

feat(subscriber): named event streams with a Morpho governance stream - #452

Merged
spalen0 merged 2 commits into
mainfrom
morpho-governance-ws
Oct 26, 2026
Merged

spalen0 merged 2 commits into
mainfrom
morpho-governance-ws

Conversation

@spalen0

@spalen0 spalen0 commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Closes #447. Part of #437.

What

The continuous subscriber from #442 can now follow more than one Envio entity. A new morpho stream observes MorphoGovernanceEvent (indexed by yearn/yearn-envio#71) for every V1 and V2 vault in protocols/morpho/config.py. It runs in the same process as the timelock pilot and uses the same WebSocket + HTTP reconciliation.

It is observation only, and sends no alerts. The daily governance.py / governance_v2.py jobs stay the alerting path. Lifecycle rules and grouped delivery are #448.

Stream Entity Inbox
timelock (default) TimelockEvent timelock_subscriber.db, timelock_sub_* (unchanged)
morpho MorphoGovernanceEvent morpho_subscriber.db, morpho_sub_*

How

  • Shared code moved from protocols/timelock/subscriber/ to utils/subscriber/. An EventStream names the entity, its address field, the field selection, the SQLite file, and the table prefix. The inbox is bound to its stream, and reconcile, the worker, notices and the report read the stream from it. Nothing hardcodes TimelockEvent anymore.
  • One process: run_workers runs every selected stream in one process.
    • One systemd READY, and a WATCHDOG ping only while every stream is alive, so a wedged stream restarts the unit instead of hiding behind a healthy one.
    • One HTTP poster with a persistent requests.Session per thread (request_with_retry gained an optional session).
    • SharedChainHeads: one chain_metadata read per cycle for all streams.
    • One WebSocket connection, one reconcile loop, its own checkpoints, and its own health per stream. A failing stream never blocks another.
    • A stream that crashes is re-raised, so run_with_alert reports it.
  • Selecting streams: EVENT_SUBSCRIBER_STREAMS (or --streams before the subcommand) picks the streams. The default is timelock, so the running deployment's behavior is unchanged. The run, reconcile-once, report and health subcommands cover every selected stream.
  • Notices: each stream posts under its own label ([timelock-subscriber], [morpho-subscriber]). Morpho notices show the vault name and generation, the sender, and the ABI-named argsJson. Timelock notice text is unchanged.
  • Schema v2: timelock_address is renamed to contract_address in place. The existing pilot DB keeps its rows and checkpoints, and a test covers the deployed v1 shape.
  • Reorg overlap and block span: added for HyperEVM (999, ~1s blocks) and Robinhood (4663, ~0.1s blocks, measured from the public RPCs). Both give about 15–20 minutes of overlap.
  • Entrypoint: the systemd unit and protocols/timelock/event_subscriber.py keep their names, so nothing changes for ops until the Morpho stream is enabled.

Rollout

The configured Envio deployment doesn't serve MorphoGovernanceEvent yet. HyperEVM and Robinhood also have no chain_metadata rows there. Enable the stream once yearn-envio#71 is deployed and has backfilled all five chains; the runbook has the steps:

uv run python protocols/timelock/event_subscriber.py --streams morpho reconcile-once   # must exit 0 (run twice: the first run only sets checkpoints)
echo 'EVENT_SUBSCRIBER_STREAMS=timelock,morpho' | sudo tee -a /etc/monitoring/.env
sudo systemctl restart monitoring-timelock-subscriber

Its first reconcile starts at each chain's indexed head, so historical events are never announced.

Testing

  • ruff and ty pass. Full suite: 2032 passed, 5 failed. The failures also happen on main and come from the local setup: 4 TestSafeApiQuota tests fail with a real SAFE_API_KEY in .env, and test_sd_notify_sends_datagram fails with AF_UNIX path too long on macOS temp paths.
  • New tests/test_event_subscriber_streams.py (16 tests):
    • Morpho query shape and scope (every configured vault, with labels).
    • Separate inboxes and tables, and independent checkpoints when one stream's range query fails.
    • One metadata read shared by two streams, plus staleness and retry for SharedChainHeads.
    • One session per thread.
    • Morpho notice text and per-stream report titles.
    • run_workers following both streams over the fake WebSocket. It sends one READY and one STOPPING; the watchdog ping is withheld while one stream is dead, and a crashed stream is surfaced.
    • CLI stream parsing and the health checks.
  • Existing timelock subscriber tests were updated for the move and the explicit stream; their behavior is unchanged. The test fakes serve both entities.
  • Live, against the configured Envio (read-only, temporary CACHE_DIR):
    • --streams timelock,morpho reconcile-once and a 15s run, stopped with SIGTERM.
    • Timelock: the WebSocket connected, the 50-row snapshot was stored, every reconcile succeeded, and the process exited cleanly.
    • Morpho: each chain failed on its own with field 'MorphoGovernanceEvent' not found or no chain_metadata row, and the WebSocket retried with backoff. Timelock was not affected.

🤖 Generated with Claude Code

Generalize the continuous timelock subscriber so one process follows
several Envio entities (#447, part of #437):

- Move the subscriber to utils/subscriber/. An EventStream names the
  entity, its address field, field selection, SQLite file and table
  prefix; the inbox, reconcile, worker, notices and report take it from
  the inbox instead of hardcoding TimelockEvent.
- Timelock keeps timelock_subscriber.db and its timelock_sub_* tables.
  Schema v2 renames timelock_address to contract_address in place.
- Add the morpho stream (MorphoGovernanceEvent from yearn-envio#71) for
  every configured V1/V2 vault, in morpho_subscriber.db. Observation
  only: no alerts, daily governance jobs stay the alerting path.
- run_workers runs every stream in one process: one READY, a watchdog
  ping only while every stream is alive, one HTTP session per thread,
  and one chain_metadata read per cycle shared by all streams.
- EVENT_SUBSCRIBER_STREAMS (default timelock) selects streams; run,
  reconcile-once, report and health cover each. Per-stream notice
  labels ([timelock-subscriber], [morpho-subscriber]).
- Reorg overlap and block span for HyperEVM (999) and Robinhood (4663).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spalen0
spalen0 merged commit decfcb9 into main Oct 26, 2026
3 checks passed
@spalen0
spalen0 deleted the morpho-governance-ws branch October 26, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generalize the continuous subscriber for named governance streams

1 participant