Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/runbooks/0100-coverage-sweep-triage.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,11 @@ three outcomes:
permanent family-wide ignore would hide the next pool of a real venue.
- **(c) A router, aggregator or other non-AMM contract.** Add a permanent
`[[contract]]` entry with `reason` and `task`. Never register it.
If the same non-AMM wasm keeps deploying new instances, each new one pages
on its own. Add one `[[wasm]]` entry with `not_a_venue = true` in place of
`until` instead (task 0336: the securities-financing ledger `f3dd3edf…`,
four new instances in one week). Do this only when the code itself rules out
a market, not because the instances seen so far are small.

Also look at:

Expand All @@ -147,7 +152,8 @@ Also look at:
`packages/coverage-sweep-probe/allowlist.toml`:

- `cargo test -p coverage-sweep-probe` validates it: reason and task on every
entry, `until` on every `[[wasm]]` entry, valid `C…` strkeys, 64-lowercase-hex
entry, `until` on every `[[wasm]]` entry except a `not_a_venue` one (which
must not have it), valid `C…` strkeys, 64-lowercase-hex
hashes, no duplicates, no unknown keys.
- The list is compiled into the binary, so an edit **ships only with an
EventBridge deploy** (§4.4).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
id: "0336"
title: "coverage-sweep-unclassified stays in ALARM: new instances of the securities-financing ledger wasm page one by one — allow-list the family as not a venue"
type: BUG
status: active
assignee: akot
related_adr: []
related_tasks: ["0100", "0323", "0334"]
tags: [layer-infra, observability, coverage, priority-high, effort-small]
links:
- "../../../packages/coverage-sweep-probe/allowlist.toml"
- "../../../packages/coverage-sweep-probe/src/allowlist.rs"
- "../../../docs/runbooks/0100-coverage-sweep-triage.md"
history:
- date: "2026-10-23"
status: active
who: akot
note: "Task created from the daily stuck-alarm digest: coverage-sweep-unclassified in ALARM for 11d 4h"
---

# New instances of the securities-financing ledger wasm keep the sweep in ALARM

## Summary

`prices-production-coverage-sweep-unclassified` has been in ALARM since
2026-10-12 07:18 CEST. [[0334]] fixed the residual it was raised for, yet the
manual run after its deploy still found 4 contracts. All four run wasm
`f3dd3edf…`, the securities-financing ledger that [[0334]] allow-listed by
one contract id (`CDU47AIU…`). The family deploys new instances every few
days, so a `[[contract]]` entry per instance would page again every Monday.

## Context

Runs from `/aws/lambda/prices-production-coverage-sweep-probe`
(`coverage sweep complete`):

| Run (UTC) | unclassified | events | Note |
| --- | --- | --- | --- |
| 2026-10-12 05:17 | 3 | 404 | Comet `CASYAKK3…` (393), `CDU47AIU…`, `CD35HSTR…` ([[0334]]) |
| 2026-10-19 05:17 | 7 | 595 | before PR #405 merged |
| 2026-10-19 09:03 | 4 | 15 | manual, after the 0334 deploy; Comet and both 0334 entries gone |

The four left on 2026-10-19 09:03, all wasm
`f3dd3edfd637a9654054181a9c47391ddafa723697b982698b3c9662a9df974e`,
`trade_opened`/`trade_closed` with shape `sym|u64|bytes -> map`:

- `CAKQ6DWL…` 8 events, ledgers 64,908,423–64,941,821
- `CCLN72NM…` 4 events, 64,958,834–64,962,719
- `CBZYHS2D…` 2 events, 64,944,905–64,959,086
- `CB4B4ELL…` 1 event, 64,978,560

Since [[0323]] the alarm holds the latest run's state, so it stays in ALARM
until a run finds none: the 2026-10-26 run, or a manual invoke after deploy.

## Design

[[0100]] D4 forbids a `[[wasm]]` entry without `until`, because a permanent
family-wide ignore would hide the next pool of a real venue. That reason does
not hold for a family whose code is not a market. Add an explicit exception:

- `[[wasm]]` takes `not_a_venue = true` in place of `until`. Both together,
or neither, fails validation.
- `f3dd3edf…` moves from `CDU47AIU…`'s `[[contract]]` entry to one such
`[[wasm]]` entry.
- Runbook §3 outcome (c) says when to use it: the wasm itself rules out a
market, not the size of the instances seen so far.

## Acceptance Criteria

- [ ] Validator: `not_a_venue` without `until` accepted; with `until`, or
`until` missing without it, refused (unit tests)
- [ ] The five known instances match `wasm:f3dd3edf…` (unit test)
- [ ] Runbook §3 describes `not_a_venue`
- [ ] EventBridge deployed; one manual invoke logs `coverage sweep complete`
with `unclassified=0`
- [ ] `coverage-sweep-unclassified` back to OK

## Notes

- The order-book wasm `4d6abb0d…` is allow-listed by two contract ids
(`CBOSXSEZ…`, `CD35HSTR…` from [[0334]]). It is the same pattern and a
candidate for `not_a_venue` if a third instance appears.
- The Comet `[[wasm]]` entry `8abc2891…` (`until = "0300"`) is stale: 0300
shipped. Not removed here.
17 changes: 11 additions & 6 deletions packages/coverage-sweep-probe/allowlist.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@
# the task that registers the family). A permanent family-wide
# ignore would hide the next pool of a real venue, so a [[wasm]]
# entry without `until` fails validation.
# The one exception is `not_a_venue = true` (task 0336), in
# place of `until`: a family whose code cannot be a market we
# price, read from the wasm, that keeps deploying new instances.
#
# Every entry needs a non-empty `reason` and `task`. Unknown keys are rejected.
#
Expand Down Expand Up @@ -188,12 +191,6 @@ wasm = "4d6abb0d5dfae9b2450c694e123f1f4f2e614bd109c6b383a03d36d49b2b6bf8"
reason = "on-chain order book of tokenised instruments (NewBuyInitiatedTrade), a further instance of the wasm above; instruments are u64 ids with no token addresses, not priceable here"
task = "0334"

[[contract]]
id = "CDU47AIU3BKKRTJBTPFHFXLVJG4BGNOXKDYUBDTKRIWAFP2RLWFKL4HV"
wasm = "f3dd3edfd637a9654054181a9c47391ddafa723697b982698b3c9662a9df974e"
reason = "securities-financing trade ledger (repo, stock loan, FX forward); trade_opened/trade_closed carry notional and rate_bps, no token price"
task = "0334"

[[contract]]
id = "CBP76I2FRMUKYKIYYBKN3DH7TSWMFAJF2WTDC7Q2OHQFBIVWP7CAAI5Q"
wasm = "76e60d025ee6c0322cb19f4846b9587ab172870802251de903e93545371678b6"
Expand Down Expand Up @@ -225,3 +222,11 @@ hash = "8abc28913035c07411ed5d134e6bfeab4723d97ddd4d1a22a0605d35c94d1a36"
reason = "Comet weighted pool, Blend backstop BLND/USDC (CAS3FL6T…), a venue not yet indexed; 53k swaps since 2024-05-02"
task = "0100"
until = "0300"

# --- non-venue families (permanent, keyed by wasm) ----------------------------

[[wasm]]
hash = "f3dd3edfd637a9654054181a9c47391ddafa723697b982698b3c9662a9df974e"
reason = "securities-financing trade ledger (repo, stock loan, FX forward); trade_opened/trade_closed carry notional and rate_bps, no token price; new instances keep appearing (CDU47AIU… in 0334, four more by 2026-10-19), so this is keyed by wasm"
task = "0336"
not_a_venue = true
73 changes: 62 additions & 11 deletions packages/coverage-sweep-probe/src/allowlist.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,23 @@ pub struct ContractEntry {
pub task: String,
}

/// A whole wasm family, allow-listed until `until` ships.
/// A whole wasm family, allow-listed until `until` ships, or for good when
/// `not_a_venue` (task 0336).
#[derive(Debug, Clone, Deserialize, PartialEq, Eq)]
#[serde(deny_unknown_fields)]
pub struct WasmEntry {
/// 64 lowercase hex characters.
pub hash: String,
pub reason: String,
pub task: String,
/// The task whose delivery removes this entry. Required (D4).
/// The task whose delivery removes this entry. Required (D4) unless
/// `not_a_venue`.
#[serde(default)]
pub until: String,
/// The family's code cannot be a market we price (a securities ledger,
/// say), so no task will ever register it. Takes the place of `until`.
#[serde(default)]
pub not_a_venue: bool,
}

/// Why an allow-list was refused.
Expand Down Expand Up @@ -87,7 +93,8 @@ impl AllowList {
/// Enforce the rules a list must meet before it may hide anything:
/// - every entry has a non-empty `reason` and `task`;
/// - every `[[wasm]]` entry has a non-empty `until` (D4: a permanent
/// family-wide ignore would hide the next pool of a real venue);
/// family-wide ignore would hide the next pool of a real venue), or
/// `not_a_venue` and no `until` (task 0336);
/// - every contract `id` is a valid `C…` strkey (an account key or a typo
/// would silently match nothing);
/// - every hash, including a contract entry's informational `wasm`, is
Expand Down Expand Up @@ -127,7 +134,13 @@ impl AllowList {
}
require(&who, "reason", &e.reason)?;
require(&who, "task", &e.task)?;
require(&who, "until", &e.until)?;
if !e.not_a_venue {
require(&who, "until", &e.until)?;
} else if !e.until.trim().is_empty() {
return Err(AllowListError::Invalid(format!(
"{who}: `not_a_venue` and `until` are exclusive"
)));
}
if !hashes.insert(e.hash.as_str()) {
return Err(AllowListError::Invalid(format!("{who}: duplicate hash")));
}
Expand Down Expand Up @@ -174,23 +187,48 @@ mod tests {
use super::*;

const SUSHI_V3: &str = "003710b383f9da7d650a7f719a7be479110266427817ebbed61d924505fcd7c7";
const SECURITIES_LEDGER: &str =
"f3dd3edfd637a9654054181a9c47391ddafa723697b982698b3c9662a9df974e";
const AQUARIUS_ROUTER: &str = "CBQDHNBFBZYE4MKPWBSJOPIYLW4SFSXAXUTSXJN76GNKYVYPCKWC6QUK";

#[test]
fn embedded_list_parses_with_the_seeded_entries() {
let list = AllowList::embedded().expect("embedded allow-list parses");
assert_eq!(list.contract.len(), 29, "{list:?}");
assert_eq!(list.wasm.len(), 3, "{list:?}");
// Every family-wide entry is temporary and names the task that ends it:
// SushiSwap V3 (0290) and the Comet BLND/USDC pool (0300).
assert_eq!(list.contract.len(), 28, "{list:?}");
assert_eq!(list.wasm.len(), 4, "{list:?}");
// Every family-wide entry is temporary and names the task that ends it
// (SushiSwap V3, 0290; the Comet BLND/USDC pool, 0300), except the one
// non-venue family (0336).
assert!(
list.wasm
.iter()
.all(|w| w.until == "0290" || w.until == "0300"),
list.wasm.iter().all(|w| if w.not_a_venue {
w.hash == SECURITIES_LEDGER
} else {
w.until == "0290" || w.until == "0300"
}),
"{list:?}"
);
}

#[test]
fn securities_ledger_family_is_allow_listed_by_wasm() {
// The 2026-10-19 sweep (task 0336): four new instances of the wasm
// 0334 allow-listed by one contract id, each paging on its own.
let list = AllowList::embedded().unwrap();
for id in [
"CDU47AIU3BKKRTJBTPFHFXLVJG4BGNOXKDYUBDTKRIWAFP2RLWFKL4HV",
"CAKQ6DWLPTKFDHYON346SBAYJJDTVO5DKTMZ7737HBF6NCGASIMOM4U2",
"CCLN72NMUDZVXK6V6W5RQ4GHGHMX437G2FMGT6YKVNB7BAAH7XFLP5SZ",
"CBZYHS2DVYAIOURIAXVUNTTAGTUXH3I4SWFWKN7FAKFKNOKLQ5LIR5YB",
"CB4B4ELLM4YTF7HLITB52OFCYPGHPT763JZBHVUOD4AN4FJR7RWVZXJ6",
] {
assert_eq!(
list.match_row(id, Some(SECURITIES_LEDGER)),
Some(format!("wasm:{SECURITIES_LEDGER}")),
"{id}"
);
}
}

#[test]
fn comet_pool_family_is_allow_listed_until_0300() {
// Phase-1 classification (task 0100): the Comet BLND/USDC pool is a real
Expand Down Expand Up @@ -276,6 +314,19 @@ mod tests {
refused(&wasm_toml(SUSHI_V3, Some(" ")));
}

#[test]
fn not_a_venue_takes_the_place_of_until() {
let entry = |extra: &str| {
format!(
"[[wasm]]\nhash = \"{SECURITIES_LEDGER}\"\nreason = \"r\"\ntask = \"0336\"\n{extra}"
)
};
AllowList::parse(&entry("not_a_venue = true\n")).unwrap();
let e = refused(&entry("not_a_venue = true\nuntil = \"0336\"\n"));
assert!(e.contains("exclusive"), "{e}");
refused(&entry("not_a_venue = false\n"));
}

#[test]
fn duplicates_are_refused() {
let e = refused(&(contract_toml(GOOD_ID, "a", "1") + &contract_toml(GOOD_ID, "b", "2")));
Expand Down
Loading