Skip to content

Harden SQLiteMetricsStore: circuit breaker, index, retention - #35

Open
lmajano wants to merge 12 commits into
developmentfrom
fix/sqlite-store-hardening
Open

lmajano wants to merge 12 commits into
developmentfrom
fix/sqlite-store-hardening

Conversation

@lmajano

@lmajano lmajano commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Description

Hardens the opt-in SQLiteMetricsStore (reviewer findings):

  1. No more error per evaluation. A broken store (missing datasource/driver, locked or full DB) used to swallow the schema failure, so WireBox resolution "succeeded" and RuleEventBus.metricsStoreFailed never tripped; every publish then ran a failing INSERT and RuleEventBus logged an error each time. The store now tracks consecutive failures itself. After circuitBreakerThreshold (default 5) it opens a circuit for circuitBreakerCooldownSeconds (default 60): one error is logged when it opens, recordEvent is a cheap no-op while open, then a single trial insert is allowed; on success one info line is logged and the circuit closes. A failed trial stays open quietly. A failed ensureSchema is remembered (schemaReady=false) and the next recordEvent re-attempts the DDL before inserting.
  2. Index + retention. CREATE INDEX IF NOT EXISTS idx_rulebox_events_rule ON rulebox_events (rulebookName, ruleName, id) backs queryRuleMetrics (called once per rule on the chain page). New settings visualizer.retentionDays (30) and visualizer.maxStoredEvents (100000), 0 disables each, enforced by a DELETE (id-range / timestamp) once every 500 successful inserts, never on every insert. A failing prune is logged as a warning and does not trip the breaker.
  3. Documented trade-off: recordEvent still performs a synchronous INSERT on the request thread. Left as is on purpose; noted in the docs and class header.

Settings are read defensively in the store (missing/invalid values fall back to defaults), so it works whether or not the settings deep-merge fix has landed. New defaults were added inside the existing visualizer = {...} block of ModuleConfig.bx. Also fixed the stale class headers in SQLiteMetricsStore.bx / IMetricsStore.bx that called SQLite "the default" (the default is InMemoryMetricsStore@rulebox), and added a short docs subsection to docs/guides/visualizer.md.

Behavior change for maintainers: recordEvent no longer throws on a DB failure (the store absorbs it), so RuleEventBus's per-event error log no longer fires for this store.

Testing: the SQL runner and clock are injectable (setSqlRunner, setClock, setPruneEvery) so breaker, schema-retry and prune-cadence logic is covered without a database; DB-backed specs use the existing skip guard. Verified locally on BoxLang 1.18.0 with a real bx-sqlite datasource using a throwaway BDD shim (the TestBox runner does not work under miniserver): against the old store 13 of 15 new no-DB specs and all 3 SQLite specs fail; with the change all 23 specs in the file pass. Repro of the original problem: 20 recordEvent calls against a missing datasource threw 20 times (so the bus logged 20 errors); now 0 throws and exactly 1 error log. CI runs the real suite.

Issues

Reviewer findings on SQLiteMetricsStore ahead of 2.0.0 (no separate GitHub issue filed).

Type of change

  • Bug Fix
  • Improvement
  • New Feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My code follows the style guidelines of this project cfformat
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes (full TestBox runner not runnable locally; verified the SQLiteMetricsStoreSpec via a shim, CI runs the suite)

Noticed, not fixed

  • Existing read queries (queryEvents, summaries, reset) still call queryExecute directly and are not behind the circuit breaker (they run on admin page requests, not per evaluation).
  • Named helper functions inside run() in a spec are hoisted into one scope, so two describe blocks cannot define the same helper name (I used distinct names).
  • The retention-by-days DELETE scans timestamp without an index (bounded by maxStoredEvents; if you disable that limit on a very high-volume table, consider a timestamp index).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX


Generated by Claude Code

claude added 3 commits October 3, 2026 11:53
ColdBox merges an app's moduleSettings over a module's defaults with a
shallow append, so the documented minimal config
`visualizer = { enabled = true }` replaced the whole default visualizer
struct and dropped metricsStore/datasourceName. RuleEventBus.onDIComplete
then threw on the missing key (its catch block re-read the same key, so
the error escaped), the bus was never built, and every RuleBook run failed
with a null publish() call.

- ModuleConfig: single visualizerDefaults() source; onLoad() deep-merges
  the defaults under whatever the app supplied.
- RuleEventBus, Visualizer handler, SQLiteMetricsStore: tolerate missing
  visualizer keys (defence in depth); resolveMetricsStore() no longer reads
  settings inside its own catch block.
- Add VisualizerSettingsSpec.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
- Count consecutive failures in the store; after visualizer.circuitBreakerThreshold (5)
  open a circuit for circuitBreakerCooldownSeconds (60). recordEvent is a cheap no-op
  while open; one error is logged on open and one info on recovery. Insert failures no
  longer escape to RuleEventBus (which logged an error per evaluation).
- ensureSchema remembers a failed state and recordEvent re-attempts it before inserting.
- Add CREATE INDEX IF NOT EXISTS on rulebox_events (rulebookName, ruleName, id).
- Add retention: visualizer.retentionDays (30) and maxStoredEvents (100000), pruned with a
  DELETE every 500 inserts (0 disables each).
- Make the SQL runner and clock injectable so breaker/retention logic is unit-tested
  without a database; add DB-backed specs behind the existing skip guard.
- Fix stale class headers that called SQLite the default store.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
ModuleConfig is now the single place that sets up and checks the visualizer
settings: onLoad() deep-merges the app's values over visualizerDefaults(), then
throws RuleBox.InvalidSettingException for a value of the wrong shape. The
handler, RuleEventBus and SQLiteMetricsStore go back to reading the settings
directly, with no scattered fallbacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
claude added 8 commits October 3, 2026 14:50
retentionDays, maxStoredEvents, circuitBreakerThreshold and
circuitBreakerCooldownSeconds get their defaults and whole-number validation in
ModuleConfig. SQLiteMetricsStore reads them directly; numericSetting() is gone.
Specs build store settings from the module's completed settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
The defaults do not depend on instance state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
The defaults do not depend on instance state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
A module is loaded once, so the defaults are a static struct instead of a
function. configure() copies it, and onLoad() fills the keys an app left out
with append( static.VISUALIZER_DEFAULTS, false ), so the static is never
written to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yc5MDcm3tyR2yqPWs9RpgX
…ge' into fix/sqlite-store-hardening

# Conflicts:
#	ModuleConfig.bx
…re-hardening

# Conflicts:
#	ModuleConfig.bx
#	test-harness/tests/specs/VisualizerSettingsSpec.bx

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants