Skip to content

DOC-7176 Document node-redis geographic failover [PARKED] - #4251

Open
andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-7176-nodejs-failover
Open

andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-7176-nodejs-failover

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Adds Client-side geographic failover for node-redis. This covers the experimental multi-database client (createMultiDbClient, createMultiDbClientPool, createMultiDbCluster, createMultiDbSentinel) introduced in redis@6.4.0-beta.0. The PR also:

The page follows the redis-py and Lettuce failover pages, with node-redis-specific sections where it behaves differently:

  • It doesn't retry commands on failover.
  • Failback is off by default.
  • initialAvailability: 'MAJORITY' means both endpoints in a two-endpoint setup.
  • It documents WATCH/transaction/scan behavior.
  • It adds a "Timeouts for unresponsive servers" section.

That last section came from testing. When a server stops responding but keeps the connection open, in-flight commands stay pending until it answers, even after the client has failed over. Only socket.socketTimeout rejects them. But the default reconnect strategy doesn't reconnect after a socket timeout, so the timeout has to be paired with a custom reconnectStrategy, or the endpoint never fails back.

Verification: every JS block on the page was extracted from the Markdown and run against redis@6.4.0-beta.0, with two redis:8.8 containers and docker pause to simulate failure. LagAwareHealthCheck only ran against unresolvable hosts, because no Redis Software cluster was available.

Warning

Do not merge. These docs are written against a beta of node-redis whose multi-db API is marked @experimental and "may change before the stable release". Wait for a stable redis@6.4.0 release — see the manifest below.

Park manifest

Ticket: DOC-7176
Parked at: 2026-10-13
Trigger to pick up: a non-prerelease node-redis release (expected redis@6.4.0) whose tag contains the multi-db merge commit ec4cd0f1c75bc01e02c8bbcfaba4d73f277a62a6 (PR #3435). Check with gh api 'repos/redis/node-redis/compare/ec4cd0f1c75bc01e02c8bbcfaba4d73f277a62a6...redis%406.4.0' --jq '{status, behind_by}' and expect behind_by: 0. Also confirm npm view redis dist-tags shows latest at that version.
Labels: parked, do not merge yet

Pinned sources (state observed at park time)

Source Snapshot at park time Re-fetch
Release redis@6.4.0-beta.0 prerelease: true, published 2026-10-12T12:25:35Z, target master, tag commit 0b9915cea5164232deea7148a233f3149c38c585. npm dist-tags: beta: 6.4.0-beta.0, latest: 6.3.0. Newest non-prerelease GitHub release: redis@6.2.1. gh api repos/redis/node-redis/releases --jq '[.[] | select(.prerelease==false)][0].tag_name'; npm view redis dist-tags
PR #3435 (multi-db client) state: closed, merged: true, merge SHA ec4cd0f1c75bc01e02c8bbcfaba4d73f277a62a6, base master, updated 2026-10-09T09:02:49Z gh api repos/redis/node-redis/pulls/3435 --jq '{state, merged, merge_sha: .merge_commit_sha, base: .base.ref}'
docs/multi-db.md (upstream guide) blob 4c548d78354ad02a8082803fdbf79a4274ae8bbd at the beta tag gh api 'repos/redis/node-redis/contents/docs/multi-db.md?ref=<tag>' --jq .sha
packages/client/lib/multi-db/config.ts (MULTI_DB_DEFAULTS, option types) blob 43da7b0fa82666e8ea6840368d7155f852d7ed8d same pattern
packages/client/lib/multi-db/events.ts (event map) blob 8485c3a67783b39bae978a6854aa533cf950fff1 same pattern
packages/client/lib/multi-db/lag-aware-health-check.ts (options) blob 28944e22cb4ca57ea23666766f4629a847f26c99 same pattern
packages/client/lib/multi-db/controller.ts (method list) blob f70003fc666dbe7f926d02736aa016ebb2b99b4b same pattern
packages/client/index.ts (public exports) blob 8fe3384444fbf327906c8076772a378b2b2ca99f same pattern
packages/client/lib/client/socket.ts (defaultReconnectStrategy) blob afcf8e3140a036443b30a195fa062dd038b14a5f; lines 524–528 return false for SocketTimeoutError same pattern

An unchanged blob SHA at the stable tag means that file needs no re-read.

Observed shape the page assumes

Semantics: HIGH. Run-verified against the beta. These are likely to survive to release:

  • The four factories each return { client, controller }. The client is the event surface, and all endpoints must be the same type.
  • Selection picks the highest weight with a CLOSED circuit. Ties go to the endpoint listed first.
  • No command retry or replay on failover.
  • Failback is off by default and only moves to a strictly higher-weight endpoint.
  • connect() is gated by initialAvailability. A rejected first connect() makes the client unusable.
  • When all endpoints are down: TemporarilyUnavailableError → PermanentlyUnavailableError plus the terminated event. Calling connect() again recovers the client.
  • Pub/Sub subscriptions move to the new database. Pools have no Pub/Sub.
  • multi(), scan iterators and legacy() are bound to the database that was active when they were created. exec() after a failover rejects with WatchError or CommandAbandonedError.
  • SELECT, MONITOR and emitInvalidate are rejected.
  • Health-check probes share the command connection.
  • An unresponsive server leaves in-flight commands pending. Only socketTimeout rejects them, and the default reconnect strategy doesn't reconnect after a socket timeout.

Identifiers: LOW. Read from the beta tag (location in brackets). Expect renames.

  • Factories and exports [packages/client/index.ts, packages/redis/index.ts]: createMultiDbClient, createMultiDbClientPool, createMultiDbCluster, createMultiDbSentinel, defaultErrorFilter, DefaultHealthCheck, LagAwareHealthCheck, TemporarilyUnavailableError, PermanentlyUnavailableError, CommandAbandonedError, WeightBasedStrategy
  • DatabaseConfig fields [config.ts]: id (default db-<n>), options, weight (default 1, range [0,1]), skipInitialHealthCheck (addDatabase only), poolOptions (pool only, on PoolDatabaseConfig)
  • Top-level options and defaults [config.ts MULTI_DB_DEFAULTS]: gracePeriod 60000, autoFallbackInterval -1, maxFailoverAttempts 10, delayBetweenFailoverAttempts 12000, initialAvailability 'MAJORITY' ('ALL'/'MAJORITY'/'ONE'), failoverStrategy, healthChecks
  • healthCheck.* [config.ts HealthCheckConfig]: interval 5000, timeout 3000 (must be < interval), numProbes 3, delayBetweenProbes 500, policy 'ALL' ('ALL'/'MAJORITY'/'ANY')
  • failureDetector.* [config.ts FailureDetectorConfig]: minNumOfFailures 1000, failureRateThreshold 10 (percent), windowSize 2000, errorFilter
  • LagAwareHealthCheck options [lag-aware-health-check.ts]: restEndpoint, bdbUid (each a value or (id) => value), credentials {username, password}, lagTolerance 5000, requestTimeout 3000, requestOptions, onProbeError(err, id)
  • Custom check interface [health-check.ts]: probe(target) → Promise<boolean>, where target is { id, sendCommand(args) }
  • Controller [controller.ts]: getActiveDatabase(), getDatabases() (descriptor {id, weight, circuitState, role}), addDatabase(), removeDatabase(), replaceDatabase(), setWeight(), setAutoFallback(ms | false), setActiveDatabase(), releasePin()
  • Events [events.ts]: connect, ready, end, terminated {attempts}, error, failover {from, to, reason}, fallback {from, to}, database-unhealthy {id, cause}, database-recovered {id}, all-databases-down {attempt, maxAttempts}, member-error {id, error}, member-ready {id}, member-end {id}
  • Failover reason values [events.ts FailoverReason]: failure-detector, health-check, connection-ended, forced, active-removed
  • Socket options used on the page [client/socket.ts]: socket.socketTimeout, socket.reconnectStrategy, SocketTimeoutError
  • client.duplicate() returns a { client, controller } pair [multi-db/index.ts]

Re-check checklist

  • ⚠️ Install note: change "v6.4.0-beta.0 and later" and npm install redis@beta to the stable version and plain npm install redis.
  • ⚠️ bannerText: keep the preview banner only if the stable release still marks the multi-db API @experimental. Check the JSDoc in config.ts and index.ts.
  • ⚠️ Defaults: diff MULTI_DB_DEFAULTS in config.ts against every default in the page tables, including the 120-second figure (maxFailoverAttempts × delayBetweenFailoverAttempts).
  • ⚠️ Socket timeout: re-check defaultReconnectStrategy in socket.ts. If it now reconnects on SocketTimeoutError, simplify "Timeouts for unresponsive servers" and the "No failback after recovery" troubleshooting item. Also check whether a multi-db fix now rejects in-flight commands on failover, which would make the section unnecessary.
  • Tick off every name in the identifier list against the stable tag (exports, options, controller methods, events, failover reasons).
  • Diff upstream docs/multi-db.md between the beta tag and the stable tag. Watch for changes to the behavior contracts, pinned surfaces and Pub/Sub sections.
  • Gap: run createMultiDbClientPool, createMultiDbCluster and createMultiDbSentinel at least once. Only createMultiDbClient was exercised. The "pools have no Pub/Sub" claim comes from upstream docs.
  • Gap: CommandAbandonedError for queued, unsent commands at failover is taken from upstream docs and wasn't reproduced. The scan-iterator and multi() cases were reproduced.
  • Gap: the LagAwareHealthCheck success path is unverified because no Redis Software cluster was available. If one becomes available, run the page's example against it.
  • Re-run all JS blocks on the page against the stable release, using the same harness: extract the blocks, substitute local URLs, run against two containers with docker pause.
  • Check whether the stable release adds features the support table should reflect, and whether the overview's "Pub/sub failover" and "OSS Cluster failover" columns still read correctly for node-redis.

On unpark, then

Run /unpark <this PR> when the trigger fires. The closing sequence is defined in /unpark Steps 4–5. /finalize is deferred until then, and the do not merge yet guard holds until /finalize completes.

🤖 Generated with Claude Code


Note

Low Risk
Documentation-only changes with no runtime or security impact; content targets a beta API and the PR is marked do-not-merge until stable release.

Overview
Adds node-redis client-side geographic failover documentation for the preview multi-database API (createMultiDbClient and related factories in redis@6.4.0-beta.0).

A new nodejs/failover page documents weighted endpoints, circuit breaker and health checks (including LagAwareHealthCheck), runtime controller APIs, Pub/Sub re-subscription, and node-specific behavior: no command retry on failover, failback off by default, initialAvailability: 'MAJORITY' requiring both endpoints in a two-node setup, transaction/scan/WATCH semantics, and pairing socket.socketTimeout with a custom reconnectStrategy for hung connections.

The shared failover overview gains a node-redis table row and related link; nodejs/produsage adds a checklist item and short geographic failover recommendation.

Reviewed by Cursor Bugbot for commit cd2bbdc. Bugbot is set up for automated code reviews on this repo. Configure here.

Add nodejs/failover.md for the experimental multi-db client in
redis@6.4.0-beta.0 (createMultiDbClient / ClientPool / Cluster /
Sentinel), add node-redis to the overview support table and
relatedPages, and link it from nodejs/produsage.md.

Structure follows the redis-py and Lettuce failover pages, with
node-redis-specific sections where behavior differs: no command retry
(a Retries section replaces the retry-config section), failback off by
default, initialAvailability MAJORITY meaning both of two endpoints,
"Transactions and connection state" (WATCH/multi/scanIterator/legacy,
SELECT/MONITOR/emitInvalidate rejected), and a connect() troubleshooting
entry.

Every JS block on the page was extracted from the markdown and run
against redis@6.4.0-beta.0 with two redis:8.8 containers on ports
6401/6402, using docker pause to simulate an unresponsive server.

The surprise: a paused server (connection still open, no replies)
leaves in-flight commands pending until it answers, even after the
client has failed over. commandOptions.timeout does not help, because
commands-queue.ts drops a command's timeout once it is written to the
socket. Only socket.socketTimeout rejects them. But
defaultReconnectStrategy returns false on SocketTimeoutError
(socket.ts:525), so socketTimeout alone makes the endpoint emit
member-end and never fail back. The page therefore pairs socketTimeout
with an always-reconnect reconnectStrategy; the full failover ->
recovery -> fallback cycle was verified with that pair, with no idle
reconnect churn while health checks run.

Running the page's own blocks also caught two draft errors:
healthCheck.timeout must be less than interval (default timeout 3000,
so lowering interval alone throws at construction), and releasePin()
does not switch back, it only re-enables fallback.

Learned: socketTimeout is the only way to reject in-flight commands on an unresponsive server, and it needs a custom reconnectStrategy
Learned: running the page's extracted code blocks caught 2 defects that reading the source had not
Gaps: LagAwareHealthCheck success path unverified (no Redis Software cluster); only constructed and probed against unresolvable hosts
Gaps: createMultiDbClientPool/Cluster/Sentinel factories not run; only createMultiDbClient was exercised
Gaps: CommandAbandonedError for queued unsent commands taken from upstream docs/multi-db.md, not reproduced
Recheck: when redis@6.4.0 stable ships, every identifier and default on the page (all were read from the 6.4.0-beta.0 tag)
Directive: keep the socketTimeout + reconnectStrategy pairing together; either one alone is wrong
Ticket: DOC-7176
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andy-stark-redis andy-stark-redis added do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead. labels Oct 13, 2026
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

DOC-7176

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Redis Memory

Found 5 related items from repository history (5 new this commit):

Memory updated at cd2bbdc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clients Client library docs do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant