feat(store/redis): namespace keys per tenant with WithKeyPrefix - #32
Merged
Merged
Conversation
If you run more than one dispatch instance against the same Redis, they
all write under dispatch:* and start dequeuing each other's jobs, treating
each other's cron names as duplicates and fighting over the leader key.
Nothing outside the store can fix that: streams, the wake channel and the
lease scripts go straight to go-redis, so a grove namespace hook never sees
them.
Every key the store touches now comes from one keys type on the Store.
redis.WithKeyPrefix("ws_acme:") puts the tenant segment outside the
dispatch namespace (ws_acme:dispatch:job:<id>), the same layout a tenant's
other Redis keys use, so one SCAN pattern covers everything it owns. An
empty prefix is byte for byte the old key, so existing deployments keep
their data.
The forge extension reads it as key_prefix in YAML or WithKVKeyPrefix from
Go and passes it through on the grove KV path only. Tests cover job, cron
and leadership isolation between two prefixed stores on one container, the
legacy key shape for an empty prefix, and YAML winning the merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run two dispatch instances against one Redis today and they share every key under dispatch:*. Each dequeues the other's jobs, sees the other's cron names as duplicates and blocks on the other's leader key. A grove namespace hook can't help because streams, the wake channel and the lease scripts talk to go-redis directly.
This adds redis.WithKeyPrefix. Every key, stream and channel now comes from one keys type on the store, and the tenant segment goes outside the namespace: ws_acme:dispatch:job:. That matches how a tenant's other Redis keys look, so one SCAN pattern finds all of it. Pass an empty prefix (or nothing) and you get the exact old keys, so nothing you already run changes.
On the forge extension you set it as key_prefix in YAML or WithKVKeyPrefix from Go. It only applies on the grove KV path; the grove database path ignores it.
Tests: two prefixed stores on one container must not see each other's jobs, cron names or leadership; an empty prefix reads what an unprefixed store wrote; YAML beats the programmatic value in the merge. The full store/redis integration suite and the extension package pass, golangci-lint is clean.
Tagged v1.6.6.