Skip to content

Federation deployment refactor - #347

Merged
klpoland merged 10 commits into
masterfrom
feature-kpoland-federation-easy-deploy
Sep 21, 2026
Merged

klpoland merged 10 commits into
masterfrom
feature-kpoland-federation-easy-deploy

Conversation

@klpoland

@klpoland klpoland commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Added scripts for streamlining federation initialization and testing flow, added templates for site/peer configuration, network config refactor


Note

Medium Risk
Changes production edge routing (Traefik split) and how federation sync URLs/secrets are injected; misconfiguration could break /sync or peer TLS, but runtime sync logic is largely unchanged.

Overview
Adds automated federation site onboarding so operators can stand up sync in production or local lab without hand-editing many files. New just recipes (federation-onboard, federation-render-config, federation-doctor, federation-add-peer) drive scripts that render gitignored site.env, federation.toml, gateway federation keys in django.env, and optional per-site Traefik /sync drop-ins from templates; the onboard flow runs doctor checks, ensures gateway secrets and init_federation_sync_token, builds/starts sync, probes http://127.0.0.1:8001/sync/health, and prints peer handoff TOML.

Production compose now loads ./site.env instead of hardcoding GATEWAY_INTERNAL_BASE_URL, FEDERATION_SYNC_SERVICE_URL, and REDIS_URL in compose.production.yaml. Peer adds append [[peers]] and restart sync; re-render merges extra peers and preserves ca_cert_path by FQDN.

Traefik dynamic config moves out of the monolithic traefik.toml into network/traefik/conf.d/ (TLS, gateway HTTP, federation sync routers); compose mounts that directory. Wizard-generated *-sync.toml files are gitignored except tracked ND routes in 00-federation-sync.toml.

Docs/README are rewritten around the wizard path; pytest covers doctor, template render, site.env quoting, and TOML peer merge. Test fixtures in sample_data.py now populate fuller RFC-shaped federated docs.

Reviewed by Cursor Bugbot for commit 8fe6892. Bugbot is set up for automated code reviews on this repo. Configure here.

@klpoland klpoland self-assigned this Sep 21, 2026
@klpoland klpoland added refactoring General code improvements network Networking and other deploy tasks federation Federation related work (sync service, document storage, peer configuration, etc.) labels Sep 21, 2026
@semanticdiff-com

semanticdiff-com Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Review changes with  SemanticDiff

Changed Files
File Status
  federation/sds_federation/testing/sample_data.py  9% smaller
  README.md Unsupported file format
  federation/.gitignore Unsupported file format
  federation/README.md Unsupported file format
  federation/compose.production.yaml  0% smaller
  federation/justfile Unsupported file format
  federation/scripts/add-peer.sh Unsupported file format
  federation/scripts/deploy-local-peer-2-peer.sh Unsupported file format
  federation/scripts/federation-doctor.sh Unsupported file format
  federation/scripts/lib/bootstrap.sh Unsupported file format
  federation/scripts/lib/ca_cert_path.sh Unsupported file format
  federation/scripts/lib/common.sh Unsupported file format
  federation/scripts/lib/gateway.sh Unsupported file format
  federation/scripts/lib/site_env.sh Unsupported file format
  federation/scripts/merge_federation_toml_peers.py  0% smaller
  federation/scripts/onboard-site.sh Unsupported file format
  federation/scripts/render-site-config.sh Unsupported file format
  federation/scripts/write_site_env.py  0% smaller
  federation/site.example.env Unsupported file format
  federation/templates/federation.toml.tmpl Unsupported file format
  federation/templates/gateway-federation.env.tmpl Unsupported file format
  federation/templates/site.env.tmpl Unsupported file format
  federation/templates/traefik-sync.toml.tmpl Unsupported file format
  federation/tests/test_merge_federation_toml_peers.py  0% smaller
  federation/tests/test_onboarding_doctor.py  0% smaller
  federation/tests/test_onboarding_render.py  0% smaller
  federation/tests/test_site_env_for_render.py  0% smaller
  federation/tests/test_write_site_env.py  0% smaller
  network/compose.yaml  0% smaller
  network/traefik/conf.d/.gitignore Unsupported file format
  network/traefik/conf.d/00-federation-sync.toml Unsupported file format
  network/traefik/conf.d/00-tls.toml Unsupported file format
  network/traefik/conf.d/01-gateway-http.toml Unsupported file format
  network/traefik/traefik.toml Unsupported file format

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread federation/scripts/onboard-site.sh
Comment thread federation/scripts/render-site-config.sh
Comment thread federation/scripts/onboard-site.sh
Comment thread federation/scripts/onboard-site.sh Outdated
Comment thread federation/templates/traefik-sync.toml.tmpl
Comment thread federation/scripts/render-site-config.sh
Comment thread federation/scripts/federation-doctor.sh Outdated
@klpoland
klpoland requested a review from mjcurran September 21, 2026 17:25

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread network/traefik/traefik.toml
Comment thread federation/scripts/render-site-config.sh
Comment thread federation/scripts/onboard-site.sh Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread federation/templates/site.env.tmpl Outdated
Comment thread federation/scripts/lib/site_env.sh
Comment thread federation/scripts/add-peer.sh

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread federation/scripts/onboard-site.sh Outdated
Comment thread federation/scripts/federation-doctor.sh
Comment thread federation/scripts/render-site-config.sh

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread federation/scripts/lib/site_env.sh Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 8fe6892. Configure here.

Comment thread federation/scripts/federation-doctor.sh
@klpoland
klpoland merged commit 917726d into master Sep 21, 2026
3 checks passed
@klpoland
klpoland deleted the feature-kpoland-federation-easy-deploy branch September 21, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

federation Federation related work (sync service, document storage, peer configuration, etc.) network Networking and other deploy tasks refactoring General code improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants