Repository navigation
feat(support): add support service with WebSockets and Yamux - #47
Open
edospadoni wants to merge 70 commits into
Open
edospadoni wants to merge 70 commits into
edospadoni wants to merge 70 commits into
Conversation
edospadoni
deployed
to
feature/support-service - my-frontend-qa PR #47
March 10, 2026 08:00 — with 
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 08:00 — with
Render
Active
Render
Active
Contributor
|
🔗 Redirect URIs Added to Logto The following redirect URIs have been automatically added to the Logto application configuration: Redirect URIs:
Post-logout redirect URIs:
These will be automatically removed when the PR is closed or merged. |
edospadoni
requested a deployment
to
feature/support-service - my-proxy-qa PR #47
March 10, 2026 08:00 — with
Render
In progress
Render
In progress
Contributor
🤖 my.nethesis.it API structural change detectedStructural change detailsAdded (13)
Modified (5)
Powered by Bump.sh |
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 08:11 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 08:15 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 08:52 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 09:32 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 09:39 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 09:54 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 10:07 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 10:22 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 10:31 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 10:39 — with
Render
Active
Render
Active
edospadoni
force-pushed
the
feature/support-service
branch
from
March 10, 2026 10:53
c62b877 to
007bd6d
Compare
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 10:54 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 13:23 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 10, 2026 14:33 — with
Render
Active
Render
Active
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 11, 2026 13:09 — with
Render
Active
Render
Active
Member
Author
tunnel-client binary (linux/amd64)Download: Quick start# Make it executable
chmod +x tunnel-client-linux-amd64
# Run it
./tunnel-client-linux-amd64 \
--url wss://my-proxy-qa-pr-47.onrender.com/support/api/tunnel \
--key <SYSTEM_KEY> \
--secret <SYSTEM_SECRET>Parameters
Service discovery modesThe tunnel-client auto-detects the environment:
Diagnostics plugin systemAt connect time, the tunnel-client collects a health snapshot and sends it to MY over the tunnel. Operators see the results directly in the support session popover — before opening a terminal or proxy — so they have immediate context on the system state. How it works:
Built-in plugin (
External plugins: any executable file placed in Each plugin must:
#!/bin/bash
# /usr/share/my/diagnostics.d/10-myservice.sh
STATUS="ok"
SUMMARY="all good"
if ! systemctl is-active --quiet myservice; then
STATUS="critical"
SUMMARY="myservice is not running"
fi
echo "{\"id\":\"myservice\",\"name\":\"My Service\",\"status\":\"$STATUS\",\"summary\":\"$SUMMARY\"}"
exit $([ "$STATUS" = "ok" ] && echo 0 || echo 2)The overall session status shown in MY is the worst status across all plugins (critical > warning > ok). If a plugin exceeds its timeout it is marked If Environment variablesAll flags can also be passed as env vars: export SUPPORT_URL=wss://my-proxy-qa-pr-47.onrender.com/support/api/tunnel
export SYSTEM_KEY=<your-key>
export SYSTEM_SECRET=<your-secret>
./tunnel-client-linux-amd64 |
edospadoni
deployed
to
feature/support-service - my-backend-qa PR #47
March 11, 2026 14:06 — with
Render
Active
Render
Active
GetSupportSessionLogs now resolves the session through the caller's org scope before returning logs, closing a cross-tenant read. Subdomain proxy token and cookie lifetime are configurable via SUPPORT_PROXY_TOKEN_TTL, defaulting to 1h instead of 8h.
main's alerts/backup release added migrations through 025, colliding with the support migrations at 018-021. Renumber the five support migrations to 026-030 (preserving order) and drop their stale self-writes to schema_migrations, which the runner already handles by filename number.
Main took 039-051 meanwhile (048 is reserved for PR #237). Also fix a leftover conflict marker in schema.sql, make 054 re-runnable, align the 052 comments with schema.sql, take formatDateTimeNoSeconds from vue-components and declare the support routes in authz/routes.yml.
Keep the users state on persistent storage, record each identity as soon as it exists and add --cleanup-only for boot-time cleanup. Send the reconnect token and stop on 410, never delete identities fetched from the server, keep secrets out of argv, require wss:// unless --dev, pin injected targets and cap streams, PTYs and header reads.
Strip internal headers before proxying, namespace appliance cookies per session, encrypt ephemeral credentials at rest and wipe them on close. Require the reconnect token to resume a session, listen on the backend auth invalidation channel, serve fetched credentials only to sibling nodes, fix keepalive and tunnel replacement, bound control streams, trust only configured proxies and validate manifest paths.
Reject cross-session writes on the subdomain proxy, re-check the operator on every request with 15m tokens and log first use per service. Drop the main-origin appliance proxy, gate the Host rewrite to debug mode, decrypt and audit credential reads, hide session ids from non-support users and open proxied UIs with noopener. Scope WebSocket settings to the terminal route, tighten the CSP and support server headers, make support_enabled a kill switch and run the prod support service as a private service behind nginx.
A dedicated server block for support.my.nethesis.it (and the QA twin) exposes only /api/tunnel and answers 404 to everything else, so the dashboard is never served on a host that shares the cookie scope of the *.support.* appliance proxy.
The proxy subdomain carried the raw service name, so long route names
or names with dots produced hosts that DNS and the wildcard
certificate reject. Use a DNS-safe label instead: a readable prefix of
at most 20 chars plus 8 hex chars of sha256(name), so
{label}--{session} never exceeds 63 chars. The support service maps
the label back to the service in the session manifest.
Member
Author
|
up deploy |
Contributor
|
🔌 Preview environment resumed Back online (last build restored):
Skipped (not found):
|
Member
Author
|
update deploy |
Auto-updated .render-build-trigger files to ensure all services are deployed in PR preview environments. 🤖 Generated by GitHub Actions
Contributor
|
🚀 Build triggers updated! All |
Local services are dialed directly, bypassing Traefik, and the proxy now strips the browser's X-Forwarded-* headers, so apps that build absolute redirects from them (FreePBX visualplan) failed with an empty target. Set X-Forwarded-Host/Proto as Traefik would and rewrite Location headers pointing at an upstream host to its proxy subdomain.
Local services were dialed on their backend port, skipping the route's Traefik middlewares (forwardAuth, injected headers, prefix stripping), so they behaved differently from a direct visit. Dial the local Traefik like the remote nodes' ones.
NS8 guards some routes (e.g. rspamd) with a Traefik forwardAuth to the cluster-admin http-basic endpoint, so they take the cluster admin credentials rather than the module's domain user. The tunnel-client flags those routes from list-routes' forward_auth and the services dialog shows the cluster admin credentials next to them.
A services request made while the tunnel or the support service was reconnecting cached an empty list for the session, hiding the Services button until a full page reload.
…tors Credential views are now logged as 'view' access entries, which never get a disconnect time, so every opening of the services dialog showed up as a permanently connected operator.
Start, expiry and node count of a system row were aggregated over the whole session history, so a closed and extended session stretched the times of the live one. Use the active/pending sessions when the system has any, otherwise the latest one.
The Nethesis light headset renders at 20x23 px against the 24x24 px of its Font Awesome solid counterpart, so the menu entry shrank when not selected. Grow it slightly; other entries render identically already.
The Font Awesome transform misplaces glyphs whose viewBox is not square (the Nethesis light headset is 24x25) and pushed the icon over the previous entry. A CSS scale keeps it centred in its box.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support Service — Architecture
How it works
A tunnel client on the customer's system opens a persistent WebSocket to our support service. The connection is multiplexed with yamux — one WebSocket carries many parallel streams. When an operator clicks "Open" in the UI, traffic flows through the tunnel to reach the remote service (web UI, terminal, API) as if it were local.
graph LR subgraph Customer System TC[tunnel-client<br/>yamux mux] --> WU[Web UI] TC --> SA[SSH/API] TC --> ETC[...] end TC ---|WebSocket<br/>single connection| SS BR[Browser<br/>operator] --> NG[nginx<br/>proxy] NG --> BE[Backend :8080<br/>sessions, auth] BE --> SS[Support :8082<br/>tunnels, yamux]Session Lifecycle
stateDiagram-v2 [*] --> pending pending --> active : WebSocket established active --> closed : operator closes active --> grace_period : disconnect grace_period --> active : reconnect (same session) grace_period --> expired : timeout (30-60s)WebSocket + yamux Multiplexing
The tunnel client opens one WebSocket to the support service. On top of it, yamux creates a multiplexed session — like having many TCP connections inside a single one.
How it connects:
GET /support/api/tunnelwith HTTP Basic Authnet.Connyamux.Serveris created over the wrapped connection (keepalive 15s)Server-initiated streams: the support service can also open streams toward the tunnel-client. These start with a
COMMAND <version>\nheader and carry a JSON payload. The tunnel-client processes the command and respondsOK\norERROR <msg>\n.On disconnect: the tunnel enters a grace period (30-60s). If the client reconnects, the same session is reused. If the grace expires, the session is closed and ephemeral credentials are wiped from the database.
Ephemeral User Provisioning
When the tunnel-client connects, it provisions temporary support users on the managed system:
*)Lifecycle:
/var/run/my-support-users.json), reports them to the server via yamuxUSERS_REPORTstreamAfter user provisioning,
users.d/plugins configure applications to accept the support credentials (e.g., creating FreePBX admin entries for NethVoice).Static Service Injection
Operators can add arbitrary
host:portservices to a running tunnel without reconnection. This is useful for services not auto-discovered via Traefik — for example the web management interface of a device on the customer's LAN (IP phone, managed switch, NAS, etc.).Example: to access a Yealink phone's web UI at
192.168.1.100:443on a customer system, add a service withtarget: 192.168.1.100:443,tls: true. The phone's interface becomes available at:…as if the operator were on the same LAN as the phone.
How the UI Proxy works (subdomain)
When an operator clicks a service link (e.g. NethVoice UI), the browser opens a new tab on a dedicated subdomain. Each service gets its own origin, so all the app's absolute paths (
/_next/,/api/,/static/) work natively.The
?token=is removed from the URL after the first request (redirect), so it never leaks in logs, referrer headers, or browser history.How the Web Terminal works (xterm.js)
The terminal needs a WebSocket from the browser, but browsers can't send
Authorizationheaders on WebSocket connections. Solution: one-time ticket exchanged beforehand.The tunnel client spawns a PTY (pseudo-terminal) directly on the customer system — no SSH daemon involved. The PTY output is forwarded as raw bytes through the yamux stream back to the browser's xterm.js.
Why TCP hijacking instead of
httputil.ReverseProxy?httputil.ReverseProxycan't handle WebSocket upgrades. After the101 Switching Protocols, the connection becomes a raw bidirectional byte stream — not HTTP. The solution ishttp.Hijacker: take control of the raw TCP socket from Go's HTTP server. Two goroutines thenio.Copybytes in both directions (browser ↔ support service) with no HTTP overhead.Access Patterns & Auth
system_key:system_secret(SHA256), 3-tier cache (memory → Redis → DB), rate-limitedconnect:systemspermission, RBAC scope verified on all operationsGETDELon use → WebSocket via TCP hijack{session_id, service_name, org_role}→ SameSite=Strict cookie on subdomainINTERNAL_SECRETX-Session-Token(64-char hex, per-session) +INTERNAL_SECRET(required, fail-fast at startup)Security Highlights
GETDEL), JWT never touches the URL?token=, stored asHttpOnly SameSite=Strictcookie, URL cleaned via redirectcrypto/subtlefor all token validationsframe-ancestors 'self'on proxied responsesInter-service Communication
Components & Files
services/support/services/support/cmd/tunnel-client/backend/methods/support.go,support_proxy.gofrontend/src/components/support/proxy/nginx.confbackend/database/migrations/009_*,018_*–022_*support_sessions,support_access_logs, diagnostics, users columnsRelated Comments
diagnostics.d/andusers.d/— Developer reference for writing diagnostic and user configuration pluginsTesting Environment
To trigger a fresh deployment of all services in the PR preview environment, comment:
Automatic PR environments:
Merge Checklist
Code Quality:
Builds: