Skip to content

About

The open-source framework for serving robot policies from remote GPUs over wide-area networks. An inference layer (CUDA graphs with the same outputs, batching within each robot's slack, replicas), a network layer (a robot's connection to its GPU), and the client for GPUs hosted by General Sequences. Supports all open-source robot policies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

21 stars

Watchers

0 watching

Forks

Latest commit

 

History

7 Commits

Folders and files

sequence-ai

tests pip install sequence-ai Python 3.9 or later license: Apache-2.0

sequence-ai is an open-source framework for serving robot policies from remote GPUs over wide-area networks (WANs). It is used with openpi, LeRobot, GR00T and Octo policies, and with any policy given as a function. It runs on your own GPU, or on one rented from General Sequences near your robot: on the robot, only what connect() is given changes.

Give it a policy server and a robot. On the GPU, it replays the module a model calls many times per inference from CUDA graphs, with exactly the same outputs, and batches several robots' requests within the slack their action chunks leave. On the connection, it keeps the TCP congestion window open across the robot's pauses, so a 24 KB upload after a pause takes one round trip where it took two, at RTTs from 12 to 101 ms. Together these are GS-WAN (General Sequences Wide-Area Network). Each component is one file that can be added to existing code.

Quick start · Three ways to run it · Architecture · The problem · Inference layer · Network layer · Results · Docs

Install

pip install sequence-ai      # both sides: serve() on the GPU, connect() on the robot, every component, the seq command

One package, two parts: the open-source framework (serve(), connect() to an address, sequence_ai.engine, sequence_ai.network) and the client of General Sequences' GPUs and hosted deployments (seq, sequence_ai.gpu, connect() to a deployment's name). Python 3.9 or later for the client and the components, on Linux and macOS; the seq command needs Python 3.10 or later. cuda_graphs uses the torch your model already has (2.4 or later). Each component is one file, and most import only the standard library, so copying a file into your project works as well.

Quick start

On the GPU, one call serves your model, given as a function:

import sequence_ai

sequence_ai.serve(policy_user, port=8000, example=observation_user)  # policy_user(observation) -> what the robot needs

On the robot, one call connects to it:

with sequence_ai.connect("http://your-gpu:8000") as policy_user:     # BBR, TCP keepalive, a ping while idle, msgpack
    policy_user.observe({"image": frame_user, "state": state_user})
    actions_user = policy_user.act()["actions"]

Both sides' defaults are on: pictures go as JPEG, large arrays are compressed where that saves time, the policy runs on a thread of its own and is warmed up before the server says it is ready. What is on by default lists each part and how to turn it off, and python -m sequence_ai.check http://your-gpu:8000 says which are on.

In a server of your own, the inference components wrap the model you already have:

from sequence_ai.engine import cuda_graphs
from sequence_ai.engine.batcher import Batcher

cuda_graphs.install(model_user, ["action_head.denoiser"])        # replayed from CUDA graphs; kept only if outputs match
batcher_user = Batcher(infer_a_batch_user, sizes=[1, 2, 4, 8])   # several robots' requests in one forward pass
action_user = await batcher_user(observation_user)               # in each request's handler

Each name ending in _user is yours: your model, your frames, your handler. Both run on a laptop with no GPU and no model:

python examples/quickstart_server.py & sleep 1 && python examples/quickstart_robot.py    # a server and a robot
python examples/quickstart_batching.py                                                   # five robots, batched

The first line each prints:

request 0: 8 actions of 7 numbers, the first [128. 128. 128.]
5 robots served in 113 ms, as batches of {4: 1, 1: 1}

Quickstart: a policy server and a robot and Quickstart: batching and replicas walk through each with the output of every step.

Three ways to run it

The code is the same on all three: serve(policy) on the GPU, connect(...) on the robot. Only where the GPU comes from changes.

Where the GPU comes from On the GPU On the robot
Your own, or any GPU the robot can reach sequence_ai.serve(policy_user) connect("http://your-gpu:8000")
Rented from General Sequences near your robot, by city sequence_ai.serve(policy_user) connect("http://<its address>:8000")
A hosted deployment of General Sequences nothing: the platform builds your policy, serves it and scores it connect("my_policy_user")

The first needs no account and costs nothing. The other two are paid services of General Sequences, billed while a GPU runs, and their commands are in this package:

seq gpu add --place "San Francisco, CA" --gpu L4 --port 8000    # a GPU near your robot, yours until seq gpu remove
seq deploy my_policy_user                                       # or a deployment: the policy in this folder

GPUs are in short supply, and few are near any given robot, which is why the second exists: How do I get a GPU near my robot? has its commands. The platform guide has the third: writing a policy class, deploying it, benchmarks and evals, teleoperation, and every seq command.

An official template runs on all three: seq serve in the folder seq init made serves it on your own GPU or a rented one. Its weights download with a General Sequences key: How do I run a template on my own GPU?

How it fits together

            robot                                                GPU
 ┌─────────────────────────┐                        ┌───────────────────────────┐
 │  your control loop      │                        │  your policy              │
 ├─────────────────────────┤      observation       ├───────────────────────────┤
 │  sequence_ai.network    │  ───────────────────▶  │  sequence_ai.engine       │
 │    connect · openpi     │                        │    serve · cuda_graphs    │
 │    window · compress    │          WAN           │    batcher · replicas     │
 │    frames · wifi_power  │                        │    slack                  │
 │                         │                        ├───────────────────────────┤
 │                         │  ◀───────────────────  │  sequence_ai.network      │
 │                         │      action chunk      │    one_write · ping       │
 └─────────────────────────┘                        │    tcp_link · compressed  │
                                                    └───────────────────────────┘

  both ends: msgpack_io · latency_stats             your own link, measured: bench/

sequence_ai.network is the connection between a robot and the GPU its policy runs on. On the robot, connect() applies the client-side components to one connection, and openpi.keep() does the same on the connection that openpi's own client holds. On the server, one_write and ping wrap a uvicorn or a WebSocket server. sequence_ai.engine is the inference layer on the GPU: serve() serves a policy with the server's side of both layers on, cuda_graphs shortens one inference, batcher and slack share a GPU among robots, and replicas runs one process per GPU. bench/ measures a link before anything is added to it.

The problem, by layer

A robot policy built on a vision-language-action (VLA) model runs each inference on a GPU, and a robot without a suitable GPU on board sends its observations over a WAN to a remote one. Each layer of that path has its own problem.

The model is most of the latency. Across thirteen model and network configurations, the model is 68 to 96% of a request's latency. The time of one inference is therefore the main lever, and cuda_graphs shortens it without changing the model's output.

One robot leaves its GPU mostly idle. One robot keeps a GPU busy 21 to 55% of the time, so several robots can share one, provided that no robot's action chunk runs out while its request waits. batcher batches the requests that are waiting, slack computes how long each may wait and how many robots a replica can serve, and replicas runs one replica per GPU.

A robot pauses, and Linux restarts its connection. A robot executes a chunk between requests, and Linux restarts a TCP connection's congestion window after an idle period. The upload after a pause then takes one extra round trip, at RTTs from 12 to 101 ms. window, ping and connect keep the window open.

A request carries more round trips and bytes than it needs. A response split across writes can wait a round trip behind a tunnel, and a camera frame is 50 to 100 KB. one_write sends each response in one write, compress compresses an observation only when that shortens the upload, and frames resizes a frame before it is sent.

GPUs are in short supply. Few are near any given robot, and every millisecond of round-trip time comes out of the slack an action chunk leaves. Getting a GPU near the robot is the routing layer: a hosted service of General Sequences, reached with this package's seq gpu commands. Three ways to run it has them.

Inference layer

Component What it does To add it Requires
serve Serves a policy, given as a function or a batch function, to robots elsewhere: on a thread of its own, pings answered at once, a warm-up, pictures and compressed arrays restored. openpi.batched(policy) makes an openpi policy a batch function. one call what sequence-ai installs
cuda_graphs Replays a torch module that your model calls many times per run from CUDA graphs captured from its own eager kernels. A graph is kept only when its output is exactly the same. one call torch 2.4 or later, a CUDA GPU
batcher Batches the calls waiting for a model into one call, in the batch sizes the model lists. one object, one await nothing
replicas Runs replicas of a model, each in its own process on the GPU assigned to it, and replaces one that dies. one object, one await nothing
slack Schedules by the slack an action chunk leaves: how many robots a replica can serve, each by its own round-trip time (RTT), how long a request may wait for a batch, and which requests are dispatched first. a few calls in your server's loop nothing
from sequence_ai.engine import cuda_graphs, slack
from sequence_ai.engine.batcher import Batcher
from sequence_ai.engine.replicas import ReplicaPool

cuda_graphs.install(model_user, ["action_head.denoiser"])         # model_user.action_head.denoiser, graphed in place
batcher_user = Batcher(model_on_a_list_user, sizes=[1, 2, 4, 8], wait_ms=2.0)
pool_user = ReplicaPool(load_user, replicas=2, devices=[0, 1])     # one replica per GPU

table_user = slack.measure(forward_of_user, sizes=[1, 2, 4])       # forward_of_user(n): one forward pass of n requests
play_user = slack.playback(hz=20, valid_len=15, replan_at=8)       # 15 valid steps, the next requested at the 8th
seating_user = slack.Seating(table_user, play_user["buffer_s"] * 1000, sizes=[1, 2, 4],
                             cycle_ms=(play_user["play_s"] - play_user["buffer_s"]) * 1000)
seats_user, over_user = seating_user.seat([("near", 30.0), ("far", 200.0)])     # (robot, its RTT in ms)
  • cuda_graphs speeds up PyTorch policies whose time goes to launching kernels; a JAX policy such as π0.5 already compiles its kernels into one program. It counts the calls each module gets over a few runs (Census), captures the module that is called many times per run with the same shapes, and checks each new shape against eager mode byte for byte. A shape whose capture fails or differs runs in eager mode from then on. The module stays the same object of the same class, with its weights, state_dict and hooks unchanged. How do I make a PyTorch robot policy faster without changing its outputs?
  • batcher dispatches a batch as soon as it is full, or wait_ms after its first call while the model is free. While the model is busy, waiting calls accumulate and are dispatched together. Only the listed sizes are used, and a batch in which the model raised is re-run call by call. How many robots can one GPU serve?
  • replicas runs each replica in its own process on its own GPU, so a forward pass never blocks the server's event loop. ReplicaError is the model's own error, and the replica takes the next call. ReplicaDied is a process that has exited, and it is replaced. Should I run two replicas of a policy on one GPU?
  • slack computes a replica's capacity from the playback time an action chunk leaves when its robot requests the next one, each robot by its own RTT, and tells a server how long the open batch may wait. slack: scheduling by the slack an action chunk leaves

How do I use the inference components with openpi, LeRobot or GR00T?

Network layer

"Client" is the robot's program and "server" is the policy server on the GPU.

Component Side What it does Requires
window client Keeps the connection's congestion window across the robot's pauses: BBR on its socket, TCP keepalive, a ping while the connection is idle. nothing
compress client Compresses an observation's large arrays only where that saves more upload time than it costs. nothing; zstd where it is installed
frames both Sends each picture as JPEG and turns it back into its array on the server; resizes a frame to the policy's input size. Pillow
wifi_power client Detects Wi-Fi power saving on the robot's link and reports how to disable it. Linux
one_write server Sends each HTTP response of a uvicorn server in one write instead of three. uvicorn
tcp_link server Reads each client's TCP state from the server's kernel: RTT, retransmitted segments, congestion window. Linux
compressed server Decompresses what the client compressed, byte for byte. numpy; zstandard for zstd
ping server A server-side ping, so that the robot's connection is not left idle while the server can answer it: the options to pass to your server. a WebSocket server
msgpack_io both Encodes observations and responses as msgpack, with numpy arrays as raw bytes. Also reads openpi's layout. nothing
latency_stats both Counters, gauges and latencies with their p50, p95 and p99, read as one snapshot. nothing

Combined, and for clients you may already use:

Side What it does Requires
connect client The client-side components on a kept connection to your own server: JPEG and compression while the server says it restores them, and an act whose window a pause restarted sent in two halves while the server joins them. httpx
lines client connect(url, lines=["eth0", "wwan0"]): each act sent over each of the robot's uplinks, each line bound to its own network interface, and the first answer used; serve() runs the policy once for the copies. How httpx; Linux 5.7 or later, or macOS
check client python -m sequence_ai.check <url>: which defaults are on between a robot and its server, and what the server is. httpx
openpi client The client-side components on the connection held by openpi's own client. openpi-client
lerobot both The Zenoh setting that keeps LeRobot's remote-inference connection from going idle. lerobot 0.7
bench/ both Measures your own link: what a pause, a ping and a camera stream cost on it. nothing
from sequence_ai.network import lerobot, openpi

kept_user = openpi.keep(policy_user)                              # openpi: one line on its WebsocketClientPolicy
path_user = lerobot.keep_alive_file("zenoh-keep-alive.json5")     # LeRobot 0.7: one Zenoh setting, given to both sides

What does each component do? gives the lines to add, the reason and the measurement for each one, and examples/components/ has a runnable file for each. How do I run π0 (openpi) on a remote GPU? | How do I reduce the latency of LeRobot remote inference? | How do I serve a robot policy from my own GPU?

Language models

sequence_ai.chat() and Conversation (an agent's history and its tool calls) take server="openrouter" with your own OpenRouter key, or the address of a server of your own with the OpenAI chat completions API:

result_user = sequence_ai.chat("qwen/qwen3-8b", [{"role": "user", "content": "Plan the next step."}],
                               server="openrouter")      # your key in OPENROUTER_API_KEY

How do I use an LLM with my own OpenRouter key?

Results

Across thirteen model and network configurations, the time a request spends outside the model ranges from 13 to 137 ms, and its share of the latency from 4 to 32%. A remote GPU is therefore close to an on-board GPU of the same type, and inference time is the main lever.

Inference layer:

Component Result Measured on
cuda_graphs 1.54× faster (224 → 146 ms), exactly the same outputs GR00T N1.7's diffusion transformer, one L4
cuda_graphs against torch.compile 1.54×, against 1.45× with different outputs (inductor) and 1.00× (cudagraphs) the same model and GPU
cuda_graphs 6.7× faster (16.1 → 2.4 ms), byte for byte a small model of many small kernels, one T4
Batching up to 1.78× the requests per second (5.99 → 10.66) π0.5, one L4, batch 16
slack 94.1% of responses in time, against 85.7% with all robots on one replica four robots, a model paced like a 15 Hz policy

Network layer:

Component Result Measured on
window, ping an upload after a pause in 1.0 round trips, not 2.0 a 24 KB upload, RTTs of 12 to 101 ms
window: BBR no extra round trip after a pause, also with 1% loss a 24 KB upload, RTT about 21 ms
ping, the server's 1.1 to 1.25 round trips, not 3.0 100 KB to 1 MB after 500 ms of silence
window.Warm, the robot's ping under 1 ms, not 303.6 ms, for an upload to arrive 162 KB uploads 3 s apart, RTT 226 ms
one_write 106.7 ms, not 189.7, for a response a GET through a tunnel
compress camera frames about 1.9 times smaller, 0.37 ms to compress frames from a public dataset's video

Measurements lists every command, and bench/ measures your own link.

Repository layout

sequence_ai/
  engine/        inference layer: serve · openpi · cuda_graphs · batcher · replicas · slack
  network/       network layer
                   client: connect · window · compress · frames · wifi_power · openpi
                   server: one_write · ping · tcp_link · compressed
                   both:   msgpack_io · latency_stats · lerobot
  chat/          chat() and Conversation: on General Sequences, or at a server of your own (server=)
  check.py       python -m sequence_ai.check: which defaults are on between a robot and its server
  ...            the client for General Sequences' GPUs and deployments: connect, gpu, teleop, record
seq/             the seq command, and the decorators a hosted policy or benchmark is written with
bench/           measures a link: a TCP client and server, WebSocket, Zenoh
examples/        the quick starts, and one runnable file per component
docs/            one page per question, the measurements, the FAQ
tests/           the components' tests: Python 3.9 to 3.12, on a machine with no GPU
tools/           writes llms-full.txt
platform.md      the guide to the hosted platform; eval.md and observability.md are its references

About 4,500 lines in sequence_ai/engine and sequence_ai/network, and 202 test functions.

Documentation

llms.txt is a map of the repository for a coding assistant, and llms-full.txt is this README and every page of docs/ in one file.

A note on the numbers

Every number in this README is given with the model, the GPU or the link it was measured on, and Measurements lists the command behind each one. What a network component saves depends on the link, so bench/ measures yours before you add anything. Each module's docstring states what the component does in one sentence, why, and what was measured.

Develop

uv venv --python 3.11
uv pip install -e ".[dev]"
.venv/bin/python -m pytest -q
.venv/bin/ruff check sequence_ai/engine sequence_ai/network tests examples bench tools

The tests are the components', and run on Python 3.9 to 3.12 on a machine with no GPU. The client for the hosted platform and the seq command are developed in General Sequences' own repository and published here, as they ship, at each release. CONTRIBUTING.md has what that means for a pull request.

Contributing

Issues and pull requests are welcome: see CONTRIBUTING.md and the Code of Conduct. To report a security problem, see SECURITY.md.

Cite

If you use sequence-ai in research, please cite it:

@software{generalsequences2026sequenceai,
  title  = {sequence-ai},
  author = {{General Sequences}},
  year   = {2026},
  url    = {https://github.com/General-Sequences/sequence-ai}
}

CITATION.cff has the same, with the version.

License

Apache-2.0. Copyright 2026 General Sequences Company. Parts of two components are adapted from another project, and the page seq eval replay serves carries others' files: THIRD_PARTY_NOTICES.md.

About

The open-source framework for serving robot policies from remote GPUs over wide-area networks. An inference layer (CUDA graphs with the same outputs, batching within each robot's slack, replicas), a network layer (a robot's connection to its GPU), and the client for GPUs hosted by General Sequences. Supports all open-source robot policies.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

21 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages