Repository navigation
Feat/docker - #7
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds containerization support (Docker + docker-compose) for running the inference proxy alongside multiple worker services, and updates the Rust binaries to accept key networking configuration via environment variables to better support deployments.
Changes:
- Added a multi-stage
Dockerfile,.dockerignore, anddocker-compose.ymlto build and orchestrate the proxy + multiple workers. - Updated proxy (
src/main.rs) and worker (src/bin/worker.rs) to read listen/connect addresses from environment variables with local-dev defaults. - Adjusted the sample client connection target in
src/bin/client.rs.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/main.rs |
Reads proxy listen address + worker addresses from env and connects to workers at startup. |
src/bin/worker.rs |
Allows configuring the worker bind address via WORKER_LISTEN_ADDR. |
src/bin/client.rs |
Updates the client’s proxy connection address. |
Dockerfile |
Adds a Rust multi-stage build producing a slim runtime image. |
docker-compose.yml |
Defines proxy + three worker services with env-based configuration. |
.dockerignore |
Excludes build and VCS artifacts from Docker build context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
10
to
+12
| println!("Connecting to the Inference Proxy server..."); | ||
|
|
||
| let mut client = InferenceServiceClient::connect("http://[::1]:50051").await?; | ||
| let mut client = InferenceServiceClient::connect("http://0.0.0.0:50051").await?; |
Comment on lines
+35
to
+43
| // Read worker addresses from env var (comma-separated), fallback for local dev | ||
| let worker_addrs_str = std::env::var("WORKER_ADDRESSES") | ||
| .unwrap_or_else(|_| "http://[::1]:50052,http://[::1]:50053,http://[::1]:50054".to_string()); | ||
|
|
||
| let worker_addrs: Vec<&str> = worker_addrs_str.split(',').collect(); | ||
| let mut worker_clients = Vec::new(); | ||
|
|
||
| for port in worker_ports { | ||
| let worker_addr = format!("http://[::1]:{}", port); | ||
| match InferenceServiceClient::connect(worker_addr).await { | ||
| for addr_str in worker_addrs { | ||
| let addr_str = addr_str.trim(); |
Comment on lines
+42
to
59
| for addr_str in worker_addrs { | ||
| let addr_str = addr_str.trim(); | ||
| match InferenceServiceClient::connect(addr_str.to_string()).await { | ||
| Ok(client) => { | ||
| info!("Connected to worker node on port {}", port); | ||
| info!("Connected to worker node at {}", addr_str); | ||
| worker_clients.push(client); | ||
| } | ||
| Err(e) => { | ||
| info!("Failed to connect to worker node on port {}: {}", port, e); | ||
| warn!("Failed to connect to worker at {}: {}", addr_str, e); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // safety check: if all workers are offline, crash the proxy | ||
| if worker_clients.is_empty() { | ||
| panic!("FATAL: No worker nodes are available. Exiting..."); | ||
| error!("FATAL: No worker nodes are available. Exiting..."); | ||
| panic!("No workers online"); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces Docker support for the project, enabling multi-container orchestration for the inference proxy and worker services. It also updates the codebase to support configuration via environment variables, making deployments more flexible and production-ready. Additionally, it improves logging and error handling for better observability.
Containerization and Deployment:
Dockerfileto build the Rust binaries and create a minimal runtime image, installing necessary dependencies and exposing required ports (50051,50052,9090)..dockerignorefile to excludetarget/and.gitfrom Docker build context, reducing build size and time.docker-compose.ymlfile to orchestrate multiple worker containers and the proxy, with environment variables for configuration.Configuration via Environment Variables:
src/main.rsandsrc/bin/worker.rsto read proxy and worker addresses from environment variables (PROXY_LISTEN_ADDR,WORKER_ADDRESSES,WORKER_LISTEN_ADDR), with sensible fallbacks for local development.Networking and Logging Improvements:
src/bin/client.rsto use0.0.0.0for better compatibility with Docker networking.