Herdr outside container

Summary

Extend the existing Herdr Dev Container Feature to support connecting to a Herdr server running on the host machine.

The goal is to enable AI coding agents running inside separate Dev Containers to discover, communicate, and coordinate with one another through a shared Herdr instance.

Follow the same developer experience as the official Docker outside-of-Docker Dev Container Feature.

Important: Extend the existing src/herdr Feature. Do not create a separate Feature.

Problem

Consider a development environment with three repositories, each running in its own Dev Container:

Repo A: GitHub Copilot CLI Agent A
Repo B: GitHub Copilot CLI Agent B
Repo C: GitHub Copilot CLI Agent C

Herdr runs on the host and manages the agents through its workspaces and terminals.

Although Herdr can launch Copilot CLI inside Dev Containers using devcontainer exec, the agents themselves cannot automatically access the host Herdr control API.

This prevents an agent in Repo A from using Herdr to discover or communicate directly with agents in Repo B or Repo C.

We need to expose the host Herdr control plane to agents running inside Dev Containers without requiring a separate Herdr server in each container.

Proposed Solution

Enhance the existing Herdr Feature with an optional outside-of-container mode.

This mode should:

Install the Herdr CLI using the existing Feature implementation.
Connect to an existing Herdr server running on the host.
Make the host Herdr control socket accessible inside the Dev Container.
Configure the CLI to communicate with the host Herdr instance.
Support agent discovery, prompting, status inspection, and response retrieval.
Preserve the existing Feature's default behavior and backward compatibility.

The preferred implementation is a Unix socket bind mount, provided Herdr supports the required client/server connection behavior.

If direct socket mounting is unsupported on a target platform, investigate a compatible relay or alternative transport rather than introducing another orchestration system.

Target Architecture

Host Machine
│
├── Herdr Server
│   ├── Workspace: Repo A
│   ├── Workspace: Repo B
│   └── Workspace: Repo C
│
├── Shared Herdr Control API / Socket
│   │
│   ├── Dev Container A
│   │   ├── Herdr CLI
│   │   └── Copilot Agent A
│   │
│   ├── Dev Container B
│   │   ├── Herdr CLI
│   │   └── Copilot Agent B
│   │
│   └── Dev Container C
│       ├── Herdr CLI
│       └── Copilot Agent C
│
└── Cross-Agent Communication
    ├── Agent discovery
    ├── Prompt delivery
    ├── Agent status
    └── Response retrieval

All three containers connect to the same Herdr control plane.

Agents must be able to communicate across repository boundaries without requiring direct network connections between containers.

Expected Developer Experience

The existing Herdr Feature remains the only Feature needed.

Example devcontainer.json:

{
  "features": {
    "ghcr.io/jsburckhardt/devcontainer-features/herdr:1": {
      "outsideOfContainer": true
    }
  }
}

Note: outsideOfContainer is a proposed new option, not an existing supported configuration.

The final naming, socket configuration, and implementation details should follow the repository's existing conventions.

Desired Behavior

With outside-of-container mode enabled:

The Herdr CLI is available inside the Dev Container.
The CLI connects to the host Herdr server.
No additional Herdr server is launched inside the container.
Agents can discover agents running in other Dev Containers.
Agents can send prompts and retrieve responses using Herdr's native commands.

Functional Requirements

1. Extend the Existing Herdr Feature

Reuse the existing src/herdr implementation.
Preserve existing installation logic.
Preserve existing options and behavior.
Add outside-of-container connectivity as an optional capability.
Maintain backward compatibility.
Do not create another Feature.

2. Host Herdr Connectivity

Investigate and implement access to the host Herdr server.

Preferred approach:

Identify the host Herdr Unix socket.
Mount the socket into the Dev Container.
Configure the Herdr CLI to connect through the mounted socket.
Support configurable host and container socket paths.
Handle filesystem permissions and UID/GID differences.
Avoid requiring privileged containers.

Validate Herdr's socket location, protocol, and CLI connection configuration before choosing the implementation.

3. Agent Discovery

From inside any connected Dev Container, the Herdr CLI should list agents running across all connected repositories.

Example:

herdr agent list

Expected behavior:

Agents from Repo A, Repo B, and Repo C are visible.
Agent IDs and names are consistent with the host Herdr instance.
Agents can be addressed using Herdr's native identifiers.

4. Cross-Container Agent Communication

An agent inside Repo A should be able to communicate with an agent inside Repo B.

Example:

# Discover available agents
herdr agent list

# Ask another agent a question
herdr agent prompt backend \
  "What API contract are you implementing?" \
  --wait --timeout 120000

# Retrieve the response
herdr agent read backend \
  --source recent --lines 100

The commands should work through the shared host Herdr instance.

Communication should be bidirectional:

Repo A → Repo B
Repo B → Repo A
Repo A → Repo C
Repo C → Repo B

The Feature should expose Herdr's existing capabilities, not implement a separate messaging protocol.

5. Agent Visibility

Agents launched inside Dev Containers should remain visible and manageable through the host Herdr interface.

For example:

HERDR_AGENT=copilot \
  devcontainer exec \
  --workspace-folder ~/repos/repo-a \
  copilot --yolo

Investigate and document how agent detection, naming, status tracking, and terminal ownership interact with the outside-of-container setup.

The Feature should not assume that mounting the socket alone automatically registers agents with Herdr.

6. Configuration

Investigate support for optional configuration such as:

Option	Purpose
`outsideOfContainer`	Enable connection to an external Herdr server
`hostSocketPath`	Configure the host Herdr socket location
`containerSocketPath`	Configure the socket location inside the container

These are proposed options; final names and defaults should follow existing Feature conventions.

Configuration should require minimal manual steps.

The implementation must account for the fact that Dev Container Feature installation scripts run during image build, while host socket mounting must be configured at container runtime.

7. Platform Compatibility

Investigate support for:

Linux with native Docker
macOS with Docker Desktop
Windows with WSL2
GitHub Codespaces

Document platform-specific requirements.

Where Unix socket mounting is unavailable, document limitations and investigate whether a relay or another supported Herdr transport is necessary.

8. Security

Treat the host Herdr control socket as a privileged interface.

Agents with access to the socket may be able to control other agents and terminals managed by the host Herdr instance.

Requirements:

Document the trust boundary.
Avoid unnecessary privileged container access.
Avoid mounting the Docker socket solely for Herdr integration.
Investigate whether Herdr supports restricted permissions or authentication.
Document the risks of granting autonomous agents cross-workspace control.
Highlight additional risks when GitHub Copilot CLI runs in YOLO mode.

Direct socket mounting should only be enabled when explicitly configured.

Acceptance Criteria

Feature Integration

Existing src/herdr Feature is extended.
Existing Feature behavior remains backward compatible.
Outside-of-container mode is optional.
Herdr CLI is available inside the Dev Container.
No additional Herdr server is required inside the container.

Connectivity

Container CLI successfully connects to the host Herdr server.
Host socket paths can be configured where applicable.
Socket permissions are handled appropriately.
Connectivity works after a Dev Container rebuild.
Unsupported host configurations fail with actionable errors.

Agent Communication

herdr agent list discovers agents across containers.
Agent A can send a prompt to Agent B.
Agent B can respond through Herdr.
Agent A can retrieve Agent B's response.
Agent communication works bidirectionally.
Agent status remains visible in the host Herdr interface.

Quality and Documentation

Automated Feature tests are provided where feasible.
A working multi-container example is documented.
Existing Feature documentation is updated.
Supported platforms and limitations are documented.
Security considerations are documented.

Implementation Approach

Phase 1 — Research and Validation

Inspect the existing src/herdr Feature.
Identify Herdr's control socket and client connection behavior.
Confirm whether the CLI supports connecting to an existing server through a configurable socket path.
Validate access to the host Herdr instance from one Dev Container.
Identify platform limitations and security implications.

Do not assume the socket-based approach is supported until validated.

Phase 2 — Feature Enhancement

Introduce optional outside-of-container configuration.
Implement required runtime mount configuration.
Configure Herdr CLI connectivity.
Preserve existing Feature installation behavior.
Add diagnostics for unavailable host connectivity.

Phase 3 — Multi-Container Validation

Test with three independent repositories and Dev Containers.

Validate:

Agents appear in the host Herdr interface.
Agents are discoverable from within containers.
Agents can send prompts across repositories.
Agents can retrieve responses.
Communication works in both directions.
Configuration survives container rebuilds.

Phase 4 — Documentation and Tests

Add Feature configuration examples.
Document supported environments.
Add automated tests where feasible.
Document security considerations.
Include troubleshooting guidance.

Out of Scope

Creating a new Herdr Dev Container Feature.
Building a custom agent orchestration framework.
Replacing Herdr's native agent management.
Implementing Foreman or RPIV workflow logic.
Creating a separate inter-agent messaging protocol.
Running independent Herdr servers inside every Dev Container.

References

Existing Implementation — Extend This

Existing published Feature:

{
  "features": {
    "ghcr.io/jsburckhardt/devcontainer-features/herdr:1": {}
  }
}

Herdr

Implementation Pattern

Definition of Done

A developer can use the existing Herdr Dev Container Feature across three independent repositories, each running GitHub Copilot CLI inside its own Dev Container, while sharing a single Herdr server on the host.

Agents can:

Discover each other.
Send prompts across repository boundaries.
Wait for responses.
Read responses.
Inspect agent status.

All communication uses Herdr's native control capabilities, without deploying additional Herdr servers or introducing a separate orchestration framework.

The implementation must verify that the proposed host connectivity mechanism is supported before treating it as the final design.

Herdr outside container #156

Description