Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,78 @@ All notable changes to Code Context Control (C3) are documented here.
The format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.129.0] - 2026-09-09

### Fixed — a broken act runner no longer reports "0 parsed failures" (#173)

`act` can fail *before any workflow step runs*: the runner image cannot be
pulled, the daemon refuses, the container never starts. `ci_runner` handed
that log to `ci_failures.parse` like any other failure, so it came back
`parser: unparsed` with a log tail — and `c3_ci(action='failures')` answered
"0 parsed failures" for what was really "the runner image could not be
obtained".

`ci_act.setup_failure()` now recognises it from act's own
`Failure - Set up job` marker and names the cause: registry auth, missing
image, dead daemon, or the daemon's own words. The job is reported with
`parser: container-setup` and that reason. Status stays `FAILED` — a job
that could not run is never a green light.

`c3_ci(action='doctor')` reports whether the runner image is present locally
(without pulling ~1 GB to find out) and says what a setup failure looks like.
Note what a pre-flight probe **cannot** do: in the reported case the image
was present and `docker pull` succeeded, while act's own pull failed because
act sends the stored registry credential where the CLI sends none. Only a
real act run surfaces that, which is why the classification lives in the
runner. `tests/test_ci_act.py`'s one container test now skips with that
reason instead of failing on `parser != 'ruff'` when no step ever ran.

### Fixed — `c3_read` reads outside the project root instead of crashing

A path outside the project root did not get a refusal. It got an uncaught
`ValueError` out of `Path.relative_to` in `handle_read`, surfaced as
`Error calling tool 'c3_read': '…' is not in the subpath of '…'`. Nothing
in the tool ever checked containment: `validate_file_path` is a ghost-path
heuristic (type names, heredoc markers, redirect artifacts) and
`access_guard.canonicalize` returns `rel=""` with **no denial** for an
outside path, which `tests/test_access_guard.py` has always asserted.

The accidental boundary was worth nothing as a control:

- Access Guard rules match the **absolute canonical path** as well as the
project-relative one (`Rule.matches` tests `canon`), so `**/.env*`,
`**/.c3/secrets.enc` and user `deny` globs already bound on outside
paths. That is where the protection lives, and it is unchanged.
- The PreToolUse hook deliberately stands down outside the root, so native
`Read` went there unimpeded. Refusing only pushed the agent off the
budgeted, guarded, telemetered tool onto the one with none of that.

It also leaked. `full.exists()` was checked *before* the throw, so an
existing outside file produced the subpath error and a missing one produced
`File not found` — a distinguishable-response probe over the whole
filesystem, which is exactly what the R2 rule quoted three lines above that
code exists to prevent.

Outside paths are now read: source, maps, symbol reads and directory maps.
Access Guard still runs first, unchanged. What the root still decides is
**indexing** — `file_memory` is keyed by project-relative path and feeds the
text index, the map cache and `prune_stale`, so an outside file is served
from a transient record (`FileMemoryStore.build_transient_record`): same
parser, same schema, never persisted, never looked up. Directory maps
outside the root extract symbols the same transient way, under the same
file/deadline/byte bounds.

Every outside response opens with
`[c3-read:external] <abs path> — outside the project root; served but not
indexed, …`. Related facts are suppressed there (they are project knowledge
keyed by project-relative path), and telemetry carries `external: true`.

A hard root boundary is available as an Access Guard rule; it is not a
property of the tool. See `docs/file-map.md` § Outside the root.

`c3_edit` already tolerated outside paths (it falls back to `rel = file_path`
for the ledger) and is unchanged here.

## [2.128.1] - 2026-09-08

Both of these were found by shipping 2.128.0 and watching the runtime, not
Expand Down
76 changes: 76 additions & 0 deletions agent_file_map_systems.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Repository Mapping Systems for AI Coding Agents: Summary & Comparison

> **Purpose:** Evaluation and ranking of file-folder and repository mapping systems to optimize agent contextual awareness, token efficiency, and codebase navigation.

---

## 🏆 Summary Ranking & Architectural Tiers

```
[ Tier 1: AST & Graph-Driven (Gold Standard) ]
├── 1. Aider Repo Map (Tree-sitter + PageRank)
└── 2. MCP Graph Servers (e.g., repo-graph, Codebase-Memory)

[ Tier 2: IDE & Subagent Indexing Engines ]
├── 3. Claude Code Native Indexer (Subagent Context Architecture)
└── 4. Cursor / Copilot Workspace Symbol Maps

[ Tier 3: Static & Baseline Mapping ]
├── 5. Static Generator CLIs (e.g., repo-map / Annotated Markdown)
└── 6. Standard Directory Trees (`tree` / Shell Script Piping)
```

---

## 📊 Feature & Performance Matrix

| Tool / System | Architecture Type | Semantic Depth | Token Efficiency | Execution Model | Best Use Case |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **Aider Repo Map** | Native AST + Graph | **Highest** (Signatures + Call Graphs) | **Extreme** (~1k token cap) | Pre-prompt Graph Truncation | CLI coding agent workflows |
| **MCP Graph Servers** | MCP Server Daemon | **High** (Dynamic Graph Queries) | **Very High** (On-demand querying) | Tool-call based AST inspection | Cross-tool / Custom agent stacks |
| **Claude Code Indexer** | Subagent Orchestration | **High** (Subagent Summaries) | **High** (Isolated context) | Dynamic subagent exploration | Large monorepos & complex tasks |
| **Cursor / Copilot** | IDE Background Index | **Medium-High** (Vector + Symbol) | **Medium** (Automated windowing) | Continuous workspace sync | IDE-native development |
| **Static Markdown Maps** | Pre-computed LLM Docs | **Medium** (Annotated Folders) | **Medium** (Static overhead) | Session boot context injection | Architecture docs & `.cursorrules` |
| **Raw Shell `tree`** | Directory Paths Only | **Very Low** (Paths only) | **Poor** (High guesswork) | Pure shell dump | Simple scripts / Tiny codebases |

---

## 🔍 Detailed Analysis of Top Mapping Systems

### 1. Aider's Repo Map
* **Mechanism:** Leverages `tree-sitter` to parse source code into Abstract Syntax Trees (ASTs) and extract classes, function signatures, and variable definitions. Uses a **Personalized PageRank algorithm** over the directed dependency graph to surface only the relevant code symbols within a strict token budget (~1k tokens).
* **Key Advantages:** Unmatched context-to-token ratio; provides precise function signatures without sending full file implementations.
* **Limitations:** Coupled to Aider's CLI runtime environment.

### 2. MCP Structural Graph Servers (e.g., `repo-graph`, `Codebase-Memory`)
* **Mechanism:** Operates as a background Model Context Protocol (MCP) server. Indexes codebase entities using AST parsers and exposes explicit query endpoints (e.g., `get_caller_hierarchy`, `trace_route`, `get_symbol_definition`).
* **Key Advantages:** Reduces token bloat by 70–90%; standardized protocol compatible with Claude Code, Cursor, Windsurf, or custom agent frameworks.
* **Limitations:** Requires maintaining an active background daemon process during development sessions.

### 3. Claude Code Native Indexer
* **Mechanism:** Implements a **subagent architecture** to decouple discovery from reasoning. Specialized background subagents explore target directory branches and return concise structural summaries to the main orchestrator agent.
* **Key Advantages:** Scales effortlessly to massive monorepos where single static maps exceed context limits.
* **Limitations:** Introduces additional API tool-call latency during initial exploration phases.

### 4. Cursor / Copilot Workspace Symbol Indexers
* **Mechanism:** Proprietary background indexing engines combining semantic vector embeddings with AST symbol graphs integrated directly into the editor UI.
* **Key Advantages:** Completely friction-free developer experience with automated real-time index synchronization.
* **Limitations:** Proprietary black-box implementation with limited customization of token allocation and parsing rules.

### 5. Static Generator CLIs (Annotated `ARCHITECTURE.md` Maps)
* **Mechanism:** Traverses repo structures and utilizes fast LLM passes to generate human- and agent-readable architectural summaries stored in root project files (e.g., `CLAUDE.md`, `ARCHITECTURE.md`).
* **Key Advantages:** Instant session cold-starts with minimal operational overhead.
* **Limitations:** Prone to drift as code changes unless tied to git pre-commit triggers.

### 6. Standard `tree` Command Dumps
* **Mechanism:** Generates plain directory hierarchy text via native CLI commands (`tree -I "node_modules"`).
* **Key Advantages:** Zero dependencies or setup required.
* **Limitations:** Lacks semantic understanding; forces the agent into high-cost file exploration loops.

---

## 🎯 Implementation Guidance for AI Agents

1. **For Multi-File Refactoring / High Precision:** Prefer **AST/Graph-based options** (Aider Repo Map or MCP Graph Servers) to ensure type and import safety across module boundaries.
2. **For Monorepo Management:** Utilize **Subagent Context Isolation** (Claude Code pattern) to avoid blowing context windows on single-pass map injection.
3. **For Agent Context Injection:** Maintain a concise `ARCHITECTURE.md` map in the root directory for instant cold-start baseline orientation.
2 changes: 1 addition & 1 deletion cli/c3.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@
# Config
CONFIG_DIR = ".c3"
CONFIG_FILE = ".c3/config.json"
__version__ = "2.128.1"
__version__ = "2.129.0"


def _compress_file_cli(compressor, path, mode="smart", **kw):
Expand Down
5 changes: 4 additions & 1 deletion cli/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -734,7 +734,10 @@ async def c3_read(file_path: str, symbols: Any = None, lines: Any = None,
"""READ source, or MAP a file when called with only file_path (read-only, plan-mode safe).
file_path: single or comma-separated. symbols: names as the map prints them (Class.method ok).
lines: int, [start,end], or list of ranges. No symbols/lines = one line per symbol with
signature and [La-Lb] range at ~10% of the file's tokens; then read the symbols you need."""
signature and [La-Lb] range at ~10% of the file's tokens; then read the symbols you need.
Paths OUTSIDE the project root are read too (sibling repo, extra working dir, installed
package source): tagged [c3-read:external], served from a transient record, never added
to this project's index or related facts. Access Guard rules still apply there."""
path_err = validate_file_path(file_path)
if path_err:
return f"[c3_read:error] {path_err}"
Expand Down
34 changes: 34 additions & 0 deletions cli/tools/_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,40 @@ def finalize_with_tokens(finalize, svc, tool_name: str, args: dict,
_GHOST_NAMES = _PYTHON_TYPE_NAMES | _HEREDOC_MARKERS


EXTERNAL_TAG = "[c3-read:external]"


def external_banner(path: str) -> str:
"""The one line every outside-the-root response opens with.

One definition so c3_read and c3_compress cannot drift apart on it.
"""
return (f"{EXTERNAL_TAG} {path} — outside the project root; served but "
f"not indexed, and absent from this project's search, map cache "
f"and related facts.\n")


def project_key(full, project_path) -> tuple[str, bool]:
"""``(key, external)`` for an already-resolved-or-not path.

Inside the root the key is the project-relative posix path — how
file_memory, the edit ledger and related facts all address a file.
Outside it there is no such key, so the absolute posix path stands in and
``external`` tells the caller to serve from a transient record rather than
the index (docs/file-map.md § Outside the root).

This used to be a bare ``relative_to`` whose ValueError escaped as a tool
crash. It is not a containment check and must not become one: Access Guard
already rules on the absolute canonical path, and it runs before this.
"""
resolved = Path(full).resolve()
try:
rel = resolved.relative_to(Path(project_path).resolve())
return str(rel).replace("\\", "/"), False
except ValueError:
return resolved.as_posix(), True


def validate_file_path(file_path: str) -> str | None:
"""Return an error message if file_path looks like a ghost-file path, else None.

Expand Down
9 changes: 9 additions & 0 deletions cli/tools/ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,15 @@ def handle_ci(action: str, job: str, run_id: str, allow_foreign: bool,
f" runs Linux jobs in a container "
f"({ci_act.DEFAULT_IMAGE}), real `uses:` actions included",
]
if state.get("warning"):
lines.append(f" WARNING: {state['warning']}")
# An image that is present locally is still not proof act can pull
# it: act sends the stored registry credential where the CLI sends
# none, so an expired token fails a public pull (issue #173). Say
# so rather than let `available` read as a guarantee.
lines.append(" a runner failure before any step runs is "
"reported as a container-setup failure, not as 0 "
"parsed failures")
else:
lines += [" act UNAVAILABLE", f" {state['reason']}"]
lines += [
Expand Down
68 changes: 50 additions & 18 deletions cli/tools/compress.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@
from pathlib import Path

from cli.tools import _grants
from cli.tools._helpers import finalize_with_tokens, show_token_ratios
from cli.tools._helpers import external_banner, finalize_with_tokens, project_key, show_token_ratios
from core import count_tokens
from services import access_guard
from services.file_map import render_map

#: Every mode name c3_compress ever accepted, and what to say about it now.
RETIRED_MODES = {
Expand All @@ -35,21 +36,24 @@ def deprecation_line(requested: str) -> str:


def map_detail(svc, rel: str, requested_mode: str, *, cache_hit: bool,
symbols: int = 0, fallback: str = "") -> dict:
symbols: int = 0, fallback: str = "", record: dict = None,
external: bool = False) -> dict:
"""Telemetry `detail` for a map served from file_memory (C0, 2.120.0).

backend names the code path, parser the extractor that built the record
(tree_sitter | regex | generic, "" for a pre-2.120.0 record), cache_hit
whether the record was already fresh. `symbols` is how many the caller
asked for and `fallback` why a read served a map instead of source.
`deprecated_mode` (2.122.0) is set when a retired mode name was asked for.
`record` is a caller-supplied transient record (an outside-the-root read,
which is deliberately absent from file_memory) and `external` flags it.
Flat so telemetry.aggregate_tool_telemetry can fold it without a schema.
"""
record = None
try:
record = svc.file_memory.get(rel)
except Exception:
record = None
if record is None:
try:
record = svc.file_memory.get(rel)
except Exception:
record = None
if not isinstance(record, dict):
record = None
detail = {
Expand All @@ -65,6 +69,9 @@ def map_detail(svc, rel: str, requested_mode: str, *, cache_hit: bool,
detail["fallback"] = fallback
if requested_mode not in ("map", "read"):
detail["deprecated_mode"] = requested_mode
if external:
detail["backend"] = "transient"
detail["external"] = True
return detail


Expand Down Expand Up @@ -136,20 +143,37 @@ def _compress_single(file_path: str, requested: str, note: str, svc, finalize) -
if not full.exists():
return ("[file_map:error] not found. To create a new file use "
"c3_edit(file_path=..., old_string='', new_string=<content>).")
rel = str(full.resolve().relative_to(Path(svc.project_path).resolve())).replace("\\", "/")
# C0 measurement: was the record already fresh before this call?
cache_hit = not svc.file_memory.needs_update(rel)
res = _build_map(svc, rel)
resolved = full.resolve()
rel, external = project_key(resolved, svc.project_path)
# Outside the root the map comes from a transient record — same parser,
# never written to an index keyed by project-relative path
# (docs/file-map.md § Outside the root). c3_read does the same thing; the
# two map surfaces must not disagree about where a file may live.
banner = external_banner(rel) if external else ""
if external:
ext_record = svc.file_memory.build_transient_record(resolved, key=rel)
cache_hit = False
body = (render_map(ext_record, max_tokens=svc.file_memory.MAP_TOKEN_BUDGET)
if ext_record
else f"[file_map] Could not build map for {rel} — unreadable.")
else:
ext_record = None
# C0 measurement: was the record already fresh before this call?
cache_hit = not svc.file_memory.needs_update(rel)
body = _build_map(svc, rel)
res = banner + body
raw_tokens = None
map_tokens = 0
try:
raw_text = full.read_text(encoding="utf-8", errors="replace")
raw_tokens = count_tokens(raw_text)
map_tokens = count_tokens(res)
if map_tokens >= raw_tokens and not res.startswith("[file_map]"):
if map_tokens >= raw_tokens and not body.startswith("[file_map]"):
# A map that costs more than the file is worth nothing
# (docs/file-map.md § Small files).
res = f"[compress:{file_path}] whole file — smaller than its map\n" + raw_text
res = (banner
+ f"[compress:{file_path}] whole file — smaller than its map\n"
+ raw_text)
map_tokens = count_tokens(res)
summary = "map"
except Exception:
Expand All @@ -162,7 +186,8 @@ def _compress_single(file_path: str, requested: str, note: str, svc, finalize) -
res, summary,
raw_tokens=raw_tokens, optimized_tokens=map_tokens or None,
response_tokens=map_tokens,
detail=map_detail(svc, rel, requested, cache_hit=cache_hit))
detail=map_detail(svc, rel, requested, cache_hit=cache_hit,
record=ext_record, external=external))


def _compress_batch(paths: list, requested: str, note: str, svc, finalize) -> str:
Expand Down Expand Up @@ -195,10 +220,17 @@ def _do_one(fp):
full = Path(fp)
if not full.exists():
return fp, None, None, None, "not found"
rel = str(full.resolve().relative_to(
Path(svc.project_path).resolve())).replace("\\", "/")
res = svc.file_memory.get_or_build_map(
rel, max_tokens=svc.file_memory.MAP_TOKEN_BUDGET)
resolved = full.resolve()
rel, external = project_key(resolved, svc.project_path)
if external:
record = svc.file_memory.build_transient_record(resolved, key=rel)
res = external_banner(rel) + (
render_map(record, max_tokens=svc.file_memory.MAP_TOKEN_BUDGET)
if record
else f"[file_map] Could not build map for {rel} — unreadable.")
else:
res = svc.file_memory.get_or_build_map(
rel, max_tokens=svc.file_memory.MAP_TOKEN_BUDGET)
try:
raw_tok = count_tokens(full.read_text(encoding="utf-8", errors="replace"))
map_tok = count_tokens(res)
Expand Down
Loading
Loading