Repository navigation
Add initial performance investigation tooling - #122
Open
lightspeed-internal-1[bot] wants to merge 13 commits into
Open
lightspeed-internal-1[bot] wants to merge 13 commits into
lightspeed-internal-1[bot] wants to merge 13 commits into
Conversation
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Contributor
Author
|
No description provided. |
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
Signed-off-by: Engineer <310755840+lightspeed-internal-1[bot]@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Original task — issue #121
Implement the initial runtime performance-investigation milestone from the published
perf/initial-profiling-findingsstarting point: a standalone OpenAI Chat Completions simulator, a TypeScript driver using the public runtime API, minimum queue/execution instrumentation, and stepped experiments for rapid agent loops, idle/active event readers, and mature histories. Publish measured findings/data on issue #121 and stop before environment profiling or a general optimization campaign. Use isolated synthetic data only; no production endpoints/data or billable model calls.Delivered
crates/llm-simulatorserving native Chat Completions responses with deterministic scenario/tool sequences, response sizes/usage, delay/jitter, scripted failures, and per-request metrics.scripts/perf/driver, fixtures, scheduler/accounting tests, run scripts and observations; minimal optional Temporal SDK metrics and SQLx pool-acquisition diagnostics.EMFILEerrors at a 1,024 soft file-descriptor limit; the simulator sustained 1,000 small requests/s. These are not production or bank-capacity claims.Validation
Passed:
cargo test -p llm-simulator --locked(4); OpenAI endpoint/client integration (6); explicitly selected local simulator/runtime-adapter/CAS integration (1); runtime config tests (9); driver scheduler/accounting tests (7); TypeScript typecheck;cargo fmt --check;npm run check:docs; local roadmap links;git diff --check; public API smoke (4/4 completed); scripted context-limit failure classification; and the exact required gatecargo clippy --workspace --all-targets --locked -- -D warnings.No ignored Temporal live suite,
runs_live_slow, credentialed provider test or environment-capacity test was run. Platform-authenticated reads, hardware CPU/allocation profiles and some CAS/context attribution remain explicit gaps. Full tables, exact commands, raw result files and diagnostic limitations will be published in the issue report with an accessible artifact link.Handoff / remaining
Implementation commit:
f5f8ae6f985b0bab1fb3bd4e7302cc5980d32dcd. This PR is for human review againstmain; do not merge automatically. The issue report and accessible raw-data asset are the final delivery and then this workstream stops pending follow-up. Luke's latest task clarification was to use the exclusively assigned VM for these experiments within the stated scope and safety limits; that was followed.