diff --git a/CHANGELOG.md b/CHANGELOG.md index 096ae98..46f983a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ Versions follow [semantic versioning](https://semver.org): while the major versi - **Instructions found in project files are reported.** Text in comments, docs, test data, or fetched pages that addresses AI assistants is never followed, and the agent tells the developer which file contains it. The report header has a required "Instructions found in project files" section, and the final summary repeats it, so the warning is not lost in report mode. - **The language file is required reading.** In every mode, `SKILL.md` names the language files next to the mode file, to be read before any code: one for each language in the code being read or written, `GENERIC.md` for a language without one, and a language met partway through is loaded before its code is read. The language's hidden costs and security traps (such as unrandomized integer hashing in Python) are no longer skipped. - **`SKILL.md` is less than half as long** (about 2,600 words, from 5,600), so every session loads less. Each mandatory rule is stated in a few sentences there, and its full text moved to a new "Rule details" section in `WORKFLOW.md`, which report and apply mode already read. The workflow overview and anti-pattern list, which repeated `WORKFLOW.md` and the rules, were removed; their examples moved into the rules. Rule numbers are unchanged. +- **Report mode loads less.** Applying and close-out (Phases 5 and 6) moved from `WORKFLOW.md` to a new `APPLYING.md`, which only apply mode reads, in the same step as `WORKFLOW.md`. Text in "Rule details" that repeated those phases or `DIFFERENTIAL-TESTING.md` was removed. - **New traps and tests:** cyclic structures (an iterative rewrite without a visited set loops forever where recursion failed fast), and round-trip tests when new code reads what existing code writes. ### Fixed diff --git a/README.md b/README.md index 660c2f8..99e2f69 100644 --- a/README.md +++ b/README.md @@ -251,7 +251,8 @@ The skill's knowledge base is modular and organized around its workflow. The age | File | Purpose | |---|---| | `SKILL.md` | The mandatory rules in brief, modes, which files to read first, instructions found in project files, working in few steps | -| `WORKFLOW.md` | Report and apply mode: setup (including the test baseline and commit authorization), survey (scope, contracts, reachability to entry points, cost along the call chain, exposure), analysis, report, apply, close-out; the full text of the mandatory rules ("Rule details") | +| `WORKFLOW.md` | Report and apply mode: setup (including the test baseline and commit authorization), survey (scope, contracts, reachability to entry points, cost along the call chain, exposure), analysis, report; the full text of the mandatory rules ("Rule details") | +| `APPLYING.md` | Apply mode only: applying each item (differential tests first, change, verify, benchmark, behavior changes held back, commits only if authorized) and close-out | | `WRITING-NEW-CODE.md` | Write mode: which rules apply to new code, sizing the workload, choosing structures, existing-code bottlenecks, tests, summary | | `ALGORITHM-ARSENAL.md` | How to find candidates: six triage questions (starting with whether the work is needed at all), complexity reference, routing by current complexity, lower bounds, reformulation, behavior-changing techniques | | `CORE-TECHNIQUES.md` | Lookup tables of core techniques with their assumptions, production patterns, database queries, bit manipulation, combinations; loaded one section at a time | diff --git a/skills/APPLYING.md b/skills/APPLYING.md new file mode 100644 index 0000000..5f4ee01 --- /dev/null +++ b/skills/APPLYING.md @@ -0,0 +1,21 @@ +# Applying + +Apply mode only: Phases 5 and 6, after Phases 1 to 4 in `WORKFLOW.md`. Read this file together with `WORKFLOW.md`, in the same step. The mandatory rules and their full text ("Rule details" in `WORKFLOW.md`) apply throughout. + +## Phase 5: Apply + +For each approved item, one at a time: + +1. **Characterize the original.** Write differential tests per `DIFFERENTIAL-TESTING.md`, with the original implementation copied verbatim as the reference, and turn the contract into executable checks ("The contract as executable properties"). Run them against the unchanged code first: they must pass, which shows the harness is wired correctly. Then check the check: the tests must catch planted differences, and pass with the original on both sides when behavior can vary between runs ("Check the check" in `DIFFERENTIAL-TESTING.md`). Keep one differential-test file and one benchmark program for the whole apply phase, and add each item to them ("Work in few, large steps" in `SKILL.md`). +2. **Change the code.** Make the smallest change that implements the chosen approach. Keep the signature and contract. Add a short comment where the technique is not obvious. +3. **Verify.** Run the differential tests and contract checks, then the full existing suite, then any bounded or exhaustive check from the verification plan. Walk the behavior traps again for this change, and confirm each is covered by a test or ruled out by analysis. Record the level reached. +4. **Benchmark.** Measure time and memory for the original and the new version at realistic `n`, at a larger `n`, and on worst-case inputs where relevant (`BENCHMARKING.md`). State the end-to-end effect: the share of the whole operation's time the changed code took before the change, and the resulting saving, measured where possible ("Component and end-to-end" in `BENCHMARKING.md`). Record the numbers as measured, with the measured growth exponent, and compare them with the Phase 3 prediction (`WORKFLOW.md`). If they disagree, find out why before keeping the change: a wrong prediction means the analysis missed a cost. If the new version is not faster at realistic sizes, do not keep it unless the developer agrees for scaling reasons; a change not kept goes under "Tried and rejected." If verification or benchmarking cannot be completed, say so, report the gain as estimated only, and do not present the change as a confirmed win. +5. **Handle unverified parts and behavior changes.** If any behavior cannot be verified, or the change alters behavior in any way, stop work on this item: restore its original code, do not commit it, add it to "Behavior changes for review," and go on to the next item. Ask about every such item at the end; an earlier "apply all" does not approve it (rule 5). If the developer approves it, apply it with the `NEEDS-TESTING` or `BEHAVIOR-CHANGE` marker from `REPORTING.md`. If the developer declines it, apply the version that keeps the behavior, when one exists, through steps 1 to 4 ("Declined is not the end of the item" under rule 5 in "Rule details" in `WORKFLOW.md`). +6. **Run again after the final edit.** If anything changed after the last test run (review fixes, comments, formatting), run the differential tests and the full suite again. Results from an earlier version of the code do not count. +7. **Commit, only if authorized.** If the developer authorized commits (Phase 1, step 4, in `WORKFLOW.md`), make one commit per optimization, with a message that states the complexity before and after, the benchmark result, the verification level, and the tests added. Do not mix bug fixes, formatting, or unrelated refactors into it. Without authorization, do not commit: record in the report the exact files and functions this item changed, so the developer can review, commit, or revert it on its own. + +If a differential test fails, the optimization is wrong: fix the optimization, never the test. If the failure shows that the original behavior is itself a bug, record it in the bug log and follow rule 4: if the best optimization removes the bug as a side effect, confirm that outputs differ only on inputs that trigger the bug and document it; otherwise keep matching the original. + +## Phase 6: Close out + +Write `APPLY-RESULTS.md` once, in one pass ("Apply results format" in `REPORTING.md`), and leave `OPTIMIZATION-REPORT.md` exactly as the analysis wrote it. Cover what was applied, skipped, or rejected; final benchmark numbers against the predictions; changes tried and rejected while applying; bugs found while applying; and the full lists of `NEEDS-TESTING` and `BEHAVIOR-CHANGE` items. State plainly which items were verified and at which level (with the evidence), which the developer accepted with unverified behavior, and which knowingly change behavior. Call an item proven only at verification level 4. Offer to keep each item's benchmark as a regression guard ("Offer a regression guard" in `BENCHMARKING.md`), and list the ones the developer kept. diff --git a/skills/REPORTING.md b/skills/REPORTING.md index 9c59194..c855e13 100644 --- a/skills/REPORTING.md +++ b/skills/REPORTING.md @@ -6,7 +6,7 @@ For a small result (one or two items), present the report in the conversation. O - `OPTIMIZATION-REPORT.md` — written in Phase 4: the ranked items, behavior changes for review, and tried-and-rejected candidates from the analysis. - `BUGS-FOUND.md` — written in Phase 4: every bug noticed during the analysis. None are fixed on purpose. -- `APPLY-RESULTS.md` — written at the end of apply mode (Phase 6): what was applied and how it was verified and measured, changes marked as needing testing, behavior changes decided, changes tried and rejected while applying, and bugs found while applying. It refers to items by the IDs in `OPTIMIZATION-REPORT.md`. +- `APPLY-RESULTS.md` — written at the end of apply mode (Phase 6 in `APPLYING.md`): what was applied and how it was verified and measured, changes marked as needing testing, behavior changes decided, changes tried and rejected while applying, and bugs found while applying. It refers to items by the IDs in `OPTIMIZATION-REPORT.md`. **Write each file once, and do not edit it afterward.** Text you write stays in the conversation, and every later step sends the whole conversation to the model again: a report's size is paid again on every step after it, and every edit adds another copy of the changed text. - Keep findings in your working notes while you analyze. Write `OPTIMIZATION-REPORT.md` and `BUGS-FOUND.md` once, after Phase 3 is complete for every candidate. diff --git a/skills/SKILL.md b/skills/SKILL.md index 68f2a65..992427b 100644 --- a/skills/SKILL.md +++ b/skills/SKILL.md @@ -11,7 +11,7 @@ Make code fast by lowering its time complexity, using the right data structures, **Then, if you know your model, check it.** This skill is built for coding-tier models such as Claude Sonnet and Opus. If you know which model you are running on and its provider offers it as a small, fast, or low-cost tier (for example Claude Haiku, or a model named "mini," "nano," or "lite"), add one line after the mode line: "Note: this skill is built for coding-tier models such as Claude Sonnet and Opus; on , expect shallower analysis and more speedups held back for review." Then continue as normal. If you do not know your model, say nothing about it. -**Then, before looking at any code, read the file for that mode.** Invoking the skill loads only this summary, and skipping the mode file is the most common way this skill fails: [WORKFLOW.md](WORKFLOW.md) for report and apply mode, following its phases in order; [WRITING-NEW-CODE.md](WRITING-NEW-CODE.md) for write mode. In every mode, also read the language file for each language in the code you will read or write, and only those, before looking at that code: `languages/PYTHON.md`, `JAVASCRIPT-TYPESCRIPT.md`, `JAVA.md`, `KOTLIN.md`, `SWIFT.md`, `CSHARP.md`, `C.md`, or `RUST.md`. For a language with no file here, read `languages/GENERIC.md`: it says to load the profile saved in the project if one exists and how to build one if not. If you come across code in another language partway through the task, read that language's file before reading that code. The language file's "Hidden costs" section lists traps, including security ones such as unrandomized hashing, that no other file repeats. Read [DIFFERENTIAL-TESTING.md](DIFFERENTIAL-TESTING.md) before writing any differential test. The mode files hold the details of the rules below and the steps that carry them out, which this file does not repeat. +**Then, before looking at any code, read the file for that mode.** Invoking the skill loads only this summary, and skipping the mode file is the most common way this skill fails: [WORKFLOW.md](WORKFLOW.md) for report and apply mode, following its phases in order, and in apply mode also [APPLYING.md](APPLYING.md), read in the same step; [WRITING-NEW-CODE.md](WRITING-NEW-CODE.md) for write mode. In every mode, also read the language file for each language in the code you will read or write, and only those, before looking at that code: `languages/PYTHON.md`, `JAVASCRIPT-TYPESCRIPT.md`, `JAVA.md`, `KOTLIN.md`, `SWIFT.md`, `CSHARP.md`, `C.md`, or `RUST.md`. For a language with no file here, read `languages/GENERIC.md`: it says to load the profile saved in the project if one exists and how to build one if not. If you come across code in another language partway through the task, read that language's file before reading that code. The language file's "Hidden costs" section lists traps, including security ones such as unrandomized hashing, that no other file repeats. Read [DIFFERENTIAL-TESTING.md](DIFFERENTIAL-TESTING.md) before writing any differential test. The mode files hold the details of the rules below and the steps that carry them out, which this file does not repeat. ## Mandatory rules diff --git a/skills/WORKFLOW.md b/skills/WORKFLOW.md index a935a4f..6375f14 100644 --- a/skills/WORKFLOW.md +++ b/skills/WORKFLOW.md @@ -1,6 +1,6 @@ # Workflow -Follow these phases in order in report and apply mode. Report mode stops after Phase 4. Write mode follows `WRITING-NEW-CODE.md` instead. "Rule details" at the end of this file holds the full text of the mandatory rules that `SKILL.md` states briefly; it applies in every phase. +Follow these phases in order in report and apply mode. Report mode stops after Phase 4. Apply mode continues with Phases 5 and 6 in `APPLYING.md`. Write mode follows `WRITING-NEW-CODE.md` instead. "Rule details" at the end of this file holds the full text of the mandatory rules that `SKILL.md` states briefly; it applies in every phase. ## Phase 1: Setup @@ -61,24 +61,6 @@ Once every candidate has been analyzed, write the report defined in `REPORTING.m In report mode, stop here and present the report. Ask which items to apply and raise any dependency requests. Ask separately about each item under "Behavior changes for review": a general answer such as "apply all" covers only items whose equivalence is verified (rule 5). -## Phase 5: Apply - -For each approved item, one at a time: - -1. **Characterize the original.** Write differential tests per `DIFFERENTIAL-TESTING.md`, with the original implementation copied verbatim as the reference, and turn the contract into executable checks ("The contract as executable properties"). Run them against the unchanged code first: they must pass, which shows the harness is wired correctly. Then check the check: the tests must catch planted differences, and pass with the original on both sides when behavior can vary between runs ("Check the check" in `DIFFERENTIAL-TESTING.md`). Keep one differential-test file and one benchmark program for the whole apply phase, and add each item to them ("Work in few, large steps" in `SKILL.md`). -2. **Change the code.** Make the smallest change that implements the chosen approach. Keep the signature and contract. Add a short comment where the technique is not obvious. -3. **Verify.** Run the differential tests and contract checks, then the full existing suite, then any bounded or exhaustive check from the verification plan. Walk the behavior traps again for this change, and confirm each is covered by a test or ruled out by analysis. Record the level reached. -4. **Benchmark.** Measure time and memory for the original and the new version at realistic `n`, at a larger `n`, and on worst-case inputs where relevant (`BENCHMARKING.md`). State the end-to-end effect: the share of the whole operation's time the changed code took before the change, and the resulting saving, measured where possible ("Component and end-to-end" in `BENCHMARKING.md`). Record the numbers as measured, with the measured growth exponent, and compare them with the Phase 3 prediction. If they disagree, find out why before keeping the change: a wrong prediction means the analysis missed a cost. If the new version is not faster at realistic sizes, do not keep it unless the developer agrees for scaling reasons; a change not kept goes under "Tried and rejected." If benchmarking cannot be run, say so and report the gain as estimated only. -5. **Handle unverified parts and behavior changes.** If any behavior cannot be verified, or the change alters behavior in any way, stop work on this item: restore its original code, do not commit it, add it to "Behavior changes for review," and go on to the next item. Ask about every such item at the end; an earlier "apply all" does not approve it (rule 5). If the developer approves it, apply it with the `NEEDS-TESTING` or `BEHAVIOR-CHANGE` marker from `REPORTING.md`. If the developer declines it, apply the version that keeps the behavior, when one exists, through steps 1 to 4 ("Declined is not the end of the item" under rule 5 in "Rule details"). -6. **Run again after the final edit.** If anything changed after the last test run (review fixes, comments, formatting), run the differential tests and the full suite again. Results from an earlier version of the code do not count. -7. **Commit, only if authorized.** If the developer authorized commits (Phase 1, step 4), make one commit per optimization, with a message that states the complexity before and after, the benchmark result, the verification level, and the tests added. Do not mix bug fixes, formatting, or unrelated refactors into it. Without authorization, do not commit: record in the report the exact files and functions this item changed, so the developer can review, commit, or revert it on its own. - -If a differential test fails, the optimization is wrong: fix the optimization, never the test. If the failure shows that the original behavior is itself a bug, record it in the bug log and follow rule 4: if the best optimization removes the bug as a side effect, confirm that outputs differ only on inputs that trigger the bug and document it; otherwise keep matching the original. - -## Phase 6: Close out - -Write `APPLY-RESULTS.md` once, in one pass ("Apply results format" in `REPORTING.md`), and leave `OPTIMIZATION-REPORT.md` exactly as the analysis wrote it. Cover what was applied, skipped, or rejected; final benchmark numbers against the predictions; changes tried and rejected while applying; bugs found while applying; and the full lists of `NEEDS-TESTING` and `BEHAVIOR-CHANGE` items. State plainly which items were verified and at which level (with the evidence), which the developer accepted with unverified behavior, and which knowingly change behavior. Call an item proven only at verification level 4. Offer to keep each item's benchmark as a regression guard ("Offer a regression guard" in `BENCHMARKING.md`), and list the ones the developer kept. - ## Rule details The full text of the mandatory rules in `SKILL.md` where it says more than the short version there. Rule numbers match `SKILL.md`. Apply these in every phase; the phases above refer to them by name. @@ -87,7 +69,7 @@ The full text of the mandatory rules in `SKILL.md` where it says more than the s **Rule 2, trading time for memory.** Replacing an O(1) hash map with an O(log n) sorted array to save memory is the kind of change rule 2 forbids. When the code exceeds a hard memory budget, techniques that trade time for memory include external sorting, recomputation (such as gradient checkpointing), and streaming. -**Rule 3, verification.** Aim for the highest verification level that is cheap to reach ("Verification levels" in `DIFFERENTIAL-TESTING.md`). Passing tests are evidence, not proof. Report the inputs and properties covered, the tests and tools actually run, and any behavior left unverified. +**Rule 3, verification.** The details are in `DIFFERENTIAL-TESTING.md` ("Verification levels," "Check the check," and "Reporting verification"). **Rule 4, bugs.** - **If the buggy code cannot be optimized** (it is already optimal, or no optimization can be verified to keep behavior unchanged), leave it exactly as it is. @@ -113,6 +95,6 @@ List each one under "Behavior changes for review" with the exact behavior before A version found this way is an ordinary item: verify it and apply it under a general approval. Report the faster version that changes behavior next to it, as an option for review. - **Declined is not the end of the item.** When the developer declines a behavior change, apply the version that keeps the behavior if one exists (once verified, it needs no further approval), and record it in the item's decision. Leave the code unchanged only when no such version exists. -**Rule 6, complexity.** Whatever the starting complexity (factorial, exponential, polynomial of any degree, or a product of several size variables), analyze the whole call chain: a function that is optimal for one call can be the inner loop of a cubic computation. Many problems have no known lower bound, or none the code can establish, especially when cost depends on databases, networks, distributed systems, hardware, or data distributions: report the bound as unknown with the reason, and give the strongest bound you can support (for example, "at least one pass over the input"). A bound below the input size (O(log n), O(1)) is possible only when the work is amortized over repeated queries, the input is already structured (such as sorted or indexed), or a correct answer does not depend on every element (for example, finding any local peak). Use `ADVANCED-TECHNIQUES.md` when the core techniques do not reach the lower bound or the problem is in a specialized domain; it is an index, so load only the `techniques/` files it routes to. When counting calls, a key function runs n times when the sort computes keys once, and once per comparison when it is wrapped in a comparator; wrapping values in objects and hashing string keys (proportional to length) count too; a doubling experiment (`BENCHMARKING.md`) settles a cost no document does. Analysis never stops early: always identify and report the best achievable approach and the lower bound. Applying is evidence-driven: recommend applying a change only when measurements or a stated workload estimate show it pays off, do not add a specialized structure only for a better asymptotic bound, and stop applying once the bottleneck is resolved or the remaining cost is at the lower bound, recording what remains. +**Rule 6, complexity.** Whatever the starting complexity (factorial, exponential, polynomial of any degree, or a product of several size variables), look for its lower bound: do not look only for quadratic code, and count the cost along the whole call chain (Phase 2, step 4). Many problems have no known lower bound, or none the code can establish, especially when cost depends on databases, networks, distributed systems, hardware, or data distributions: report the bound as unknown with the reason, and give the strongest bound you can support (for example, "at least one pass over the input"). A bound below the input size (O(log n), O(1)) is possible only when the work is amortized over repeated queries, the input is already structured (such as sorted or indexed), or a correct answer does not depend on every element (for example, finding any local peak). Use `ADVANCED-TECHNIQUES.md` when the core techniques do not reach the lower bound or the problem is in a specialized domain; it is an index, so load only the `techniques/` files it routes to. When counting calls, a key function runs n times when the sort computes keys once, and once per comparison when it is wrapped in a comparator; wrapping values in objects and hashing string keys (proportional to length) count too; a doubling experiment (`BENCHMARKING.md`) settles a cost no document does. Analysis never stops early: always identify and report the best achievable approach and the lower bound. Applying is evidence-driven: recommend applying a change only when measurements or a stated workload estimate show it pays off, do not add a specialized structure only for a better asymptotic bound, and stop applying once the bottleneck is resolved or the remaining cost is at the lower bound, recording what remains. -**Rule 8, measurement.** Big-O describes growth, not speed at a given size. "Provably tiny" means a bound visible in the code or data (a fixed enum, a constant-size table, a validated limit), not a guess about typical use; "not hot" means the code does not run per item, per request, per frame, or inside another loop. When a measurement disagrees with the prediction, revise the analysis before trusting either. Benchmark time and memory before and after every applied change, at realistic and larger sizes (`BENCHMARKING.md`). A 10× speedup of code that takes 2% of the operation saves under 2%; say so plainly. If verification or benchmarking cannot be completed, state the limitation and do not present the optimization as a confirmed win. +**Rule 8, measurement.** Big-O describes growth, not speed at a given size. "Provably tiny" means a bound visible in the code or data (a fixed enum, a constant-size table, a validated limit), not a guess about typical use; "not hot" means the code does not run per item, per request, per frame, or inside another loop. A 10× speedup of code that takes 2% of the operation saves under 2%; say so plainly.