diff --git a/CLAUDE.md b/CLAUDE.md index 8caffb4d8..0c7978aa5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -204,10 +204,11 @@ pytest -m "level_1 and tier_a" # quick pytest -m "tier_a or tier_b" # full validation ``` -Tier A is hardened and reviewed — safe to build code around. Tier C is not mature -enough to drive coding. Every bug fix ships the regression test that would have -caught it, written first and shown to fail. Validate a new test's own correctness -before changing library code to satisfy it. +Tier A is hardened and reviewed — safe to build code around. Every bug fix ships +the regression test that would have caught it, written first and shown to fail; +such a proven test starts at tier B, so it gates. Tier C is only for a test not yet +proven, so it cannot derail the code. Validate a new test's own correctness before +changing library code to satisfy it. Every test file must be reachable from a glob in `scripts/test.sh`; `scripts/check_test_coverage.py` enforces it. diff --git a/docs/developer/TESTING-RELIABILITY-SYSTEM.md b/docs/developer/TESTING-RELIABILITY-SYSTEM.md index 4e8d072c8..a6d6f5ee8 100644 --- a/docs/developer/TESTING-RELIABILITY-SYSTEM.md +++ b/docs/developer/TESTING-RELIABILITY-SYSTEM.md @@ -291,11 +291,15 @@ pytest -m "not tier_c" ### When Writing a New Test -1. **Start at Tier C**: All new tests begin as experimental +1. **A proven test starts at Tier B**: a regression test written first and shown to + fail without the fix it guards, or a test asserting a hard baseline whose own + correctness has been checked, gates from the start. Tier C is for a test that is + not yet proven, so that it cannot derail the code it was written against (the C2 + population above); it is otherwise close to obsolete (Louis, 2026-10-11). 2. **Document Intent**: Clear docstring explaining what behavior is tested 3. **Use xfail Appropriately**: If testing unimplemented feature, mark with xfail 4. **Don't Break CI**: Tier C tests with xfail won't break automated testing -5. **Promote Deliberately**: Don't rush promotion - let tests prove reliability +5. **Promote Deliberately**: Tier B → A is the review step described above ### When a Test Fails diff --git a/tests/test_0025_jit_compile_flags.py b/tests/test_0025_jit_compile_flags.py index 5fe304997..006144840 100644 --- a/tests/test_0025_jit_compile_flags.py +++ b/tests/test_0025_jit_compile_flags.py @@ -14,7 +14,7 @@ import underworld3 as uw -pytestmark = [pytest.mark.level_1, pytest.mark.tier_c] +pytestmark = [pytest.mark.level_1, pytest.mark.tier_b] def _compile_args(bundles):