[maximusunc]

translator-testing-model 0.7.0 (TranslatorSRI/TranslatorTestingModel#44) adds skip and skip_reason to test assets and test cases, for tests that always fail for a known reason. CI will fail at install until translator-testing-model==0.7.0 is on PyPI.

Harness

• Not run: a skipped asset isn't queried, and nothing is sent for a test case whose assets are all skipped. An asset that shares its query with a skipped one is still queried. A skipped performance test doesn't start HelmsDeep.
• Still reported: each skipped test is reported as SKIPPED, with the message Skipped in the suite: <reason>, in Slack, the CSV, Zebrunner and the Information Radiator. The radiator already leaves skips out of pass rates, and its diff files a change to or from skipped under "skipped", not as a regression.
• --include_skipped: runs the skipped tests anyway, to see whether the issue behind them is fixed. test-harness-cycle and test-harness-sweep pass it on when it's given after the --.
• README: a new "Skipped tests" section, and a note in the bundled suites' README.

Information Radiator

• New field: a performance result can carry a message, via radiator_schema and migration 0009. It's an addition, so the schema version stays 2.
• A skipped load test:
  • its run page says "Not load tested" with the reason, instead of "no stage met the SLO";
  • on the cycle board it shows the reason and raises no flags;
  • it's never the baseline the next run is compared with, so that run is compared with the last one that measured something.

Testing

• Harness: 205 passing, including tests/test_skip.py, which checks:
  • only the runnable assets are queried;
  • a fully skipped case sends nothing;
  • the reason reaches the radiator record;
  • --include_skipped runs everything;
  • a skipped performance test never starts HelmsDeep.
• Radiator: 103 passing against Postgres, including a skipped performance test on the board, its page and the next run's comparison. alembic check is clean.
• One test-file fix: tests/test_skip.py imports test_harness.main first, like the real entry point. gevent's monkey-patching has to come before anything starts a thread; otherwise the file, run on its own with output to a pipe, hung at exit.

Rollout

1. Release translator-testing-model 0.7.0.
2. Deploy the radiator (migration 0009 runs on startup), and this harness.
3. Only then set skip in NCATSTranslator/Tests data, since an older harness can't read a suite that uses it. The Tests repo also gets test_generators/sync_asset_copies.py, which copies an asset edit, eg skip: true, into the copies the suites carry, keeping every test case ID.

---

Generated by Claude Code

[maximusunc]

Run pytest is red only because pip install -r requirements.txt can't find translator-testing-model==0.7.0: it isn't on PyPI yet (the latest there is 0.6.0). No test ran. The other checks pass.

The fix is the model release, not a change here: merge TranslatorSRI/TranslatorTestingModel#44 (green on 3.10–3.12), then create the v0.7.0 release, then re-run this job.

Checked: with a 0.7.0 wheel built from #44's head (90b1b97), the way the release workflow builds it, this job's steps pass on this PR's head (93745bf) on Python 3.11. All three requirements files install, and pytest gives 205 passed.

---

Generated by Claude Code