Add pkcs11-provider process exit regression test

[bukka]

Follow-up to #897, which fixed the crash from #729 and #780 where OpenSSL's atexit cleanup (via pkcs11-provider) called C_CloseSession on an already destroyed SoftHSM instance. That PR did not come with a test, so this adds one so we don't regress.

The test is based on the script from #864 by @petrovr (added as co-author), reworked to fit the test infrastructure that has landed in src/bin/util/test since then:

• The module / softhsm2-util / openssl discovery and token directory setup are moved out of import-key-test-common.sh into a new test-common.sh, so the import tests and this one share the same code (including the Windows handling). The import tests behave exactly as before.
• Same style as the sibling scripts: bash, BSD-2-Clause header, fail() reporting that dumps the token log.
• Not gated on the crypto backend. The crash is backend independent, so the test runs for Botan builds too and simply skips (77) when OpenSSL has no provider support or pkcs11.so is not in MODULESDIR. PKCS11_PROVIDER_MODULE can override the lookup.
• Registered in both automake and CMake (with SKIP_RETURN_CODE 77). The OpenSSL 3.0 CI job installs the pkcs11-provider package so the test actually runs there; the other jobs don't have a provider and skip.
• Also adds .gitattributes forcing LF endings on *.sh (needed for bash on Windows) and ignores the tokens-* directories the tests create.

Checked locally with OpenSSL 3.5 and pkcs11-provider 3.5.7 against -O2 builds: the test fails with exit status 139 on main before #897 and passes on current main. ML-DSA / ML-KEM import tests still pass after the refactor.

Summary by CodeRabbit

• Tests
  • Added regression coverage for OpenSSL operations through the PKCS#11 provider, including RSA key import, signing, and signature verification.
  • Expanded automated test setup to handle supported environments and skip tests when required capabilities are unavailable.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 2cfd2a21-072b-4897-b14f-293c1ae75c99

📥 Commits

Reviewing files that changed from the base of the PR and between 9ad8734 and a931c45.

📒 Files selected for processing (8)

• .gitattributes
• .github/workflows/ci.yml
• src/bin/util/test/.gitignore
• src/bin/util/test/CMakeLists.txt
• src/bin/util/test/Makefile.am
• src/bin/util/test/import-key-test-common.sh
• src/bin/util/test/p11prov-test.sh
• src/bin/util/test/test-common.sh

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds shared setup and failure helpers for SoftHSM shell tests, adds a test for signing through the OpenSSL PKCS#11 provider, and registers that test in the build and CI configuration.

Changes

PKCS#11 provider test coverage

Layer / File(s)	Summary
Shared SoftHSM test setup and existing test integration src/bin/util/test/test-common.sh, src/bin/util/test/import-key-test-common.sh	The shared script locates the SoftHSM module and OpenSSL, configures and cleans up token directories, and reports failures. The key-import test now uses these helpers and skips Botan-module tests.
OpenSSL PKCS#11 provider signing test src/bin/util/test/p11prov-test.sh	The test checks OpenSSL and provider availability, configures a SoftHSM token and OpenSSL providers, then signs data with an imported RSA key and verifies the signature with the software key.
Test registration and build support src/bin/util/test/CMakeLists.txt, src/bin/util/test/Makefile.am, .github/workflows/ci.yml, .gitattributes, src/bin/util/test/.gitignore	Registers and distributes the new test, marks exit code 77 as a skip in CTest, adds pkcs11-provider to the OpenSSL 3.0 CI job, sets shell scripts to LF line endings, and ignores root-level tokens-* entries.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant Test as p11prov-test.sh
  participant OpenSSL
  participant SoftHSM as SoftHSM token
  Test->>SoftHSM: Initialize token and import RSA key
  Test->>OpenSSL: Sign data using PKCS#11 provider
  OpenSSL->>SoftHSM: Request RSA signature
  SoftHSM-->>OpenSSL: Return signature
  Test->>OpenSSL: Verify signature using software key

Loading

Suggested reviewers: jschlyter

Merge Risk: ⚪ Minimal · up to a931c

The new test detects the targeted process crash, and the shared setup preserves the existing configuration-path behavior. No actionable merge-blocking issue remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files. (5 skipped: 5…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a regression test for process exit behavior with pkcs11-provider.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files. (5 skipped: 5 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}