Skip to content

Fix SIGABRT crash in test suite by catching unhandled std::runtime_error in threads - #28

Merged
dstebila merged 2 commits into
open-quantum-safe:mainfrom
AKG451:fix-test-concurrency-crash
Oct 31, 2026
Merged

dstebila merged 2 commits into
open-quantum-safe:mainfrom
AKG451:fix-test-concurrency-crash

Conversation

@AKG451

@AKG451 AKG451 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

When building liboqs-cpp against liboqs:main (0.16.0), the test suite completely crashes with a SIGABRT (core dumped) during the oqs_Signature tests, halting the entire test runner.

Root Cause:
Certain algorithms (specifically the new ML-DSA-*-extmu variants) are failing their signing operations and throwing a std::runtime_error("Can not sign message"). Because test_sig.cpp runs these tests concurrently inside std::thread, this unhandled exception immediately triggers std::terminate(), blowing up the GoogleTest process and preventing developers from seeing the rest of the test results.

The Fix:
I have wrapped the core cryptographic operations inside test_sig_correctness, test_sig_correctness_with_ctx_str, test_sig_wrong_signature, and test_sig_wrong_public_key in try/catch blocks.

If an algorithm throws an exception during testing, it is now gracefully caught and piped to FAIL(). The test suite now safely completes its run and cleanly reports exactly which algorithms failed without crashing the OS process.

Signed-off-by: AKG451 <akg451154@gmail.com>
@AKG451
AKG451 force-pushed the fix-test-concurrency-crash branch from 37cf972 to 6c4c11e Compare October 27, 2026 06:34
@AKG451

AKG451 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Just to add some context on why the exceptions were being thrown:
I dug into the liboqs C core and saw that OQS_SIG_ml_dsa_87_extmu_sign strictly checks if message_len == OQS_SIG_ml_dsa_87_extmu_length_mu (64 bytes).

The standard test message in test_sig.cpp is "This is our favourite message to sign"_bytes (37 bytes). The C library instantly rejects it, throws OQS_ERROR, the C++ wrapper throws a runtime error, and the thread terminates.

I can either leave this PR as purely a stability fix (catching the exception), or I can push one more commit to bump the test message length to 64 bytes so the extmu algorithms actually pass. Let me know what you prefer.

The ML-DSA-*-extmu variants sign an externally computed mu, which must be
exactly 64 bytes, so signing the 37-byte test message failed and threw.
Use a random 64-byte message for those variants, as liboqs's own
test_sig does.

Replace the per-helper try/catch blocks with a single guard around the
functions run in std::threads, so that any exception is reported as a
test failure instead of calling std::terminate() and aborting the run.

With the abort gone, SNOVA overflowed the 512KB stack of a secondary
thread on macOS, so add it to no_thread_sig_patterns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Douglas Stebila <dstebila@uwaterloo.ca>
@dstebila

Copy link
Copy Markdown
Member

Thanks for tracking this down, and for finding the extmu message-length check. I've pushed a commit that extends your patch a bit:

  • Fixes the extmu failures. ML-DSA-*-extmu signs an externally computed mu, which must be exactly 64 bytes. The tests now use a random 64-byte message for any algorithm whose name contains -extmu, as liboqs's own tests/test_sig.c does. The extmu variants don't support context strings, so CorrectnessWithContextString already skips them.
  • Replaces the four per-helper try/catch blocks with one guard. test_sig_in_thread wraps the function each std::thread runs, catches std::exception and ..., and reports ADD_FAILURE(). An unexpected exception in a worker thread now shows up as a test failure instead of aborting the run, and future helpers are covered without their own try/catch.
  • Adds SNOVA to no_thread_sig_patterns. With the abort gone, my macOS run crashed with SIGBUS next: SNOVA_III_K_neon_sk_expand overflows the 512 KB stack that macOS gives secondary threads. SNOVA now runs on the main thread, like Falcon and MAYO.

I also ran clang-format and restored the trailing newline. All 8 unit tests pass locally against liboqs main on macOS arm64; CI will cover Linux and Windows.

@AKG451

AKG451 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Thank You @dstebila for the review and commit.

@dstebila
dstebila merged commit 9045e0c into open-quantum-safe:main Oct 31, 2026
5 checks passed
@AKG451
AKG451 deleted the fix-test-concurrency-crash branch October 31, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants