Skip to content

refactor(engine): replace magic numbers with named constants - #802

Merged
Christian Walter (walterchris) merged 1 commit into
mainfrom
engine/magic-numbers
Oct 26, 2026
Merged

Christian Walter (walterchris) merged 1 commit into
mainfrom
engine/magic-numbers

Conversation

@walterchris

@walterchris Christian Walter (walterchris) commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #731. Removes the magic numbers from the engine's production code. No behaviour change except one unreachable path (below).

Changes

  • Return codes.
    • OpenSSL 1/0 results go through ossl_check / ossl_ok / RetCode (core error.rs). ossl_check is now pub and converts its error only on failure, so a &'static str message costs nothing on success.
    • Ctrl results use ctrl_ok: any positive value is success (GET_RSA_OAEP_LABEL, for one, returns a length).
    • Panic fallbacks return RetCode::Fail; void cleanups use ().
  • Named conventions: CTRL_UNSUPPORTED (-2), CTRL_NOT_IMPLEMENTED (-1), ANY_KEYTYPE / ANY_OPTYPE (-1).
  • OpenSSL's own values:
    • RSA_PSS_SALTLEN_*, now allowlisted; the hand-copied ones are gone.
    • EVP_PKEY_HKDEF_MODE_* and NID_undef.
  • Named domain values:
    • DEFAULT_RSA_KEYGEN_BITS (OpenSSL's default) and RSA_UNWRAPPING_KEY_BITS (the HSM's fixed key): both 2048, different meanings.
    • PSS_ENCODING_OVERHEAD, CREDENTIALS_SIZE / _HEX_SIZE, AES_KEY_BITS, PKCS8_VERSION_V1, FILE_OUTPUT_SIZE_HINT.
    • The HMAC key size is now derived from the digest size.
  • One definition each:
    • SECRET_FILE_MODE, MAX_STORAGE_FILE_SIZE and OBK_LEN live in the resiliency crate; the engine re-exports them.
    • new_ex_index() replaces three copies of the ex_data index registration, and openssl_free() replaces four copies of OPENSSL_free.
  • Messages that state a value now format it from the constant. Where a test matches the text, it renders the same as before.
  • Tests use the constants they restate, including the 8192 kdflen in the Rust tests, the C-API tests and hkdf.sh (from the HKDF over masked secrets through the engine #694 review).

Behaviour change

When OpenSSL's built-in method is missing, the RSA pkey ctrl returned 0. It now returns CTRL_UNSUPPORTED (-2), like the EC and HKDF methods. This path is not reachable in practice.

Testing

All run in the u2004 container (OpenSSL 1.1.1f):

  • cargo fmt --check
  • cargo clippy --all-targets -- -D warnings for engine, engine+mock, the integration-test crates and resiliency
  • Unit tests: core 54, engine 55, resiliency 31
  • CLI 9/9
  • C-API 11/11, each test in its own process

Copilot AI balanced review requested due to automatic review settings October 21, 2026 06:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no blocking issues in the refactoring or documented fallback change; tests were not rerun.

No open issues 🎉

What changed in this PR

Replaces magic numbers in the AZIHSM OpenSSL engine with named constants and shared helpers, addressing #731.

Changes:

  • Centralizes OpenSSL return-code handling, memory cleanup, and index registration.
  • Names domain values and shares constants across engine, resiliency, and tests.
  • Changes the documented missing-method RSA control fallback from 0 to CTRL_UNSUPPORTED.
File Description
plugins/​ossl_engine/​integration-tests/​openssl-cli/​testfiles/​hkdf/​hkdf.sh Uses the named buffer size.
plugins/​ossl_engine/​integration-tests/​openssl-cli/​testfiles/​env.sh Defines the CLI buffer-size constant.
plugins/​ossl_engine/​integration-tests/​openssl-capi/​src/​engine_capi_tests.rs Names derive-size expectations.
plugins/​ossl_engine/​integration-tests/​masked-keygen/​src/​main.rs Reuses secret-file permissions.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​sign.rs Uses shared return-code checking.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​rsaload.rs Reuses return-code and index helpers.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​rsaimport.rs Uses the unwrapping-key size constant.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​logging.rs Formats permissions from their constant.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​lib.rs Reuses resiliency file permissions.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​keyload.rs Shares file limits and OpenSSL helpers.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​keygen.rs Uses shared return-code checking.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​integration.rs Exposes secret-file permissions.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​hkdf.rs Names AES sizes; derives HMAC size from digest.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​engine_impl.rs Names the incompatible-version result.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​derive.rs Uses shared OpenSSL cleanup.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​context/​tests.rs Reuses constants in tests.
plugins/​ossl_engine/​azihsm_ossl_engine/​src/​context.rs Names credential and unwrapping-key sizes.
plugins/​ossl_engine/​azihsm_ossl_engine_sys/​build.rs Allows OpenSSL PSS salt-length constants.
plugins/​ossl_engine/​azihsm_ossl_engine_resiliency/​src/​lib.rs Exposes shared storage constants.
plugins/​ossl_engine/​azihsm_ossl_engine_resiliency/​src/​config.rs Reuses permission constants in tests.
plugins/​ossl_engine/​azihsm_ossl_engine_resiliency/​src/​callbacks.rs Exposes the OBK length constant.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​rsa_pkey_method.rs Names RSA conventions and updates control fallback.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​rsa_method.rs Uses shared return-code handling.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​pkey_method.rs Reuses result and derive-size constants.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​method_table.rs Exposes derive sizes and names dispatch results.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​mem.rs Adds shared OpenSSL cleanup.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​lib.rs Exports memory helpers and derive sizes.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​hkdf_method.rs Uses named modes, sizes, and result helpers.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​exdata.rs Centralizes index registration.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​error.rs Adds result helpers and deferred error conversion.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​engine.rs Avoids eager error-message allocation.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​cipher_method.rs Shares control constants and result helpers.
plugins/​ossl_engine/​azihsm_ossl_engine_core/​src/​asn1_method.rs Names encoding values and reuses helpers.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The refactor preserves existing contracts apart from the documented RSA fallback change, with no blocking issues identified.

No open issues 🎉

Review effort: Balanced

Comment thread plugins/ossl_engine/azihsm_ossl_engine_core/src/exdata.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The substitutions preserve existing checks and ownership handling, and no blocking issues remain in the shared helpers or documented RSA fallback change.

No open issues 🎉

Review effort: Balanced

Check OpenSSL 1/0 results with ossl_check/ossl_ok and RetCode, ctrl
results with ctrl_ok (any positive value is success), and name the ctrl
conventions and wildcards. Use OpenSSL's own values where they exist
(RSA_PSS_SALTLEN_*, EVP_PKEY_HKDEF_MODE_*, NID_undef) and name the rest:
RSA key sizes, PSS overhead, credential length, AES key sizes and the
derive size-query answers. Define shared values once: the secret file
mode, file-size cap and OBK length in the resiliency crate, ex_data index
registration and OPENSSL_free in core. Tests use the constants they
restate.

The RSA pkey ctrl now returns CTRL_UNSUPPORTED, like the EC and HKDF
methods, when the built-in method is missing.

Signed-off-by: Christian Walter <christian.walter@9elements.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The refactor preserves existing values and cleanup paths apart from the documented fallback change, with no blocking issues identified in static review.

No open issues 🎉

Review effort: Balanced

Merged via the queue into main with commit d689a47 Oct 26, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove magic numbers from the engine

3 participants