Skip to content

fix(mem): Restore mlock allocator defaults on init and cleanup#5848

Open
alexw91 wants to merge 2 commits into
aws:mainfrom
alexw91:fix-mlock-after-cleanup
Open

fix(mem): Restore mlock allocator defaults on init and cleanup#5848
alexw91 wants to merge 2 commits into
aws:mainfrom
alexw91:fix-mlock-after-cleanup

Conversation

@alexw91

@alexw91 alexw91 commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

Goal

Restore the mlock/MADV_DONTDUMP allocator defaults on both init and cleanup, so that a s2n_cleanup_final/s2n_init cycle does not permanently drop memory hardening.

Why

The original s2n_mem_cleanup (2016, 7486f4785) reset use_mlock back to its default enabled state, so a subsequent s2n_init would start with the hardened allocator. The 2020 callback refactor (0de8c411e) replaced the use_mlock flag with function pointer callbacks but changed cleanup_impl to unconditionally select the no_mlock variants, while init_impl only conditionally overwrote the callbacks when S2N_DONT_MLOCK was set or in unit tests. After a s2n_cleanup_final/s2n_init cycle in a normal process, the no_mlock callbacks set by cleanup persisted through init, silently dropping mlock and MADV_DONTDUMP hardening for all subsequent key and secret allocations.

How

Extract the callback selection logic into a single s2n_mem_set_default_callbacks helper that evaluates S2N_DONT_MLOCK and s2n_in_unit_test(), then selects the appropriate mlock or no-mlock callbacks. Both s2n_mem_init_impl and s2n_mem_cleanup_impl now call this helper, restoring the original reset-to-defaults behavior and eliminating the asymmetry introduced by the refactor.

Callouts

  • The 2020 refactor's cleanup_impl unconditionally selecting no_mlock does not appear to have been an intentional design choice. The original 2016 commit message explicitly states "This would be the least surprising behavior for the caller" about resetting to defaults. This PR restores that intent.
  • The file-scope initializers still set the callbacks to mlock_impl, which is correct for the very first s2n_init call (before any cleanup has run).
  • s2n_mem_set_default_callbacks is static void since it cannot fail.

Testing

Added a regression test in s2n_init_test.c that:

  1. Saves the current memory callbacks.
  2. Overrides malloc/free with distinctive sentinel callbacks.
  3. Sets s2n_in_unit_test(false) and unsets S2N_DONT_MLOCK to simulate a production environment.
  4. Cycles s2n_mem_cleanup/s2n_mem_init directly (avoids triggering real mlock-backed allocations that could fail under restrictive RLIMIT_MEMLOCK in CI).
  5. Verifies the callbacks are no longer the sentinels, proving that init restored the defaults.
  6. Restores unit-test mode and original callbacks.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

The original s2n_mem_cleanup (2016, 7486f47) reset use_mlock to
its default enabled state so that a subsequent s2n_init would start
with the hardened allocator. The 2020 callback refactor (0de8c41)
replaced the use_mlock flag with function pointer callbacks but
changed cleanup to unconditionally select no_mlock, while init only
conditionally overwrote the callbacks when S2N_DONT_MLOCK was set
or in unit tests. This meant a s2n_cleanup_final/s2n_init cycle in
a normal process permanently dropped mlock and MADV_DONTDUMP
hardening for all subsequent key and secret allocations.

Extract the callback selection logic into s2n_mem_set_default_callbacks
and call it from both init and cleanup, restoring the original
reset-to-defaults behavior. Add a regression test that installs
sentinel callbacks, simulates a non-unit-test reinit cycle, and
verifies the callbacks are reset.
@alexw91 alexw91 force-pushed the fix-mlock-after-cleanup branch from 4bc193f to 6e585fc Compare April 24, 2026 14:35
@alexw91 alexw91 requested review from jmayclin and maddeleine April 24, 2026 14:39
@alexw91 alexw91 enabled auto-merge April 29, 2026 02:16
EXPECT_OK(s2n_mem_get_callbacks(&ignored_init_cb, &ignored_cleanup_cb,
&observed_malloc_cb, &observed_free_cb));
EXPECT_EQUAL(observed_malloc_cb, s2n_mem_sentinel_malloc_cb);
EXPECT_EQUAL(observed_free_cb, s2n_mem_sentinel_free_cb);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is intentional, but you're not checking cleanup_cb and init_cb here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants