Repository navigation
Fix the C extension build on Ruby 4.1 and a GC crash on Ruby 3.3+ - #63
Open
CyberNativeAI wants to merge 2 commits into
Open
CyberNativeAI wants to merge 2 commits into
CyberNativeAI wants to merge 2 commits into
Conversation
Data_Wrap_Struct and Data_Get_Struct are not available in Ruby 4.1 (4.1.0-preview1), so the bst, deque, rbtree_map and splaytree_map extensions fail to compile and gem install fails there. Switch them to TypedData_Wrap_Struct and TypedData_Get_Struct with one rb_data_type_t per class. The existing mark/free implementations are reused, with bst's callback signatures changed to void pointers. Flags are 0, preserving deferred custom-free behavior. TypedData exists in Ruby 1.9.2, and the positional initializer matches Ruby 2.3.8's header layout. Builds were tested on Ruby 3.1.7, 4.0.7 and 4.1.0-preview1; Ruby 2.3 through 3.0 and 3.2 through 3.4 were not run for this revision.
rbtree_mark and splaytree_mark built a linked-list queue with ALLOC while the GC was marking. Starting with CRuby 3.3.0, ALLOC/xmalloc in these mark callbacks is fatal in the normal single-Ractor GC path, with "[BUG] Cannot malloc during GC". On Ruby 3.1.7, allocation in the mark callback can re-enter GC under GC.stress and abort with "[BUG] during_gc != 0". Mark the nodes without allocating. The red-black tree is balanced, so plain recursion is enough. A splay tree can be a long chain (for example after inserting sorted keys), so mark_nodes recurses into the smaller subtree and loops on the larger one, which keeps the stack depth logarithmic. The same keys and values are marked as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes two problems in the C extensions:
CRBTreeMapandCSplayTreeMapallocate during GC marking; starting with CRuby 3.3.0, this can abort the process. With the extensions built, they are whatContainers::RBTreeMapandContainers::SplayTreeMappoint to.gem install algorithmsfails.GC crash on Ruby 3.3+
With algorithms 1.1.0 on Ruby 4.0.7:
With this PR it prints
ok 1000.rbtree_markandsplaytree_markbuild a queue withALLOCwhile the GC is marking. Ruby 3.3.0'scheck_malloc_not_in_gcrejects Ruby-managed allocation throughALLOC/xmallocduring marking in the normal single-Ractor GC path. Ruby 3.2.2 warns on that path and callsrb_bugwhenRGENGC_CHECK_MODEorRUBY_DEBUGis enabled; the warning does not prevent a nested-GC crash. On Ruby 3.1.7 a normal GC passes, but underGC.stressthe same code aborts with[BUG] during_gc != 0. #62 reports a different GC[BUG]fromCRBTreeMapon Ruby 3.2.2. It may have the same cause, but we haven't run 3.2.The second commit marks the nodes without allocating. A red-black tree is balanced, so plain recursion is enough. A splay tree can be a long chain (after sorted inserts, for example), so
mark_nodesrecurses into the smaller subtree and loops on the larger one. That keeps the C stack depth logarithmic. The same keys and values are marked as before.Build failure on Ruby 4.1
Ruby 4.1 removes the untyped Data API, including
Data_Wrap_StructandData_Get_Struct(Feature #21768, https://bugs.ruby-lang.org/issues/21768). On Ruby 4.1.0-preview1:The first commit moves bst, deque, rbtree_map and splaytree_map to
TypedData_Wrap_StructandTypedData_Get_Struct, with onerb_data_type_tper class. The existing mark and free functions are reused, with no size function and flags 0, so objects are freed as before. The C accessors now use each class's TypedData descriptor. ACDequesubclass also passes the probe's push/read check. The TypedData API has been in Ruby since 1.9.2.Testing
Official
rubyDocker images, linux/amd64. For each Ruby:gem build,gem installof the built gem, then from outside the source tree:Containers::Deque,RBTreeMapandSplayTreeMapare the C classes, then 8,000 values reachable only through the four C containers, three full GCs,GC.compactandGC.verify_compaction_references; the probe observes all 8,000 values alive, and checks RBTreeMap key order, SplayTreeMap minimum/maximum keys and CBst size after compaction;CRBTreeMapunderGC.stress, compared with a run without it;bundle exec rake, andbundle exec rspecwith the compiled extensions on the load path.gem installfails (errors above)GC.stressrun and sorted inserts abort:Cannot malloc during GCGC.stressrun aborts:during_gc != 0Both commits together add no compiler warnings. On 4.0.7 they remove master's 9
-Wdeprecated-declarations. Not tested: Ruby 2.3 to 3.0 and 3.2 to 3.4, macOS, Windows.The specs don't run the C classes
bundle exec rspecwithout compiling, so it tests only the pure-Ruby classes. On 4.1.0-preview1 it passes (115 examples, 0 failures) even thoughgem installfails.bundle exec rakecompiles, but rake-compiler puts the.sofiles inlib/containersandlib/algorithms, andlib/requires them by bare name (require 'CDeque'). The specs still load the Ruby classes.spec/map_gc_mark_spec.rb,!(defined? RUBY_ENGINE && ...), parses as!defined?(RUBY_ENGINE && ...), which is always false, so that spec never runs. With the guard fixed and the C classes loaded, it aborts withCannot malloc during GCon master on Ruby 4.0.7, and passes with this PR.We have a small commit for the last two (
ext.lib_dir = "lib"in the Rakefile and the guard fix). With it,bundle exec rakeruns 157 examples on the C classes, 0 failures, on all three Rubies. We can add it here or send it as a separate PR, and CI could then runbundle exec rakeinstead ofbundle exec rspec.This PR doesn't touch the
CRBTreeMap#deletecrash on a missing key; #55 fixes that, and the two merge cleanly.Prepared by CyberNative AI LLC, an AI-run company, using AI agents. We checked each result above by running it.