Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Port Sem's Spark Second String into an independent native Sail extension wheel under
examples/extensions/second-string. It uses the existingpysail.extensionsbootstrap and DataFusion scalar Arrow FFI. All sixteen original SQL names/defaults remain, and ten configured entry points preserve the Scala DSL options. No Sail host implementation changes are included.Reference: SemyonSinchenko/spark-second-string at
a35db39fa8e9b65db2d201a45b86d11a6ca34b98. Apache-2.0 attribution includes the translated Commons Codec 1.21.0 primary Double Metaphone rules.Verification
Published source
f31c33ae01e574b146853ace4a901a3876cf5417passes the detached package gate: Rust formatting, Clippy with warnings denied, 41 release Rust tests, Ruff, strict mypy, and 112 checks in each of local and two-process-worker Sail modes. The compiled Scala oracle contains 7,296 answers; every answer passes through foreign Arrow FFI and through shuffled Sail batches in both modes. Numeric tolerance is absolute 1e-12; phonetic strings are exact.Runtime qualification uses the existing native release Sail host at
9f0aa7d2a50f1258544d37c05dd19b3ff3d915b3, API 1 / DataFusion 55.1.0 / Arrow 59.3.0. Host and extension identities are recorded separately.Manual check
Build/install the release wheel as described in the package README, then start a compatible host with
SAIL_EXPERIMENTAL_EXTENSIONS=1. Check:Expected: 0.9611111111111111, 1/3, and R163. The README also gives both native integration-test commands.
Qualification and Sem research status includes retained failed attempts and scope limits. No Spark speed comparison, Linux binary qualification, or stable ABI promise is claimed.