Repository navigation
Conversation
…way the engine does The engine looks a column up through Schema, which ignores case, so a setting that names `Name` reads the column `name`. The exported script indexed the frame with the name as written and raised a KeyError. StandaloneCodeGenerator.columnAsStored resolves a name against the input schema to the table's spelling. Projection, Filter, Keyword Search, Regular Expression, Substring Search, Dictionary Matcher and Unnest String read their column through it. Projection still names its output by the alias, as the engine does. Type Casting matches a column by its exact name in the engine, so a unit whose column differs in case casts nothing there. The script now skips that unit too. Closes apache#9004 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Automated Reviewer SuggestionsBased on the
|
Contributor
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
The engine looks a column up through
Schema, which ignores case. So a setting that namesNamereads the columnname, and the workflow runs in Texera. The exported script indexed the pandas frame with the name as written and stopped with aKeyError.StandaloneCodeGeneratorgainscolumnAsStored, which resolves a setting's column name against the input schema and returns the table's spelling. A name the schema does not hold comes back unchanged, so the script still fails where the run would. Seven operators read their column through it: Projection, Filter, Keyword Search, Regular Expression, Substring Search, Dictionary Matcher and Unnest String. Projection still names each output column by its alias, which falls back to the name as the setting writes it. That matchesProjectionOpExec.Type Casting is the exception. Its schema propagation and its executor both match the column by its exact name, so in the engine a unit whose column differs in case casts nothing and the column passes through unchanged. The export now skips that unit too, instead of raising a
KeyError.Operators that run as Python in the engine already compare names exactly on both sides, so they are left alone.
Any related issues, documentation, discussions?
Closes #9004. @carloea2 found the mismatch while reviewing #8918.
#8980 rewrites Filter's export and looks up the declared type by the column's name. Whichever of the two merges second should look that type up under the resolved name as well.
How was this PR tested?
Each of the eight operators'
OpDescSpecgains a test with a setting in a different case from the table. It asserts that the generated code reads the table's spelling. For Type Casting, the test also checks that the engine's output schema leaves that column's type unchanged, and that the export skips the unit. All ofWorkflowOperator/testpasses. Merged with the other open export PRs, the verification suite still passes all 150 cases (3 ignored, as before).Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
🤖 Generated with Claude Code