You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Export the JSON Lines and Arrow sources as Python #8702
The JSON Lines and Arrow sources implement StandaloneCodeGenerator. Arrow is read into pandas' nullable dtypes, since the file states of every value whether it is there. JSONL states nothing, so each column is typed the way the engine types it, and a timestamp is read the way the engine reads one.
Matching Arrow also changes the engine. A TIMESTAMP reaches a Python worker as a zoneless microsecond Arrow timestamp instead of a millisecond one labelled UTC. That is what the Python side already declares, and what a script holds.
Step 12b in #8325. It needs #8341 for the shared source code and #8597 for the timestamp helpers.
Activity