Skip to content

Preserve column order in generated orm.yaml - #136

Merged
yhong123 merged 1 commit into
mainfrom
yhong123/135-parquet_csv_column_order
Sep 11, 2026
Merged

yhong123 merged 1 commit into
mainfrom
yhong123/135-parquet_csv_column_order

Conversation

@yhong123

@yhong123 yhong123 commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • yaml.dump in make_tables_file defaulted to sort_keys=True, alphabetising all dict keys in the generated orm.yaml — including each table's columns.
  • This ordering propagated downstream through dict_to_metadata/dict_to_table, causing exported CSV/parquet column order to not match the source parquet/database schema.
  • Pass sort_keys=False to yaml.dump so the original column order (already correctly preserved by parquet2orm.py) is retained through to the exported files.

Fixes #135

Test plan

  • Generate synthetic data from a parquet file with a known non-alphabetical column order and confirm the exported CSV/parquet columns match the source order.

@yhong123
yhong123 requested a review from tim-band September 10, 2026 11:18

@tim-band tim-band left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great

@yhong123
yhong123 merged commit f4e3843 into main Sep 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

yaml.dump alphabetises columns in generated orm.yaml, causing CSV/parquet export column order to not match the source data

2 participants