refactor(exporters): ExporterConfig + registry — uniform ctor for buffered exporters#180
Merged
Conversation
…fered exporters Behaviour-neutral restructuring so cross-cutting exporter changes stop touching N files (the exportUri change had to edit 5 constructors + 5 factory calls + the base): - ExporterConfig (frozen dataclass) bundles the 5 params every buffered file exporter shares: setup_context, product_name, chunk_size, encoding, export_uri. - UnifiedBufferedExporter.__init__(exporter_type, config) reads them from the config. - Every file exporter now has the uniform ctor __init__(config, params) and pulls its OWN format-specific options (delimiter, sheet_name, root_element, use_ndjson, ...) from params - self-contained, no repeated common params. - _BUFFERED_EXPORTERS registry (name -> class) replaces the get_exporter_by_name if/elif chain; the factory builds one config and dispatches. Non-buffered exporters (Console/Log/DB/Mongo/Memstore/TestResult) keep their branches - genuinely different. Also fixes a latent bug: the old fieldnames-is-str branch was elif-chained to the exporter dispatch, so a string fieldnames returned no exporter. Now a common setting = one field on ExporterConfig; a new buffered exporter = one registry entry. Existing exporter unit tests route construction through a small make_exporter helper (old kwargs -> config). 798 passed; lint clean, mypy at baseline.
DbUnit predated the ExporterConfig refactor and was the last file exporter built via a special elif with the old positional ctor (broken against the new UnifiedBufferedExporter signature after rebase). Now it takes the uniform (config, params) ctor — forcing chunk_size=None via dataclasses.replace, a dbunit dataset is one document — and registers in _BUFFERED_EXPORTERS; the registry gate covers it. Also: 'Target not found' message lists the registry keys instead of a hardcoded name list, and the never-buildable JSONSingle constant is purged.
ake2l
force-pushed
the
refactor/exporter-config
branch
from
July 3, 2026 20:18
d310a44 to
1d4c088
Compare
…redundant _task_id The shared factory carried CSV-only knowledge (fieldnames literal_eval) plus a params copy just to hold the parsed value — CSVExporter owns fieldnames, so it parses them. CSV/JSON also re-set self._task_id that the base ctor already sets identically.
… registered exporter One descriptor writes 10 records to all buffered targets (CSV,JSON,XML,XLSX,TXT,DbUnit) via multi-target + exportUri; each format is read back and must yield ids 1..10. Self-extending: the reader map must equal _BUFFERED_EXPORTERS, so registering a new exporter without matrix coverage fails the gate. Would have caught the DbUnit ctor break instantly.
A JSON export is read back as a source across page boundaries: ordered+cyclic (exact wraparound sequence), random+cyclic (each source-length block of the global shuffle is a permutation), unique (each row exactly once), and non-cyclic exhaustion (stops at source length). All asserts are seed-independent. Note: absent distribution defaults to RANDOM, so ordered is explicit.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A cross-cutting exporter change touched too many files: adding
exportUri(#179) had to edit 5 constructors + 5 factory calls + the base, because the common params (setup_context, product_name, chunk_size, encoding, export_uri) are repeated in every exporter signature and the factory is a hand-written if/elif chain.What (behaviour-neutral)
ExporterConfig(frozen dataclass) bundles the 5 shared params.UnifiedBufferedExporter.__init__(exporter_type, config)reads them from the config.__init__(config, params)and pulls its own format-specific options (delimiter, sheet_name, root_element, use_ndjson, …) fromparams— self-contained._BUFFERED_EXPORTERSregistry (name → class) replaces the factory if/elif chain; the factory builds one config and dispatches. Non-buffered exporters (Console/Log/DB/Mongo/Memstore/TestResult) keep their branches — genuinely different (no config, lazy, client-based).fieldnames-is-str branch waselif-chained to the exporter dispatch, so a stringfieldnamesfell through and returned no exporter.Payoff (measured)
The
exportUrichange would now be one field onExporterConfig+ one line in the base instead of 11 edits. A new buffered exporter = one registry entry.Scope kept honest
Format-specific params stay per-exporter (they are different). Non-buffered exporters keep their factory branches. This is composition/config cleanup, not forcing everything into one mold.
Tests
Existing exporter unit tests route construction through a small
make_exporterhelper (old kwargs → config) — same assertions, new ctor. New registry unit test asserts every entry is aUnifiedBufferedExporterwith the uniform(config, params)signature. 798 passed, lint clean, mypy at baseline.