Docs update#72
Open
trbKnl wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by Human (Niek)
Summary
This PR lands several interconnected changes that have been developed across multiple branches, plus updated documentation to match.
Code changes
Tightened archive types (
SeekableBinaryReaderrefactor):ZipArchiveReader,validate_zip, and all platform extractors now receivea
SeekableBinaryReaderobject directly from the browser upload. Zip files are never materialised to a path or Pyodide's heap (AD0007).This affected every platform module and the upload pipeline.
Docstring-driven config generation: Each extractor function now carries a
Table config::JSON block in its docstring.scripts/generate_port_config.pyparses these via AST (no Python import needed) and writespackages/python/port/configs/<platform>_config.json. The generator refuses to overwrite an existing config file (AD0012, AD0014).Per-platform config files and
VITE_PLATFORMrequired in dev:release.shnow discovers platforms by globbingconfigs/*_config.json— no hardcoded platform list.VITE_PLATFORMis required in dev mode; starting without it shows an error with ahint to run
pnpm generate-config <platform>(AD0005 amendment, AD0013).Standard platform module interface: Every platform module exposes
EXTRACTOR_REGISTRY,extraction(), aFlowBuildersubclass, andprocess().script.pyis now fully platform-agnostic (AD0013).example.py: Added a minimal but fully working example platform as the canonical starting template for new platforms.Extractor integration test framework:
ExtractorSpecdataclass +conftest.pyfor running extractors against real DDP zips storedlocally. Tests skip cleanly when DDP data is absent from
tests/ddp/(AD0004).Removed
poetry.lock: The donation task runs inside Pyodide; package versions are dictated by Pyodide, not by the local lock file.Documentation
04-flowbuilder.mdand05-extraction.mdto reflect the current architecture (stream-based archive, config-driven extraction,single-platform builds).
pnpm generate-configand the new multi-platform release flow.integration tests).
VITE_PLATFORMrequirement in dev.Review Checklist
Architecture / correctness
validate_zip — no path strings sneak through
UI)
commands during extraction (WhatsApp, Netflix)
Config generation
overwrite
Platform modules
defined and all entries resolve to callable functions
config file name
Tests
Documentation
UI metadata for extractor functions" — these should match
environment — check docs.yml workflow)
CI / workflow
dependencies are pinned somewhere)