Skip to content

Documents the parallel-column exit - #121

Merged
johnnyt merged 1 commit into
mainfrom
ece-cwg6-parallel-column-exit
Sep 30, 2026
Merged

johnnyt merged 1 commit into
mainfrom
ece-cwg6-parallel-column-exit

Conversation

@johnnyt

@johnnyt johnnyt commented Sep 30, 2026

Copy link
Copy Markdown
Member

What

The migrate-from-cloak guide gains a second exit, "The other exit: a parallel column", beside the in-place walk (which keeps every word; the diff removes no line of the guide). A host adds a <field>_encrypted binary column beside each encrypted one, its schema declares both fields (the old on the legacy type, the new on this package's type with no legacy:), and its changeset writes both. A plan reads each old column through its legacy type and writes into: the new one (from: the legacy type, to: this package's type, into: the new column, scope_from, source_authenticated:). Reads stay on the old column through the backfill and the verification, one deploy cuts reads over as a step of its own, and only then do the old columns and the legacy library go. The steps are numbered 0 to 8 so a rehearsal can assert each one.

The guide also says:

  • how parallel step 0 is checked: the census a parallel plan renders reads the new column, so before the pass it shows no legacy rows; the format query run over the old column is given;
  • why the dual write must be live before the pass: the compare-and-swap compares the new column only;
  • why the shape needs no reverse plan: the pass never writes the old column, and the dual write keeps it current until the drop;
  • that a field renamed back onto the new column pins column:, because the declared context column is derived from the field name.

No code change

The shape runs on the package as it is: Encryptor.Ecto.Migrator.pass!/5 takes the target column from into:, verify/2 goes through the same pass, and Encryptor.Ecto.Migrator.Census's rewrite_queries/2 reads the into: column. So there is no change under lib/, no Note on ADR-0002, no changelog fragment (documentation and test fixtures are excluded), and the plan field options are unchanged. Ruled by the operator, 2026-09-29: a recipe, with code only where the tip showed a gap; none was shown.

Tests

test/encryptor/ecto/runbook_parallel_test.exs runs the parallel plan against the runbook's integrations table, which a new fixture migration (Encryptor.Ecto.TestMigrationRunbookParallel, added to TestRepo's migration list; no existing migration edited) gives three _encrypted columns. The tests show: the dual write writing both formats; the pass leaving every old column byte for byte and writing the new one in this package's format, a NULL source leaving its pair NULL; a dual-written row counted already_target and not rewritten; verify/2 red before the pass and {:ok, _} after it over the new columns; the census reading the new column; the new columns readable under the old field names with the column: pin, the old columns still readable through the legacy schema, and the cut-over schema reading every row after the old columns are dropped. Each test carries its sabotage note.

Provenance

  • The fixture modules (ParallelIntegration, CutOverIntegration, ParallelMigration) are added to Encryptor.Ecto.TestRunbook beside the in-place ones; the in-place walk in runbook_test.exs is untouched.
  • The dual-write test and the cut-over tests read through a helper that returns a refused decrypt as a value, so a failed read is an assertion failure rather than a crash.

Review

Own in-turn review (no lib/ change, so no cold pass): I re-read the diff against the bead's description, acceptance and notes, and checked each guide claim against the code by anchor: pass!/5 sets target_column from into: and builds the target's params for it; Keyset.swap_query/5 compares only the target column (unchanged/3); a NULL source row counts :null (Pass.row/4); Census.rewrite_queries/2 reads the into: column and its integrity/2 emits source_non_null, target_non_null and target_empty; Encryptor.Ecto.Binary's derive_column/1 derives the context column from the field name, and a field-level column: pin wins over it. The in-place steps keep every word (zero removed lines in the guide).

Gate

mix quality on the rebased head: Format, Compile (warnings as errors), Doc links, Dependencies, Credo, Docs, Dialyzer and Tests (875 of 875 passed, 95.3% coverage) all passed; Doctor, Gettext and Sobelow skipped as not installed.

Refs: ece-cwg6

The migrate-from-cloak guide gains a second exit beside the in-place
one: a <field>_encrypted column per encrypted field, written by the
host's changeset beside the old one, backfilled by a plan that reads
the old column through its legacy type and writes into: the new one,
verified over the new column, then read, then the old column and the
legacy library dropped. Steps 0 to 8 are numbered for a rehearsal to
assert; the in-place steps keep every word.

The tip already carries the shape: into: is honoured by the pass,
verify/2 and the census for a legacy encrypted source, so no lib/
code changes, no record changes and no changelog fragment. The guide
says why the shape needs no reverse plan: the pass never writes the
old column, which stays current until the drop.

A new fixture migration adds the three _encrypted columns to the
runbook table, and a new test file runs the plan against it.

Refs: ece-cwg6
@johnnyt
johnnyt merged commit 1bb3624 into main Sep 30, 2026
1 check passed
@johnnyt
johnnyt deleted the ece-cwg6-parallel-column-exit branch September 30, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant