debezium/dbz#2059 Freeze _debezium_cdc_log contract and add SqliteTestHelper with trigger generator#3
Draft
siddhantcvdi wants to merge 5 commits into
Draft
Conversation
siddhantcvdi
force-pushed
the
dbz-2059-cdc-log
branch
2 times, most recently
from
aae2f98 to
9a22d7f
Compare
siddhantcvdi
force-pushed
the
dbz-2059-cdc-log
branch
2 times, most recently
from
a8b16c6 to
20240c9
Compare
Define the _debezium_cdc_log table in one place: the table name, the six column names, the c/u/d operation codes, and the CREATE TABLE DDL, all as named constants in CdcLog. The DDL is built from the column constants so it cannot drift from the names later code reads. Point SYSTEM_TABLES_FILTER at CdcLog.TABLE_NAME so the name lives in exactly one spot. CdcLogTest runs the DDL against an in-memory SQLite database through the framework's JdbcConnection and asserts, via PRAGMA table_info and sqlite_master, that the columns, types, NOT NULL flags, and the autoincrement primary key match the contract. Signed-off-by: siddhantcvdi <dev.siddhantcvdi@gmail.com>
…ion tests Add SqliteTestHelper, a closeable test double built on the framework's JdbcConnection. Creating one makes a temporary .db file, switches it to WAL journal mode and verifies SQLite reported wal back, then runs the frozen CdcLog DDL so _debezium_cdc_log exists. Closing it closes the connection and deletes the temp file and its -wal/-shm side files. Bind the Failsafe plugin to the integration-test and verify phases. The parent only manages Failsafe in pluginManagement, so without this binding no *IT.java would run. SqliteTestHelperIT is the first integration test and self-tests the helper: a WAL-mode database with an empty _debezium_cdc_log, and temp-file cleanup on close. Signed-off-by: siddhantcvdi <dev.siddhantcvdi@gmail.com>
Add TriggerGenerator, a test-side SQL builder that stands in for a real change-event
writer. Given a source table and its columns, it builds the AFTER INSERT, AFTER UPDATE,
and AFTER DELETE triggers that write one correctly shaped row into _debezium_cdc_log for
each change, capturing row data with SQLite's json_object() over the NEW and OLD aliases.
Insert records the new row under code c, update records both rows under code u, and
delete records the old row under code d. The committed_at value uses unixepoch('now',
'subsec') * 1000 for epoch milliseconds.
TriggerGeneratorTest installs the generated triggers in an in-memory database through
JdbcConnection, so SQLite confirms they parse, and asserts the operation codes and the
json_object column references. Wiring the generator into the helper and proving the
end-to-end c/u/d loop is the next commit.
Signed-off-by: siddhantcvdi <dev.siddhantcvdi@gmail.com>
…d Testing.Files Add src/test/resources/logback-test.xml so test logging uses the same console layout and levels as the other connectors, instead of logback's noisy default. Switch SqliteTestHelper from java.nio Files.createTempFile to io.debezium.util.Testing.Files, which places the temporary database under target/data where it is cleaned by mvn clean and easy to find when debugging. This adds the debezium-util test-jar dependency that Testing lives in, the same dependency the other connectors declare. Signed-off-by: siddhantcvdi <dev.siddhantcvdi@gmail.com>
…e the CDC loop Add SqliteTestHelper.installTriggers(table), which reads the table's columns from PRAGMA table_info, builds the insert, update, and delete triggers with TriggerGenerator, and runs them. CdcCaptureIT drives a plain SQL insert, update, and delete on a source table and asserts the matching c, u, and d rows land in _debezium_cdc_log with the right table name and JSON row data. This is the Phase 0 exit gate: a real write produces a correctly shaped CDC row, so the development and test loop is proven end to end. Signed-off-by: siddhantcvdi <dev.siddhantcvdi@gmail.com>
siddhantcvdi
force-pushed
the
dbz-2059-cdc-log
branch
from
1f67806 to
aa30e77
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes debezium/dbz#2059
This PR freezes the
_debezium_cdc_logcontract, the single table the connector reads change events from, and builds the test harness that fills it. Every later phase depends on this table's shape and on being able to run against a real SQLite file, so both land here before any read logic is built on top.The main changes:
CdcLog: the table name, column names, operation codes, andCREATE TABLEDDL, with the DDL built from the column constants so it cannot drift from names later code reads.SqliteTestHelper, a closeable WAL-mode temporary database built on the framework'JdbcConnection. This also binds the Failsafe plugin, which the bootstrap POM never did,*ITtests run for the first time.json_object(). It stands in for the change-event writer so Java side is not blocked on the native extension.logback-test.xmland a switch toTesting.Files, matching the connectors.installTriggers(table)plus an integration test that runs a plain insert, update, and and asserts the matchingc,u, anddrows land in the log. This is the Phase 0 exit gate.A sample _debezium_cdc_log table created on top of a
customerstable: