Production-grade CDC pipeline: MySQL → Debezium → Kinesis → S3 → AWS Glue (PySpark) → Redshift + Postgres + OpenSearch. Multi-sink fanout with SCD2, idempotency tracking, and 13 modular Terraform modules.
mysql python aws postgres streaming etl terraform kinesis pyspark data-engineering redshift data-pipelines cdc change-data-capture debezium opensearch step-functions aws-glue scd2 multi-sink
-
Updated
Apr 23, 2026 - Python