Skip to content

About

Payments backend: transactional outbox, reserve-then-settle transfers, event-sourced ledger design. Java 21, Spring Boot, PostgreSQL/jOOQ, Kafka, Testcontainers.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

13 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Distributed Ledger Platform

A payments backend built around one question: what happens to the money when a service dies halfway through a transfer?

Moving funds between two accounts owned by two different services cannot use a single database transaction. The operation still has to be atomic in effect — no sequence of crashes, restarts, or redelivered messages may create money, destroy money, or leave two services permanently disagreeing about a balance. This project implements the patterns production payment systems use to make that true: a transactional outbox, a choreographed saga with compensation, an event-sourced ledger of record, and an independent reconciliation job.

Status: the reserve step of the transfer flow is implemented and tested end to end. The ledger service and reconciliation are designed but not yet built. This README distinguishes throughout between what is implemented and tested and what is designed but not yet built. Nothing below is claimed as working unless it is. The full design is in docs/design.md; the task-by-task build plan is in docs/implementation-plan.md.


What works today

mvn clean verify

28 tests, all passing — unit tests plus integration tests running against a real PostgreSQL container via Testcontainers.

Component State What it does
common — Money ✅ implemented, 6 tests Integer-only monetary type: long minor units + ISO-4217 currency. Overflow throws instead of wrapping; mixed-currency arithmetic is rejected.
contracts — TransferEvent ✅ implemented, 3 tests Sealed event hierarchy (TransferReserved, TransferPosted, TransferRejected) with verified JSON round-tripping.
accounts-service — schema ✅ implemented, 1 test Flyway migrations, jOOQ type-safe access generated from the DDL, Testcontainers harness proving migrations and queries reach the same database.
accounts-service — Account ✅ implemented, 12 tests Reserve-then-settle balance rules, with compensation proven as an algebraic property.
accounts-service — POST /transfers ✅ implemented, 6 tests The transactional outbox: balance change, transfer row, and outbox event commit together. Idempotency-Key honoured, including under a concurrent retry.
Outbox poller → Kafka ⬜ designed, not built Rows are written; nothing drains them yet.
ledger-service — event-sourced ledger ⬜ designed, not built
Reconciliation job ⬜ designed, not built
CQRS read model, Kubernetes, load/chaos testing ⬜ designed, not built

Why a reservation instead of a debit

A transfer does not subtract from the sender and add to the receiver. It places a hold: funds move from available to reserved, and only later either settle (the money leaves) or release (the hold returns to the owner).

The reason is what compensation becomes in each design. Debit-then-reverse makes the failure path a second mutation that can itself fail or race. Reserve-then-settle makes it the exact inverse of an operation that already happened. That is not an assertion in a comment — it is asserted as a property:

@Test
void releasingIsTheExactInverseOfReserving() {
    var before = accountWith(1000, 0);
    var after = before.reserve(Money.of(400, "EUR"))
                      .releaseReservation(Money.of(400, "EUR"));
    assertThat(after).isEqualTo(before);
}

Card authorization works the same way, for the same reason.

Both inverse paths are guarded: settling or releasing more than is actually held throws rather than silently driving a balance negative. An early review caught that reserve was guarded while its two inverse paths were not — an asymmetry that would have surfaced much later as money quietly appearing.

The invariants

Two, and the distinction between them is the design's core.

INV-0 — internal to the ledger. Every posting group balances: the sum of debits equals the sum of credits. Double-entry, cheap to verify continuously.

INV-1 — across services. For each account, at rest, accounts.available + accounts.reserved == ledger.balance. While transfers are in flight the two sides legitimately differ, and the difference must be fully attributable to specific non-terminal transfers.

What reconciliation is allowed to do

(Designed; the job itself is not yet built. Included here because it is the reasoning the rest of the design exists to support.)

The reconciliation job computes per-account drift and tries to attribute each unit of it to a transfer:

  • Attributable, within the window — a transfer legitimately in flight. Healthy. Leave it alone.
  • Attributable, past the window — a lost event or a service that died between steps. Repair by re-driving the missing event from its source of truth. Safe precisely because every consumer is idempotent, so re-driving an already-processed message is a no-op.
  • Unattributable — alert, and do not touch it.

That last rule is the one that matters. A reconciliation job that adjusts balances until the books balance is inventing money to hide a bug it does not understand, and it is strictly worse than one that stops and raises an alarm. Detection without a correct explanation is a reason to page a human, not a reason to write to the ledger.

Documented trade-offs

Three places where this design knowingly departs from what an unconstrained production system would do.

One PostgreSQL instance, three logical databases. Each service gets its own database and its own credentials, so no service can read another's tables. But they share one instance — which means one blast radius and one scaling ceiling regardless of where the logical boundaries sit. An instance-level failure or a noisy-neighbour workload takes down all three at once, which is exactly what per-service isolation is supposed to prevent. This is a resource constraint of a solo project on one laptop, not a preference. With three instances the change is purely operational: separate connection strings, separate backup and failover policy per service, and independent scaling of the ledger's write path. No application code would change.

Reconciliation reads both databases. This breaches per-service data ownership deliberately. It is a back-office auditor holding read-only credentials on both, because auditing a system through the API of the system being audited defeats the purpose. Real back-office reconciliation runs this way for the same reason.

Reconciliation is its own deployable. Folding it into the Accounts service would tie its failure mode, deploy cycle, and blast radius to a service it exists to check. One extra deployable is a fair price for that independence being true rather than asserted.

Architecture

                    ┌──────────────────┐
   Client ─────────▶│   API Gateway     │  rate limiting
                    └────────┬──────────┘
                             │
              ┌──────────────┴───────────────┐
              ▼                              ▼
      ┌────────────────┐            ┌──────────────────┐
      │ Accounts Svc    │            │  Query Svc        │
      │ command side    │            │  read model       │
      │ accounts_db     │            │  query_db + Redis │
      │ + outbox        │            └────────▲──────────┘
      └───────┬─────────┘                     │ projects
              │ publishes                     │
              ▼                               │
      ┌──────────────────────────────────────┴───────┐
      │                  Kafka                        │
      │  keyed by transfer id → per-transfer ordering │
      └───────┬───────────────────────────────────────┘
              ▼
      ┌────────────────┐
      │ Ledger Svc      │  event-sourced, append-only
      │ ledger_db       │  balances derived by replay
      └───────┬─────────┘
              ▼
      ┌────────────────┐
      │ Reconciliation  │  independent, read-only on both
      └─────────────────┘

The transfer flow

Step 1 — reserve. ✅ Implemented and tested. POST /transfers with an Idempotency-Key. In one transaction, Accounts inserts the transfer row, moves funds from available to reserved, and writes a TransferReserved event to an outbox table. There is no window where the balance moved but the event was lost — that is the entire point of the outbox pattern, and it is what an integration test asserts directly.

Three properties this step guarantees, each covered by a test:

  • Insufficient funds writes no event. The transfer is recorded REJECTED, the balance is untouched, and the outbox stays empty — there is nothing for a downstream service to undo.
  • The destination is not credited yet. Money is held, not moved, until the ledger posts.
  • A replayed Idempotency-Key does not reserve twice. Uniqueness is enforced by a database constraint, not just a read-then-write check. Two concurrent requests with the same key can both pass the pre-check; one wins the constraint and the other's transaction rolls back entirely, then returns the winner's stored result rather than an error. A retry gets the original answer, which is what the header is for.

Steps 2-4 — designed, not yet built. An outbox poller drains unpublished rows with FOR UPDATE SKIP LOCKED — letting every service instance poll the same table without blocking each other or double-publishing — and publishes to Kafka keyed by transfer id. Ledger appends a balanced double-entry pair to its append-only store and emits its result through its own outbox. Accounts then settles the hold, or runs the compensating release.

Every consumer will write a processed_event row in the same transaction as its state change. That is what turns Kafka's at-least-once delivery into exactly-once effect.

Tech stack

Java 21 · Spring Boot 4.1.0 · PostgreSQL with jOOQ · Flyway · Apache Kafka · Testcontainers · JUnit · ArchUnit · Maven multi-module · Docker

Running the tests

Requires Docker (integration tests start real PostgreSQL and Kafka containers).

mvn clean verify

Repository layout

contracts/          event records shared by every service
common/             Money, and the outbox/inbox mechanism
accounts-service/   command side: balances, reservations, transfer API, outbox
docs/design.md      full design, invariants, trade-offs
docs/implementation-plan.md   task-by-task build plan

About

Payments backend: transactional outbox, reserve-then-settle transfers, event-sourced ledger design. Java 21, Spring Boot, PostgreSQL/jOOQ, Kafka, Testcontainers.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages