Skip to content

Feature/2962 reserved domains audit trail - #2967

Merged
vohmar merged 14 commits into
masterfrom
feature/2962-reserved-domains-audit-trail
Oct 9, 2026
Merged

vohmar merged 14 commits into
masterfrom
feature/2962-reserved-domains-audit-trail

Conversation

@OlegPhenomenon

@OlegPhenomenon OlegPhenomenon commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2962

Depends on #2964 and #2965. This branch is built on top of both (merge commit da92faea0), so until they are merged the diff also shows their changes (expire_at_for, release_expired, the expire_at index migration). Please review/merge #2964 and #2965 first; this branch will then be rebased on master.

What

  1. Every reserved domain change records where it came from and why. New audit columns on log_reserved_domains are filled for every create/update/destroy.
  2. Admin page with the full reservation history: Settings → Archive → Reserved domains history. It covers active, expired, released to auction, deleted and removed reservations, with filters, a change timeline and CSV export.
  3. No behaviour change for the Business Registry API, EPP, REPP, the billing callback or the reservation logic. Responses, status codes, routes, expiry rules, password rotation and dispute sync are unchanged. Existing API tests pass unmodified (only assertions were added).

Audit data

New nullable columns on log_reserved_domains: source, reason, reason_note, domain_name, registrar_id.
They are written through PaperTrail meta from ReservedDomain::Audit (ActiveSupport::CurrentAttributes, block-scoped with Audit.set { }). Nothing was added to reserved_domains. creator_str/updator_str keep their existing string format, so Versions#creator/updator work as before.

Writer whodunnit / creator_str source reason
BR API POST reserve_domains Business Registry API business_registry free_reservation
BR API long_reserve_domains(_status) (paid) Business Registry API business_registry paid_reservation
EIS billing callback EIS billing callback eis_billing paid_reservation
Daily cleanup cron (#2965) unchanged #2965 string expiry_job reservation_expired
Lazy expiry in availability check (#2965) unchanged #2965 string availability_check reservation_expired
EPP/REPP domain create on a reserved name ApiUser registrar (+ registrar_id) domain_registered
Dispute password sync caller dispute dispute_password_sync
Admin create / update / delete AdminUser admin admin_created / admin_updated / admin_deleted
Admin release to auction, auction create, auction CSV import AdminUser admin released_to_auction

If no context is set, source falls back to a value derived from the whodunnit prefix (AdminUser/ApiUser/console-/rake-, else unknown). Audit never raises and never blocks a write.

Admin behaviour change (from the issue: "Manual admin modification (with reason)")

  • Edit reserved domain: a Reason field is required. Without it nothing is saved, and the form keeps the entered values.
  • Delete reserved domain: the GET delete route is unchanged. The Delete button now asks for a reason (prompt) and passes it as reason_note. Without a reason nothing is deleted.
  • Create and release to auction do not require a note; the reason is recorded automatically.

History page and performance

History lives in a table, reserved_domain_lifecycles, with one row per reservation. Rows are derived by a single SQL function, reserved_domain_lifecycle_rows(bigint[]), from log_reserved_domains plus live reserved_domains rows. Derived fields: name, created at/by/source/reason, last change at/by/source/reason/note, expire_at, ended at, end reason, registration_recorded and live. Status (active / expired / released_to_auction / deleted / removed) is computed at read time, because it depends on the current time.

The table is never written by reservation write paths. It is kept current by:

  • catch-up on admin read (ReservedDomain::Lifecycle.sync!): re-derives only reservations changed since the previous sync, with a 10-minute safety margin. It is serialized by a pg_try_advisory_xact_lock, so a busy lock skips the sync and the page serves the current data. A failed sync is logged and shown as a warning; it never breaks the page;
  • nightly rebuild (cron, 03:15, registry cron group): ReservedDomain::Lifecycle.rebuild! repairs rows changed without versions;
  • rake reserved_domains:rebuild_lifecycles, also run at the end of reserved_domains:backfill_audit.

The first implementation was a plain SQL view. It was measured and replaced:

synthetic data SQL view table + catch-up
history page, 100k reservations 6.2 s —
history page, 300k reservations / 700k versions did not finish in 5 min 16–80 ms
filters (status / reason / name), 300k ~0.9 s already at 100k 3–160 ms
full CSV export 92 s at 100k 5.2 s at 300k (streamed, rendered in SQL)
catch-up on page load — 0.1–0.4 s
full rebuild, 300k — 10–30 s (cron / rake only)

Other details:

  • Filters: name (unicode or punycode), status, created/last-changed date ranges, creation/last source, last reason, created by / last changed by, registration recorded.
  • Detail page: full version timeline with source, reason, note, registrar and field changes. Password values are masked.
  • CSV export: all audit fields with UTC ISO 8601 timestamps. Passwords are never exported, and cells that a spreadsheet would treat as a formula are prefixed with '.
  • Access and navigation: admin role only (can :read). Each row of the current reserved domains list gets a History link.

Backfill of existing history

rake reserved_domains:backfill_audit is batched and idempotent:

  • It fills domain_name for old rows, event-aware.
  • It fills source/reason only where the whodunnit proves them: AdminUser: → admin, ApiUser: → registrar, the Release expired domain reservations daily #2965 cleanup strings → expiry_job / availability_check, console-/rake-.
  • Everything else stays unknown; nothing is guessed from expire_at.
  • whodunnit, object and object_changes are never modified.
  • It rebuilds the history table at the end.

Deployment

  1. Migrations (run before restarting web and Sidekiq):
    • 20261005090000: adds nullable columns, metadata-only in PG13.
    • 20261005090100: indexes, built CONCURRENTLY.
    • 20261005090200: creates the function and the two tables, and fills the history table. It only reads the log tables; expect tens of seconds on large history.
  2. bundle exec rake reserved_domains:backfill_audit (once).
  3. Regenerate crontab on the registry server for the new 03:15 rebuild job, the same way as for Release expired domain reservations daily #2965 (mina pr cron:setup / whenever --update-crontab ... cron_group=registry).

Rollback: db:rollback STEP=3 drops the history table, the function, the indexes and the audit columns.

Tests

  • New/extended:
    • model tests for the audit context: meta columns, fallbacks, nested/exception restore;
    • lifecycle derivation, every status, sync!/rebuild!, lock contention, CSV;
    • version backfill and the rake tasks;
    • an integration test per writer (BR API, billing callback, EPP and REPP create on a reserved name, dispute, admin, auction);
    • admin history controller: filters, show, CSV, failing sync.
  • Run in docker: related suites 607 runs, 82 admin system tests, green.

Known issues not fixed here (separate PRs)

A reservation used to expire at the exact activation timestamp plus the
period, so a domain reserved at 13:43 was released at 13:43 seven days
later. The period is now counted in full calendar days in the app time
zone: the activation day is not counted and the reservation ends at
23:59:59 of its last day.

ReservedDomain.expire_at_for is the single place computing the deadline
and is used for both free (7 days) and paid (1 year) reservations.

Closes #2961
The business registry billing callback created reserved domains without
expire_at, which made them permanent. Use the same 1-year full-day
expiry as the status endpoint.

Refs #2961
Add ReservedDomain.release_expired, run by cron every day at 00:35, which
removes reservations whose expire_at has passed. Permanent reservations
(expire_at NULL) are kept.

Each record is locked and re-checked before removal. The reason is stored
in updator_str and whodunnit before destroy, so the PaperTrail destroy
version in log_reserved_domains carries the process, reason and timestamp.
Lazy removal on availability check reuses the same path.

A failing record is logged and reported to Airbrake without stopping the
run. WHOIS refresh is enqueued after commit. Add index on expire_at.

Closes #2963
…y-expiry' into feature/2962-reserved-domains-audit-trail

# Conflicts:
#	CHANGELOG.md
#	test/models/reserved_domain_test.rb
Add source, reason, reason_note, domain_name and registrar_id to
log_reserved_domains and fill them through PaperTrail meta from
ReservedDomain::Audit (CurrentAttributes). Without an explicit context
the source falls back to the whodunnit prefix, so writes never fail
because of missing audit data.
Business Registry API, EIS billing callback, expiry cleanup, domain
registration, dispute password sync and admin actions now set the audit
context, so each log_reserved_domains row says where the change came
from and why. Admin update and delete require a reason note; the delete
route stays GET and takes the note as a query parameter. API responses
and reservation logic are unchanged.
reserved_domain_lifecycles builds one row per reservation from
log_reserved_domains plus live rows: name, creator, last change, expiry,
end reason, recorded registration and derived status. The
reserved_domains:backfill_audit task fills domain_name and, where the
whodunnit proves it, source and reason on rows written before the audit
columns existed.
Settings > Archive > Reserved domains history lists every reservation
lifecycle with filters by name (unicode or punycode), status, dates,
source, reason and author, a detail page with the full change timeline
(passwords masked) and a CSV export of all audit fields. Each row of the
reserved domains list links to its history.
The lifecycle view recomputed every reservation on each request: with
100k reservations the history page took 6 s and the CSV export 92 s.
History now lives in reserved_domain_lifecycles, derived by the
reserved_domain_lifecycle_rows() SQL function. Admin reads catch up
only the reservations changed since the previous read, a nightly cron
and the rake tasks rebuild the whole table, and the CSV is streamed and
rendered in SQL. Reservation write paths are not touched.

With 300k reservations the page loads in under 100 ms, filters in up to
0.5 s and the full CSV export takes about 5 s.
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

🚀 Deploy Complete!

Property Value
App registry
Slot 1
URL https://registry1-dev.cloud.tld.ee
Namespace registry1-dev

(Environment ready for testing)

The expired reservation was created one day before the real current
time but released at a fixed past moment, so the test failed once the
real date moved past it.
…ed-domains-audit-trail

# Conflicts:
#	CHANGELOG.md
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

🚀 Deploy Complete!

Property Value
App registry
Slot 1
URL https://registry1-dev.cloud.tld.ee
Namespace registry1-dev

(Environment ready for testing)

The button sits next to 'New reserved domain' and opens the lifecycle
history list. If a name search is active, it is carried over as the
domain name filter, so a domain missing from the reserved list can be
looked up in its history in one click.
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

🚀 Deploy Complete!

Property Value
App registry
Slot 1
URL https://registry1-dev.cloud.tld.ee
Namespace registry1-dev

(Environment ready for testing)

@vohmar
vohmar requested a review from maricavor October 8, 2026 14:36
@vohmar
vohmar merged commit 0408e6b into master Oct 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement proper audit logging and UI for reservations list

3 participants