Skip to content

feat: platform-wallet backend rewrite (spec + implementation)#860

Draft
lklimek wants to merge 347 commits into
v1.0-devfrom
docs/platform-wallet-migration-design
Draft

feat: platform-wallet backend rewrite (spec + implementation)#860
lklimek wants to merge 347 commits into
v1.0-devfrom
docs/platform-wallet-migration-design

Conversation

@lklimek

@lklimek lklimek commented May 18, 2026

Copy link
Copy Markdown
Contributor

Goal

Dash Evo Tool carried its own large SPV/wallet stack that duplicated logic now maintained upstream. This PR rewrites DET's wallet backend on top of the upstream platform-wallet crate (dashpay/platform), which owns SPV chain sync, address derivation, identity/asset-lock handling, and the shielded coordinator.

DET is now a thin adapter over that crate: its bespoke SPV stack (src/spv/) and the legacy RPC wallet mode are gone, chain sync and derivation come from upstream, and the shielded subsystem routes through the upstream coordinator. The BackendTask action/channel contract is preserved, so the UI layer is largely unchanged. Existing wallets are migrated onto the new model on first launch (marker-gated, with a *.premigration backup). Net effect: far less wallet code to maintain in DET, a single source of truth shared with the rest of the platform, and upstream fixes inherited automatically.

Bugs fixed

Funds-safety

  • Funds sent to addresses beyond the SPV gap window — via Receive → New Address, the asset-lock deposit address, or platform top-up change — were never watched and never appeared in the balance. All address derivation now comes from the SPV-watched pool.
  • The asset-lock deposit QR flow could hang at "Waiting for funds…" forever (the event that advances it had no producer).
  • Incoming DashPay contact payments were never detected or credited (the detection chain had no callers).

Shielded confirmation-safety

  • Shield-from-asset-lock and the four sibling shielded spends reported success / marked notes spent even when the transaction might never have confirmed. They now surface typed *ConfirmationUnknown errors instead of a false success.

Secret hygiene

  • The passphrase was prompted at startup for wallets the user hadn't chosen to unlock; it is now requested just-in-time per operation and dropped immediately after use.
  • A single-key private key was kept in plaintext for the whole session — removed.
  • DashPay ECDH and HD encryption keys are now zeroized on drop.

Data-safety

  • Password-protected single-key wallets that silently vanished after upgrade are preserved.
  • Wallets left unloaded after the cold-start migration (which required a manual restart) are now rehydrated automatically.

Identity / sync

  • User-set identity aliases are preserved during auto-discovery (previously silently overwritten).
  • Platform/identity sync no longer starts before chain quorum is ready (which previously caused a DAPI ban storm).

Known gaps

  • Dependency pin — merge-blocker (PROJ-005). The upstream platform-wallet / dash-sdk crates are pinned to an unreleased branch (fix/wallet-core-derived-rehydration), not a released tag. Must be re-pinned to a released version before merge-to-ship.
  • Non-wallet data not migrated (PROJ-034). App settings (network/theme/paths), top-up history, and scheduled DPNS votes reset on upgrade; scheduled votes carry a vote-window deadline risk. Deferred to a follow-up.
  • Single-key wallets partial (PROJ-007). Balance refresh and SPV send for single-key wallets still return SingleKeyWalletsUnsupported, pending upstream watch-only support. Import / data-loss guard / password-restore are done.
  • Minor / deferred: token "stop tracking" is undone by a refresh (PROJ-041); no in-app notice for the removed QR-direct-fund path (DOC-003); external user-docs update pending (PROJ-018). Merge-time action: fill the ADR floor SHA placeholder (PROJ-019).

Testing

  • cargo build / cargo build --features headless clean; cargo clippy --all-features --all-targets -- -D warnings clean.
  • Library + UI (kittest) + e2e suites green (889 lib tests at latest push).
  • Funds-safety / security review on the migration, shielded retirement, and address-derivation changes; per-workstream design + QA reports under docs/ai-design/.

Attribution

Additional fixes folded in (2026-06-17)

Two independently-verified fixes were folded onto this branch.

Wallet platform-balance source-of-truth (a21e224628bac801)

  • Problem: The Wallets page selector showed a platform balance up to ~227× too high (e.g. 0.21674054 DASH vs the real 0.00039605), diverging from the per-address section.
  • Cause: A direct sync_address_balances query wrote every found balance — including non-owned "orphan" addresses — into the balance-bearing map; the selector summed the whole map while the address section summed only watched addresses.
  • Fix: Platform balance is now sourced from the upstream coordinator's push (owned addresses only), mirroring the shielded-balance pattern. The selector total and the per-address tab derive from the same data and are equal by construction (unit-tested). Balance updates automatically on each coordinator pass — no manual Refresh.

DashPay contact receiving-account registration (a11c8312c163b67b)

  • Problem: Incoming payments to established DashPay contacts weren't watched — funds appeared only after a manual refresh, and on fresh installs not at all.
  • Cause: registration ran against watch-only wallets (hardened DashPay derivation needs the seed); the local wallet-manager was never told about sent/accepted contact requests, so contacts never auto-established.
  • Fix: registration moved to the unlock/bootstrap path (seed in scope) and delegates to upstream platform-wallet account derivation; accepting a request now records both the incoming and the sent contact request so the contact establishes in-process; SPV rebuilds its bloom filter (within ~100 ms once synced) to watch the contact's pay-to-us addresses.

Both fixes verified by QA; the combined branch builds green (894 lib tests pass, clippy clean).

🤖 Co-authored by Claudius the Magnificent AI Agent

Additional fixes folded in (2026-06-18)

Wallet-backend lifecycle + adapter fixes layered on the rewrite. Combined branch builds green; clippy --all-features --all-targets -D warnings clean; wallet_lifecycle suite (39 tests, incl. new gates) passes.

Reconnect "wallet already open" (B + B-2) — 09540cf8, 976ad0d4

  • Problem: Disconnect → Connect failed with WalletStorageError::AlreadyOpen on spv/<net>/platform-wallet.sqlite; the second Connect could never reopen the wallet DB.
  • Cause: the SQLite persister path lives in a process-global open-set, released only on SqlitePersister::Drop. On disconnect the SPV run loop (B) and the three upstream sync coordinators each kept an Arc<SqlitePersister> alive past teardown, so the path stayed registered.
  • Fix (B): WalletBackend::shutdown() now stops the SPV run loop before tearing down the manager.
  • Fix (B-2, interim): a deterministic persister-release barrier in stop_spv waits (bounded, 5s) until the path is actually free before disconnect completes, closing the common coordinator-thread drain race. Verified as an interim stopgap.
  • Known residual / follow-up: the barrier does not bound DET-owned wallet subtasks blocked on a dead-network DAPI call past the timeout. A durable cancel + await-exit shutdown primitive (StructuredShutdown = hierarchical CancellationToken + an mpsc all-senders-dropped exit barrier) — covering both the DET-owned subtasks (no upstream dependency) and the upstream coordinator threads — is designed as the follow-up that retires the barrier. The architecturally clean variant keeps the WalletBackend alive across reconnect and restarts SPV/coordinators in place (confirmed feasible upstream), removing the open/close registry race by construction. An upstream rs-platform-wallet issue (make coordinator quiesce() join, not cancel-and-drain) will be filed.

Watch-only identity funding (C/D) — 98bc4913

  • Problem: On a reloaded (seedless watch-only) HD wallet, "Create Identity → Wallet balance" and "send funds core → identity" failed with AssetLockTransaction("Watch-only wallet has no private key").
  • Cause: identity funding-account provisioning called add_account(type, None), deriving from the live wallet's absent root private key — before any seed was in scope.
  • Fix: provisioning now runs inside the just-in-time seed scope, derives the account xpub from a seed wallet, and inserts via the Some(xpub) path — mirroring the DashPay contact-account pattern. One fix covers both reported flows plus the non-identity asset-lock path. Deterministic cold-boot regression guard added.

ensure_spv_synced made event-driven (②) — a98186af

  • The MCP ensure_spv_synced gate replaced its poll loop with a tokio::watch subscription on ConnectionStatus's SPV status, waiting on SpvStatus::Running (not the DAPI-gated Synced). Wallet tools no longer busy-poll.

Inspector / Core-P2P removal (①) — 1c4f4dcd

  • Removed the masternode-list-diff inspector and the legacy Core P2P handler (dead/duplicated since the platform-wallet rewrite owns chain sync).

Tests — 43ede282

  • Added backend-e2e coverage: SPV reconnect (B) and cold-boot identity funding from wallet balance (C/D).

@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7015e7a9-096b-4148-8f8c-e9677178a680

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/platform-wallet-migration-design

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@lklimek lklimek changed the title docs: platform-wallet migration design docs: platform-wallet backend — clean-slate rewrite spec May 18, 2026
@lklimek lklimek changed the title docs: platform-wallet backend — clean-slate rewrite spec feat: platform-wallet backend rewrite (spec + implementation) May 19, 2026
lklimek added a commit that referenced this pull request May 29, 2026
refactor: complete data.db unwire — shielded + DashPay (stacked on #860)
lklimek and others added 8 commits May 29, 2026 16:31
…003)

Eight `Err(...)` arms in `finish_unwire.rs` matched `msg.contains("no
such table")` to silently treat a missing legacy table as "no rows" —
fragile string parsing, bypasses the type system, breaks on rusqlite
message drift. Adams' note already pointed at `legacy_table_exists`
and `wallet_table_has_core_wallet_name` as the typed primitives to
mirror.

Factor `legacy_table_exists_named(conn, table: &'static str)` that
queries `sqlite_master` directly and propagates the static table name
into `MigrationError::LegacyDbRead`. Every `migrate_*_rows_from_conn`
body now calls it before `conn.prepare(sql)` and the prepare
failure path becomes a single typed map_err — no
`Err(SqliteFailure)` or `Err(SqlInputError)` arms with string parsing.

Net effect: a rusqlite error from `prepare` after the pre-check passes
is a hard error rather than getting silently buried. The
`missing_*_table_yields_zero_outcome` tests already cover the
pre-check path and continue to pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PROJ-001 + PROJ-002. Adds is_wallet_touching() and short-circuits any
wallet-family backend task (Wallet/Core/Identity/DashPay/Shielded) with
TaskError::WalletStorageNotReady while the cold-start migration is
running, so the UI shows the "data is still being updated" banner
instead of a misleading SDK timeout. Shielded tasks additionally consult
legacy_shielded_present_but_sidecar_empty() — the NFR-4 pre-flight gate
that detects legacy shielded rows not yet mirrored into the sidecar.

Bilby-A's prior commits (a7a3d3b8 → 9b05ab77 → 77175534, now
cc887fe23be433ce676fa on this branch) plus this commit close
Phase-3 findings SEC-001, PROJ-001, PROJ-002, PROJ-003, PROJ-004, and
QA-002/DOC-002. SEC-002 (per-key passphrase) is the remaining HIGH and
will land in a follow-up commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Inserts a blank doc-comment line so the prose after the numbered list
in `sentinel_is_per_network_mainnet_then_testnet` parses as a new
paragraph. Fixes `doc_lazy_continuation` clippy lint introduced by
SEC-001 (cc887fe). Pure cosmetic; no behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… typed seed-length error

SEC-005 (LOW): leading version byte on the upstream-vault stored seed
envelope payload. Reader accepts both the tagged form (current) and a
legacy bare-bincode form, so existing entries keep round-tripping while
future shape changes can dispatch on the tag.

SEC-007 (LOW): row-level length guard on legacy `wallet` migration —
salt must be 16 bytes when uses_password=true, nonce 12 bytes, both
empty when uses_password=false. Corrupted rows are counted as failed
and skipped (logged), matching the existing per-row policy; the
migration still proceeds for the rest.

SEC-008 (LOW): cold-boot hydration of a non-password envelope whose
encrypted_seed is not 64 bytes now surfaces
`TaskError::SeedLengthInvalid { wallet_label, got, expected }` instead
of silently degrading the wallet to a closed state. The user sees
WHICH wallet was skipped and WHY.

Tests: 553 lib tests pass (was 551 baseline); two added — sec_005
version-byte round-trip + legacy fallback, sec_008 typed-error
surfacing on a 16-byte (wrong-length) seed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes SEC-002 (HIGH). Each imported WIF gets its own optional
passphrase at import time — mirrors the HD per-wallet password model
already shipping. The fixed empty-passphrase open of the upstream
SecretStore is preserved (file-perm-gated vault floor) and the
per-key passphrase encrypts the inner entry on top.

Storage shape: new `SingleKeyEntry { has_passphrase, salt, nonce,
ciphertext, passphrase_hint, public_key_bytes }`, encoded as
`[version_byte || bincode(entry)]` and stored under the same
`single_key_priv.<addr>` label. Backward-compat: a legacy 32-byte raw
payload is recognised by length and treated as `has_passphrase = false`.

Unprotected entries store the raw 32 key bytes inside the struct
(unchanged vault-encryption posture). Protected entries AES-256-GCM
encrypt the raw key under an Argon2id-derived key, reusing the same
`encrypt_message` helper the HD seed path uses. `public_key_bytes`
lets the cold-boot picker rebuild a locked wallet without unlocking.

UI: `ImportSingleKeyDialog` gets a "Protect with passphrase" checkbox
that reveals passphrase + confirm + hint fields. Validation enforces
≥ 8 chars and matching confirm — both surfaced as typed
`TaskError::SingleKeyPassphraseTooShort` / `SingleKeyPassphraseMismatch`.

Unlock-on-use: `WalletBackend` holds an in-process
`BTreeMap<address, [u8; 32]>` cache of unlocked plaintexts. New
`SingleKeyView` methods:
  - `has_passphrase(addr)`
  - `unlock_with_passphrase(addr, passphrase)`
  - `forget_unlocked(addr)`
`sign_with` consults the cache first; an empty cache for a locked
entry surfaces `TaskError::SingleKeyPassphraseRequired { addr }`. Cache
drops on shutdown — never persisted.

UI integration: the passphrase prompt modal that the sign-time flow
will use is NOT wired here — that touches many backend tasks
(register_identity, send_funds, asset-lock-signer, ...). This commit
ships the storage + import + cache + API; the per-task prompt flow is
left as a follow-up. TODO comment placed at the WalletBackend
single_key view docs.

Tests: 561 lib tests pass (was 553 after SEC-005/007/008). Eight added
across `single_key_entry` and `single_key`:
  - protected/unprotected round-trip + legacy 32-byte fallback
  - passphrase-protected sign requires unlock first
  - wrong passphrase → `SingleKeyPassphraseIncorrect`
  - short passphrase rejected at import
  - legacy raw payload still signs (no migration regression)
  - vault payload is ciphertext, not the WIF plaintext bytes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the project rule that says: when a checklist step cannot be
completed (UI work that touches every backend signing path), add a
TODO comment in the relevant source file marking the incomplete step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hPayTask in matrix

SEC-009 (Smythe re-QA, LOW): SingleKeyEntry and StoredSeedEnvelope
derived Debug over a ciphertext field that, for unprotected entries,
holds raw private key bytes. Replaces the derive with a manual impl
that emits "[redacted]" for the secret-bearing field. New unit tests
assert known plaintext does not appear in the Debug output.

QA-R-001 (Marvin re-QA, LOW): wallet_touching_matrix_is_stable
docstring claimed every task family was pinned; only 3 of 5 were
asserted. Adds IdentityTask and DashPayTask to the assertion list so
a future drop from the matches! arm is caught at test time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Migration sentinel: correct key from global `det:migration:finish_unwire:v1`
  to per-network `det:migration:finish_unwire:<network>:v1`; document the
  `sentinel_key_for` function and `SENTINEL_KEY_PREFIX` / `SENTINEL_KEY_VERSION`
  constants (no single `SENTINEL_KEY` exists).
- Single-key metadata sidecar: add `has_passphrase: bool` and
  `passphrase_hint: Option<String>` fields added by SEC-002 Option C.
- HD seed envelope value encoding: document the leading
  `STORED_SEED_ENVELOPE_VERSION` byte prepended by the storage layer
  before the bincode body (SEC-005).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lklimek

lklimek commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Doc sync — 686430a4 (docs/kv-keys.md)

Trillian audited the k/v key reference against current source and corrected three stale entries. All align the doc with code changes already described in this PR:

  • Migration sentinel — was documented as a single global det:migration:finish_unwire:v1 with constant SENTINEL_KEY. Now correctly det:migration:finish_unwire:<network>:v1 (per-network, via sentinel_key_for(network)), consts SENTINEL_KEY_PREFIX + SENTINEL_KEY_VERSION. Tracks SEC-001.
  • Single-key metadata sidecarImportedKey field list extended from 3 → 5 (has_passphrase, passphrase_hint). Tracks SEC-002 Option C.
  • HD seed envelope encoding — value is now [ version byte | bincode::serde(payload) ], not bare bincode::serde. Tracks SEC-005.

All other sections (17 platform-wallet keys, DashPay sidecar, settings, wallet selection, summary counts) verified accurate — no drift. Doc-only; no source touched.

🤖 Co-authored by Claudius the Magnificent AI Agent

lklimek and others added 5 commits June 1, 2026 12:17
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Connect button and auto-start path both called AppContext::start_spv(),
a P0.5 no-op stub returning Ok(()), so SPV never started and the indicator
stayed idle. Delegate to WalletBackend::start() (spawns the SPV run loop),
surfacing init failures via ConnectionStatus and the typed
WalletBackendNotYetWired error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
start_spv() callers (GUI auto-start, manual Connect, MCP boot, network
switch) invoked it before ensure_wallet_backend wired the backend, so each
silently fast-failed with WalletBackendNotYetWired. Trigger SPV start from
wiring completion instead via a new async chokepoint
(ensure_wallet_backend_and_start_spv); surface Connect via a typed
AppAction::StartSpv with a progress banner; make network-switch and MCP
paths wire-then-start; drop the DAPI-gate masking of SPV Error state and
fix the overclaiming docstrings.

QA-001: boot auto-start folded into the eager wallet-backend init task so
it cannot fire before wiring; redundant synchronous boot call removed.
QA-002: Connect button returns AppAction::StartSpv (handled where the
TaskResult sender lives); the dead AppAction::Custom path and the
i18n-violating concatenated error string are gone.
QA-003: resolve::ensure_spv_synced lazily wires + starts SPV on first call
— single chokepoint covering stdio standalone, HTTP swap, and post-switch.
QA-004: SwitchNetwork task and finalize_network_switch wire-then-start;
spv_started now reflects actual backend readiness.
QA-005: cached-context restart logs an explicit "already running" at info
instead of a silent debug no-op.
QA-006: SPV Error precedence hoisted above the DAPI gate in refresh_state;
docstrings corrected to match reality.

Tests: offline gating tests for start_spv()/the chokepoint (wired/unwired)
and two refresh_state regression tests for the SPV-error/DAPI precedence.
StartLatch tests left intact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Connect handler only logged a warning when ensure_wallet_backend_and_start_spv
failed at the (fallible) wiring step, leaving the user with no feedback and the
indicator on Disconnected. Flip the SPV indicator to Error and show an actionable
banner on failure; this also makes the previously-dead error-surfacing path
reachable via wiring failures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lklimek

lklimek commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

SPV-start fix (PROJ-001, CRITICAL) + consolidated gap report — 8d35933e

Bug: the Settings "Connect" button did nothing and SPV stayed idle. Root cause: AppContext::start_spv() was an inert Ok(()) compile-floor stub and WalletBackend::start() (the only spawner of the SPV run loop) had zero callers — so SPV / platform-address / identity sync never started in any path (Connect, auto-start, MCP, network switch).

Fix (3 commits):

  • 42388c4b — wire start_spv()WalletBackend::start(); add a per-instance StartLatch so SPV can't be double-driven.
  • 3165f98c — route all callers through one idempotent async chokepoint AppContext::ensure_wallet_backend_and_start_spv() (wire-then-start); fixes the boot auto-start race (sync constructor fired before the backend was wired), MCP ensure_spv_synced, and network-switch restart; hoist SpvStatus::Error above the DAPI gate so chain-sync failures aren't masked as "Disconnected".
  • 36f5a982 — surface start/wiring failures to the user (indicator → Error in the chokepoint; actionable banner at the Connect handler); replaces the dead AppAction::Custom no-op.

QA: two full Marvin rounds (caller-integration focus). All findings resolved; 2 LOW residuals (user-facing wiring-failure feedback + a forward-compat dead branch) closed by 36f5a982. 571 lib tests pass (+8), clippy/fmt clean. 8 new offline tests cover the start-path gating. The one kittest failure (tc_sk_004) is confirmed pre-existing.


Consolidated gap audit — 2313089a (+ status update 8d35933e)

docs/ai-design/2026-06-01-pr860-gap-audit/gaps.md — a whole-PR audit by project-reviewer Adams. 21 confirmed gaps (1 CRITICAL, 4 HIGH, 6 MEDIUM, 7 LOW, 3 INFO), each with file:line. Highlights:

  • PROJ-001 (CRITICAL) — resolved on-branch (above).
  • PROJ-005 (HIGH) — now the sole remaining open merge-blocker: Cargo.toml pins platform to the unreleased #3625 rev (release gate G1).
  • 5 net-new findings, notably PROJ-004 (HIGH) — DashPay outgoing contact-request xpub derivation uses placeholder seed material (let wallet_seed = sender_private_key;), the substrate behind the TC-037/043 association symptom.
  • 4 seeded "gaps" verified already-fixed (eager-init hard-failure → graceful warn, TC-019 precedence, core_backend_mode removal, the 5-manager readiness gate).

Remaining open gaps are catalogued in the report (not filed as separate issues by request).

🤖 Co-authored by Claudius the Magnificent AI Agent

lklimek and others added 7 commits June 1, 2026 13:31
TaskError::WalletStorage rendered "check disk space and restart" for every
storage failure, including a forward-version database (schema written by a
newer build), where that advice is wrong and leaves the user stuck. Add a
dedicated typed variant matched on the upstream WalletStorageError, with an
actionable message telling the user to update the app. Disk/IO failures keep
the original copy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…stream-owned caches

Bump platform to 08b0ed9 and adapt DetKv to the ObjectId-scoped KvStore via a
DET-side DetScope (Global/Wallet wired; Identity/Token reserved for Wave 2).
Isolate the redundant platform-address and token-balance caches behind
PlatformAddressView/TokenBalanceView seams with reserved upstream-reading impls
(stubbed pending platform todos e817b66a / f5897abd); caches stay active, no
behaviour change. Removal is a one-line swap once upstream exposes public readers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ance seam

Map the upstream divergent-migration open failure to an actionable typed error
instead of the misleading "check disk space" message (extends the WB-001
pattern). Route the devnet token sweep through TokenBalanceView and dedupe the
key prefix so the seam is honoured everywhere. Add a dev note on the on-disk
schema break requiring a local wallet-DB reset on this branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…alance reader)

The kv module dropped ObjectKind and KvError::ObjectNotFound upstream:
meta_* writes now succeed without a parent row (AFTER DELETE triggers reap
orphans), so DET's ObjectNotFound-promotion path is dead. Removed the
ObjectKindLite mirror + variant, collapsed map_kv_error to a plain Store
wrap, and repurposed the K9 test to assert the generic Store arm.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ream

Upstream (35e4a2f) now exposes IdentitySyncManager::state_for_identity/all_state
for per-(identity,token) balances, so DET's duplicate det:token_balance cache is
removed and TokenBalanceView reads upstream via a blocking snapshot bridge.
Pre-sync balances show "syncing" (not zero); DET no longer persists/zero-seeds.

The upstream identity-sync registry was never populated (start() ran but no
register_identity), so reading all_state() alone would be empty forever. The
balance backend tasks now register each local identity's watched-token set,
force a sync pass, and republish a lock-free DET-typed snapshot the egui frame
reads infallibly. Token mutation tasks apply their proof-derived balance into
the snapshot for instant feedback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the upstream meta_* FK relaxed (35e4a2f), move det:identity / top_ups /
scheduled_vote to Identity(id) and dashpay private/address_index to
Identity(owner). Add a Global identity-id index for enumeration; rely on the
upstream soft-cascade trigger for meta_identity cleanup where DET deletes the
upstream identity row, explicit cleanup otherwise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DetScope::Identity is now active (identities, top-ups, scheduled-votes,
dashpay private/address_index); det:token_balance removed (read upstream);
add the two enumeration-index keys and fix kv-keys.md summary counts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lklimek

lklimek commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

DetKv per-object refactor + platform bump to 35e4a2f129d54d0

Reseats DET's KV layer onto upstream's per-object KvStore and retires two of three duplicate caches. 7 commits:

  • 549ddfa1 — honest error when wallet data is from a newer app version (WalletDataTooNew, replaces the misleading "check disk space" for schema-version skew).
  • dbd94356 — reseat DetKv on the 08b0ed9 per-object KvStore via a DET-side DetScope { Global, Wallet, Identity, Token } (maps to upstream ObjectId only inside kv.rs — no type leak); isolate the platform-address and token-balance caches behind PlatformAddressView/TokenBalanceView seams.
  • 845ff685 — honest error for an incompatible on-disk schema (WalletDataIncompatible, divergent-migration case); finish the token-balance seam.
  • fb50c044 — bump platform to 35e4a2f (meta_* FK relaxed to soft-cascade triggers; public per-(identity,token) balance reader).
  • f6119de7delete the det:token_balance cache; balances now read live from upstream IdentitySyncManager via a lock-free snapshot bridge (pre-sync shows "syncing", not 0).
  • d7ac9a06promote identities + DashPay overlays to Identity scope (identities/top-ups/scheduled-votes → Identity(id); dashpay private/address_indexIdentity(owner)); Global enumeration indices; explicit cleanup (the upstream soft-cascade trigger doesn't fire for DET-only KV removals); hardened private-key Debug redaction.
  • 129d54d0 — docs synced to the per-object model.

Principle applied: DET's KV stores only DET-specific metadata; canonical object state upstream owns (token balances) is read, not duplicated.

QA (Marvin, combined tree): PASS — cargo build --all-features ✓, 598 tests (597 pass, 1 ignored) ✓, clippy -D warnings ✓, fmt ✓. Zero code defects; M-DONT-LEAK-TYPES intact.

Remaining duplicate: the platform-address cache (det:platform_addr/det:platform_sync) stays, staged behind PlatformAddressView — one-line swap once upstream exposes a public per-address nonce reader (the only open platform-side dependency; the FK relaxation and token-balance reader both landed in #3625 @ 35e4a2f).

🤖 Co-authored by Claudius the Magnificent AI Agent

After the platform-wallet migration the SPV sync UI read a hardcoded
SpvStatusSnapshot::default() — an inert, empty snapshot — so the progress
bars, phase summary, peer count and status labels stayed blank even while
chain sync was actively running.

EventBridge::on_progress already receives the full upstream SyncProgress
(per-phase current/target heights, percentage) but only collapsed it into a
coarse SpvStatus and threw the heights away. Now it publishes the live
SyncProgress into ConnectionStatus, keeping ConnectionStatus the single
source of truth for connection health.

The network and wallet screens read the live snapshot via
ConnectionStatus::spv_status_snapshot(), driving the existing determinate
per-phase progress bars (full height/target → "X / Y, NN%"). Also fixed two
related dead-default reads in the network chooser: the network selector now
disables mid-sync and the developer "Clear SPV Data" control now gates on
live status.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lklimek and others added 3 commits June 17, 2026 12:03
…QA-001)

The all-wallets sweep skips a wallet that is locked at Platform-ready
time, but no path re-ran discovery on unlock — so a password-protected
wallet was never auto-discovered for the whole session, contradicting
IDN-015, the design, and the sweep's own doc comment. Add
queue_unlocked_wallet_identity_discovery, called from
handle_wallet_unlocked after the seed is promoted:

- latch-independent: it does NOT consult identity_autodiscovery_fired
  (that guards the all-wallets sweep, not per-wallet unlock);
- Platform-ready-gated: a no-op when the masternode list is not yet
  Synced, so an early unlock defers to the upcoming sweep (the wallet is
  now open and will be included);
- prompt-free: it runs past the seed-promotion guard, so allow_prompt=true
  resolves from the session cache without a modal.

Also extract the shared open-only wallet snapshot into open_wallets(),
used by both queue_all_wallets_identity_discovery and
init_missing_shielded_wallets (DRY; gives the filter a tested entry point).

Add offline AppContext tests (QA-002, QA-003): the discovery latch is
one-shot until stop_spv re-arms it; open_wallets() excludes a locked
protected wallet; a re-discovery update preserves the user alias and the
wallet binding (the F-1 carry-over regression guard).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(QA-004/005/006/007)

- QA-004: build_qualified_identity_from_wallet now returns
  Result<_, TaskError> and propagates with `?`, so the typed chain
  (including AuthKeyUnlockRequired) survives end-to-end. Drops the
  `.to_string()` hops and the WalletInfoDeterminationFailed { detail }
  flatten — no new String-typed error seam.
- QA-005: the per-index Progress event carries a meaningful soft total
  (the current rolling-window upper bound, clamped to the hard cap) and a
  message with an honest "of about N" denominator, instead of total ==
  current (always "N of N").
- QA-006: add the network-equality guard the design required —
  upsert_discovered_identity skips the store if the active network changed
  since the scan started, so an in-flight pass never writes under the
  wrong network scope.
- QA-007: correct the gate-nudge comment — a full-channel try_send drop is
  tolerated (large channel, manual discovery available); there is no
  next-tick re-delivery.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "up to index" field lost its old "max 29" cap, so a fat-finger seed
(e.g. 4000000000) launched a full hard-cap-deep scan. Add a pure model
validator validate_search_index (single source of truth, unit-tested) that
rejects indices above MAX_IDENTITY_SEARCH_INDEX, and apply it in the
By-Wallet screen with a clear, i18n-ready error. Also surface a friendly
message for non-numeric input.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lklimek

lklimek commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

NEW since last push — 0484bcb6 post-migration identity auto-discovery (By-Wallet) + rolling gap-limit

Why: After the platform-wallet migration, a wallet's identities were only loaded if the user manually opened Load Identity → By Wallet and searched. This push discovers them automatically once Platform connectivity is ready, with a rolling gap-limit so newly-created identities are always found, and preserves user-set aliases on re-discovery.

What it does:

  • Auto-discovery on Platform-ready. Once the SPV masternode list reaches Synced (the point at which Platform/DAPI is safe to query — querying earlier gets DAPI nodes banned), DET runs By-Wallet identity discovery across all open wallets. Two triggers: a global once-per-session sweep (AtomicBool latch, cleared on SPV stop so it re-arms on reconnect) and a per-wallet, latch-independent trigger fired when a wallet is unlocked.
  • Rolling gap-limit (IDENTITY_GAP_LIMIT = 5). Scans to max(existing identity index) + 5, extending the window on each discovery so 5 empty indices always trail the highest found index (mirrors the BIP-44 address gap-limit). Bounded by a hard cap of 100. Pure decision logic in model/identity_discovery.rs (10 unit tests); one shared scan fn used by both the UI By-Wallet path and the auto-trigger.
  • Alias preservation on re-discovery. Re-discovered identities are updated in place, but DET-only metadata (alias, wallet binding) is carried forward; top-ups are unaffected (stored under a separate key).

Safety items fixed in this push (surfaced by review):

  • (High, pre-existing bug) the single-index By-Wallet search silently wiped a user's alias on every search — fixed (both single-index and gap-limit paths now carry the alias forward).
  • (High, UX hazard) a background sweep over a locked protected wallet would have popped a passphrase modal unprompted (identity-auth keys are hardened-to-leaf, so a cold cache miss triggers a secret prompt); the background path now runs allow_prompt=false and skips locked wallets, and the eventual unlock re-triggers discovery for that wallet.

Verification: headless build clean · clippy -D warnings zero · 889 lib tests pass (4 new offline guards: latch one-shot + stop_spv re-arm, locked-wallet exclusion, alias preservation, search-index cap). Design + QA reports added under docs/ai-design/2026-06-17-identity-autodiscovery-gap-limit/.

🤖 Co-authored by Claudius the Magnificent AI Agent

lklimek and others added 11 commits June 17, 2026 15:34
…watched

PROJ-027's 2026-06-11 fix wired the incoming-contact-payment detection path
(EventBridge -> detect_incoming_contact_payments) but left the watch-registration
half it depends on unwired: `register_dashpay_contact` had zero callers, so a
contact's pay-to-us addresses were never registered upstream. With nothing
watching them, SPV never emitted a transaction event for a real incoming contact
payment and the detection path stayed dormant on a live wallet — funds a contact
sent arrived at addresses DET didn't watch, invisible in the balance. tc_045
passed only because it injects the sidecar mapping and feeds synthetic outputs.

Open the faucet: `load_contacts` now registers every established (mutual)
contact's `DashpayReceivingFunds` account via `watch_established_contact_accounts`
-> `WalletBackend::register_dashpay_contact` on every contact-list load/sync.
xpub-only (no seed, no passphrase prompt — safe on a locked wallet), idempotent
(re-registering overwrites with an identical account), best-effort (a per-contact
failure is logged and skipped so contact loading still completes).

Add a backend-e2e step that drives the real registration seam against a live
wallet and asserts idempotency — the address-watching proof tc_045 cannot give.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ments

Reworks the v1 approach (a11c831): registration moves to the JIT-unlock
bootstrap path where the seed is in scope (hardened DashPay derivation needs
the private key), re-registers idempotently each unlock, skips locked wallets,
and delegates to upstream platform-wallet types for account construction.

The DIP-15 receiving path m/9'/coin'/15'/0'/owner/friend is hardened, so the
earlier approach (calling IdentityWallet::register_contact_account on a
watch-only boot wallet) silently failed. v2 derives the account xpub from a
seed-built signable wallet and inserts the ManagedCoreFundsAccount directly
(seed-bearing dual-insert, sibling to provision_identity_funding_account),
then bumps monitor_revision so dash-spv mempool sync rebuilds the peer bloom
filter with the newly derived addresses in-session.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…t query

The wallet selector summed the whole platform_address_info map (including
non-owned orphan entries from the direct sync_address_balances query), while
the addresses section summed only watched/owned addresses, producing a ~227x
inflated total. Mirror the shielded push pattern: the coordinator's
platform-address sync writes per-wallet owned-only balances into a frame-safe
AppContext snapshot that the selector reads, and the orphan-bearing direct
query is retired as a balance source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…stration

QA-021 (MED): e2e step_register_contact_receiving_accounts silently no-oped
because established_contact_pairs reads DashpayView which reads the upstream
wallet-manager's established_contacts map — only populated by dashpay_sync,
never called in tc_037. Fix: call backend.dashpay_sync() before the bootstrap
step to populate the map, then assert dashpay_receiving_account_count > 0.

QA-022 (LOW): add assertion count_after_first > 0 and idempotency check that
count_after_second == count_after_first.

QA-023 (LOW): only bump monitor_revision for newly-inserted accounts (track
len delta of dashpay_receival_accounts BTreeMap) — not on every re-run, which
would cause a spurious bloom-filter rebuild on each cold boot/unlock.

QA-024 (LOW): return newly_inserted (0 on idempotent re-run) instead of total
successful inserts, so the "Registered" info log in wallet_lifecycle.rs only
fires when the contact set actually grew.

QA-020 (MED): update doc comment and gaps.md to disclose that the 100ms
bloom-filter rebuild only fires when the mempool manager is in SyncState::Synced;
before sync completion the contacts are already in the wallet at the initial
FilterLoad sent during FiltersSyncComplete activation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…A-B-001)

On cold start the selector showed the coordinator-push total (correct)
while the per-address Platform Payment tab showed 0 because
`account_summary.rs` reads `Wallet.platform_address_info`, which was
only populated by the manual Refresh (direct-query) path. The two data
stores were updated by different triggers and could diverge permanently.

**Fix**

`EventBridge::on_platform_address_sync_completed` now emits a new
`BackendTaskSuccessResult::PlatformAddressSyncPushed` result in addition
to writing the frame-safe total-balance snapshot. `AppState::update()`
handles it by calling `AppContext::apply_platform_address_push`, which
converts each 20-byte P2PKH hash to a `dashcore::Address` (using the
active network) and writes it into every loaded wallet's
`platform_address_info`. After the first coordinator pass (~15 s) both
the selector total and the per-address account tab derive from the same
`AddressSyncResult.found` data — equality is guaranteed by construction.

The pure extraction logic lives in `summary_ok_platform_entries` (no I/O,
fully unit-testable). `PlatformAddressEntry` / `PlatformAddressUpdates`
type aliases in `model/wallet/mod.rs` keep complex nested tuple types out
of public API surfaces.

**Also addressed**

- QA-B-002: `summary_ok_platform_entries_extracts_only_successful_wallets`
  — happy-path test asserting Ok outcomes produce correct entries and Err
  outcomes are skipped.
- QA-B-006: updated `collect_account_summaries` docstring to accurately
  describe both the coordinator-push and direct-query data flows.
- QA-B-003 (accepted): sub-duff integer truncation is intentional and
  documented with an inline comment (consistent with direct-query path,
  < 0.001 DASH per address max).
- QA-B-004 (accepted): stale `platform_balances` entry on wallet removal
  documented as a known memory-leak (consistent with `shielded_balances`;
  not a display bug). See SEC-003 in the grumpy review for the shared
  cleanup track.
- QA-B-005 (accepted): cold-start zero-balance window documented as an
  intentional trade-off (no orphan inflation > brief 15 s zero).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…p (QA-B2-001/003)

**QA-B2-001 — Selector total arithmetic unified with per-address tab**

The selector total previously used truncate-then-sum:

  Σ floor(credits_i / 1000)

which can under-count by up to n−1 duffs when per-address balances have
non-zero sub-duff remainders. Classic example: two addresses × 500 credits
→ truncate-then-sum = 0 duffs; sum-then-truncate = 1 duff.

The per-address tab accumulates raw `platform_credits` into each
`AccountSummary` group and converts to duffs at display time, which is
equivalent to floor(Σ credits_i / 1000). Unifying to the same formula
means both views always agree on the true held duffs regardless of
per-address remainder distribution.

Fix: compute `total_credits = Σ credits_i`, then `total_duffs =
total_credits / 1000`. One sub-duff remainder (< 0.001 DASH) is accepted
for the wallet total.

Test added: `platform_total_duffs_uses_sum_then_truncate_not_truncate_then_sum`
demonstrates both formulae on the pathological 500+500 case and asserts the
correct result (1 duff) with the bug value (0 duffs) as a comment.

**QA-B2-003 — Deduplicated seed_hash_for lookup**

Previously `on_platform_address_sync_completed` called
`self.snapshots.seed_hash_for(wallet_id)` twice — once in the total
snapshot update loop and once when building the per-address push vec.

Restructured to a single `summary_ok_platform_entries` → `filter_map`
pass that resolves each `WalletId → WalletSeedHash` once and produces
`resolved: PlatformAddressUpdates`. Both the Mutex write (step 1) and
the channel send (step 2) iterate the same already-resolved vec.

**QA-B2-002** — left; brief frame-loop wallet-write comment is optional
and the lock-hold duration is already documented in the field docs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…atform_address_push (QA-B2-002)

Added a three-line comment in `AppContext::apply_platform_address_push` noting
that the wallet write lock is held for a pure BTreeMap update only — no I/O,
no network, no await — consistent with the codebase's existing frame-loop
write pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-025)

DET's send_contact_request_with_proof called sdk.document_create but never
updated ManagedIdentity.sent_contact_requests locally.  The upstream
add_incoming_contact_request gate only promotes a contact to
established_contacts when sent_contact_requests[peer] already exists; without
this call dashpay_sync never auto-populates established_contacts, so
established_contact_pairs() always returns empty and contact receiving-account
registration is silently skipped for every user who sends from DET.

Fix: after document_create succeeds, call WalletBackend::record_sent_contact_request
which acquires the wallet-manager write lock, finds the ManagedIdentity by
owner_id, and calls managed.add_sent_contact_request(contact_request, persister).
The persister write is best-effort (logs on failure); the state-transition was
already committed to Platform so this is non-fatal.

accept_contact_request delegates to send_contact_request_with_proof and
therefore inherits the fix automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents the root cause and resolution of QA-025 (HIGH): DET's custom
send path never wrote to sent_contact_requests, silently breaking the
auto-establishment gate in dashpay_sync for all users who sent contact
requests from DET.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…process (QA-025 Option A)

sync_contact_requests skips a received document when sent[sender] already
exists (guard: sent || incoming || established → continue). After d34ffae
populated sent[A] via record_sent_contact_request, dashpay_sync would skip
A's incoming CR, leaving established_contacts empty and registration at 0.

Fix (Option A, accept path only): in accept_contact_request, BEFORE calling
send_contact_request, parse the fetched incoming document into a ContactRequest
and call WalletBackend::record_incoming_contact_request — which calls
managed.add_incoming_contact_request(A_cr, persister) so incoming[A] is
populated.  Then when send_contact_request_with_proof fires
record_sent_contact_request → add_sent_contact_request(B→A), it finds
incoming[A] and auto-establishes established_contacts[A] in-process.

auto-establish sequence:
1. record_incoming_contact_request → incoming[A] populated
2. send_contact_request_with_proof → record_sent_contact_request →
   add_sent_contact_request(B→A) → incoming[A] found → established[A] set
3. established_contact_pairs() → [(owner, A)] → registration runs → count > 0

Only applied to the accept path where an incoming CR exists. Plain outbound
send does not have a pre-existing incoming entry and uses a separate
auto-establishment path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lklimek and others added 5 commits June 18, 2026 10:56
The Connect → Disconnect → Connect sequence was failing with
`WalletStorageError::AlreadyOpen` on the second connect.

Root cause: `WalletBackend::shutdown()` called only `pwm.shutdown()`,
which quiesces coordinators and joins the event adapter but does NOT
stop the SPV background task.  That task holds `Arc<SpvRuntime>`, which
transitively keeps `Arc<SqlitePersister>` alive via the chain:

  SpvRuntime → event_manager → BalanceUpdateHandler → wallets
    → Arc<PlatformWallet> → WalletPersister::inner

The upstream `platform-wallet-storage` crate uses a process-global
`REGISTRY` (`OnceLock<Mutex<HashSet<PathBuf>>>`) that registers a path
on `SqlitePersister::open` and removes it only on `Drop<SqlitePersister>`.
While the SPV task runs the path stays registered, so the reconnect's
`WalletBackend::new` fails with `AlreadyOpen`.

Fix: call `self.inner.pwm.spv().stop().await` **before**
`pwm.shutdown()`.  `SpvRuntime::stop()` takes the SPV client, signals
the run loop to exit, and joins/aborts the background task (with a 15 s
abort fallback).  Once the task is gone, its transitive ref chain is
released; when the `WalletBackend` itself drops the remaining structural
refs (inside `PlatformWalletManager`) drop synchronously, clearing the
REGISTRY entry before the reconnect opens the same path.

Ordering is safe: stopping the SPV producer before the coordinator
consumers is correct — no new events can arrive, and any in-flight
`sync_now` pass completes before `quiesce()` returns.

Also improve the regression test:
- Replace the misleading `Weak<WalletBackend>::strong_count` polling
  band-aid (it was watching the wrong Arc — the SPV task never held
  `Arc<WalletBackend>`) with a direct `SqlitePersister::open` assertion
  immediately after `stop_spv()`.
- Add a comment explaining the offline limitation (the task exits fast
  offline, so the test may pass without the fix in offline mode; the
  authoritative guard is the online backend-e2e suite).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…h-only fix)

`provision_identity_funding_account` was calling `kw.add_account(account_type, None)`
which internally calls `root_extended_priv_key()` on the live watch-only wallet
to derive a hardened account path.  Watch-only wallets have no private key, so
this fails with "Watch-only wallet has no private key" every time.

Fix (mirrors `register_contact_receiving_accounts`):
- Add `seed: &[u8; 64]` parameter to `provision_identity_funding_account` and
  `ensure_identity_funding_accounts`.
- When the account is not yet in the watch-only wallet, build a short-lived
  `UpstreamWallet::from_seed_bytes` (signable), derive the hardened xpub via
  `seed_wallet.derive_extended_public_key(&account_type.derivation_path(network))`,
  and pass it as `Some(account_xpub)` to `kw.add_account`.  The seed wallet
  is dropped at end of scope.
- Move the `ensure_identity_funding_accounts` call INSIDE the
  `with_secret_session` closure in all three callers (`create_asset_lock_proof`,
  `register_identity`, `top_up_identity`) so the HD seed from the session is
  available for derivation.  Each caller extracts the seed via
  `session.plaintext().expose_hd_seed().ok_or(WalletStateInconsistent)?`.
  If the scope is `HdSeed` (which all three callers enforce via `hd_scope`),
  `expose_hd_seed()` always returns `Some`; `None` is an unreachable invariant
  violation mapped to `WalletStateInconsistent`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…count provisioning (C.7)

Adds an offline deterministic test that verifies `ensure_identity_funding_accounts`
succeeds on a watch-only wallet (the live state DET always operates in).

Before commit 98bc491 the function called `kw.add_account(account_type, None)` on
the seedless upstream wallet, reaching the hardened-derivation gate and failing:
  "Watch-only wallet has no private key"

After the fix it derives the account xpub from a short-lived signable wallet built
from the provided seed bytes and calls `kw.add_account(account_type, Some(xpub))`,
which succeeds on any wallet regardless of private-key availability.

Deterministic: the live wallet is unconditionally watch-only — no timing dependency,
no network required. Also verifies idempotency (second call is a no-op).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous test (b851790) was incorrect: `register_wallet_from_seed`
creates a FULL upstream wallet with private keys, so `add_account(None)`
succeeds and the pre-fix failure was never reproduced.

Replace with a proper two-boot cold-boot test:

Boot 1 — writes wallet-meta sidecar (xpub bridge for fund-routing gate)
  and upstream persister from seed (`WalletAccountCreationOptions::Default`
  creates IdentityRegistration in the manifest, but NOT IdentityTopUp{n}).

Boot 2 (cold) — `WalletBackend::new` over a copied data dir runs
  `load_from_persistor_seedless` → upstream `Wallet::new_watch_only`: the
  wallet has its BIP44/BIP32/IdentityRegistration accounts from the
  persisted manifest but **no root private key**.
  `IdentityTopUp{3}` is absent from the manifest, so the provisioning
  branch is taken.

Verified fails-before / passes-after:
  Before fix: FAILED with
    WalletBackend { source: AssetLockTransaction(
      "Invalid parameter: Watch-only wallet has no private key") }
  After fix: PASSED in 21s offline; idempotency call also Ok.

Deterministic: cold-booted wallet has no root private key by design;
IdentityTopUp{n} is absent from the default manifest on every first use.
No network or timing dependency.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…andler

Delete the SPV-dead MN-list-diff screen (4481 LOC), all MnListTask
backend variants, CoreP2PHandler, backend-e2e tests, and every
reference across app.rs, ui/mod.rs, left_panel.rs, and the
tools-subscreen chooser.

All remaining Tools subscreens are SPV-safe, so the requires_core_rpc
gate and its unit tests are also removed.

Files deleted (5 559 LOC):
  src/ui/tools/masternode_list_diff_screen.rs
  src/backend_task/mnlist.rs
  src/components/core_p2p_handler.rs
  src/components/mod.rs
  tests/backend-e2e/mnlist_tasks.rs
  tests/backend-e2e/framework/mnlist_helpers.rs

build+clippy+tests green (998 tests, 0 failed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@lklimek

lklimek commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Session update — 5 commits pushed (d2f4fd87..1c4f4dcd)

Three fixes landed against the platform-wallet backend, each root-caused from a live testnet symptom:

Reconnect crash (09540cf8)

  • Connect → disconnect → connect failed with WalletStorage(AlreadyOpen { … platform-wallet.sqlite }).
  • Cause: WalletBackend::shutdown() never stopped the SPV run loop, so the persister path stayed registered in the process-global open-set.
  • Fix: shutdown() now calls spv().stop().await before tearing down the wallet manager.

Watch-only asset-lock failure (98bc4913, tests b8517905 + acdd1336)

  • Both "create identity" and "send funds core→identity" failed with AssetLockTransaction("Watch-only wallet has no private key").
  • Cause: wallets reload seedless (watch-only); identity funding-account provisioning called add_account(type, None), which has no key material on a cold-booted watch-only wallet.
  • Fix: provisioning now derives the account xpub from the seed (derive_extended_public_key) inside the with_secret_session scope and calls add_account(type, Some(xpub)). Regression guard uses a two-boot cold-start scenario (write persister from seed → copy data dir → cold-load seedless → assert provisioning succeeds + is idempotent).

Legacy MN-list-diff inspector + Core P2P handler removed (1c4f4dcd, −5,559 LOC)

  • The Masternode-List-Diff inspector was the only RPC-only tool and is dead in SPV-only mode. Removed the screen (~4.5k LOC), core_p2p_handler.rs, MnListTask, TaskError::P2P, all wiring, and the backend-e2e mnlist tests.
  • Verified independently: git grep clean of all removed symbols; surviving chainlock paths use only CoreTask::GetBestChainLock (no dependency on the removed task/handler).

Verification: cargo build (default + --features headless) ✅ · clippy --all-features --all-targets -D warnings ✅ zero · test --all-features --workspace998 passed, 0 failed · +nightly fmt --check ✅ clean.

🤖 Generated with Claude Code

lklimek and others added 3 commits June 18, 2026 12:52
…C/D) tests

Scenario B — `spv_reconnect_succeeds_without_already_open`
  Isolated AppContext; drives connect → stop_spv → reconnect via
  ensure_wallet_backend_and_start_spv. Asserts no AlreadyOpen and SPV
  peers found on both legs. Regression for the WalletBackend::shutdown
  fix that joins the SpvRuntime run-loop before re-opening the persister.
  Funding: none — only needs testnet egress (TCP 19999).

Scenarios C+D — `cd_cold_boot_identity_register_and_topup`
  Shared harness; creates a funded test wallet, registers an identity
  via RegisterIdentityFundingMethod::FundWithWallet (scenario C), then
  tops it up via TopUpIdentityFundingMethod::FundWithWallet (scenario D).
  Both paths call ensure_identity_funding_accounts which provisions
  IdentityTopUp{n} on a watch-only upstream wallet — the pre-fix
  add_account(type, None) path that failed with "Watch-only wallet has
  no private key" (fix: 98bc491). Deterministic offline regression
  lives in wallet_lifecycle.rs (acdd133); this e2e test validates the
  full BackendTask integration on a live testnet.
  Funding: ≥ 0.6 tDASH in framework wallet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ch event wait

The old loop polled overall_state (Synced) every 1s. In headless/MCP mode
the DAPI availability counter — gated by overall_state == Synced — is only
refreshed by the GUI frame-loop trigger_refresh call, which never runs there.
Result: ensure_spv_synced blocked for the full 600s backstop even after the
chain was fully synced (visible as MCP tool hangs on headless det-cli).

Fix: wait on SpvStatus::Running, which is push-based (set by the wallet-
backend EventBridge on_progress callback, not by the poll loop). SpvStatus::
Running is the correct gate: it means chain headers + filters are synced and
proof-verifying DAPI calls can proceed. DAPI availability is a separate
concern not required at this chokepoint.

Changes:
- connection_status.rs: add tokio::sync::watch::Sender<SpvStatus> field;
  set_spv_status() broadcasts every transition (deduped via send_if_modified);
  subscribe_spv_status() creates on-demand receivers; reset() mirrors Idle
  for network-switch coherence; begin_spv_stop bypasses (Stopping is not
  a waited-on state — intentional).
- resolve.rs: subscribe before calling ensure_wallet_backend_and_start_spv
  (no lost wakeup); borrow_and_update loop → Running=Ok, Error=Err, others
  keep waiting; tokio::time::timeout(600s) backstop preserved; remove
  SPV_WAIT_POLL_INTERVAL and OverallConnectionState import.

Unit tests (all synchronous, no network):
- subscribe_spv_status_wakes_on_running
- subscribe_after_running_reads_immediately (lost-wakeup guard)
- subscribe_spv_status_wakes_on_error
- reset_mirrors_idle_to_watch

Edge-case parity preserved (no behavior change flagged):
- Stopping is bypassed in the watch (begin_spv_stop writes atomic directly);
  waiters see Stopping only if it passes through set_spv_status — it does not,
  so they keep waiting to the backstop. Identical to the old poll behavior.
- Network switch reset → Idle sent to watch; waiters continue.
- Multiple concurrent waiters: watch broadcasts (fine).
- Error returns immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause (Marvin §B-2): WalletBackend::shutdown() stops the SPV
run-loop (releasing its Arc<SqlitePersister>), but
PlatformWalletManager::shutdown() only quiesce()s the three sync
coordinators (identity_sync, platform_address_sync, shielded_sync).
quiesce() is cancel-and-drain, NOT join — each coordinator runs on a
detached std::thread that is never joined and transiently holds
Arc<SqlitePersister>. After shutdown().await returns and the
Arc<WalletBackend> drops, those threads are still winding down, so
SqlitePersister::Drop has not run and the path stays registered in the
process-global REGISTRY. An immediate reconnect's
SqlitePersister::open(path) then fails with WalletStorageError::AlreadyOpen.

Fix: add await_persister_released() — a bounded-poll barrier that
probes SqlitePersister::open(path) after shutdown + Arc drop. A
successful open proves the coordinator threads dropped their last Arc
clone (REGISTRY is clear); the probe is dropped immediately (re-freeing
the path) and stop_spv proceeds. AlreadyOpen → sleep 20ms + retry.
Any other error (IO, schema, etc.) is not this barrier's concern —
logged and returned so the reconnect path surfaces it properly. Bounded
at 5 s / warn + proceed to never hang the disconnect path.

Structural matching only: matches WalletStorageError::AlreadyOpen { .. }
— no string parsing.

New unit test: await_persister_released_waits_for_registry_release
  — holds a SqlitePersister open (path in REGISTRY), spawns the barrier
    concurrently, proves barrier is still looping while held, releases,
    proves barrier returns promptly, proves path is free after. Fully
    deterministic — no reliance on coordinator-thread timing.

TODO(upstream): make PlatformWalletManager::shutdown() join coordinator
threads (keep JoinHandle / signal oneshot at thread end) so the barrier
is unnecessary. Filed as code comment in await_persister_released.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@lklimek

lklimek commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Pushed 1c4f4dcd..976ad0d4 (3 commits): backend-e2e coverage (B reconnect + C/D cold-boot), event-driven ensure_spv_synced (②), and the interim persister-release barrier for the reconnect AlreadyOpen regression (B-2).

The B-2 barrier is verified as an interim stopgap (closes the common coordinator-thread drain race). The durable fix — a reusable cancel + await-exit shutdown primitive (StructuredShutdown) covering both DET-owned wallet subtasks and the upstream coordinator threads, with the architecturally clean variant being keep-the-WalletBackend-alive + SPV/coordinator restart-in-place — is designed and will retire the barrier. An upstream rs-platform-wallet issue (make coordinator quiesce() join, not cancel-and-drain) will be filed. See the "Additional fixes folded in (2026-06-18)" section in the description.

🤖 Co-authored by Claudius the Magnificent AI Agent

* docs(overlay): requirements + UX spec for blocking progress overlay

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): test case specification

49 TCs covering FR-1..FR-10, NFR-1..NFR-6, and R-7 kittest checklist.
Items depending on the FR-10 concurrent-overlay architecture decision
(stack vs. replace vs. reject) and the stuck-overlay threshold (R-4)
are marked [depends on 1d] for Nagatha to resolve.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(overlay): development plan and architecture decisions

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(overlay): generic button facility + Component trait conformance

Folds in two user-mandated redesigns of the blocking progress overlay
that the prior session did not land:

Redirect 1 — generic button facility (no first-class Cancel). The overlay
knows nothing about cancellation. `OVERLAY_CANCEL_ACTION_ID`, `with_cancel`,
`CANCEL_LABEL`, and the Esc->Cancel routing are gone. A caller attaches a
generic button via `OverlayConfig::with_button(id, label)` /
`OverlayHandle::with_button(id, label)`, choosing its own opaque action id
and label. A click enqueues the id; the owning screen drains it via
`take_actions` and runs whatever logic it wants — including its own
cancellation. Esc/Tab/Enter are swallowed so a hard block is never
keyboard-dismissable.

Redirect 2 — `Component` trait conformance (placement legitimacy for
`src/ui/components/`). `ProgressOverlay` is now a struct holding
`state: Option<OverlayState>`; `Component::show` renders that instance's
card and returns `ProgressOverlayResponse` (`DomainType = String`, the
clicked action id), with `current_value()` reporting the last clicked id.
The global `render_global` path is preserved as the production entry point;
the instance `show()` is additive, mirroring `MessageBanner`.

Also: clamp the card to the window so it never runs off-screen in a narrow
window (FR-6); settle the centered card in the kittest click/focus cases
before interacting (anchored CENTER_CENTER needs a few frames to cache its
size). Docs: dev-plan gains a post-outage note superseding D-5/FR-7;
test-spec reframes the Cancel-specific cases to the generic-button model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): align D-5 and risk notes to the generic-button redesign

Rewrites the D-5 decision body and §8 risk #3 in place to drop the stale
`with_cancel`/`OVERLAY_CANCEL_ACTION_ID` framing and describe the generic
`with_button(id, label)` facility instead — consistent with the post-outage
note added at the top of the plan. Documentation only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(overlay): add ignored probe proving button-less keyboard-block gap (QA-001)

TC-OVL-029 only exercises a with-button overlay, where the first button steals
focus on raise, so typing is blocked incidentally rather than by the overlay's
input handling. This probe raises a button-less hard block over an
already-focused field (the J-2 broadcast / J-4 migration case) and asserts
FR-8 AC-8.2: typed input must not reach the field beneath.

The probe currently FAILS — render_global filters Tab/Enter/Esc only after the
beneath widgets have consumed input that frame, and a button-less overlay has
no first button to steal focus, so keystrokes leak into the focused field
beneath. Marked #[ignore] so the suite stays green; un-ignore once the overlay
claims keyboard focus / consumes text while active.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): frame-start input claim (QA-001) + clear action queue on switch (SEC-007)

Implements two QA-wave findings from the design addendum (§1 A-2, §2 A-4):

- QA-001 (HIGH) — button-less keyboard/text leak. `render_global`'s key filter
  runs at end-of-frame, one frame too late: a button-less hard block raised over
  an already-focused field let typed characters reach the field beneath (the
  J-2 broadcast / J-4 migration case). New `ProgressOverlay::claim_input(ctx)`,
  called near the top of `AppState::update` (before the panels) and gated on no
  active secret prompt, releases beneath text-edit focus and strips `Event::Text`
  plus the navigation/confirm keys (Tab, Enter, Escape, Space, arrows). The
  `#[ignore]`d probe `qa_buttonless_overlay_blocks_typing_into_focused_field_beneath`
  is un-ignored and now passes.

- SEC-007 — `clear_all_global` (network switch) now also drains the action queue,
  so a click queued just before the switch cannot survive into the new context
  and be mis-dispatched.

Adds inline unit tests: `claim_input` strips text + nav/confirm keys while a
block is up and is a no-op when idle; `clear_all_global` clears the queue.

Scope note: this is a partial pass on the QA list. The end-of-frame filter in
`render_global` is kept as belt-and-suspenders and is NOT yet gated on a secret
prompt (marked TODO at the call site — blocker #2's full fix removes it and
routes the keyboard tests through `claim_input`). Still outstanding from the
addendum / task: A-1 no-progress watchdog, A-3 keyed `OverlayHandle::take_actions`
+ `sweep_orphan_actions`, instance `Component::show` focus-trap separation,
secondary-button styling, 30s clock seam, Foreground layering, and doc sync.
Also adds Nagatha's `04-design-addendum.md` (the authoritative spec).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): QA-wave hardening — watchdog, keyed dispatch, secondary buttons, Foreground, focus separation

Implements the design addendum (§1/§2) plus the rest of the QA fix list and the
three cross-finding reconciliations. All on top of the earlier claim_input/SEC-007 pass.

Addendum §1 (safety-valve / A-1):
- 120 s no-progress watchdog: STUCK_OVERLAY_WATCHDOG_THRESHOLD, OverlayState
  { last_progress_at, watchdog_logged }, watchdog_tripped() clock seam, escalated
  STUCK_WATCHDOG_REASSURANCE (replaces the soft line, never stacks), one-shot
  tracing::error! (no flaky time-based panic). last_progress_at is bumped on a real
  content change, reusing log_overlay_state's change detection, so a progressing
  multi-step flow never trips it.

Addendum §2 (action-dispatch / A-3, SEC-007/A-4):
- Actions are keyed: OverlayAction { key, action_id }. OverlayHandle::take_actions()
  drains only its own ids (FIFO); clear() purges its key's pending ids; the static
  take_actions is demoted to sweep_orphan_actions() (dead-owner ids only). app's
  drain logs orphans. clear_all_global already clears the queue (SEC-007).

Reconciliations (lead brief):
1. SEC-004/F-1 — claim_input is gated on no active secret prompt at the app site,
   and render_global no longer strips keyboard at all (the gated claim_input is the
   sole keyboard block); release-beneath-focus is button-less only (stop_text_input
   clears ANY focus, which would steal a button's focus otherwise).
2. QA-002 — claim_input strips Space (and render_global's removal means the kittest
   keyboard path runs through claim_input). TC-OVL-044 now also presses Space.
3. QA-003 — render_card/render_buttons take trap_focus; the instance Component::show
   passes false so it never seizes the host screen's focus or installs the lock.

Rest of the list:
- SEC-002: overlay dim/sink/card raised to Order::Foreground (above ComboBox /
  autocomplete / SelectionDialog popups); passphrase modal also raised to Foreground
  so it stays above the overlay (R-1, TC-OVL-048).
- F-3/4/7: ButtonStyle { Primary, Secondary }, with_secondary_button on
  OverlayConfig/OverlayHandle/instance, ConfirmationDialog-style right_to_left layout
  (primary right, secondary left).
- SEC-005: corrected the Send+Sync note to the real invariant (UI-thread-only ops).
- F-6: Elapsed uses a named placeholder. SEC-006: log-content doc note on show_global.
- QA-007: instance clear() makes the empty-response path reachable.
- QA-008: TC-OVL-013b asserts elapsed >= 2s; TC-OVL-021 also bounds vertically.

Tests: un-ignored qa_buttonless probe; new inline tests (watchdog threshold/clock-reset/
one-shot, keyed FIFO/isolation/orphan-sweep, QA-007); new kittest reconciliations
(render_global keeps keyboard for the prompt; instance show leaves host focus navigable).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): sync requirements/dev-plan to the shipped design + add UX user story

- 01-requirements-ux.md: add a supersession callout flagging the Cancel-era
  items now overtaken by the generic-button + watchdog + claim_input redesign
  (FR-7, AC-7.3/7.4, NFR-3 AC-3b, AC-8.4, AC-10.5, J-1/J-2/J-3, §6.3-6.5), pointing
  at the dev-plan post-outage note, the addendum, and the code as source of truth.
- 03-dev-plan.md: drop OVERLAY_CANCEL_ACTION_ID from the §2 re-export row; mark the
  §3 API block superseded (real surface is with_button/with_secondary_button, keyed
  take_actions/sweep_orphan_actions, OptionOverlayExt::raise, the watchdog); fix the
  §4.1 drain comment; update the §9 D-4/D-5 rows.
- user-stories.md: add UX-001 (blocking please-wait overlay; cannot fire a
  conflicting second action), tagged across personas, [Implemented].

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(overlay): close re-QA coverage residuals RQ-1/RQ-2/RQ-3

RQ-1 (security) — the app.rs secret-prompt gate had no test; deleting
`if self.active_secret_prompt.is_none()` left every test green. Extracted the
gate into `AppState::claim_overlay_input` (called from `update`) and added a
`#[cfg(feature = "testing")]` seam (`AppState::test_set_secret_prompt_active`,
`ActivePrompt::test_stub`). New AppState-level kittest
`rq1_appstate_secret_prompt_gate_keeps_prompt_typeable_over_overlay` drives the
REAL `update()` loop with a prompt active over a button-less overlay and asserts
the prompt input keeps focus AND accepts typed text (types a passphrase + Enter,
the prompt submits and closes). Deleting the gate makes `claim_input`
(button-less → `stop_text_input`) steal focus and strip the keys, failing both
assertions. Extended `tc_ovl_048` to assert prompt interactivity (submit button
renders + input holds focus), not just visibility.

RQ-2 — added a `#[cfg(feature = "testing")]` clock seam `OverlayHandle::backdate`
(shifts `created_at` + `last_progress_at` into the past). New kittest
`tc_ovl_047b_threshold_reveals_via_clock_seam` renders past 30 s and 120 s and
asserts: the soft "This is taking longer than usual." line + Elapsed
force-reveal, then `STUCK_WATCHDOG_REASSURANCE` REPLACING the soft line (never
both) — the addendum §1 obligation that was previously only flag-checked.

RQ-3 — reframed the `tc_ovl_047` doc comment (the escape-hatch button is a
deliberate v1 non-feature per addendum §1, not a deferred T7 TODO); added a
"(superseded)" note to 01-requirements-ux.md's "what to reuse" list where it
still cited `with_cancel`/`with_action`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): close QA residuals — README catalog entry, requirements Cancel reconciliation, T7 TODO

Post-gate cleanup on the blocking progress overlay (gate green):

- README: add a ProgressOverlay row to the Feedback Components table,
  covering show_global/render_global, with_button(id, label), the 120s
  watchdog, and companions OverlayConfig/OverlayHandle/OptionOverlayExt/
  ProgressOverlayResponse.
- 01-requirements-ux.md: reconcile the remaining literal-Cancel acceptance
  criteria (intro line, AC-7.3, AC-8.4, the §6.5 "Visible, cancelable" row,
  R-3) to the shipped generic-button model, matching the top supersession
  callout — Esc/Tab/Enter/Space are swallowed and there is no built-in Cancel.
- app.rs: mark drain_overlay_actions with a TODO(T7) recording that an overlay
  button can only stop waiting (not abort) until the BackendTask system gains
  cooperative cancellation; until then the 120s watchdog (see
  progress_overlay.rs) bounds every block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(overlay): hard-block the UI during startup/Connect SPV sync

Raises the blocking ProgressOverlay while a startup- or Connect-initiated SPV
sync runs, and lowers it when the chain becomes usable (Synced) or fails (Error).

Honors the overlay's C1/C2 caller contract. SPV sync is UNBOUNDED — it can wait
indefinitely for peers — so a button-less block would trap the user. The block
therefore carries a "Continue in the background" escape
(`SYNC_CONTINUE_BACKGROUND_ACTION`); clicking it lowers the block while sync
proceeds safely in the background (read-only — nothing is stranded). C1: the
block also always lowers on its own at a terminal state.

- `AppState`: `sync_overlay`/`sync_block_active`/`sync_overlay_dismissed` fields;
  armed on boot auto-start and on the manual `StartSpv` (Connect); reset on
  network switch so the handle never goes stale.
- New per-frame `update_sync_overlay` driver (called beside
  `update_connection_banner`) applies a pure, unit-tested policy `sync_block_step`
  (Block / Release / Idle) and drains the escape click.
- Pure decision + descriptions are i18n-clean single sentences.

Tests: 6 inline unit tests of `sync_block_step` (inactive→Idle; active+not-usable
→Block; terminal→Release for both dismissed states; dismissed→Idle; stable action
id; sentence descriptions). New `#[cfg(feature = "testing")]` integration kittest
`task9_sync_overlay_blocks_lowers_on_synced_and_on_escape` drives the real
`update_sync_overlay` against a forced connection state: asserts the block raises
while connecting, lowers on Synced (C1), and lowers on the escape click (C2 — user
never trapped). Adds `ConnectionStatus::set_overall_state` + AppState
`test_activate_sync_block`/`test_drive_sync_overlay` test seams.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(overlay): align SPV-sync block to the approved spec

Reworks the SPV-sync overlay wiring (introduced in the previous commit) to the
user-approved design. Net behaviour: while the active context is Connecting or
Syncing the overlay hard-blocks the UI, lowering when the chain becomes usable
(Synced), fails (Error), or drops (Disconnected).

Changes vs the first cut:
- Keyed purely to the live connection state + a per-episode dismissal flag — drops
  the separate "armed" flag, so any sync episode (startup, Connect, or reconnect)
  blocks. Pure policy renamed `sync_block_step` -> `spv_block_step`
  (Block/Release/Stand); Disconnected now Releases + re-arms.
- Escape is now an always-visible SECONDARY button "Continue in the background"
  (id renamed `spv:sync:continue_background`); fields renamed to
  `spv_overlay`/`spv_overlay_dismissed`; method renamed `update_spv_overlay` and
  driven BEFORE `update_connection_banner`.
- Live content: description = `spv_phase_summary(progress)` (else a generic
  connecting line), plus a "Step N of 5" counter via new
  `connection_status::spv_phase_step` (Headers=1 … Blocks=5). Raises once per
  episode, then updates in place.
- Suppresses the redundant Connecting/Syncing connection-banner text while the
  overlay is up (don't double-shout); keeps Error/Disconnected banners.

C1/C2 contract preserved: SPV sync is UNBOUNDED, so the escape (lower while sync
continues safely in the background — read-only, nothing stranded) guarantees the
user is never trapped; episode-ending states always release.

Tests updated: 4 inline `spv_block_step` unit tests; the integration kittest
`task9_spv_overlay_blocks_lowers_on_synced_and_on_escape` now also asserts the
secondary escape button, re-raise for a fresh episode, no re-raise within a
dismissed episode, and re-raise after the episode ends. Test seams renamed to
`AppState::test_drive_spv_overlay` (+ `ConnectionStatus::set_overall_state`).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): reconcile SPV-sync block decision (F-SPV-1) + phase-step test (F-SPV-2)

F-SPV-1 — the user-authorized SPV-sync hard-block + always-visible "Continue in
the background" escape contradicted three docs written for the standalone
overlay. Reconcile the docs to the decision (the feature is correct; the docs
were stale) so a future dev does not "correctly" remove the button per old docs:

- docs/user-stories.md: carve out the SPV-sync exception in UX-001's "no
  background/dismiss button" guarantee, and add UX-002 — the blocking SPV-sync
  overlay with the always-on "Continue in the background" escape (tagged across
  personas, [Implemented]).
- 01-requirements-ux.md §5: supersession note — the user chose to block the
  startup/Connect get-connected sync; the power-user concern is mitigated by the
  escape (sync is read-only and safe to background); this is the overlay's first
  adopter.
- 04-design-addendum.md A-1: record that A-1's "ship NO dismiss/background button
  in v1" was scoped to unsafe-to-interrupt ops whose safety rests on boundedness;
  for the unbounded-but-read-only SPV-sync adopter the C2 "never trap the user"
  guarantee is met by the always-on escape, which must NOT be removed.

F-SPV-2 — the granular phase progress (spv_phase_summary description +
"Step N of 5" via spv_phase_step) was already wired in the previous commit; adds
a unit test locking the active-phase → step mapping and the summary text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): scope SPV block to user-initiated sync + de-jargon copy (F-SPV-A/B/E)

F-SPV-A (sev-2/1 regression, introduced by the prior refactor) — the SPV block
fired on ANY Connecting/Syncing, so an ambient mid-session reconnect, or the SPV
engine flipping Synced→Syncing as it processes each new block (event_bridge
on_progress maps !is_synced() → Syncing), would hard-block a working user.
Re-introduce a startup/Connect-SCOPED arming gate:
- `spv_block_armed` flag, armed only on boot auto-start and the Connect button
  (AppAction::StartSpv); reset on network switch.
- `spv_block_step(armed, dismissed, state)`: !armed → Idle (never block); armed +
  Synced/Error → Disarm (lower + clear armed); armed + Connecting/Syncing/
  Disconnected → Block (or Stand if dismissed). Once disarmed, ambient sync never
  re-blocks until the next user-initiated episode.

F-SPV-B (sev-2) — the block description showed blockchain jargon ("Headers:
12345 / 27000 (45%)") to the Everyday User. Replace with plain complete
sentences ("Connecting to the Dash network." / "Syncing with the Dash network.");
keep the jargon-free "Step N of 5" counter (via spv_phase_step) as the
determinate granularity. spv_phase_summary stays (still used by wallets_screen);
it is just no longer the overlay description. UX-002 acceptance criterion updated
to stop enshrining the jargon.

F-SPV-E (sev-4) — AppAction::StartSpv set an orphaned Info banner whose handle was
dropped (could not be cleared by the overlay's banner suppression). Dropped it;
the block conveys "connecting" and the error path still surfaces via replace_global.

Tests: spv_block_step unit tests rewritten around the arming gate —
`unarmed_never_blocks` is the regression guard (ambient sync never blocks);
`armed_terminal_state_disarms`; jargon-free-description test. The integration
kittest is rewritten to `task9_spv_overlay_armed_scope_disarm_and_escape`: an
un-armed Connecting does NOT block, an armed one does, Synced disarms, ambient
sync afterward does NOT re-block, the escape lowers without re-raising, and only a
fresh armed episode re-blocks. New `AppState::test_arm_spv_block` seam.

is_synced() finding: `EventBridge::on_progress` (event_bridge.rs) does map
`!is_synced()` → `SpvStatus::Syncing`, so overall_state CAN flip Synced→Syncing on
per-block catch-up — the arming gate makes that harmless (disarmed after the
initial episode).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): address review findings — deterministic elapsed test, SPV phase-count constant, input-claim hardening, doc drift

- Replace the 2.1s wall-clock sleep in tc_ovl_013b with the deterministic
  `backdate` clock seam (gated behind `testing`), mirroring tc_ovl_047b — zero
  wall-clock waiting; asserts the elapsed readout counts up to a concrete 2s.
- Add `SPV_SYNC_PHASE_COUNT` next to `spv_phase_step` as the single source of
  truth for the "Step N of 5" total; reference it at both app.rs call sites and
  guard the max step with a `debug_assert!` so it cannot silently drift.
- Delete the misplaced orphan-sweeper paragraph from `claim_overlay_input`'s doc
  (it belongs to `drain_overlay_actions`, which already carries it).
- Reconcile the `Order::Middle` → `Order::Foreground` doc drift: supersession
  callouts in the dev plan §4.2/§4.3 and the kittest module doc, citing SEC-002.
- Drop the dead `CONNECTING_MSG`/`replace_global` swap in the StartSpv failure
  path (the "Connecting…" banner was removed in F-SPV-E) for a plain
  `set_global(...).with_details(e)`; fix the now-stale comment.
- Extend `claim_input`'s per-frame strip to also drop Backspace, Delete, Home,
  End, PageUp, PageDown and the Copy/Cut/Paste clipboard events; add a kittest
  locking the new classes via event survival + the field-beneath contract.
- Strengthen the SEC-001 lifecycle rustdoc on `show_global` /
  `show_global_spinner_only` (button-less blocks need a frame-driven reconcile
  owner or an escape; the watchdog only logs).
- Nits: UX-001 "developer warning" → "developer error"; "while a armed" →
  "while an armed". Add deferred TODOs (SEC-002-pointer, SEC-001, RUST-006).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): close one-frame SPV block gap, fix slow-phase watchdog, align API to MessageBanner

Three changes to the blocking progress overlay + SPV-sync hard-block:

A — Close the one-frame interactive gap. `update_spv_overlay` now runs at the
top of `AppState::update`, BEFORE `claim_overlay_input`, the visible screen
`ui()`, and `render_global`. A freshly-armed episode therefore raises, claims
input, AND paints on the same frame; previously the block was raised only after
`render_global`, leaving the frame right after Connect/arming fully interactive
(effective at frame N+2). The connection banner still reads the block state
afterwards, so its Connecting/Syncing suppression is unchanged.

B — Stop the 120s no-progress watchdog from falsely escalating on slow phases.
A single SPV phase running >120s (e.g. Headers on a slow link) wrote a constant
(description, step), so `log_overlay_state` never reset `last_progress_at` and
the watchdog tripped — swapping to the STUCK copy and firing the one-shot
dev-error, the exact false signal the SPV escape was meant to avoid. A hidden,
monotonic `progress_token` (step in the high 32 bits, advancing height in the
low 32) is threaded from `ConnectionStatus` into the overlay; an advancing token
resets the watchdog even when the shown (description, step) is unchanged. The
token is NEVER rendered — copy is byte-for-byte unchanged and the jargon-free
test stays green. Distinct from TODO(SEC-001), which is left in place.

C — Align the overlay public API toward MessageBanner so migrating from the
banner is a name-for-name swap. One-way (overlay → banner), no capability loss:
  with_button(id, label)            -> with_action(label, action_id)
  with_secondary_button(id, label)  -> with_secondary_action(label, action_id)
  show_global(...)                  -> set_global(...)  (return type kept)
  show_global_spinner_only(...)     -> set_global_spinner_only(...)
`OptionOverlayExt::raise` keeps its name: renaming to `replace` (the banner
analogue) would be shadowed by the inherent `Option::replace`, so every
`slot.replace(ctx, desc, config)` call would fail with E0061 (verified). A doc
note records why. `render_global`, `claim_input`, the watchdog, `OverlayConfig`,
and all handle progress methods are untouched. Rustdoc, the README catalog row,
and the design-doc API references are updated to the new names; the banner's own
`MessageBanner::show_global(ui)` render path is left alone.

Tests: new real-AppState kittest for the one-frame gap (same-frame paint), new
backdate kittest + unit tests for the token-driven watchdog reset, and a
`spv_progress_token` monotonicity unit test. fmt + clippy clean; kittest 138
passed; lib 926 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(overlay): keyboard-reachable escape for the SPV hard block (QA-002 refinement)

Resolves the TODO(RUST-006) marker: the SPV-sync hard block's "Continue in
the background" escape was mouse-only, stranding keyboard-only / assistive-tech
users behind the UNBOUNDED block. Hard blocks strip Enter/Space every frame
(the deliberate QA-002 rule, guarded by TC-OVL-044), so the escape could not be
activated by keyboard.

Add a per-block opt-in — `OverlayConfig::with_keyboard_escape(action_id)` and
`OverlayHandle::with_keyboard_escape(action_id)` — that designates ONE action as
the single keyboard-reachable escape. The general rule is unchanged: a block
with no designated escape stays fully keyboard-blocked.

- claim_input: when the active block designates an escape AND that escape button
  is *confirmed* to hold focus (its egui id was recorded by last frame's
  render_buttons and still matches the focused widget), Enter/Space pass through;
  every other key, and the raise frame (focus not yet confirmed), stays stripped.
  So the passthrough can never reach a widget beneath.
- render_buttons: for an opt-in block, pin focus to the designated escape (match
  by action id) — re-requested every frame and locked — and record its id for the
  claim_input gate.
- SPV adopter (update_spv_overlay): mark "Continue in the background" as the
  keyboard escape; it remains unconditionally present whenever the block is up.

Tests (egui_kittest — the reliable check for input/focus):
- TC-OVL-051/052: Enter / Space activate the focus-pinned escape.
- TC-OVL-053: a TextEdit beneath never receives Enter; Tab and a backdrop click
  cannot move focus off the escape.
- task9_spv_escape_is_keyboard_activatable: the REAL SPV block lowers on Enter.
- TC-OVL-044 and the keyboard-block tests stay green (general rule intact).
- Unit tests for the opt-in API + the claim_input safety gate.

Docs: QA-002 design note + NFR-3 accessibility ACs, test-spec, user story UX-002,
and the public rustdoc updated to state the refined rule.

cargo +nightly fmt: clean. clippy --all-features --all-targets -D warnings: 0.
kittest --all-features: 142 passed. lib --all-features: 928 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): activate keyboard escape at frame start (SEC-001, SEC-002)

The opt-in keyboard escape used to "keep" Enter/Space in `i.events` only
while the escape button was confirmed-focused, and `render_buttons`
re-requested that focus every frame. Two bugs fell out of it:

- SEC-001: `render_global` runs before `render_secret_prompt`, so the
  per-frame focus re-request stole focus from a passphrase modal raised
  above the block — the field went un-typeable and Enter fired the escape
  instead of submitting. Realistic on a cold-start migration prompt over
  the startup SPV auto-sync block.
- SEC-002: the kept Enter/Space reached the beneath screen's `ui()` (which
  runs before `render_global`), so a focus-independent global key handler
  beneath (info_popup / selection_dialog / address_input) observed the key
  — a single Enter/Space leaked through the "hard" block.

Unified fix: move escape activation to frame start in `claim_input`. When
a block designates a `keyboard_escape_action` and Enter/Space is pressed,
enqueue that action directly (the same queue a click feeds) and strip the
key with all the others. Activation no longer needs the button focused
(SEC-001) and the key never survives to a widget beneath (SEC-002). Focus
on the escape is now purely visual and is suppressed while a secret prompt
is active — `render_global` takes a `secret_prompt_active` flag mirroring
the `claim_overlay_input` gate. A non-opted block still strips Enter/Space
and activates nothing; Esc still never dismisses.

Drops the now-dead `escape_focus_id` field and confirmed-focus logic.

Also in this rework:
- SEC-003 residual: TODO documenting the narrow constant-height >120s
  watchdog false-alarm (benign log + accurate copy, no abort) pending a
  coarser SDK liveness signal.
- RUST-001: `keyboard_escape_action.clone()` -> `as_deref()` in
  render_buttons (no per-frame String alloc).
- RUST-002: corrected the stale `log_overlay_state` call comment to note
  the watchdog also resets on a hidden progress_token advance.
- PROJ-001: render_global rustdoc now cross-references
  `MessageBanner::show_global` for the set_global/render_global vs
  set_global/show_global asymmetry.

Tests (egui_kittest, the authority for input/focus):
- sec001_* drives the real AppState loop with an escape block beneath an
  active secret prompt: the prompt keeps focus, Enter submits it, the
  escape action is never enqueued.
- sec002_* a focus-independent `key_pressed(Enter)` sentinel beneath an
  escape block never fires; the Enter is stripped and routed to the escape.
- Replaced the obsolete confirmed-focus unit test with one asserting the
  frame-start enqueue + strip. TC-OVL-044/048/051/052/053, rq1, and
  task9 escape tests stay green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(overlay): document hidden progress_token watchdog reset; pin cross-phase token invariant

- RUST-003: strengthen `spv_progress_token_advances_with_height_and_is_monotonic`
  with a cross-phase assertion — a later phase (masternodes, step 2) at
  height 0 must out-rank an earlier phase (headers, step 1) near the u32
  ceiling, pinning the high-bits-dominate invariant the test name claims.
- DOC-001: design-addendum §1 now documents the hidden progress_token
  watchdog reset — `last_progress_at` resets on a shown (description, step)
  change OR a token advance; the token is never rendered and its reset is
  decoupled from the once-per-content-change log (NFR-5). Corrected the
  now-wrong "only when content changes" instructions and the test note, and
  the superseded confirmed-focus escape description.
- DOC-002: dev-plan §3 superseded block — dropped `with_action` from the
  "there is no ..." list (it is the real shipped builder), resolving the
  self-contradiction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): keep escape button mouse-clickable after a backdrop press

The blocking ProgressOverlay rendered its dim/pointer sink and its content
card as peer Order::Foreground areas. egui auto-raises any interactable Area
to the top of its Order on a pointer press (area.rs bring-to-front), so a
single click on the dim backdrop floated the full-window sink above the card
and permanently buried its buttons beneath the click-absorbing sink. For the
unbounded SPV-sync block that meant the "Continue in the background" escape
became unclickable with the mouse — force-quit was the only exit.

Pin the card as a sublayer of the sink (ctx.set_sublayer): egui places a
sublayer directly above its parent after the per-frame order sort, so the
card-above-sink z-order now holds by construction, immune to the bring-to-
front race. The sink still blocks every widget beneath, and the secret-prompt
window still wins above the overlay.

Add TC-OVL-054: press the backdrop, then click the escape at its own position
and assert the action enqueues. It fails before this change (the sink eats the
click) and passes after. Existing button-click tests never press the backdrop
first, so they missed this path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(overlay): arm the SPV-sync block on the post-onboarding auto-start path

The blocking SPV-sync overlay only shows for an *armed* episode
(spv_block_step returns Idle when !armed). Two production paths armed it
— boot auto-start (via the constructor: spv_block_armed = boot_auto_start_spv)
and the Connect button (AppAction::StartSpv) — but the third did not:
AppAction::OnboardingComplete calls try_auto_start_spv(), which spawned
ensure_wallet_backend_and_start_spv WITHOUT setting spv_block_armed.

So a fresh user who enabled auto-start and then finished onboarding
(onboarding_completed was false at boot, so boot_auto_start_spv was false
and the flag stayed false) would sync with no blocking overlay at all —
exactly the journey the overlay exists to cover.

Fix: arm the block inside try_auto_start_spv when the start actually
fires (spv_block_armed = true; spv_overlay_dismissed = false), mirroring
AppAction::StartSpv. This is the single correct arming point for that
caller — the method takes &mut self now, and the active context is cloned
up front so the mutation does not alias the borrow. Boot auto-start is
untouched (it arms via the constructor and inlines its own start).

Test: fspv_a_onboarding_auto_start_arms_spv_block drives the REAL
try_auto_start_spv via a testing-only seam and asserts both the armed
flag flips and that an armed Connecting sync then raises the overlay.
Verified the test fails when the arm is removed and passes with it.

Verified: cargo clippy --all-features --all-targets -D warnings (0
warnings), cargo test --test kittest (146 ok).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants