loopin: cancel the probe invoice when initiation fails

[starius]

Before a Loop In, the server probes the route to the client by paying a probe invoice whose preimage nobody knows. The client's probe watcher cancels the invoice once the probe HTLC is accepted, which fails the probe back. The server probes within the NewLoopInSwap call, and the client only accepts a successful call if the probe reached it, so a swap never starts before its probe is done.

This PR is about initiations that fail. The watcher stops when the call returns, but a failed initiation does not mean that the probe is finished:

• The client gave up while the server was still probing, for example after its 30 second server call timeout or on shutdown.
• The server gave up on a probe that is still in flight and returned an error.
• The call succeeded, but the probe had not reached the client, which fails the initiation with "probe error".

A probe HTLC that arrived after that was accepted by the still open hold invoice and held. lnd does not expire an accepted hold invoice by time. It only cancels the HTLC shortly before its CLTV expiry: about 60 blocks later with lnd's defaults, an 80-block invoice CLTV delta and an 18-block hold expiry delta. The server's payment stayed pending along the route for those roughly ten hours.

The client now cancels the probe invoice whenever initiation fails after the invoice was created, including on a probe error. A probe HTLC that arrives later is failed right away, and one that already arrived is failed back.

• The cancellation uses a context that is detached from the possibly canceled initiation context, with a 10 second timeout.
• The probe invoice can never be settled, and canceling an already canceled invoice succeeds, so this is always safe.
• If no probe was ever sent, for example because the server rejected the request before probing, the cancellation only cleans up the unused invoice.

This matters only when a probe outlives the initiation call. It protects the liquidity of the server and the routing nodes, not the user's funds.

Tests

• TestLoopInCancelsProbeInvoiceOnInitiationFailure: the probe invoice is canceled when the server call fails, and when the call succeeds without the probe having reached the client.
• TestAwaitProbeCancelInvoiceUsesLiveContext

Not covered

The cancellation is attempted once. If it fails, it is logged. The open invoice then expires after its hour, and a probe HTLC that arrives before that is held as described above. Retrying is a separate follow-up.

Pull Request Checklist

• Add an entry to docs/release-notes/release-notes-next.md, or apply the
  no-changelog label (required by CI)

[hieblmi]

To catch all future cases where we need to cancel we clould also...

initiationSucceeded := false
defer func() {
	if !initiationSucceeded {
		cancelProbeInvoice(
			globalCtx, cfg.lnd.Invoices, probeHash,
		)
	}
}()

and in the end

initiationSucceeded = true

return &loopInInitResult{
	swap:          swap,
	serverMessage: swapResp.serverMessage,
}, nil

[starius]

To catch all future cases where we need to cancel we clould also...

initiationSucceeded := false
defer func() {
	if !initiationSucceeded {
		cancelProbeInvoice(
			globalCtx, cfg.lnd.Invoices, probeHash,
		)
	}
}()

and in the end

initiationSucceeded = true

return &loopInInitResult{
	swap:          swap,
	serverMessage: swapResp.serverMessage,
}, nil

Great improvement! Applied.

Note that it could cancel the invoice in some other failing paths where it is not needed, before the server has seen the invoice or after the watcher has already canceled it. Another attempt can return "no such invoice" if the server runs with --gc-canceled-invoices-on-the-fly option and removes the invoice after the first cancel.

So I decided to apply the following rule: memorize (in RAM) if the invoice was successfully cancelled and skip the second attempt in that case. I made a shared function for that. This complicated the code a bit, but I think it is still manageable.

@hieblmi Please take another look!