CLI: a create that times out client-side still creates the session, and the retry is invisible until it is too late

[jiashuoz]

Separate from #82 — this one is about recovery, not ergonomics.

What happened

rainier new --detach returned a client-side timeout:

$ rainier new --name probe --detach -- /bin/sh -c '...'
timeout; check server connectivity and rerun rainier status

rainier status reported the cell healthy and the runner connected, and rainier ls immediately afterwards did not list a session by that name. Both signals point the same way: the create did not land.

So I retried with the same name. It succeeded.

Some minutes later, addressing the session by name failed:

$ rainier attach probe
ambiguous name "probe" matches 2 sessions: sess_aaaa1111... , sess_bbbb2222... — use the session id

The timed-out create had landed. Two sessions were running the same command, and I had been billed for a runner slot I did not know existed.

What is already right

The ambiguity refusal is documented and deliberate:

a name … resolved against your sessions; a name matching more than one is refused, with every match's id listed, rather than guessed

That behavior is correct and is what surfaced the problem at all. This issue is not a request to change it, and not a request to enforce name uniqueness — the refusal is the safer design.

The actual gap

A timed-out new is indistinguishable from a failed new at the moment you have to decide what to do about it:

1. The client reports a timeout, which reads as failure.
2. rainier ls right afterwards does not show the session. (The starting state exists — "queued or booting" — so this may be a race with registration, or ls may need --all here. Either way, the check an operator would reach for came back empty.)
3. The only safe recovery — retry — is precisely what creates the duplicate.

This matters much more for scripted and agent-driven use than interactive use. A human notices two sessions in ls eventually. A retry loop creates one runner-consuming session per attempt, and only discovers it later when name addressing breaks.

Possible directions

• Idempotency on create. An optional client-supplied key, so a retry with the same key returns the existing session rather than making a second one. Most robust, and it makes safe retry loops possible.
• Make the timeout honest. If the request may have been accepted, say so — "timed out waiting for the session; it may still have been created, check rainier ls --all before retrying" — rather than a bare timeout, which reads as "nothing happened".
• Make the post-timeout check reliable. Whatever state a just-accepted session is in, rainier ls (or a documented --all) should show it, so "did it land?" has a trustworthy answer.

The second is cheap and removes most of the harm on its own: the failure here was not that a duplicate is possible, but that every signal available pointed at "safe to retry".

Repro sketch

Hard to force deterministically since it needs a real client-side timeout against a healthy cell. Observed once with --detach and a /bin/sh -c command; the session was created and running normally, only the client's wait expired.

(Identifiers above are synthetic.)

[AmirF194]

Checked this against current main (ca7c6cc): a new failure can't produce the "timeout; check server connectivity..." text you quoted any more, that string only comes out of status/doctor now. #70 already routes a create failure through "create was not confirmed; inspect rainier ls, or retry the same request with --idempotency-key %s".

What's still open: that flag isn't automatic. createSession mints a fresh random key whenever --idempotency-key is empty, so a plain retry, no flag, sends a different key each time and the server has no way to tell it's the same intent. Wrote a quick test against createSession confirming two bare retries get two different keys.

So direction 2 from your list is done, direction 1 only works if you remember to pass the key by hand. Want a deterministic default key (name+cmd+env), or is that too surprising for a re-create under the same name after a delete?