Skip to content

fix(tty,vt,console): prevent deadlocks and hangs in terminal I/O#3420

Open
michaellukashov wants to merge 1 commit into
elfmz:masterfrom
michaellukashov:prevent-deadlocks-and-hangs-in-terminal
Open

fix(tty,vt,console): prevent deadlocks and hangs in terminal I/O#3420
michaellukashov wants to merge 1 commit into
elfmz:masterfrom
michaellukashov:prevent-deadlocks-and-hangs-in-terminal

Conversation

@michaellukashov

Copy link
Copy Markdown
Contributor

Summary

Fixes for six deadlock and indefinite-hang scenarios in the TTY backend, VT
shell, and console input layer. Each fix addresses a distinct root cause where
a thread could block indefinitely, spin-loop at 100% CPU, or hold a mutex
across a blocking modal call.

Changes

1. Console input: dispatch inter-thread calls on NOOP_EVENT

File: far2l/src/console/keyboard.cpp

Problem: GetInputRecordInner peeked NOOP_EVENT (written by
InterThreadCall to wake the input queue) but did not recognize it. The peek
returned true, the loop broke out of the inner block, WaitConsoleInput
returned immediately (NOOP still pending), and the outer loop peeked the same
NOOP again — a tight spin loop burning 100% CPU. DispatchInterThreadCalls()
was never called, starving all inter-thread delegates (including the VT input
reader during mouse scroll).

Fix: Added a NOOP_EVENT handler inside the peek block: consume the NOOP
with ReadInput, dispatch pending inter-thread calls, check for pending Ctrl
events, and continue the loop to process the real event now in rec.

2. TTY WriterThread: skip tcdrain() during shutdown

File: WinPort/src/Backend/TTY/TTYBackend.cpp

Problem: tcdrain(_stdout) blocks until all buffered output is
transmitted. When the terminal is disconnected (_deadio) or the process is
exiting (_exiting), the kernel tty buffer can never drain, causing
pthread_join on the writer thread to hang forever during backend teardown.

Fix: Guard tcdrain with if (!_exiting && !_deadio).

3. TTY Far2lInteract RPC: timeout and shutdown awareness

File: WinPort/src/Backend/TTY/TTYBackend.cpp

Problem: Far2lInteract called pfi->evnt.Wait() — an indefinite block.
If the far2l peer disconnected mid-RPC, the ReaderThread hung forever.
Additionally, the method did not check _deadio at entry.

Fix: Replaced with a TimedWait loop (10 s total timeout, 500 ms poll
granularity). Checks _exiting || _deadio on entry and on each poll iteration,
so terminal disconnect is detected within 500 ms. The 500 ms interval balances
shutdown responsiveness against loop overhead.

4. TTY signal handlers: correct SIGTSTP/SIGCONT restoration swap

File: WinPort/src/Backend/TTY/TTYBackend.cpp

Problem: In the WinPortMainTTY cleanup path, SIGCONT was restored to
the old SIGTSTP handler and vice versa. If the original handlers were both
SIG_DFL, this was functionally harmless by coincidence (SIG_DFL dispatches
by signal number). If the parent process had installed non-default handlers,
the swap would cause incorrect terminal state manipulation on exit.

Fix: Swapped the assignments so each signal is restored to its correct
original handler.

5. TTY ReaderThread: detect ForkTTYChild wrapper exit

Files: WinPort/src/Backend/TTY/TTYBackend.cpp, TTYBackend.h,
TTYRevive.cpp, utils/src/LocalSocket.cpp

Problem: When the ForkTTYChild wrapper exited (terminal closed / SIGHUP),
the orphaned child hung indefinitely in the revival loop — TTYReviveMe
waited for a peer that would never connect because the session manager was gone.

Fix:

  • Track _parent_pid (std::atomic<pid_t>, captured at construction) and
    _is_forktty_child (notify_pipe != -1).
  • In the suspend loop and idle state, check getppid() != _parent_pid. On
    parent death, set _exiting, inject CTRL_CLOSE_EVENT, and break out.
  • Moved kickass-fd byte consumption from TTYReviveMe catch block into
    LocalSocketServer::WaitForClient at the throw site — all callers benefit
    from automatic drain before LocalSocketCancelled is thrown, preventing
    immediate rethrow on the next select iteration.

6. VT shell: fix OnConsoleLog deadlock

File: far2l/src/vt/vtshell.cpp

Problem: OnConsoleLog (called from the input thread) used try_to_lock
on _inout_control_mutex — silently dropping log requests under contention.
When it did acquire the lock, it held it across a blocking InterThreadCall
(modal console log dialog). The main thread, which services InterThreadCall,
may need to call StartIOReaders/StopIOReaders — which also acquire
_inout_control_mutex. Classic lock-order deadlock.

Fix: Three-phase lock discipline:

  1. Lock → suspend VTA → stop output reader → unlock (VTA restored by dtor)
  2. InterThreadCall with no locks held
  3. Lock → restart output reader → unlock

This eliminates silent drops (no more try_to_lock) and the deadlock (mutex
released before the blocking call). VTA being active during the
InterThreadCall is safe — the output reader is stopped so no ANSI sequences
are parsed, and VTA only gates parsing state, not terminal output.

Testing

  • Reproduced hangs with the TTY backend under ForkTTYChild mode by closing
    the terminal window while far2l was running.
  • Verified clean shutdown path with tcdrain disabled during teardown.
  • Confirmed NOOP_EVENT dispatch resolves the input-reader spin loop observed
    during rapid mouse scroll in TTY mode.
  • Smoke test suite passes as a regression gate.

@michaellukashov michaellukashov force-pushed the prevent-deadlocks-and-hangs-in-terminal branch from d514eb7 to 9dcf088 Compare June 4, 2026 13:15
@unxed

unxed commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Interesting. Great work!

I encountered hangs when closing far2l, but not in other situations.

@michaellukashov michaellukashov force-pushed the prevent-deadlocks-and-hangs-in-terminal branch 2 times, most recently from dc427fd to 179dbce Compare June 12, 2026 12:33
Orphan detection uses prctl(PR_SET_PDEATHSIG, SIGUSR1) instead of
getppid() polling, eliminating the PID reuse race and removing a
syscall every 10ms from the suspend loop.

Guard tcdrain, clipboard RPC, and Far2lInteract entry with _deadio
to prevent indefinite blocking on dead file descriptors during
shutdown. Far2lInteract uses a 10s TimedWait with _exiting/_deadio
early-out to avoid blocking when the far2l peer disconnects.

Fix SIGTSTP/SIGCONT handler swap in WinPortMainTTY cleanup.

Two-phase lock in OnConsoleLog: release _inout_control_mutex
before the blocking InterThreadCall to prevent main-thread
starvation on mouse scroll during TUI app execution.

Dispatch inter-thread calls and check pending ctrl events when
NOOP_EVENT is received, preventing infinite spin when a delegate
dispatch is pending.

Drain kickass cancel byte inside LocalSocket::WaitForClient
instead of in caller catch blocks to prevent rethrow loops
for callers that retry after LocalSocketCancelled.
@michaellukashov michaellukashov force-pushed the prevent-deadlocks-and-hangs-in-terminal branch from 179dbce to 502a276 Compare June 12, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants