Skip to content

Stream OpenVMM test transcripts to disk incrementally #176

Description

Problem

The NVX microVM test harness buffers OpenVMM/guest transcript bytes in memory and persists them only during normal cleanup paths.

In scripts/nvx_tools/openvmm_process.py, OpenvmmProcess accumulates output in _output and writes _log_path from wait() or close(). Similar helpers in scripts/nvx_tools/benchmark.py, including run_guest_script(), write their buffered output from finally blocks.

This works for ordinary failures, but it weakens diagnostics for intermittent failures. If the Python process, Actions job, or runner is terminated before cleanup completes, the transcript may be absent or truncated even though output had already been read. The queue-drain-at-close model can also make the final bytes dependent on reader timing.

Proposed change

Stream transcript bytes to their destination file as they are consumed, while preserving the existing in-memory behavior needed for marker detection and callers that inspect captured output.

Apply the behavior consistently to the shared OpenVMM process/transcript paths rather than fixing a single scenario.

Requirements:

  • Open the transcript destination before or immediately after starting the child process.
  • Append each consumed chunk in exactly the received order.
  • Preserve binary output byte-for-byte; do not add timestamps or prefixes to the raw transcript.
  • Flush periodically and at phase/cleanup boundaries, without calling fsync for every small chunk.
  • Preserve already-written bytes when a timeout, exception, or forced process termination occurs.
  • Avoid duplicate bytes when normal cleanup runs after incremental writes.
  • Surface transcript I/O failures explicitly rather than silently falling back to memory-only capture.
  • Keep existing marker-search, output, timeout, and process-termination behavior compatible.

A separate structured sidecar can be considered later for timestamps and phase events; this issue should keep the raw transcript format unchanged.

Acceptance criteria

  • A transcript file is created and grows while a test process is still running.
  • Bytes consumed before an exception or timeout remain in the transcript.
  • Normal completion produces exactly the same raw transcript bytes as the process output, with no duplication or truncation.
  • Binary and partial-line output are preserved exactly.
  • Linux PTY and Windows pipe execution paths are covered.
  • Tests cover normal completion, timeout/exception cleanup, forced termination, and delayed reader output.
  • Existing microVM marker and byte-exact console tests continue to pass.

Non-goals

  • Enabling more verbose OpenVMM or guest logging.
  • Changing CI retry or pass/fail behavior.
  • Uploading new workflow artifacts; artifact retention can be handled separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions