Skip to content

fix(deepseek): preserve JSONL results on non-UTF-8 streams - #82

Open
U1oSun wants to merge 1 commit into
AMAP-ML:mainfrom
U1oSun:codex/fix-deepseek-jsonl-encoding
Open

U1oSun wants to merge 1 commit into
AMAP-ML:mainfrom
U1oSun:codex/fix-deepseek-jsonl-encoding

Conversation

@U1oSun

@U1oSun U1oSun commented Sep 10, 2026

Copy link
Copy Markdown

Summary

The DeepSeek bridge can crash with UnicodeEncodeError when model output contains characters that its piped stdout cannot encode. This loses the dsh.result record, even when the agent succeeded. A Unicode preparation or launch error can also fail while being written to stderr, before the structured error is emitted.

Emit ASCII-escaped JSONL so decoding restores the original Unicode text without changing the stream configuration. For stderr diagnostics, preserve the original text when encodable and fall back to ASCII escapes only on UnicodeEncodeError. Result fields, exit codes, newline framing, and stdout flushing remain unchanged.

Fixes #78.

Reproduction

Run this against the base revision to reproduce without a DeepSeek installation or API key:

import io
from contextlib import redirect_stdout
from lh_harness.adapters.deepseek_runner import _emit_result

with io.TextIOWrapper(io.BytesIO(), encoding="cp1252", errors="strict") as stream:
    with redirect_stdout(stream):
        _emit_result(text="中文 ≠ 😀", is_error=False, exit_code=0)

The base raises UnicodeEncodeError; the fix emits a JSON record whose decoded text is unchanged. This addresses bridge output encoding, separate from the subprocess decoding changes in #71 and the broader Windows support in #57.

Validation

Tested on Windows with Python 3.13.5:

  • New runner regression suite: 15 failed / 5 passed on the base, 20 passed after the fix. Covers ASCII, cp1252, GBK, and UTF-8; JSONL framing and log parsing; real Python child pipes; successful and nonzero agent results; Unicode preparation and launch errors.
  • python -m pytest tests/test_deepseek_runner.py tests/test_provider_errors.py tests/test_runtime_signals.py -q: 30 passed.
  • Existing tests/test_deepseek_harness_adapter.py: 4 passed / 4 failed on both base and fix. The same pre-existing Windows failures require executable POSIX shell stubs or unavailable SIGHUP/killpg primitives.
  • Python 3.10 syntax compatibility check and git diff --check passed.

The tests use mocked dsh results and real Python subprocesses.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant