[Bug]: Windows non-UTF-8 locale — Path text I/O decodes structured state files with the locale encoding

[GZY-SUPER-HACKER]

Summary

On a Windows host whose ANSI code page is not UTF-8 (a zh-CN host with cp936/GBK),
LoopX code paths that read structured state files through Path.read_text() without an
explicit encoding decode them with locale.getpreferredencoding(False), which is gbk.
JSON (RFC 8259) and TOML are UTF-8 by specification, so any file carrying a non-ASCII
character is either rejected or silently mis-decoded.

• bytes GBK cannot decode -> UnicodeDecodeError, which is a ValueError subclass, so
  callers that catch (OSError, ValueError) swallow it and report a misleading secondary error;
• bytes GBK can decode -> silent wrong data: 用户任务 reads back as 鐢ㄦ埛浠诲姟.

Same class as #4155, different code path. #4396 pinned UTF-8 for every remaining text-mode
subprocess read and closed #4155; the Path.read_text() / Path.write_text() path was not
covered by that sweep.

Expected behavior

Text reads and writes of structured state files (.json / .toml) decode and encode as
UTF-8 regardless of the host's ANSI code page. Concretely:

• loopx goal-portfolio --manager-view portfolio --format json succeeds on a cp936 host
  whose registry.json carries a non-ASCII knowledge_root / state_file, instead of
  returning {"ok": false, "error": "manager_evidence_unavailable_or_invalid", "rows": []};
• a record written by collaboration/inbox.py can be read back by the same module on the
  host that wrote it.

Reproduction — real command, real output

A project registered under a non-ASCII path is sufficient: registry.json carries filesystem
paths (project_record.knowledge_root, goal_record.state_file), and non-ASCII directory
names are ordinary for a zh-CN user.

loopx goal-portfolio --manager-view portfolio --format json

{ "ok": false, "error": "manager_evidence_unavailable_or_invalid", "rows": [] }

Exit status 1. Nothing in the output points at encoding.

The only variable is the encoding default. Same fixture, same command, same machine:

Invocation	Result
loopx goal-portfolio ...	{"ok": false, "error": "manager_evidence_unavailable_or_invalid", "rows": []}
PYTHONUTF8=1 loopx goal-portfolio ...	{"ok": true, "view": "portfolio", ...}

Root cause: loopx/capabilities/manager_context/evidence_export.py:12 reads the registry
without an explicit encoding.

Scope — 10 sites in loopx/

Measured with an AST scan over the package (a grep count is misleading here, because the
tree uses multi-line calls):

Kind	Count
Path.read_text() without encoding	9
Path.write_text() without encoding	1

Most consequential:

• loopx/control_plane/collaboration/inbox.py:56 — the module's own _write uses
  encoding="utf-8" and ensure_ascii=False, so a record written by LoopX cannot be read
  back by LoopX on a non-UTF-8 host.
• loopx/capabilities/connector_registry/core.py:149 (write side) — save_connector_registry
  writes with the locale encoding while load_connector_registry reads as UTF-8. On cp936
  the write produces GBK bytes, the read raises, except (OSError, ValueError) swallows it,
  and the loader silently falls back to built-in defaults.
• loopx/codex_cli_runtime_probe.py L172/183/190/197 — the same file where fix(runtime): pin UTF-8 decoding for every remaining text-mode subprocess read #4396 pinned
  L222/275 (subprocess) but left these four (file read) untouched.

Planned change

Add an explicit encoding="utf-8" to those 10 product-code call sites — one keyword per site,
no control-flow change. I am not proposing to touch the ~449 matching call sites under
tests/, examples/ and benchmark/ in the same change; they are a separate slice and I am
happy to take them separately if you would rather.

Separate decision, not in this change

CPython ships EncodingWarning for exactly this pattern (PEP 597). Run pytest with
-X warn_default_encoding plus filterwarnings = error::EncodingWarning and the class is
caught rather than the instances. Measured on this host:

• against the unfixed call sites under benchmark/tests/, it fails the same 5 tests that
  PYTHONUTF8=1 fixes — so it does detect exactly this defect;
• against two test files whose modules are already fixed (17 passed + 18 passed before),
  it reports 18 failed / 7 passed / 10 errors — the currently-passing tests go red, because
  the ~449 unfixed sites under tests/, examples/ and benchmark/ are inside the same run.

So enabling the gate is a CI-wide decision that first requires that cleanup to land. Flagging
it here, not proposing it. (ruff's PLW1514 covers part of it — 6 of these 10 — but it is
preview-only today.)

If you would rather have the ~449 sites fixed first and the gate enabled in one go, I am happy
to take that as a follow-up slice instead of this one.

[GZY-SUPER-HACKER]

Two updates after rebasing onto eb16c54 — one correction to what I wrote above, and one severity upgrade.

Correction: the site count is 9, not 10

Current main replaced the direct read in loopx/capabilities/manager_context/evidence_export.py with load_registry / registry_codec, which reads bytes (path.read_bytes() → json.loads(bytes)) and is therefore already encoding-safe. I verified by running the documented command on a cp936 host with a project in a Chinese-named directory:

loopx goal-portfolio --manager-view portfolio --format json
-> {"ok": true, "view": "portfolio", ...}

That already succeeds on eb16c54 without my change, so the registry read is no longer part of this class. The remaining scope is 9 sites in loopx/: 8 reads + 1 write, across 6 modules. An AST scan over the package now reports zero remaining read_text() / write_text() calls without an encoding.

Severity upgrade: one of the nine is worse than described above

I described connector_registry/core.py:149 as a write-side site where the loader "silently falls back to built-in defaults". That understates it, and the reason is worth stating precisely.

save_connector_registry writes with the locale codec; load_connector_registry reads the same file as UTF-8 inside try/except (OSError, ValueError): pass. On a cp936 host:

1. loopx connector register <id> returns {"ok": true, ...};
2. the file on disk is not valid UTF-8 — the first Chinese character lands as byte 0xd0;
3. the next loopx connector list cannot decode it, the UnicodeDecodeError is swallowed, and the registry silently reverts to the built-in catalog. Exit status 0, nothing on stderr;
4. registering a second connector loads (fails silently), then saves — which overwrites the file and permanently destroys the first entry.

So on a non-UTF-8 host this registry is effectively write-only: every write is discarded by the next read, and a subsequent write erases what came before.

The reason this needs no unusual user input: BUILTIN_CONNECTOR_CATALOG itself carries 37 non-ASCII field values (新浪日线行情, ARK 联网搜索, 同花顺财务摘要, …). The write side therefore emits GBK on a cp936 host for any state, including a registration the user typed entirely in ASCII. I confirmed the ASCII-only variant reproduces identically.

An independent consequence: three tests already in this repository are red on such a host for exactly this reason and go green with the fix — test_register_use_rank_persists, test_builtin_connector_updates_survive_catalog_resync, and test_cli_path_override_reads_the_registry_it_writes (3F → 0F, with the full per-test failure list diffed against the base tree and no other outcome changed). They pass on a UTF-8 locale, so CI on Linux does not currently observe this.

PR: #4942.