fix: tolerate non-utf8 worker output#2350
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| await self._broadcast(_decode_worker_output(line).rstrip("\n")) | ||
|
|
||
| # Handle out message | ||
| try: |
There was a problem hiding this comment.
🟡 json.loads(line) on raw bytes raises uncaught UnicodeDecodeError for non-UTF-8 stdout lines
On line 339, _decode_worker_output(line) safely handles non-UTF-8 bytes for the broadcast. However, on line 343, json.loads(line) still receives the original raw bytes. When those bytes contain invalid UTF-8 (e.g., \x97), json.loads raises UnicodeDecodeError (confirmed: it is NOT a subclass of json.JSONDecodeError). This exception escapes the inner except json.JSONDecodeError: handler at line 362 and propagates to the outer except Exception as e: at line 367, terminating the entire read loop and putting the session into error state. The intent of the inner handler is to gracefully skip unparseable lines, but this case slips through. The fix is to pass the already-decoded string to json.loads so that invalid-UTF-8 lines produce a json.JSONDecodeError (due to the replacement character making invalid JSON), which IS caught by the inner handler, allowing the loop to continue.
(Refers to lines 339-343)
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
Fixes #2313.
The web session runner decoded worker stdout and crash stderr with strict UTF-8. On Windows, child processes can still emit locale-encoded bytes such as cp1252 smart punctuation. A single invalid byte then hides the real worker failure behind a UnicodeDecodeError.
This change decodes worker output with replacement characters at the process boundary, matching the existing uploaded text-file behavior. Wire JSON is still parsed after decoding, but the runner no longer crashes while trying to report a worker error.
To verify
uv run pytest tests\web\test_session_error_recovery.py -quv run ruff check src\kimi_cli\web\runner\process.py tests\web\test_session_error_recovery.pypython -m py_compile src\kimi_cli\web\runner\process.py tests\web\test_session_error_recovery.pygit diff --check