Repository navigation
fix(core): the LLM stream idle ceiling — a hung read fails through the transport path, the turn unwinds, busy frees - #1759
Conversation
…er an idle ceiling A provider socket that dies without a clean FIN/EOF parked the stream read forever, leaking the turn, its run-coordinator registration, and the session's busy state for hours (#1739). Race every stream read against a configurable idle ceiling (AMICODE_STREAM_IDLE_TIMEOUT_MS, default 90s) — reset-on-read, so healthy slow generations never trip it — and fail the step with an LLMError(TransportReason) through the existing turn-failure path, which unwinds the turn, publishes the failed assistant message, frees the busy registration, and lets a re-prompt resume. The one explicit llm.stream(request) call per provider turn is untouched; the ceiling wraps the read loop only.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Superseded: the consolidation #1833 merged live-test/stack-e2e into main (2026-10-21, Aaron's window) — this branch's content is contained in main (ancestor-verified during the drain-plan analysis). Closing with this crosspointer; the work is landed. |
Closes #1739. Part of #1731 (panel-reliability lane — the idle-timeout doctrine's third leg: client #1752, upstream #1758, now the engine). Stacks on #1721 (cut from
jev-routed-pool@f20ee140).The fix — the lifecycle family's last member
withStreamIdleCeilingwraps the ONEllm.stream(request)(preserved — the wrapper consumes the same single stream object):Stream.fromPull(Stream.toPull(stream))with each pull raced byEffect.timeoutOption(idleMs)— reset-on-read, never a busy-wait. On a hung read the stream fails withLLMError(TransportReason{kind:"idle-timeout"})— no new error vocabulary: it enters the exact branch that already handled transport failures (failAssistant+failCause), so the durable machinery unwinds identically — the turn errors, the run-coordinator registration frees, the busy state clears, and a re-prompt resumes to a next provider turn (all pinned).Config:
AMICODE_STREAM_IDLE_TIMEOUT_MS, default 90,000 (inside the issue's 60–120s band — reasoning deltas arrive continuously, so 90s of dead air is dead transport), read per-turn at call time, invalid/non-positive falls back to default. Mirrors theAMICODE_MAX_STREAM_*knob family (#775/#1716).The repro it kills: today's live case #3 — a session went mid-turn silent (reasoning/text/tool in flight, zero event frames for 34s, engine otherwise healthy), held busy until
POST abortcleared it; the issue's original case held busy 8 hours. With this ceiling, the same hang fails the step within 90s and the session recovers on re-prompt.Gates (orchestrator-run, baselines captured pre-edit)
server-sdk.tsxTS2339) — zero delta from this slicematerialize.mjsmanifest verified at exact hashes (hand-patched sha256s, no auto-refresh noise);drift_gate.mjsPASSfinish:"error"with flushed partial content; re-prompt resumes to a second provider turn; 60ms-gap stream under a 300ms ceiling never trips (reset-on-read); the RED pre-fix run confirmed the leak (resume parked forever, bounded only by the test's own 5s timeout)Honest notes
Pullis an Effect object, not callable (Effect.suspendis the per-run wrapper);it.effectruns under TestClock so wall-clock timing tests must useit.live.