Repository navigation
perf(transcript): coalesce streaming projections to one pass per frame - #1289
Conversation
Every native transcript event reduced synchronously and then projected into the message store in the same task, so a reply delivered as one burst blocked the renderer for the whole projection series: a 400-delta reply measured a single ~1.1s long task at ~2.8-4ms per delta, freezing input on local and remote transcripts alike (NeuralNomadsAI#1017 explicitly left frame-level batching as follow-up work). Events still reduce immediately and in order through the SDK reducer; only the projection into the store and the re-render it causes moves to the frame level. stores/opencode-projection-frame.ts runs at most one projection per animation frame per session, always from the reducer's newest state: same-frame schedules merge, the last event decides pending-send confirmation, and usage updates accumulate. Hidden windows fall back to a bounded 250ms timer (they pause animation frames), and environments without a frame scheduler keep the previous synchronous projection. Store readers lag at most one frame. Ordering and lifetime edges: a session selection write and a visibility transition flush pending pages first; transcript eviction and instance teardown cancel them so no later frame resurrects a destroyed store; destructive events (session.deleted, inbox cancellation, revert commit) and forced authoritative publications still project synchronously and subsume the pending frame; an event whose session stopped being active at frame time invalidates the loaded page so the next activation reads authoritatively. The reference-anchored projection-skip prohibition and hydrateMessages record reuse are untouched. Covered by opencode-projection-frame.test.ts (manual frame clock) and tests/browser/streaming-projection.test.ts (real SessionView with a 400-delta burst: projections 400 -> 1, burst dispatch ~1094ms -> ~25ms, no long task); compaction-responsiveness settles the compaction row's creation frame before resetting its counters.
Two gaps the native pruning UI check exposed on a quiet page:
An idle compositor stops producing frames (a hidden or headless page with
nothing to draw), so a schedule armed only on requestAnimationFrame could
hang a pending projection forever — no projection means no DOM change,
which means no next frame. Every visible-window schedule now also arms a
32ms fallback; whichever wake wins cancels the other, so a live page
still projects exactly once per frame and an idle one projects within
the fallback window. The hidden-window timer stays at its calmer 250ms.
An execution settlement (session.execution.{succeeded,failed,interrupted})
now projects synchronously like the destructive events: it is the stable
point interactive reads synchronize on — part deletion rejects a row
whose status still reads "streaming" — and it fires once per turn, so
the synchronous cost is one projection at settlement.
Verified by reproducing the failing native pruning UI run locally: the
traced page showed the final event batch scheduled but never projected
on an idle compositor, and the deletion click reading a settled row a
frame behind. Both fixed runs pass all nine UI checks twice; unit tests
cover the idle-compositor fallback and the same-task settlement status.
|
Fixed both root causes and pushed What the failure exposed (reproduced locally by tracing the fixture page):
Verified: the failing native pruning UI run passes all nine UI checks twice locally; |
session.tool.progress renders through the coalesced frame projection, so the call list assertion must wait for the frame it lands on instead of reading the DOM in the same task as the event.
|
Pushed
|
|
Remaining reds on
|
pascalandr
left a comment
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan I reviewed 534bdfb7. The performance direction checks out: the burst uses one projection instead of 400, and the warmed-up browser probe avoids the long task seen on the base. Three correctness regressions remain. Each reproduces on this head and passes on the base; isolated scratch fixes also pass.
P2: a synchronous settlement drops the pending usage update
packages/ui/src/stores/instances.ts:2314-2322
Deliver session.step.ended with usage, immediately followed by session.execution.succeeded before the frame runs. The step schedules updateUsage: true; settlement cancels that entry and projects synchronously, but skips updateSessionInfo because execution settlement is absent from USAGE_EVENT_TYPES. The message store reports 118 combined tokens while session-info reports 0 (or remains absent in a cold store). Running another frame cannot recover the cancelled obligation.
Please preserve the pending usage update when a synchronous publication subsumes it. Updating session info at synchronous barriers passes the reproduction.
P2: a pending frame can cross a hard reconnect/source reset
packages/ui/src/stores/opencode-projection-frame.ts:135-139
The entry retains its old Data without a lifetime/generation fence. The instance callers added here cancel frames, but reconnect's resetInstance: destroyOpenCodeData goes directly through the shared destruction boundary. Once that deletes the transcript entry, projectOpenCodeMessages' conditional current-entry guard permits the old data.
Reproduction: queue streamed text, receive server.connected from a new source, complete a forced authoritative empty load, then run the old frame. The authoritative empty store regains old rows ([] becomes msg_000, msg_001, msg_live in the store probe; the browser production-dispatch probe also resurrects msg_live).
Please cancel at the shared destroy/reset lifetime boundary, or fence publication on the captured lifetime. Central cancellation passes the reproduction.
P2: queued replay misses the settlement barrier and can lose a completed reply
packages/ui/src/stores/instances.ts:2280-2283
publishTranscriptChunk always schedules, even for a chunk containing execution settlement. The synchronous-event set protects only the immediate event path.
Reproduction: fill the reducer's 200-message window, then admit another reply and settlement while rotation queues events. Replay drains and retires the reducer before the reply reaches the display store. If another execution starts before the frame, its different Data replaces the pending entry. The next reply becomes visible while the previous completed reply remains absent. The base preserves both.
Please synchronously publish replay chunks containing settlement before retirement, subsuming the pending frame just as on the immediate path. This passes the back-to-back-execution reproduction.
Test follow-ups
- P3:
opencode-projection-frame.test.tsis missing from the workflow's explicit UI unit lists. Add it to the browser-conditioned integration step; browser-suite discovery does not run store unit files. - P3:
streaming-projection.test.ts:25-28answers/api/eventswith the generic JSON fallback. A strict console collector catches the EventSource MIME error. Please usefixtureEventStreamand retain console/network failure checks, as in the hardened #1284 fixture. history-navigation.test.ts:985assumes the scheduled row is already rendered. It fails intermittently on this head and passes on the base. Await actual publication, then recheck the original marker's identity so the assertion still protects the scroll/DOM invariant.- The new scheduler unit tests do not catch removal of usage publication, the latest-window gate, hidden-window timer behavior or pending-send confirmation semantics. The spaced-stream browser assertion also passes on the base: check complete text and the intended projection behavior, rather than only its suffix.
Validation
UI typecheck passes. The 35-file transcript/instance/replay/settlement/reconnect/idle/compaction/authority/pruning selection passes 460/460 twice. New browser suite passes three runs; stream-delta-churn, tool-images, compaction-responsiveness, part-durations, project-tab-restore (50/50) and focused scrolling (34/34) pass. The independent store/browser probes expose the regressions above despite those green suites.
I also checked the two red CI jobs: the Design activation failure comes from a native fixture outside this UI scheduler's import path; the Electron preview snapshot test passes locally on both head and base. Those failures do not account for the three reproduced regressions here.
Adopting the review's three reproductions, each now covered by a unit case in opencode-projection-frame.test.ts: A synchronous settlement or destructive event now adopts the pending frame's usage obligation instead of dropping it (a step end with usage followed by execution.succeeded in one task used to leave session info at zero while the message store carried the tokens). destroyOpenCodeData cancels coalesced frames at the shared lifetime boundary — a hard reconnect's reset previously deleted the transcript entry while a pending frame kept the old Data, and the missing entry slipped past the projection guard, resurrecting the pre-reset rows over an authoritative empty load. The frame module now reaches the store only through the registered gates, so the reducer module can cancel without an import cycle. A replayed chunk carrying an execution settlement publishes synchronously through the same barrier as the immediate path, so a drained-and-retired rotation can no longer lose the completed reply it held to a later frame or a newer execution's Data. Also per review: the frame test joins the workflow's browser-conditioned unit list; streaming-projection borrows the quiet fixture event stream and keeps strict console/network failure checks (which immediately caught the unmocked Yolo probe); its spaced-stream case now asserts the complete coalesced text and fewer passes than deltas; and history-navigation awaits the scheduled row's actual publication before rechecking the distant marker's identity. Verified: 252/252 across the transcript/instance unit families, typecheck clean, streaming-projection 3/3, compaction-responsiveness 5/5, tool-images 6/6, and the isolated native pruning UI run passes all nine checks on this head.
|
Thanks @pascalandr — all three reproductions are fixed and unit-covered in
Test follow-ups, also in Validation on this head: 252/252 across the transcript/instance unit selection (twice), typecheck clean, |
|
|
pascalandr
left a comment
pascalandr
left a comment
There was a problem hiding this comment.
@heunghingwan Round 2 on d90d7506: all three original reproductions are fixed, and independently reverting each fix breaks its corresponding regression. The CI registration, persistent SSE, pending-confirmation checks and hidden-window coverage are also addressed. One side effect of the shared cancellation boundary remains blocking.
P2: normal retirement cancels the final tool publication
packages/ui/src/stores/opencode-data.ts:1036, called by retireDrainedTranscript at :924
Reproduced through the production dispatcher, in both a schema-validated store probe and a real browser:
- Settle an execution while one of its tools is still running.
- Let the settlement read return the still-running tool; its reducer correctly remains resident.
- Deliver
session.tool.successafterward. - The event schedules the completed page, then
finishOpenCodeDataEventretires the now-drained reducer. - The new central cancellation drops that pending final page.
After subsequent frames the display still says running and has no successful output. This fails on the new head and passes on both the first-round head and the original base.
Please publish this session's pending final page and usage obligations before normal retirement, while keeping resets, eviction and destructive teardown cancel-only. A scratch publication-before-retirement fix changes the browser result from running to completed and preserves all three original fixes. This should fit the existing publication machinery rather than require another scheduling path.
P3: the history regression still does not retain the original marker
packages/ui/tests/browser/history-navigation.test.ts:988-990
The publication wait fixes the timing issue, but marker is a Locator: its later evaluate resolves the currently matching element, not the original one. Replacing the original marker with a clone during the frame still passes the test, even though an independent witness confirms the original node is disconnected. Retain an ElementHandle before streaming, then check that same node after publication.
Validation
- Original usage, hard-reset and full-window replay reproductions all pass. Reverting usage adoption, reset cancellation or the replay barrier independently makes its reproduction fail.
- Typecheck passes; the 35-file transcript/instance/authority/pruning selection passes 466/466 twice. Browser suites pass: streaming-projection three runs, stream-delta-churn 2/2, tool-images 6/6, compaction-responsiveness 5/5, project-tab-restore 50/50 and the full history-navigation file 70/70.
- Performance direction remains positive: 400 burst projections become one, with no burst long task observed in the warmed harness.
- The three current macOS job failures are runner-acquisition failures: their annotations say the jobs were not acquired by a hosted runner. All browser shards and both runtime-contract lanes passed.
Round-2 review: the shared destroy boundary cancelled the page a late tool settlement had just scheduled, so retiring the drained reducer right after session.tool.success dropped the completed output and the display kept showing the running row. Normal retirement now publishes the session's pending page (and its usage obligations) through a scoped flush before the entry goes away, while resets, eviction and destructive teardown stay cancel-only. The scoped drain leaves other sessions on their armed frame. The history-navigation marker check pins the original node with an ElementHandle, so a clone replacing it during the published frame can no longer satisfy the identity assertion the way a re-resolving Locator did. Covered by "a retirement-scoped flush publishes the pending final page and its usage". Verified: 259/259 across the transcript/instance unit families, typecheck clean, streaming-projection 3/3, compaction -responsiveness 5/5, the history case green, and the isolated native pruning UI run passes all nine checks.
|
Round-2 items are fixed in
Verified on this head: 259/259 across the transcript/instance unit selection (including the frame suite), typecheck clean, |
…n flush A normal reducer retirement publishes its own session's pending page through a scoped flush. That flush cancelled the shared wake and armed a fresh one for the sessions it left pending, so in a hidden window every retirement elsewhere pushed an unrelated session's 250ms publication back again; a steady trickle of late tool results could postpone it indefinitely. A scoped drain now leaves the armed wake alone and only disarms once nothing remains pending; projections scheduled during the drain still request a wake, which is a no-op while one is armed. Tests: the retirement case now goes through real events (a tool still running at settlement, then its late success), so dropping the flush before retirement fails it; a hidden-window case retires late tools in four other instances 100ms apart and requires the waiting session to publish on its original 250ms wake, which fails with the re-arming drain.
|
Thanks for the direct fix — |
|
@heunghingwan Thanks to you for all these nice optimisation PR ! |
What
Frame-level coalescing for event-driven transcript projection. Native transcript events still reduce synchronously and in order through the SDK reducer, but the projection into the message store — and the Solid re-render it triggers — now runs at most once per animation frame per session, always from the reducer's newest state.
This is the follow-up #1017's review explicitly deferred ("if the trace still shows a need"): even after #1284, every
session.text.deltastill paid ~2.8ms of synchronous reducer→projection→store→render, so a reply arriving as one burst still blocked the renderer for the whole series.Measurements
Same browser harness on real
SessionView+ realsseManager.handleEvent, 400-delta reply delivered in one task (tests/browser/streaming-projection.test.ts):How
stores/opencode-projection-frame.ts: one pending entry perinstanceId\0sessionId; same-frame schedules merge (the last event decidesconfirmPending; usage updates accumulate); runs onrequestAnimationFrame, on a bounded 250 ms timer while the window is hidden (hidden windows pause rAF), and synchronously where no frame scheduler exists (node harnesses keep the previous behavior).instances.tsregisters gates (active session, latest window, inactive-load invalidation,updateSessionInfo);publishTranscriptChunk(its replay-ordering identity unchanged) and ordinary event projections schedule instead of projecting inline.session.deleted,session.inbox.cancelled,session.revert.committed) and forced authoritative publications still project synchronously and subsume the pending frame.visibilitychangeflush pending pages; a normal reducer retirement publishes its own session's pending page first, leaving other sessions on their armed wake. Resets, eviction, deletion and instance teardown cancel at the shareddestroyOpenCodeDataboundary — no later frame can resurrect a destroyed store.Invariants kept: no reference-anchored projection skip (the reducer mutates rows in place);
hydrateMessagesrecord reuse and the identical-snapshot early return stay the cheap path; transcript queue publication/onAppliedordering barriers untouched.Tests
opencode-projection-frame.test.ts(unit, manual frame clock): same-task burst projects exactly once at the next frame; selection switch flushes; teardown cancels without resurrecting stores; destructive events subsume the pending frame.tests/browser/streaming-projection.test.ts(borrowFixture, real SessionView): burst coalescing, zero HTTP message reads during streaming, one-frame reader convergence, visibility/switch flushes; timings logged as observations, not CI thresholds.compaction-responsiveness.test.tssettles the compaction row's creation frame before resetting its counters.Verified:
typecheckclean; transcript/instance unit families (opencode-data, replay, settlement, reconnect, idle, compaction, native events, request authority, pruning, catalog… 437+ tests) pass; browser suitesstreaming-projection,part-durations,compaction-responsiveness,render-cost,message-scroll(128/128) pass;markdown-latest-requestfails one pre-existing test, identical on base.Docs:
dev-docs/TRANSCRIPT.md(agent conventions),dev-docs/CACHE_REFRESH_CONVENTIONS.md(transcript pages row).