Skip to content

sparse decode attention: CUDA graphs, P/D, asynchronous decode, token pipelines and preemption - #164

Open
0z5a wants to merge 40 commits into
InferMatrixLab:mainfrom
0z5a:pr157-stack-on-head
Open

0z5a wants to merge 40 commits into
InferMatrixLab:mainfrom
0z5a:pr157-stack-on-head

Merge main and keep sparse decode off FP8 KV caches

238f197
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 warning and 1 notice
lint and CPU unit tests
succeeded Nov 18, 2026 in 2m 22s