Skip to content

[Klaud Cold] Update DeepSeek-V4.1-Flash FP4 GB200 SGLang AgentX DSpark image to nightly-dev-cu13-20261118-ecd3d75a / [Klaud Cold] 将 DeepSeek-V4.1-Flash FP4 GB200 SGLang AgentX DSpark 镜像更新至 nightly-dev-cu13-20261118-ecd3d75a - #5170

Draft
Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-7aa8d6ea5943ecc7-8d058c57662ba827

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-cu13-20261027-5884e4fe@sha256:61b9083bdcb7cf3741366f487cb996c05ea33ad2ddddc820d10cbc1c0cfb0a86 to lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514.
Baseline: 2026-11-06 · lmsysorg/sglang:nightly-dev-cu13-20261027-5884e4fe@sha256:61b9083bdcb7cf3741366f487cb996c05ea33ad2ddddc820d10cbc1c0cfb0a86
Mean latency · Sources: API 1, API 2, API 3, API 4, API 5

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c1 TP4 EP1 21299a 7,346.33 45.01 693.03 1.6
AgentX c4 TP4 EP1 b7520b 9,735.9 67.29 473.37 1.73
AgentX c8 TP4 EP1 92e82e 16,542.81 115.1 510.65 1.81
AgentX c16 TP4 EP4 3a364a 28,651.92 181.9 1,183.06 2.09
AgentX c32 TP4 EP1 5cddaa 56,317.92 321.83 3,966.81 2.73
AgentX c64 TP4 EP4 0b1766 118,711.77 830.96 1,111.38 3.95
AgentX c96 TP4 EP4 381f1c 157,731.22 1,166.07 1,080.06 4.48
AgentX c128 TP4 EP4 fae0e9 190,506.62 1,458.12 1,128.28 5.01
AgentX c160 TP4 EP4 c6b210 240,522.68 1,738.02 1,153.75 5.66
AgentX c256 TP4 EP4 beca52 355,295.93 2,498.38 1,597.07 8.24
Eval Score ↑ Samples
gsm8k/em_strict · c16 97.35% 1,319
gsm8k/em_strict · c32 97.27% 1,319
gsm8k/em_strict · c256 97.57% 1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-cu13-20261027-5884e4fe@sha256:61b9083bdcb7cf3741366f487cb996c05ea33ad2ddddc820d10cbc1c0cfb0a86 更新至 lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514。
**基线:**2026-11-06 · lmsysorg/sglang:nightly-dev-cu13-20261027-5884e4fe@sha256:61b9083bdcb7cf3741366f487cb996c05ea33ad2ddddc820d10cbc1c0cfb0a86
平均延迟 · 来源: API 1, API 2, API 3, API 4, API 5;数值及异常说明见上表。
**停止:**随时关闭此 PR 即可立即停止该候选:Klaud 会取消其运行并删除其分支。

…dev-cu13-20261118-ecd3d75a

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 44866417161 / attempt 1 · 2026-11-21 05:12 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · bd334b4f0d3b
Change: Bumped the master image to SGLang nightly-dev-cu13-20261118-ecd3d75a (sgl-project/sglang ecd3d75a on main, 1744 commits after 5884e4fe, arm64 label verified), the newest nightly that still accepts the recipe's --enable-dp-attention flag and Engram host-table env vars, which 7946e589 (in the 20261119+ nightlies) removed.

Next: The TP4/EP4/DPA c64 engine became healthy, but its sglang-router frontend failed with ModuleNotFoundError: No module named 'sglang_router' because upstream b947a615 (#50671) stopped shipping sgl-model-gateway in the image; c1, c16 and the evals were fail-fast cancellations, so repair 1 runs the frozen sglang-router 0.3.2 from its official lmsysorg/sgl-model-gateway:v0.3.2 image.

中文

初次尝试 · 失败 · Run 44866417161 / attempt 1 · 2026-11-21 05:12 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · bd334b4f0d3b
**变更:**将主配置镜像更新为 SGLang nightly-dev-cu13-20261118-ecd3d75a(sgl-project/sglang main 分支 ecd3d75a,比 5884e4fe 新 1744 个提交,已核实 arm64 标签);这是仍接受配方中 --enable-dp-attention 参数和 Engram host-table 环境变量的最新 nightly,7946e589(包含于 20261119 及之后的 nightly)已将其移除。
**下一步:**TP4/EP4/DPA c64 引擎已健康启动,但其 sglang-router 前端报错 ModuleNotFoundError: No module named 'sglang_router',原因是上游 b947a615(#50671)不再在镜像中提供 sgl-model-gateway;c1、c16 和评测均为 fail-fast 取消,因此修复 1 改用官方 lmsysorg/sgl-model-gateway:v0.3.2 镜像运行冻结的 sglang-router 0.3.2。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 1 is waiting on 44866417161. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 44866417161。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions github-actions Bot added the klaud-parked Klaud parked until its queued or running benchmark runs end label Nov 21, 2026
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 2 个会话将在其规划运行的新运行中继续。

@github-actions github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Nov 21, 2026
…gateway image

SGLang images after b947a615 (#50671) no longer ship sgl-model-gateway, so the
DP-attention points' router failed with No module named 'sglang_router'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Passed · Run 44869949123 / attempt 1 · 2026-11-21 06:14 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · bd334b4f0d3b · Mean latency
Change: At 05d6710, the five TP4/EP4/DPA overrides set frontend.container_image to the official lmsysorg/sgl-model-gateway:v0.3.2 index (sha256:83711812..., arm64 present) so the frozen sglang-router 0.3.2 runs unchanged; uncertainty: that May 2026 release predates 22 gateway commits in the 5884e4fe build (for example cache_aware random tie-breaking, #33138) under the same 0.3.2 version.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c1 TP4 EP1 c50e69 46.49 (N/A) 468.94 (N/A) 1.47 (N/A)
AgentX c16 TP4 EP4 d314f2 149.94 (N/A) 1,131.1 (N/A) 1.86 (N/A)
AgentX c64 TP4 EP4 0591b1 573.36 (N/A) 5,894.18 (N/A) 3.48 (N/A)

Note: All rows: Δ N/A: no matched baseline.

Eval Score ↑ Samples
gsm8k/em_strict · c16 97.73% (+0.38 pp) 1,319 each
gsm8k/em_strict · c32 97.73% (+0.45 pp) 1,319 each
gsm8k/em_strict · c256 97.65% (+0.08 pp) 1,319 each

Next: All three smoke points (c1, c16, DPA c64 with the gateway frontend) and the c16/c32/c256 gsm8k evals passed with zero request errors; smoke deltas use the 20-minute profile and are not comparable, so append the perf-changelog entry and dispatch the final sweep.

中文

修复 1/5 · 已通过 · Run 44869949123 / attempt 1 · 2026-11-21 06:14 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · bd334b4f0d3b · 平均延迟
**变更:**在 05d6710 中,五个 TP4/EP4/DPA override 将 frontend.container_image 设为官方 lmsysorg/sgl-model-gateway:v0.3.2 索引(sha256:83711812...,含 arm64),使冻结的 sglang-router 0.3.2 保持不变运行;不确定性:该 2026 年 5 月发布版本比 5884e4fe 构建中的网关早 22 个提交(例如 cache_aware 随机平局打破,#33138),但版本号同为 0.3.2。实测数值及异常说明见上表。
**下一步:**三个 smoke 点(c1、c16、使用网关前端的 DPA c64)以及 c16/c32/c256 gsm8k 评测均通过,请求错误为零;smoke 使用 20 分钟配置,差值不可比较,因此追加 perf-changelog 条目并触发最终 sweep。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 2 is waiting on 44869949123. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 44869949123。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions github-actions Bot added klaud-parked Klaud parked until its queued or running benchmark runs end and removed klaud-in-progress Klaud is working on this candidate or will resume it labels Nov 21, 2026
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 3 个会话将在其规划运行的新运行中继续。

@github-actions github-actions Bot added klaud-in-progress Klaud is working on this candidate or will resume it and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Nov 21, 2026
…park image update

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 21, 2026
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Running · Run 44874695526 / attempt 1 · 2026-11-21 06:16 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · c268c4eaba6b
Change: Appended the perf-changelog entry at c268c4e (image unchanged from repair 1) and applied full-sweep-fail-fast after check-final confirmed all 10 frozen baseline points and default evals.

Next: Wait for the full final sweep of all 10 AgentX points and the c16/c32/c256 gsm8k evals, then compare against the frozen baseline gates.

中文

最终完整 sweep · 运行中 · Run 44874695526 / attempt 1 · 2026-11-21 06:16 UTC
lmsysorg/sglang:nightly-dev-cu13-20261118-ecd3d75a@sha256:2a7e6d43ada9311fcb7d7dd5781bda10aee1a24c40186a956c223f2036212514 · c268c4eaba6b
**变更:**在 c268c4e 追加 perf-changelog 条目(镜像与修复 1 相同),并在 check-final 确认全部 10 个冻结基线点和默认评测后添加 full-sweep-fail-fast 标签。
**下一步:**等待全部 10 个 AgentX 点及 c16/c32/c256 gsm8k 评测的完整最终 sweep,然后对照冻结基线门槛进行比较。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 3 is waiting on 44874695526. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 3 个会话正在等待 44874695526。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions github-actions Bot added klaud-parked Klaud parked until its queued or running benchmark runs end and removed klaud-in-progress Klaud is working on this candidate or will resume it labels Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) klaud-parked Klaud parked until its queued or running benchmark runs end

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant