Repository navigation
refactor(harness): rename harness/scenarios to harness/trials - #563
Conversation
Removing infra and model left three sentences in module-layout.md that still state Infra rules in the present tense. They now say what changed. The architecture directory map gains the Apps delivery package, the Apps root compatibility exports and the modules added since the map was last updated, so every package on disk is listed. The contributor guide sent new models to harness/scenarios, whose one module runs any external command as a trial and is the same for every model; it now names the serving and Benchmark locations instead. The Harness, scenarios and views package docstrings now describe what the packages hold, including the search evidence that views writes for campaign import-search. No code changes. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
…ry test #457 merged after #498 removed infra/, and its new recovery test still imported digest from inferrsi.infra.artifacts, so full-check fails on main. The value it forges is compared by harness.optimization, which takes digest from harness.artifact_primitives, so the test now imports that copy. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
The package's one module, external.py, runs an external command as one trial and retains its record (execute_trial, collect_trial_checks). It holds no scenario logic and is the same for every model, so the old name pointed contributors to the wrong place for new models. The move uses git mv and keeps both files byte for byte. The four importers, the runtime identity entry, the tests and the docs use the new path. There is no forwarder: the gate retires inferrsi.harness.scenarios as a whole, with import and returning-file controls, and the installed-wheel check requires the old names to be absent and the new ones present. A session planned before this change is refused before any operation and must be resumed with the runtime that planned it. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
…pre-change session The fixture pre-trials-rename-session is a session that the runtime just before the rename started and interrupted with its own installed wheel, unedited: a completed execution receipt for an unsettled round, calls.log at one candidate run, and a plan that names harness/scenarios/external.py. Its bytes match the H200 run that made it. The checkout test is now parametrized over both pre-change sessions, and the installed-wheel check reads every *-session directory under tests/fixtures/optimization-session through INFERRSI_PRE_CHANGE_SESSIONS, which replaces INFERRSI_PRE_INFRA_SESSION in the ready, host and release jobs. Both require the receipt state, the refusal of status and recover, and no file change. The README records the paired-wheel result for each session: the new wheel exits 2 and changes nothing, and the old wheel recovers from the receipt without running the candidate. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
Independent review of 97952bf47e3d1f0e7e8a16e101c426bcbd438755 for #558. I isolated the rename from stacked #559 by comparing cce2159c97d7a76d82620f05c1b19179c0e87eb6 to this head, then checked the full PR against main@10ba7a55. This is the first review of #563.
Required before approval
- Function ownership, line 19 changes the path but still attributes
read_metric/validate_metrictotrials/external.py. That module definesexecute_trialandcollect_trial_checks; document those as the Harness functions and keep semantic metric validation under Benchmark. This is part of #558’s explicit ownership-doc acceptance. - Restack on a base containing the canonical #556 import fix without carrying #559’s duplicate local import. In a three-way check of
tests/test_project_explanation.py,mainremoves the localdigestimport and this stack replaces the same line, producing a conflict. The rename is not the cause, but the conflicting line is present in this PR as submitted.
Verified: both moved source blobs match the stack parent byte-for-byte; the old package is gone, import callers and the runtime-identity key use trials, and the retired-path gate and installed-wheel checks cover the new layout. I read all 14 fixture files and checked the completed zero-exit receipt, calls.log = 2, and that the old identity hash matches the pre-rename source. I did not run the paired wheels, full suite, or inspect CI; the old-wheel recovery result is the author’s record, not my rerun.
Non-blocking: qualify the changelog’s old-session refusal statement to plans covered by this runtime_identity check. Service-reuse dispatches before this comparison; the rename should not change its behavior.
#556 fixed the same stale import in tests/test_project_explanation.py with a module-level import from harness.artifact_primitives. The merge keeps main's file, so this branch no longer changes that test and its diff is the docs and docstrings only. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
The base branch merged main, which carries #556; this merge keeps main's tests/test_project_explanation.py, so the rename branch no longer changes that test. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
|
Reviewed the rename-only range
The package move, importer / runtime-identity updates, retired-path gate and installed-wheel name checks look consistent with #558. On this head I only see draft-lane |
… refusal Review of 97952bf: the function-owner row for harness/trials/external named read_metric and validate_metric, which live in benchmark/metrics.py. It now lists execute_trial and collect_trial_checks as Harness functions and keeps semantic metric validation with Benchmark. The CHANGELOG limits the refusal to benchmark and AB/BA sessions whose plan carries the runtime identity; service-reuse campaigns are dispatched before that check. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
Incremental review: 97952bf47e3d1f0e7e8a16e101c426bcbd438755 → e75dbd3b8b0e4a207c0b6450e599b1e56216ed85. Both blockers in the previous review are resolved. The ownership table now lists execute_trial and collect_trial_checks under Harness and locates semantic validation in Benchmark. The changelog correctly limits the compatibility break to benchmark and AB/BA plans that carry this runtime identity; service reuse is unchanged.
The head contains the updated #559 stack and main@10ba7a55. tests/test_project_explanation.py is byte-identical to that base and is absent from the PR diff, so the earlier import overlap is gone. I checked the exact three-file increment and verified the runtime, fixture and checkout/installed-wheel probes are unchanged versus the previous review. The 14-file fixture and completed receipt were checked previously. The paired-wheel recovery result is the author’s record; I did not rerun the wheels, full suite or inspect CI. No remaining code-review blockers. This approval is not a merge or Issue acceptance; #559 is still a separate stacked dependency.
|
Thanks. Pushed
The H200 A/B was run on |
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed the increment 97952bf47e3d1f0e7e8a16e101c426bcbd438755 → e75dbd3b8b0e4a207c0b6450e599b1e56216ed85.
Both previous blockers are resolved: docs/contracts/function-owners.md:19 now names execute_trial / collect_trial_checks as Harness functions and keeps semantic read_metric / validate_metric under Benchmark in benchmark/metrics.py; the stack includes the updated #559 head caf2517c and main@10ba7a55, and tests/test_project_explanation.py is byte-identical to main (no longer in the PR-versus-main diff). src/ is unchanged in this increment. The changelog now limits the old-session refusal to benchmark and AB/BA plans that carry this runtime identity, and notes that service-reuse campaigns are dispatched before the check.
Ready full-check (3.10/3.11) was still in progress when I reviewed; the other Ready checks on this head had passed. I did not rerun the paired wheels or the full suite. This approval is for the rename follow-up on this head; #559 remains a stacked dependency. Accuracy/performance comparison evidence is not applicable (docs plus a test-import alignment with main).
Main gained #559 as a squash commit and #541's release workflow. The docs and the scenarios initializer conflicted only because this branch already carried #559's text with the rename on top; the merge keeps this branch's versions, and trials/__init__.py equals main's scenarios/__init__.py byte for byte. release.yml merged cleanly and still passes INFERRSI_PRE_CHANGE_SESSIONS to its one check_installed run. Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=xhigh] (author) Signed-off-by: GUOGUO <xwdong1998@163.com>
10bb12b
|
#559 merged, so I merged The conflicts were only in
#541's |
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
Incremental review: e75dbd3b8b0e4a207c0b6450e599b1e56216ed85 → 10bb12bb56255daffd15b369988b6957e0d6e81c. The prior review cleared the ownership-table and stacking issues. This head merges main@15f17037afb6a57a7a51348f73979f81f5e0721f after #559 landed; those fixes are still intact.
I checked both merge parents and compared the full file trees and the release workflow diff. The rename, import gate, saved-session fixture and regression tests are unchanged from the reviewed head. Five changed release files match main exactly. In the one overlapping file, .github/workflows/release.yml, the only difference from the new main is the already-reviewed fixture path passed to the isolated installed-wheel check. The weekly-release settings come from main, not this PR.
No new blockers or non-blocking findings. I did not rerun the installed-wheel or hardware tests on this merge commit or check CI.
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed the increment e75dbd3b8b0e4a207c0b6450e599b1e56216ed85 → 10bb12bb56255daffd15b369988b6957e0d6e81c.
Reviewed base 15f17037afb6a57a7a51348f73979f81f5e0721f and head 10bb12bb56255daffd15b369988b6957e0d6e81c.
This head merges main@15f17037 after #559 landed (and includes #541). The five release files added in this increment match that main exactly. In the one overlapping file, .github/workflows/release.yml, the only difference from main remains the already-reviewed INFERRSI_PRE_CHANGE_SESSIONS fixture path.
The previously cleared blockers remain intact: docs/contracts/function-owners.md still names execute_trial / collect_trial_checks as Harness functions and keeps semantic metric validation under Benchmark; the changelog still limits old-session refusal to benchmark and AB/BA plans that carry this runtime identity. Against main, docs/architecture.md and docs/contributing.md differ only in the scenarios → trials wording, and src/ is unchanged from the previously reviewed head.
lint-and-smoke, installed-package, and host-compatibility have passed on this head; Ready full-check was still running at review time.
There was a problem hiding this comment.
Technical / third-party tip COMMENT (cursor[bot]; collaborator permission none — not write+ / dual-tip / merge-ready).
Reviewed base 15f17037afb6a57a7a51348f73979f81f5e0721f and head 10bb12bb56255daffd15b369988b6957e0d6e81c (Dong1017-authored #558 rename). Tip zengchuang-hw Approve present. Holding formal Approve until Ready full-check is green on this head (currently pending).
Clean-code / repo style (provisional): PASS
- Byte-identical
git mvofharness/scenarios→harness/trials(external.py/__init__.pydigests match main). - No compatibility forwarder;
RETIRED_MODULES+ import-gate controls cover absolute/relative old imports and a returningharness/scenarios/path. - Live importers/docs (
runner,project_service,optimization_session,apps/legacy/run, architecture/module-layout/contributing/moe-forward) useharness.trials. - Remaining
scenariosstrings are intentional: frozen evidence pins, pre-change session fixtures, retirement tests, and historical CHANGELOG lines — not live API drift. - Release-boundary fixture
pre-trials-rename-session/+INFERRSI_PRE_CHANGE_SESSIONSwiring matches the #498/#558 evidence pattern.
Will re-check tip + Ready and submit formal LGTM Approve if unchanged and green.
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 15f17037afb6a57a7a51348f73979f81f5e0721f and head 10bb12bb56255daffd15b369988b6957e0d6e81c.
Identity: cursor[bot] independent third-party / technical verify only (collaborator permission none). Not write+ / dual-tip / merge-ready under docs/agent-collaboration.md. Dong1017 is the author — this vote does not substitute for write+ independents (zeng + amy tip Approves already present).
Clean-code / repo style: PASS
- Byte-identical
git mvharness/scenarios→harness/trials(no forwarder). RETIRED_MODULES+ import-gate cover old absolute/relative imports and a returningharness/scenarios/path.- Live importers/docs on
harness.trials; leftoverscenariosonly in frozen evidence, pre-change fixtures, retirement tests, historical CHANGELOG. - Release-boundary
pre-trials-rename-session/+INFERRSI_PRE_CHANGE_SESSIONSmatches the accepted #558/#498 pattern.
Verified: Ready required checks green on this tip (full-check 3.10/3.11, declarations, installed-package, lint). Prior tip COMMENT held only for Ready; tip unchanged. Not CI-green alone — rename tree and gate/docs were read earlier this round.
Governing Issue and delivery slice
Refs #558
harness/scenarios/to match its trial-execution role #558 as accepted by the goal owner (Dong1017) on 2026-11-21: renameharness/scenarios/toharness/trials/with no forwarder, and carry the same release-boundary evidence as task: retire theinfra/andmodel/packages into their owning modules #498 slice 5.10bb12bmergesmain15f1703, so the diff againstmainis the rename alone.git mv src/inferrsi/harness/scenarios src/inferrsi/harness/trials: both files byte for byte.harness/runner.py,harness/project_service.py,harness/optimization_session.pywith three imports,apps/legacy/run.py), the_runtime_identity()entry, eight test files and five docs use the new path.RETIRED_MODULESaddsinferrsi.harness.scenariosas a whole. New controls cover absolute and relative imports of the old module and a returningharness/scenarios/file, and the not-retired control importsharness/trials.tests/check_installed.py: the old names are absent from the wheel, the new ones present.tests/fixtures/optimization-session/pre-trials-rename-session/is a session that the runtime just before the rename started and interrupted with its own installed wheel, unedited. The checkout test is parametrized over both pre-change sessions. The installed-wheel check now reads every*-sessiondirectory throughINFERRSI_PRE_CHANGE_SESSIONS, which replacesINFERRSI_PRE_INFRA_SESSIONin the ready, host and release jobs.execute_trialorcollect_trial_checks; no other rename. The Harness root regrouping is deferred (task: renameharness/scenarios/to match its trial-execution role #558 scope), and theapps/views/split is a suggestion on task: Apps 应用层详细设计与迁移计划(CLI / Python / Plugin) #371.Knowledge authorship declaration
No path protected by
docs/knowledge-model-policy.mdchanges. The declaration below is disclosure only.Changed artifacts and evidence
git mv;RETIRED_MODULEStest_target_module_import_gate.pyimport and returning-file controls;check_installed.pyharness.scenarios/harness/scenariosoutside history (CHANGELOG, fixtures, retired-name tests) is emptyarchitecture.md,contributing.md,module-layout.md,contracts/function-owners.md(the row now listsexecute_trial/collect_trial_checksas Harness functions and leavesread_metric/validate_metricwith Benchmark),moe-forward-benchmark.mdPaired-wheel record on H200 (CPU only, Python 3.12; each wheel built from its commit's own tree into a clean venv). Old =
75b9872(main7a7ab84plus #559's docs commit; same runtime identity asmain), new =997d92d(the rename commit before the rebase ontocce2159; samesrc/as this head).start0,diagnose0, oneroundinterrupted after its child completed, 130. 14 files,calls.log2,plan.jsonnamesharness/scenarios/external.py, no host user name or/datapath in the session.status2 andrecover2, "Optimization runtime changed, use its frozen runtime to resume"; the SHA-256 of all 14 files is unchanged.recover0, thenstatus0 withcompleted_rounds1;calls.logstill2, so recovery used the receipt and did not run the candidate again.The untouched session is the fixture; its 14 SHA-256 values match the H200 manifest. The fixture README records this and the slice-5 record side by side.
Checks and dependencies
test_target_module_import_gate.py,test_trial_checks.py,test_legacy_run_split.py,test_architecture_closed_loop.py,test_benchmark_operations.py,test_moe_benchmark.py,test_package_root_inventory.pyandtest_knowledge_compatibility_shells.py: 269 passed, 2 skipped, 97 subtests. The two pre-change session tests, the pre-serving test and the runtime-identity test: 4 passed. Ruff passes on every changed file; the 35 F811 hits it reports elsewhere are onmaintoo.75b9872and the candidate7cbac18, comparing everyFAILED,ERRORandSUBFAILEDline: 0 candidate-only failures. Base 433 failed / 5342 passed, candidate 432 failed / 5349 passed (the 7 extra passes are the new gate controls and the second pre-change session case). The one base-only failure is the known flakytest_dsh_plugin.py::test_remote_ssh_connection_and_scoped_operations. The other 432 fail the same way on both sides for host reasons: H200's git 2.34 rejects--no-lazy-fetch, and some tests needplaywright,nodeor network.7cbac18is this branch's tree before later commits; it differs from this head only intests/test_project_explanation.py(nowmain's Use the Harness digest in the interrupted recovery test #556 version),docs/contracts/function-owners.mdandCHANGELOG.md.installed-packageand the host jobs runcheck_installed.pyin CI; I did not run it locally.inferrsi.harness.scenariosmust move to the new path when they rebase. At the time of writing two drafts do: harness: add bounded VATS/GVO runtime and benchmark adapters #488 importsexecute_trialfrom it in three files and also changesharness/scenarios/external.py, which now lives atharness/trials/external.py; Add an optional AISBench service benchmark adapter #326 imports it in one test.Remaining work and Issue closure
harness/scenarios/to match its trial-execution role #558.Provenance
Written in a Claude Code session reported as
claude-opus-5-5, effortxhigh. Each commit has DCO sign-off and anAgent-assisted-byline.🤖 Generated with Claude Code