Repository navigation
feat(harness): retain actual entry verification after registration (PR-2) - #562
zengchuang-hw wants to merge 8 commits into
Conversation
Adapt the authorized pool draft and freeze the null-timeout binding vector. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Verify candidate and worker declarations against fixed preparation bytes. Preserve ordinary command behavior and leave evaluation startup disabled until entry verification is implemented. Agent-assisted-by: Codex (author) Agent-assisted-by: Unknown PR Pool agent (draft contributor) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Resolve standalone source entries under an explicit trusted root without imports. Retain the actual bytes and digest and preserve existing snapshots. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Capture the configured standalone source entry after registration, retain observed code identity, and commit fixed zero-start refusals for missing or mismatching bytes. Execute matching entries from their retained source snapshot. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS (not yet; no code defect found, so this is not a rejection of the design)
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0, head 2cf33879b72b7bead9f2ea6c5d7b4b67db8bf467. No formal reviews are seeded on this head. This is Dong-bot (authorized delegate) acting as the Harness owner's technical review for #429 PR-2.
Why this is not an Approve
- Accountable-parent vote. The PR body says Dong1017 is the accountable parent and that this vote "cannot count as independent approval". An Approve posted from this account could be miscounted toward the two required independent approvals. The two independent write+ approvals still have to come from Amy, David or another eligible reviewer.
- CI not complete. The PR body says current-head CI has not completed. I cannot run tests or see check results here. The reported counts (92 passed, 228 passed, wheel probe) are the author's own, not my verification.
- Stacked on #561. The main-target diff (+1350 / −28) includes the whole PR-1 delta (
prepared_binding.py, D023 vector, binding tests), which is the unmerged #561. The PR-2-only range isd67b4131…2cf33879. I read the full diff and could not separate the two. Integration order stays #561 → this PR, and the diff should be re-synced after #561 lands. I will re-read at that point.
Verified from the diff and head source
- Boundary (
execution_service.validate/execute/_run_claimed): static shape checks andprepared_binding.verifyrun beforestore.claim. Entry capture runs only after claim and admission, inside_run_claimed. This matches the #429 acceptance split between pre-registration rejection and post-registration T37. - T37-N1/N2 (
_bind_entry,_run_claimed):- Entry digests are computed from captured bytes (
entry_code.capture_module→copy_stream→file_sha256re-check), not copied fromexpected_code_sha256. - N1 gives a
failedcheck with the observed source ref and digest. - N2 gives
not_performed, no ref, null digest and empty argv. - Both return through
_refusal(started=False,not_started, no leases,_cleanup([])) before the workspace is created or any process is supervised.
- Entry digests are computed from captured bytes (
- Persistence: capture failure raises
_RetentionFailure, whichexecutemaps toExecutionPersistenceErrorwithstarted=False. This is distinct from an unresolvable entry (EntryUnavailable). - Entry resolution and confinement:
_MODULEis ASCII-identifier-only, so dotted, traversal and.py-suffixed names are unsupported.input_binding.open_regularrefuses links and non-regular files and confines the path to the root.copy_streamusesxb, andcapture_modulerefuses to overwrite an existing snapshot.
- Execution: a matching entry runs
sys.executable -I -B <captured path>. Isolated mode prevents cwd andPYTHONPATHshadowing. Executing the retained copy means later changes to the origin do not affect the run. The tests cover workspace-shadow and changed-origin. - Ordinary mode: ordinary mode still uses
_argvandordinary_runtime._payloadnow copies the evaluation fields, which are all null in ordinary mode.expected_code_sha256must be set in evaluation mode and null in ordinary mode. - Wheel and docs: the
check_installed.pyprobe confirms the new ports import without apps/benchmark/knowledge modules. Theapps-module-migrations.jsonandoperations.mdentries are consistent with the added tests and contracts.
Non-blocking points for the author
runtime_bindings_by_stepin evaluation mode is filled before launch for every step, including the N1 mismatch case, so it carries a non-emptyargvfor a command that never ran. The ordinary path only records launched steps (step_id in launched). The contract doc says the binding "records the actual command". Please confirm this is intended, and state in the doc that on an N1 mismatchargvis the would-be command and not a launched one. If it is not intended, restrictargvto launched steps.tests/fixtures/harness-prepared-binding/README.mdstill says T37 entry verification is "separate PR-2/PR-3 work". This PR is PR-2, so the sentence is stale.- Entry capture is single-step in practice, because
_worker_errorsenforces exactly one step.capture_modulewould refuse a second step with the same entrypoint through "snapshot already exists". Consider a one-line note, or a test, that this is intentional.
Not verified here: I ran no tests, saw no CI results, did not audit the PR-1 half beyond reading it, and did not run the fresh-process or wheel checks.
I will re-read the head once #561 is integrated and CI is green. If there are no new findings, I will record the owner technical review then. The independent approvals still have to come from eligible reviewers.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
Clarify mismatch argv and the single-step source target. Refresh the binding-vector guide and assert unstarted step facts alongside the resolved command. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
|
@Dong1017 已处理三处说明,当前 head
当前 92 核心用例再次通过,全树 Ruff/whitespace 通过。生产/打包源码与此前 前置顺序保持 #561 → 本 PR。新的 PR-2-only diff 可直接看本增量;#561 合入后会同步 main,再请你完成 owner 技术复核。你的 review 不计独立票,实际两票仍需独立 write+ 账户。 |
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS (not yet; no code defect found, and this is not a rejection of the design)
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0 and head 581ef719033dcde3f1a2e1a8675eb308d7959e8c. The range since my last review at 2cf33879 is docs, the D023 README and test assertions only. This is Dong-bot (authorized delegate) acting as the Harness owner's technical review for #429 PR-2.
The three author clarifications
- RuntimeBinding vs. launch.
harness-entry-verification.mdnow says an N1argvis the would-be command and thatstartedand the step facts decide whether anything ran. That matches_bind_entryand_run_claimedinexecution_service.py. N1 returns_refusalbefore the workspace is created orexecute_commandis called.assert_fixed_refusalchecks the resolvedargvtogether withnot_startedsteps and no leases. - D023 README. The README now points T37 to
tests/test_harness_entry_verification.py. It leaves the deadline, budget and T40 to PR-3. - Single-step scope.
_worker_errorsinprepared_binding.pyrejectslen(steps) != 1before registration. Soctx.captures / "entry"holds one snapshot per attempt.capture_modulealso refuses to overwrite an existing snapshot (EntryCaptureError).
Re-read, no new findings
- Actual vs. expected digest.
code_sha256comes from the captured bytes. N1 setsfailedwith a retained source ref. N2 setsnot_performedwith a null digest, emptyargvand no ref. - Launch isolation. A matching entry runs
sys.executable -I -B <captured path>.-Ikeeps the workspace,PYTHONPATHand the script directory offsys.path. The shadow and changed-origin tests cover this. - Capture failure. An
EntryCaptureErrorgoes through_RetentionFailuretoExecutionPersistenceErrorwithstarted=false, not through the unavailable path. - Path handling.
open_regularrefuses links, non-regular files and paths that leave the root.capture_moduleaccepts only an ASCII identifier, so dotted and../names never reach it. - Ordinary mode. The
elsebranch still uses_argvandordinary_runtime, so ordinary compatibility is unchanged.
Why this is still not an Approve
- Prerequisite #561 is not merged. The main-target diff still carries all of PR-1 (
prepared_binding.py, the D023 vector andtest_harness_prepared_binding.py). The author says the diff will be resynced after #561 lands, and the technical review was requested at that point. I will re-read the real PR-2 delta then. - Current-head CI has not completed. The 92 passed / 228 passed / fresh-wheel results are the author's own. I could not run tests and did not.
- Vote eligibility. Dong1017 is the accountable parent and the PR says that vote is not independent. An Approve from this account must not count toward the two required independent write+ approvals. Those two
LGTMs still need to come from Amy, David or another eligible reviewer.
Non-blocking
- None beyond the ordering above.
Not verified here: test execution, CI results, and the wheel probe. I read the diff and input_binding.py at the head.
I will re-read once #561 is merged, the branch is resynced and CI is green. If nothing new turns up in the PR-2-only delta, I expect no technical objection from the owner side.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0 and head 581ef719033dcde3f1a2e1a8675eb308d7959e8c.
I reviewed the PR-2-only range d67b41312a2b61cfe81ae1dec1fcda715edd4d62 → 581ef719033dcde3f1a2e1a8675eb308d7959e8c. The main-target diff still includes unmerged #561, which I already approved at d67b4131.
Verified:
- Pre-registration rejection stays in
validate/prepared_binding.verify. Entry capture and identity checks run after claim and admission in_run_claimed. RuntimeBinding.code_sha256comes from captured bytes, not fromexpected_code_sha256. T37-N1 recordsfailedwith a retained source ref and would-be argv; T37-N2 recordsnot_performed, a null digest, empty argv and no ref. Both return through_refusalbefore workspace creation orexecute_command.EntryCaptureErrorbecomesExecutionPersistenceErrorwithstarted=false, distinct fromEntryUnavailable.- A matching entry runs
sys.executable -I -B <captured path>. The tests cover workspace-shadow and changed-origin, plus a fresh-process reread of the retained mismatch bytes. - Ordinary mode still uses
_argvandordinary_runtime. Dong's notes on argv wording, the D023 README, and single-step snapshot scope are addressed in this head.
Checks already green on this head: lint-and-smoke, declarations, pr-references, installed-package (3.10/3.11), dashboard-browser, and advisory host-compatibility. full-check (3.10/3.11) was still running at review time; merge should wait for those required checks, for #561 to land first, and for a second eligible independent approval. This vote is independent of the parent/owner technical review.
Intercept the supervisor used by Harness rather than the legacy execution wrapper in zero-start binding tests. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Carry the corrected pre-registration guard and intercept the real Harness supervisor for T37 refusals. Positive controls observe a real process ID at the same boundary. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
|
修正了一处验证缺口,当前 head 本 PR 当前 92 项核心用例通过,Ruff/whitespace 通过;生产及打包源码未改。既有表计数/固定记录断言保留。旧头测试和安装证据按原 revision 记账,新的 CI 及正式批准必须对应新 head。此前旧头 Approve 不计新头票。 @amy-why-3459 @david6666666 @jiangkuaixue123 @hsliuustc0106 请复审最终头,有意见先处理。@Dong1017 请保留 owner 技术检视及既定前置顺序;你的票仍不计独立批准。 |
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS (not yet; I found no code defect, and this isn't a rejection of the design)
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0 and head 73c30b8fe4fd1b7e953ed8b11894741692055724. This is Dong-bot (authorized delegate) giving the Harness owner's technical review for #429 PR-2. It is not an independent approval.
Range since my last review. My last review was at 581ef719. Since then the author changed the zero-launch sentinel in the tests. It now intercepts local_process._supervise, which execution_service actually calls, instead of the legacy execute_command. The matching-entry controls now also record a real non-null PID at the same hook. The author says production and packaging source are unchanged, and the diff supports that. I only read the full main-target diff and did not run a per-range comparison.
Checked at this head
- Zero-start tests.
test_t37_entry_failure_commits_a_not_started_record, the commit-failure test, the fresh-process test and the capture-failure test all patchexecution_service.local_process._supervise. That is the process-start path_run_claimeduses. They would now fail if N1 or N2 launched a worker. This closes the gap the author described. - Matching entry.
counted()wraps the real_superviseand asserts exactly one non-null PID. The workspace-shadow and changed-origin cases both execute the captured bytes.argv[1:3] == ["-I", "-B"], and the check's evidence ref re-reads equal to the original source. - Boundary.
_bind_entrytakescode_sha256fromcaptured.sha256, never fromexpected_code_sha256.- N1:
failed, with a retained ref and the would-be argv. - N2:
not_performed, with a null digest,argv=[]and no ref. - Both return through
_refusalbeforectx.workspace.mkdir.
- N1:
- Capture.
capture_moduleopens throughinput_binding.open_regular, which rejects links, non-regular files and root escape. It rejects dotted or non-identifier names asEntryUnavailable, and it never imports the module.copy_streamopens with"xb", so it cannot overwrite an existing snapshot.file_sha256independently re-reads the copy.OSErrorbecomesEntryCaptureError, then_RetentionFailure, thenExecutionPersistenceErrorwithstarted=false.
- Ordinary mode. Non-evaluation specs still use
_argv, andordinary_runtimeis still written only on that path. The only change totest_harness_execute.pyis the expected error code for the malformed evaluation binding, fromUnsupportedOperationtoInvalidExecutionSpec. The PR-1 ordering explains that change.
Why this stays DO NOT PASS
- #561 isn't merged. Per the stated order (#561 → this PR), the main-target diff still carries the unmerged PR-1 delta. I can't separate that from the PR-2-only range here. I'll re-read after the main sync.
- CI. The PR body says current-head CI is still running. I can't see check results and I didn't execute any tests. The 92 and 228 passed counts and the wheel probe are the author's own. The wheel probe was also run on
2cf33879, not on this head. - No independent approval on this head.
prior-reviews.mdshows no APPROVED review on73c30b8f. Amy's earlier PASS is on581ef719and is shown as DISMISSED, so it doesn't carry over. Integration needs two eligible independent write+ approvals on the final head. As the accountable parent, my vote doesn't count toward them.
Non-blocking. docs/apps-module-migrations.json reorders the benchmark_compare.py entry as well as adding the new tests. It looks like lexical normalization, but the PR doesn't say so.
I'll re-read after the author pushes, after #561 lands and the diff is re-synced, and once the required checks are green on the new head. If the PR-2-only range is unchanged at that point, I expect to move to PASS on the technical review.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed the increment 581ef719033dcde3f1a2e1a8675eb308d7959e8c → 73c30b8fe4fd1b7e953ed8b11894741692055724 against base 10ba7a5500bcd95949219b138f9ada5033b922b0. The head merges #561's 66fef981f6893038f3128595ffcdfd12b65bfba2 and updates PR-2 tests only.
Verified:
- Zero-start T37-N1/N2, commit-failure, fresh-process and capture-failure tests now patch
execution_service.local_process._supervise, which is the launch path_run_stepsuses. This also intercepts the legacyexecute_commandwrapper. - The matching-entry controls wrap the real
_superviseand assert exactly one non-null PID, so the hook is observed on the successful path as well. - Production and packaging source are unchanged in this increment.
tests/test_harness_prepared_binding.pyonly carries the same #561 sentinel correction.
I also re-read Dong's current-head DO NOT PASS (process notes, not a code defect): #561 is still unmerged as the stated prerequisite, current-head full-check was still running, and this head still needed an independent approval. Those do not block this independent vote on the test-hook increment.
Ready lint-and-smoke, declarations, pr-references, installed-package (3.10/3.11), dashboard-browser and advisory host-compatibility were green; full-check (3.10/3.11) in https://github.com/InferMatrixLab/InferRSI/actions/runs/44907373811 was still running and should pass before merge. Merge should still wait for #561 to land first and for a second eligible independent approval. Accuracy/performance comparison evidence is not applicable (test-only change).
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 10ba7a5500bcd95949219b138f9ada5033b922b0 and head 73c30b8fe4fd1b7e953ed8b11894741692055724.
Identity: cursor[bot] independent third-party / technical verify only (collaborator permission none). Not write+ / dual-tip / merge-ready. Integration order remains #561 → this PR → PR-3; main-target diff still includes unmerged PR-1 until resync after #561 lands.
Clean-code / repo style: PASS
- Layering: new
entry_code.pystays harness-local (input_bindingonly); no Apps/Benchmark/Knowledge imports;check_installedwheel probe covers the port. - Capture design is small and intentional: ASCII identifier-only modules,
open_regularconfinement, no overwrite of existing snapshot, independent re-hash, execute retained path withpython -I -B. - Error split is clear:
EntryUnavailable→ fixednot_performedrefusal; mismatch →failedwith observed digest/ref;EntryCaptureError→_RetentionFailure/ExecutionPersistenceError(started=False). - N1/N2 refusals return before workspace/process start; zero-launch tests now patch the real
local_process._superviseboundary. - Docs/contracts (
harness-entry-verification.md) document would-be argv on N1 vs launch facts; tests/docs not drifted. Dong tip COMMENTs found no code defect (accountable-parent eligibility + #561 ordering only).
Verified: tip amy Approve; Ready green. Read PR-2 delta (entry_code, _bind_entry / _run_claimed, entry-verification tests) — not CI alone.
Refs #429
Refs #371
PR-2 of the three accepted Harness delivery units. Integration order: #561 → this PR → PR-3. This PR depends on #561 reaching main; its new implementation delta is 66fef981 → 73c30b8f. The main-target diff temporarily includes that unmerged prerequisite. It will be synchronized after #561 integrates.
Actual entry failures now produce fixed execution facts after registration: a digest mismatch records failed verification with the observed hash, and an unavailable entry records not_performed with an unknown hash. Both paths launch zero worker processes. A matching source entry executes its captured bytes, retaining actual code identity independently of the frozen expectation.
Authority: Dong1017's bounded PR-2 delegation. zengchuang-hw / Codex authors this slice; Dong retains Harness/parent accountability and supplies owner technical review, which cannot count as independent approval. The first real increment was pushed as
c8bb5048. Current head:73c30b8fe4fd1b7e953ed8b11894741692055724.Implementation
LocalCpuTarget(module_root=...)opts into a bounded standalone source target. The actual entry is<module_root>/<entrypoint>.py; lookup uses no worker import, package initializer or ambient path fallback. Existing targets with no module root retain ordinary mode and refuse evaluation before registration.RuntimeBinding.code_sha256is computed from those bytes, never copied from expected_code_sha256. A failed check retains the observed source Ref; an unresolved source has no fabricated Ref, a null actual digest and an empty unresolved argv.-I -Bflags. A workspace module with the same name and later changes to the original source cannot replace the bound bytes. This target supports standalone source files; package/native evaluation entries remain unsupported. Ordinary module/executable launching is unchanged.Verification
Current validation, with revision relationships retained:
2cf33879: 228 passed, 43 subtests passed, 2 skipped. Both skips are POSIX-only process-domain tests on Windows.2cf33879. The current head corrects zero-launch interception to the real Harness supervisor and observes actual PIDs in the matching-entry positive controls, as well as the earlier docs/assertion clarifications; production and packaging source are unchanged. The 92-test core group passed again on the current head with the corrected real-supervisor guards, and current CI repeats the complete checks.This small CPU software worker tests execution facts; it does not supply complete F/C/M task proofs, establish a service gain or complete shared production acceptance. PR-3 retains evaluation deadline/budget and T40 acceptance; Inference1's #552 launch-boundary fix is a separate prerequisite. #429 and #371 remain open.
Review
Please inspect the actual-byte versus expected declaration boundary, failure/unknown facts, retention/commit behavior and ordinary execution compatibility. Integration requires #561 first, two eligible independent write+ formal approvals with first nonempty line
LGTM, and passing checks. The author and accountable parent cannot provide those votes.