Skip to content

task: P1 真实执行证据生产与固定记录对接 #565

Open

Description

@zengchuang-hw

Current state: Accepted capture boundary v1 (design, acceptance/delegation); Apps delegation is recorded in 7064458592. The real p1-cpu-smoke@1 / p1-cpu-smoke-rules@1 task and B worker/profile delegation are accepted in 7064895332, with the limited default-clock revision accepted in 7065967644. Dong1017 retains #429/#434 accountability; zengchuang-hw / Codex authors these bounded slices.

Delivery slice Review artifact Status and remaining work
B report codec and pure projection #566, 0ef4b078 Ready CI passed; Amy independent approval; second independent approval and #560 integration needed
H capture and producer sealing #567, b5199316 Complete H-only diff reviewed by Amy and Dong; earlier incomplete-diff blocker explicitly cleared; David approval and dependencies needed
Apps read-only role adapter and fixed views #568, abc6de4d Ready CI passed; Amy independent approval and Dong owner-boundary review; second independent approval and dependencies needed
Real CPU worker/profile #569, 2e203ae6 Source/input pins published before cases; 25 focused / 630 expanded tests passed (expanded includes focused); 7 skips / 3 existing tarfile warnings; fresh wheel passed; Dong owner review clears prior blockers; Amy/David independent review pending

Recorded failures: CPU revision 1 produced C passed / samples=null with the coarse default clock and remains insufficient M evidence. Revision 2's initial pipeline run found omitted registration of the accepted task in the result validator (16 passed / 1 failed); the registration fix preserves worker/input pins and all claim/timing rules. These failures remain in #569's record, not relabeled PASS.

Next action: Obtain Amy/David independent formal approvals, address findings, integrate prerequisites in order, then sync review-only stacks back to main and run final-head CI with fresh approvals. #567/#568/#569 must not merge into temporary review bases. Shared production CPU acceptance still waits for actual main integration, #552's separately owned finite-step-timeout fix, and a fresh integrated run. No task, timing or claim decision is waiting now; acceptance evidence and integration remain open.

Goal and acceptance

Make the first frozen P1 task produce real, retained, attributable evidence that the production offline assessor can consume, while Apps reads and presents one committed RecordRef.

Reproduced gap

An isolated combination of #560 head 4887f1e94f39c1d4b675e965cfcc0a6e932d50a2 and #564 head d34a0451f0d4edd76c604632b44d673f4f5d43ae was tested. The local validation-only merge resolves the consumer inventory by set union; it is not a new implementation or a merged main revision.

  • Both configured worker cases actually start once with a real PID; actual source SHA matches the frozen recipe, the grader output is available and retained, and cleanup is confirmed.
  • One candidate yields a worker-produced C proof with status=passed, another with status=failed. Both proof byte payloads satisfy the current B proof parser against the actual candidate digest. This diagnostic worker freezes synthetic attempt/candidate IDs before prepare; it is deliberately not the production runtime-context solution.
  • H commits only entry_code_identity and input_identity CheckObservations. The raw grader output is an ArtifactObservation.
  • B assessment._observations matches only record.checks, requiring format, grader, samples, cleanup. It therefore never selects either real grader output. Both cases return F/C/M INCONCLUSIVE, C missing_evidence=["grader"], and Apps exit 9.
  • Apps prepare/execute/assess all publish standard manifest/result files. The fixed assessment and trace use the same RecordRef. Reentry returns that original Ref with one attempt, one record, and zero additional worker starts.

Evidence: 2 diagnostic tests passed. This establishes the integration gap and safe fallback; it does not establish complete F/C/M, L4, service gains, or whole-Issue acceptance.

Sources: H output retention, B proof selection, frozen P1 recipe/evidence requirements.

Accepted ownership and functional boundary

  1. Benchmark: own the versioned P1 worker-report format, task-specific pure validation/projection, grader implementation and exact resource closure. Specify how real candidate bytes, format results and ordered AB/BA samples become the required proof bytes. No invented measurements or caller-declared success.
  2. Harness: own the capture lifecycle and actual runtime context. Before committing, retain raw reports and any derived proof bytes, associate checks with the actual attempt/input/worker, and seal producer identity/control domain/capture route from observed facts. Format/cleanup receipts must have the frozen producer kind and actual control domain; worker JSON cannot impersonate Harness. No import of Benchmark/Apps and no grader rule interpretation in Harness: configure any task-specific pure observer/projector at the host boundary.
  3. Apps: compose the configured ports and read the resulting immutable wire records. Keep report, Dashboard and trace on the same fixed RecordRef/AssessmentRef. Apps must not add evidence to an already committed record or recalculate grades.

The accepted capture contract specifies the concrete callback/wire, runtime context, validation/refusal behavior, provenance and pre-commit retention. It preserves existing CheckObservation matching. Raw ArtifactObservations are not treated as equivalent checks.

Completion criteria

  • Owner records accepted scope, concrete H↔B host boundary, accountable parent, author and acceptance authority: 7063820381.
  • Publish the first versioned worker-report / pure projection contract; freeze its supported task, schema, producer and rule revisions.
  • Real execution produces the four required evidence families from actual observations, with the actual attempt/input/entry/domain binding and retained raw/derived refs before record commit.
  • Real positive and negative task cases can distinguish the frozen independent claims; missing, malformed, mismatched, tampered or conflicting evidence keeps the existing conservative V21/V22 behavior.
  • Unknown report/rule revisions refuse explicitly; worker-provided identity or cleanup assertions cannot override H facts.
  • No archive reopening executes a worker; completed/running/no-record/conflict reentry retains original attempt, digest, deadline and budget.
  • Apps manifest/result, archived assessment, Dashboard and trace agree on the fixed references; ordinary CLI and plugin compatibility remains covered.
  • Run the shared D023 / t108-v4 CPU matrix (including T35/T36/T37/T39/T40/V23 as applicable) only with the actual integrated dependencies. Report PR readiness and whole-goal acceptance separately.

Scope and constraints

In scope: the first supported P1 production evidence bridge. Out of scope: new generic service platforms, paid-budget providers, service/E2E performance claims, revising frozen historical vectors, weakening evidence requirements, or importing B/Apps business code into H.

Dependencies: #530 prepare is merged; #560 offline assessor and #561 → #562 → #564 execution/replay are under review. Inference1 retains the separate #552 finite-step-timeout fix. This new task does not take over that change or silently extend the three already delegated Harness PRs.

Ownership and authorization

Reproducible diagnostic source (two real worker cases)

Combine the two pinned PR heads in an isolated checkout, retain both canonical-consumer lists, save the following outside tracked source, then run PYTHONPATH=src:. python -m pytest <probe_file> -q. Windows uses its normal semicolon-separated PYTHONPATH. The source has no private identifiers or host configuration. Successful tests assert the current gap; they are not a fix.

"""Integration diagnosis only: real worker bytes, no synthetic CheckObservations."""

import json
import sqlite3
from copy import deepcopy
from hashlib import sha256
from pathlib import Path

import pytest

from inferrsi.apps import api
from inferrsi.apps.storage.archive import read_prepared
from inferrsi.apps.storage.artifact_reader import ArtifactReader
from inferrsi.benchmark import assessment, preparation as benchmark
from inferrsi.contracts.json_v1 import decode
from inferrsi.harness import execution_service, run_store
from inferrsi.harness.api import Harness, LocalCpuTarget
from tests.test_benchmark_preparation import preparation as preparation


@pytest.mark.parametrize("candidate_bytes,worker_status", [
    (b"accept\n", "passed"), (b"reject\n", "failed"),
])
def test_real_retained_worker_output_has_no_matching_check_carrier(
    preparation, tmp_path, monkeypatch, candidate_bytes, worker_status,
):
    _, prepare_request, sources, protocol, _, config, archive = preparation
    vector = decode((Path.cwd() / "tests/fixtures/harness-prepared-binding/"
                     "d023-null-timeout.json").read_bytes())
    worker = b"""import hashlib, json, pathlib, sys
candidate = pathlib.Path(sys.argv[1]).read_bytes()
status = 'passed' if candidate == b'accept\\n' else 'failed'
proof = {'schema_version': 1, 'attempt_id': 'att-1', 'operation': 'evaluate',
         'validation_scope': 'p1', 'candidate_id': 'cand-1',
         'actual_sha256': hashlib.sha256(candidate).hexdigest(),
         'claim_id': 'C', 'checker_revision': '1', 'status': status,
         'reasons': [] if status == 'passed' else ['probe_rejection']}
output = pathlib.Path(sys.argv[2])
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(proof), encoding='utf-8')
"""
    sources["worker-code"].write_bytes(worker)
    protocol = deepcopy(protocol)
    protocol["worker_recipe"]["code_sha256"] = sha256(worker).hexdigest()
    pin = next(x for x in protocol["resources"] if x["role"] == "worker-code")
    pin["artifact"].update(sha256=sha256(worker).hexdigest(), size_bytes=len(worker))
    for required in protocol["required_evidence"]:
        required["allowed_control_domains"] = [execution_service._CONTROL_DOMAIN]
    service = benchmark.PreparationService(
        protocol, resources=lambda role: sources[role].open("rb"), worker_role="worker-code",
    )
    prepared_output = api.prepare(
        {"schema_version": 1, "request": prepare_request}, workspace=tmp_path,
        storage_config=config, prepare_benchmark=service.prepare,
        encode_prepared=benchmark.encode, validate_prepared=benchmark.validate_prepared,
        output_id="probe-prepare",
    )
    assert prepared_output.result["exit_code"] == 0
    prepared_ref = prepared_output.result["prepared_ref"]
    prepared = read_prepared(config, prepared_ref)
    (archive / "candidate.txt").write_bytes(candidate_bytes)
    candidate_ref = {"location": "archives/local/candidate.txt", "media_type": "text/plain",
                     "sha256": sha256(candidate_bytes).hexdigest(),
                     "size_bytes": len(candidate_bytes)}
    proposal = {k: deepcopy(vector[k]) for k in ("candidate", "inputs", "generation_record_ref")}
    proposal["inputs"][0].update(source=candidate_ref["location"], source_ref=candidate_ref,
                                 expected_sha256=candidate_ref["sha256"])
    request = {
        "schema_version": 1, "prepared_ref": prepared_ref,
        "attempt": {key: vector[key] for key in execution_service.ATTEMPT_FIELDS},
        "proposal": proposal, "output_root": vector["output_root"],
    }
    modules = tmp_path / "modules"
    modules.mkdir()
    (modules / "fixture_worker.py").write_bytes(worker)
    store = run_store.RunStore(tmp_path / "store", "test")
    target = LocalCpuTarget("fixture-cpu", "1", work_root=tmp_path / "work", module_root=modules)
    reader = ArtifactReader(config)
    harness = Harness(store, target, artifact_reader=reader)
    supervise = execution_service.local_process._supervise
    pids = []

    def counted(*args, **kwargs):
        result = supervise(*args, **kwargs)
        pids.append(result["pid"])
        return result

    monkeypatch.setattr(execution_service.local_process, "_supervise", counted)
    executed = api.execute(request, workspace=tmp_path, storage_config=config,
                           execute_harness=harness.execute, output_id="probe-execute")
    assert executed.result["exit_code"] == 0, executed.result
    record = executed.result["execution"]
    record_ref = executed.result["record_ref"]
    assert record["started"] is True and record["process_outcome"] == "exited"
    assert len(pids) == 1 and pids[0] is not None
    assert record["cleanup"]["status"] == "confirmed"
    assert record["target_binding"]["control_domain"] == execution_service._CONTROL_DOMAIN
    assert record["target_binding"]["runtime_bindings_by_step"][0]["code_sha256"] == sha256(worker).hexdigest()
    assert {x["check_id"] for x in record["checks"]} == {"entry_code_identity", "input_identity"}
    output = next(x for x in record["artifacts"] if x["role"] == "grader")
    assert output["status"] == "available"
    with reader.open(output["artifact"]) as stream:
        raw = stream.read()
    proof = json.loads(raw)
    assert proof["status"] == worker_status
    required = next(x for x in prepared["required_evidence"] if x["check_id"] == "grader")
    # Diagnostic only: the bytes themselves satisfy B's existing proof parser.
    assert assessment._proof(raw, required, record, candidate_ref["sha256"]) == proof

    assessed = api.assess(
        {"schema_version": 1, "record_ref": record_ref,
         "primary_scope": "p1", "primary_claim_id": "C"}, workspace=tmp_path,
        storage_config=config, assess_benchmark=assessment.assess,
        validate_assessment=assessment.validate_assessment, encode_assessment=assessment.encode,
        output_id="probe-assess",
    )
    assert assessed.result["exit_code"] == 9, assessed.result
    result = assessed.result["assessment"]
    assert [(x["claim_id"], x["verdict"]) for x in result["claims"]] == [
        ("F", "INCONCLUSIVE"), ("C", "INCONCLUSIVE"), ("M", "INCONCLUSIVE"),
    ]
    assert result["claims"][1]["missing_evidence"] == ["grader"]
    assert result["record_ref"] == record_ref
    view = api.read_attempt_view(record_ref, storage_config=config,
                                 assessment_ref=assessed.result["assessment_ref"], include_trace=True)
    assert view["record_ref"] == record_ref
    assert view["assessment"]["claims"] == [
        {k: x[k] for k in ("claim_id", "validation_scope", "verdict", "reasons", "missing_evidence")}
        for x in result["claims"]
    ]
    assert {x["check_id"] for x in view["trace"]["checks"]} == {"entry_code_identity", "input_identity"}
    assert len(list(tmp_path.rglob("manifest.json"))) == 3
    assert len(list(tmp_path.rglob("result.json"))) == 3

    def forbidden(*args, **kwargs):
        pytest.fail("completed attempt was implicitly executed again")

    class Never:
        open = forbidden

    monkeypatch.setattr(execution_service.local_process, "_supervise", forbidden)
    replay = Harness(store, LocalCpuTarget("retired", "2", work_root=tmp_path / "retired"),
                     artifact_reader=Never())
    replayed = api.execute(request, workspace=tmp_path, storage_config=config,
                           execute_harness=replay.execute, output_id="probe-replay")
    assert replayed.result["record_ref"] == record_ref
    assert replayed.result["execution"]["request_digest"] == record["request_digest"]
    with sqlite3.connect(store.root / run_store.INDEX_NAME) as conn:
        assert conn.execute("SELECT count(*) FROM attempts").fetchone()[0] == 1
        assert conn.execute("SELECT count(*) FROM records").fetchone()[0] == 1

Activity

  1. added
    kind:taskBounded delivery task with a delegated slice
    area:harnessHarness module: profile, A/B loop, quality gates, evidence reports
    priority:P0Decide or start now; blocks other work. Only with status:proposed or status:accepted
    status:proposedWaiting for the decision named in the status block
    area:benchmarkBenchmark line: preparation ports, metric contracts, validators, and isolated workers
    on Nov 21, 2026
  2. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted as a bounded follow-up, keeping CheckObservation matching and relaxing no frozen P1 rule. Dong-bot (authorized delegate) records this; it is not ready to code until the capture contract below is written.

    Accepted

    Open before implementation

    1. The concrete H↔B boundary: callback signature, runtime-context inputs and refusal codes, as a short design comment or doc PR.
    2. Named implementation author(s) and an independent reviewer. A repository member claims a part with @Dong1017 claim <part>; I am not assigning anyone.
    3. The synthetic att-1/cand-1 IDs in the probe must be replaced by runtime binding.

    Next step: post the boundary design, then claim the Benchmark and Harness slices separately.

    Details
    • Not verified: I did not run the probe or read the feat(benchmark): add frozen P1 offline assessment #560/fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564 heads. The gap analysis (B assessment._observations matches only record.checks; H commits only entry_code_identity and input_identity) rests on the issue text.
    • The two-PR set-union merge is validation-only. Re-run the probe on real merged heads before treating it as evidence.
    • Keep PR readiness and whole-goal acceptance (D023/t108-v4 CPU matrix) as separate reports.
    • Suggested negative tests: worker JSON claiming producer=harness, a cleanup assertion that contradicts H facts, an unknown report or rule revision, and an AB/BA sample-order conflict.
  3. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    H↔B capture boundary proposal v1

    @Dong1017 依据 接受范围但暂不编码的决定,先提交可实施的接口设计。建议接受以下边界后分别委派 B / H 两个切片;不改现有 ExecutionSpec / CheckObservation / P1 proof 的 v1 字段或匹配规则。

    1. 配置入口与调用时序

    Harness(store, target, *, artifact_reader=None, evidence_capture=None)
    
    # H owns this duck-typed host capability; no B/Apps import.
    evidence_capture.capture(
        phase,                  # "preparing" | "finalizing"
        facts,                  # defensive snapshot of H-owned capture context
        *, artifacts,           # read-only access to this attempt's retained bytes only
    ) -> (draft_wire, proof_blobs)
    
    # B owns pure functions; an Apps host adapter performs reads/field conversion.
    benchmark.proofs.project(
        prepared, protocol_bytes, phase, facts, *, candidate_bytes, report_bytes
    ) -> (draft_wire, proof_blobs)
    • evidence_capture=None preserves current behavior. Ordinary mode never calls it. Configuration is trusted host code, not a callback/class name supplied by the candidate or ExecutionSpec.
    • Reentry retains fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564's ordering: static validation → canonical request snapshot/digest → existing attempt table. Completed/conflict/running/no-record paths never read/call the capture capability.
    • Fresh execution: existing admission → actual source/input capture → preparing capture before the worker starts → existing worker/supervisor → output retention and cleanup/lease observation → finalizing capture before the one record commit.
    • The capture operation is a bounded observer/projector: no subprocess/network, no new deadline/budget/resource admission, no additional worker or retry. H checks original cancellation/deadline at the existing launch boundary. Capture work is part of the original attempt, not a new window.

    2. H-owned context and read capability

    facts is a fresh snapshot of the existing typed record fields needed for correlation: request/run/attempt IDs, request digest, prepared_ref/protocol_digest/operation/scope/candidate, actual BoundInputs, actual TargetBinding/runtime pins, started-step observations, and (finalizing only) outcome/cleanup/resource_leases. Fields unavailable at a phase stay explicit null/empty; no guessed terminal fact.

    artifacts.open(ref) authorizes only the exact Refs actually retained by this attempt at this phase, independently checks size/hash, returns bytes/read-only streams, and never reads an ambient path or arbitrary archive. No writable context/store handle is exposed. H snapshots both incoming and returned wire values; callbacks cannot mutate internal spec, budget, binding or terminal facts.

    The Apps adapter maps these existing H wire fields explicitly, reads the referenced captured candidate/report, then calls B's pure projector. There is no new shared Python business DTO package. B imports neither H nor Apps.

    3. Versioned raw worker report and proof draft

    Freeze the previously nominal output fixture-grader@1 as a closed worker-report format:

    {
      "schema_version": 1,
      "grader": {"status": "passed", "reasons": []},
      "samples": [
        {"sample_id":"A1","role":"baseline","round":1,"value":"<positive Decimal>","unit":"ms"},
        {"sample_id":"B1","role":"candidate","round":1,"value":"<positive Decimal>","unit":"ms"},
        {"sample_id":"B2","role":"candidate","round":2,"value":"<positive Decimal>","unit":"ms"},
        {"sample_id":"A2","role":"baseline","round":2,"value":"<positive Decimal>","unit":"ms"}
      ]
    }

    The two keys are required but may be explicit null when that family was not completed. The raw report carries no producer/domain, cleanup confirmation, attempt/candidate identity or Ref. Unknown keys, duplicate keys, floating/nonfinite numeric tokens and unsupported revisions refuse that report. In particular a worker cannot label itself harness_capture. Actual IDs/hashes come from H facts when B builds the existing proof envelopes; no synthetic IDs, environment injection or change to the frozen worker argv is needed.

    Draft wire v1 is closed:

    {schema_version:1, checks:[{
      check_id, checker_revision, subject, phase, artifact_schema,
      status, reasons, proof_key, source
    }], errors:[{code, check_ids, reason}]}
    

    proof_blobs is a separate in-process map proof_key -> bytes; no paths, precommitted Refs, producer or attempt fields. Valid sources are candidate_capture, worker_report, cleanup_capture. H validates the envelope/keys and source availability, snapshots bytes, retains each proof itself, computes each Ref, and creates the actual CheckObservation. The raw report remains separately retained and linked through the existing ArtifactObservation.

    • Preparing: B validates the captured UTF-8 candidate and creates the existing format-proof@1; H seals source=candidate_capture as harness_capture.
    • Finalizing: B validates the actual frozen worker's raw report, projects existing grader-proof@1 and samples-proof@1 from its completed families, and binds their IDs/hash to H's actual candidate. H seals source=worker_report as task_worker only when the observed entry/code pin and started input-bound step agree.
    • Cleanup: B formats the existing cleanup-proof@1 solely from H's final cleanup/lease facts. H seals source=cleanup_capture as harness_capture. A worker cleanup assertion is rejected; unknown/failed H cleanup never becomes confirmed.

    The CheckObservation producer revision/identity/control_domain/capture_route comes from the configured capture implementation and H's actual capture/runtime facts, never from worker JSON. Domains are frozen into a fresh preparation before execution; historical fixture-domain vectors remain untouched.

    4. Refusal and failure behavior

    • Unsupported configured task/grader/rule or capability schema is rejected during host capability construction with the existing UnsupportedRuleVersion / UnsupportedSchema; no attempt or worker start.
    • Wrong callback wire, attempted identity override or incompatible actual input/entry is explicitly InvalidExecutionSpec / IdentityMismatch as appropriate. No caller-supplied proof is accepted as an observed fact.
    • After a worker has run, malformed/unknown-version/missing report families do not erase execution facts or trigger a rerun. Commit matching not_performed checks with empty proof refs and explicit reasons; retain the raw report and record a corresponding UnsupportedRuleVersion, UnsupportedSchema or InvalidRequest execution error. B keeps the affected claims INCONCLUSIVE; independently valid families remain independent.
    • A retained proof/status mismatch is not silently normalized. Contradictory valid reports remain separate trustworthy proofs so V21 yields INCONCLUSIVE/conflict.
    • Failure to retain proof bytes or commit the record follows the existing ExecutionPersistenceError / partial facts / retained refs path with the actual started state. No valid RecordRef before commit, no fallback direct RunStore write from Apps.
    • Unknown host programming exceptions remain visible; do not convert them to evidence unavailable or success. Reentry for an uncommitted attempt remains RecoveryRequired.

    5. B task semantics and delivery boundaries

    This proposal specifies transport, provenance and existing proof projection. It does not invent an actual grader predicate, baseline workload, AB/BA timings or success markers. The documentary P1 rules define how trustworthy proofs are assessed; they are not an implementation of a real task library (current contract).

    The B slice must specify and freeze its actual worker/grader/sampler behavior and complete code/input resource pins before shared candidate runs. If a new predicate or baseline needs a new task/grader revision, record and support that version explicitly; do not reinterpret p1-rules@1 or use label/decimal parameters as an invented gate. Missing actual task semantics remains an explicit acceptance gap even when the codec/capture PRs are ready.

    Proposed bounded deliveries:

    1. B: worker-report codec, pure projector and strict proof/correlation tests; specify the real worker/grader/sampling resource contract.
    2. H: optional capture lifecycle, snapshot/read capability, retention/sealing, replay/commit-failure tests. No B/Apps imports.
    3. Apps integration/acceptance: configure the adapter, run actual supplied/frozen worker on merged dependencies; manifest/result, assessment, Dashboard and trace must agree on the same refs. No Apps business grading.

    Validation includes worker producer/cleanup spoofing, actual-ID/hash mismatch, missing/unknown report versions, malformed bytes, contradictory and reordered samples, partial independent outcomes, retention/commit failures, all replay states with zero new starts, cold imports, installed packages and ordinary CLI/plugin compatibility. Current combined regression on the validation-only B+H merge is 367 tests, 0 failures/errors/skips, including the two real-worker diagnostic cases; it is not the final merged-head acceptance.

    Please record corrections/acceptance of this boundary and the separate B/H authorship claims before implementation. Proposed independent reviewers: @amy-why-3459 and @david6666666, subject to their availability; no review is assigned or counted by this design comment.

  4. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    @Dong1017 claim Benchmark:zengchuang-hw / 本 Codex 会话申请 #565 的 B 切片,范围为上面的版本化 worker-report codec、纯 proof projector、严格负例测试及真实 worker/grader/sampling 资源规格。编码以你确认 capture boundary 与任务语义为前置;不会自行发明 P1 grader 或 AB/BA baseline。请明确限定委派和接受人。拟请 Amy/David 做独立检视,未把拟请求当作已批准。

  5. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    @Dong1017 claim Harness:zengchuang-hw / 本 Codex 会话另行申请 #565 的 H 切片,范围为上面可选 capture 回调、当前 attempt 只读 capability、提交前留存/producer 封存及 replay/commit-failure 回归。不改冻结 wire 字段、不导入 B/Apps、不接管 #552。编码等待边界接受和本切片明确委派;董仍为 parent/端口责任人,owner 技术票不计独立批准。

  6. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted: capture boundary v1 as written, with the B and H slices delegated to @zengchuang-hw. #429/#434 stay the accountable parent. This builds on the earlier scope decision and replaces nothing in it. (Dong-bot, authorized delegate.)

    Delegation

    Gate, not accepted yet: the real worker/grader/sampler semantics and resource pins. The codec and capture work can proceed. Shared D023/t108-v4 acceptance stays blocked until that spec is posted and accepted separately.

    Next: open the H and B PRs separately. Neither may close #371/#429/#430/#434.

    Details

    Accepted clauses

    • Optional evidence_capture on Harness, with None keeping current behavior.
    • Two phases, preparing before the worker starts and finalizing before the single commit.
    • Replay, conflict, running and no-record paths never call the capture.
    • H-owned facts snapshot and a read-only artifacts.open(ref) limited to this attempt's retained Refs.
    • Closed worker-report v1 with no identity, producer or cleanup fields.
    • Closed draft wire v1, with proof_blobs kept separate.
    • H alone retains proofs, computes Refs and seals producer, domain and route.
    • The §4 refusal and failure rules. After a worker has run, commit not_performed checks and an execution error, with no rerun.
    • Existing CheckObservation matching is kept, and raw ArtifactObservation equivalence stays rejected.

    Notes for the PRs

  7. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-opus-5-5).

    Claimed for the PR pool (scope: Harness:zengchuang-hw / 本 Codex 会话另行申请 #565 的 H 切片,范围为上面可选 capture 回调、当前 attempt 只读 capability、提交前留存/producer 封存及 replay/commit-failure 回归。不改冻结 wire 字段、不导入 B/Apps、不接管 #552。编码等待边界接受和本切片明确委派;董仍为 parent/端口责任人,owner 技术票不计独立批准。). A coding agent (claude-opus-5-5, effort xhigh) first plans the work: when it does not fit one run of 120 minutes, it posts a suggested split here instead of starting. Otherwise it opens a PR, or explains here why it cannot. The PR is merged only after 2 independent approvals.

  8. added
    status:acceptedThe decision owner accepted it in a recorded comment
    and removed
    status:proposedWaiting for the decision named in the status block
    on Nov 21, 2026
  9. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-opus-5-5).

    The PR pool already has a task here that covers this part: 565 (Harness:zengchuang-hw / 本 Codex 会话另行申请 #565 的 H 切片,范围为上面可选 capture 回调、当前 attempt 只读 capability、提交前留存/producer 封存及 replay/commit-failure 回归。不改冻结 wire 字段、不导入 B/Apps、不接管 #552。编码等待边界接受和本切片明确委派;董仍为 parent/端口责任人,owner 技术票不计独立批准。, status running). Claim a different part, or wait until it ends.

  10. amy-why-3459 commented on Nov 21, 2026

    @amy-why-3459
    Collaborator

    I have read this proposed P1 evidence-bridge follow-up, the diagnostic write-up, and Dong-bot’s 7063759183.

    The gap as stated is consistent with the current ports: on the isolated #560 head 4887f1e94f39c1d4b675e965cfcc0a6e932d50a2 plus #564 head d34a0451f0d4edd76c604632b44d673f4f5d43ae combination, Harness retains raw grader output as an ArtifactObservation while Benchmark _observations selects only record.checks, so the assessor correctly stays INCONCLUSIVE with missing_evidence=["grader"] rather than inventing a claim. Treating that two-PR set-union checkout as a validation-only probe, not a merged main revision, is the right framing; please re-run it on the real #561 → #562 → #564 + #560 integration before citing it as production evidence.

    I agree with the bounded acceptance already recorded: keep CheckObservation matching, do not treat a raw ArtifactObservation as a check, keep #429/#434 as the accountable parent, and do not let a child PR close #371/#429/#430/#434. The issue body and labels still say Proposed / “Decision needed”; please sync Current state to that acceptance and to the three items still open before implementation:

    1. The concrete H↔B capture entry (callback or wire, runtime-context inputs, refusal codes) as a short design comment or doc PR.
    2. Named implementation author(s) and an independent reviewer, claimed separately for Benchmark and Harness.
    3. Replace the probe’s synthetic att-1 / cand-1 IDs with runtime binding.

    One clarification I would like pinned in that boundary note: the probe’s C worker JSON only exercises the grader carrier, yet F, C and M all come back INCONCLUSIVE. Please state explicitly whether F and M are inconclusive solely because their required format / samples / cleanup families were also absent as checks (expected for this diagnostic), and how the first production slice will produce all four required families from actual observations, not only promote the grader artifact. The completion criteria already require the four families; the first implementation claim should not be limited to C/grader.

    I have not re-run the embedded probe. No relaxation of frozen P1 evidence rules is needed or accepted here.

  11. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-opus-5-5).

    The PR pool agent did not take this (Harness:zengchuang-hw / 本 Codex 会话另行申请 #565 的 H 切片,范围为上面可选 capture 回调、当前 attempt 只读 capability、提交前留存/producer 封存及 replay/commit-failure 回归。不改冻结 wire 字段、不导入 B/Apps、不接管 #552。编码等待边界接受和本切片明确委派;董仍为 parent/端口责任人,owner 技术票不计独立批准。):

    #565 H capture slice: blocked on unmerged Harness dependencies

    Why this cannot be done well on main (15f1703)

    The accepted capture boundary v1 (issue comments 7063812024 / 7063820381) is written
    against the evaluation-mode Harness from #561 → #562 → #564. Those PRs are still under
    review, and main does not have what the slice needs:

    • Evaluation mode is refused. execution_service.validate raises
      UnsupportedOperation("evaluation mode is not implemented in this slice")
      (src/inferrsi/harness/execution_service.py:340-341). It is tested in
      tests/test_harness_execute.py:439. The boundary says ordinary mode never calls the
      capture. Its preparing facts (prepared_ref, protocol_digest, operation, scope,
      candidate) and B's projector input (prepared) only exist in evaluation mode. On
      main the hook would never run and its P1 format/grader/samples/cleanup checks could
      not be tested end to end.
    • No entry-code pin. runtime_bindings_by_step[*].code_sha256 is hard-coded None
      ("T37 is a later slice", lines 888-889), and LocalCpuTarget has no module_root.
      The worker_report → task_worker seal is allowed only "when the observed entry/code
      pin and started input-bound step agree", so it has no fact to check.
    • Replay ordering is fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564's. The accepted text keeps "fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564's ordering: static
      validation → canonical request snapshot/digest → existing attempt table" with
      completed/conflict/running/no-record paths. The in-scope replay regressions are
      written against that version of execute and run_store, not main's.

    Writing the slice on main would mean one of two things:

    1. Re-implementing evaluation mode, T37 and replay. That takes over feat(harness): verify frozen preparation before registration #561/feat(harness): retain actual entry verification after registration (PR-2) #562/fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564, and
      the issue explicitly forbids "silently extend[ing] the three already delegated
      Harness PRs".
    2. Shipping a hook that is never called and cannot be tested. Its diff would also clash
      with those PRs in the same functions (validate, _run_claimed, execute,
      LocalCpuTarget).

    Neither is a reviewable, PR-ready change. This run cannot merge or rebase onto the PR
    heads, and it has no git access.

    What would unblock it

    One of these:

    Also worth deciding before coding: when the preparing capture returns a valid draft
    whose format check is failed or not_performed, does H still start the worker? Or
    does it commit the check and refuse to start? Dong-bot's note only rules out a rerun.

    Suggested split once unblocked

    The whole slice is unlikely to fit one 120-minute run. Suggested order:

    1. H capture lifecycle. Optional evidence_capture on Harness and execute
      (None = current behavior, never called on reentry). preparing call before launch
      and finalizing call before the single commit. Defensive facts snapshot.
      Read-only artifacts.open(ref) limited to this attempt's retained Refs, with
      size/hash checks. Strict draft-wire v1 and proof_blobs validation. H-side proof
      retention, Ref computation and CheckObservation creation. Unit tests with a fake
      capture.
    2. H sealing, failure semantics and regressions. Producer/domain/route sealing for
      candidate_capture, worker_report (entry/code pin agreement) and
      cleanup_capture. Spoof and cleanup-contradiction refusal. Post-run
      malformed/unknown report → not_performed checks plus an execution error, with no
      rerun. Proof retention and commit failure → ExecutionPersistenceError path.
      Completed/conflict/running/no-record replay with zero capture calls and zero worker
      starts. No B/Apps import check.
  12. amy-why-3459 commented on Nov 21, 2026

    @amy-why-3459
    Collaborator

    I've seen the description update. Current state now records the 7063820381 acceptance of capture boundary v1 and the limited B/H delegation (B: codec / pure projector / negative tests only; H: capture slice), with Dong remaining accountable for #429/#434. Decision needed correctly keeps the real worker/grader/sampler semantics and resource pins as a separate, still-unaccepted gate, so shared D023/t108-v4 acceptance stays blocked until that spec is posted and accepted. That matches the recorded decision.

    For the new Next action — opening B and H as separate Ready PRs, and requesting Amy and David as required independent reviewers without pre-counting approvals — please also carry the accepted PR notes from 7063820381 into the PR descriptions: replace the probe's att-1/cand-1 with runtime binding; add the listed negative tests; and in the H PR, clarify what happens when preparing format validation fails (the recorded expectation is a committed failed or not_performed format check, still with no rerun). If a PR is stacked on the still-unmerged #560/#561/#562/#564 heads, please get explicit stacking approval from the #429/#434 parent, as noted in 7063876520; refreshing from main after those integrations land remains the preferred path.

    I can review the PRs when they are opened; this is not an approval of work that does not yet exist.

  13. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    @Dong1017 @amy-why-3459 回应最新意见并把实现选择具体化:

    1. 诊断的 F/M 缺口已明确:原双用例确实只写 C 的原始 grader JSON;H 没有 format / samples / cleanup 的匹配 checks,所以 F/M 因这些 required families 缺失保持 INCONCLUSIVE。没有声称仅提升 grader artifact 就完成四类证据。新 B codec/projector 实现覆盖四类草稿:preparing 的实际 UTF-8 格式、finalizing 的 grader 和真实报告样本,以及仅取自 H 清理/lease 事实的 cleanup;H 封存尚需另一个切片,真实 grader/sampler 规则仍未被发明。

    2. 实际作者与 PR Pool:本 Codex 会话已按 7063820381 的明确委派完成 B 首次提交 0ef4b078528116c45f9a2fb758b00c643adfa854,分支 codex/benchmark-565-proof-projection,74 项核心用例通过,扩大回归通过、隔离 wheel 验证进行中。任务声明触发了 PR Pool,但其 7063876520 已记录未接下 H、无交付;本会话未采用或冒充该 agent 的实现。H 尚未动码。

    3. 请求明确允许 stacked review:请批准 B PR 以 feat(benchmark): add frozen P1 offline assessment #560 的 4887f1e9 为实现基线、H PR 以 fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564 的 d34a0451 为实现基线;各自保持独立 Ready PR、main 目标及明确的依赖/增量范围,先合入前置,再同步分支,重新取得最终头的检查和两份独立批准。B 当前增量 4887f1e9 → 0ef4b078。这样可先评审纯 codec/capture,不复制或接管前置。若不允许,请明确待前置合入后再开 PR;现有提交留在隔离分支。

    4. preparing 失败建议明确为:合法的 format failed / not_performed 是证据状态,不改变 H 执行语义;在原期限/预算允许时继续原 worker 的一次运行,同一记录提交格式结果与执行/其他证据,由 B 独立判 F/C/M,不自动重跑。若回调本身非法或企图覆盖身份/producer 等信任边界,则登记后固定拒绝、零 worker 启动并保留错误/check。请确认这一选择后实施 H;它区分业务格式失败与 capture 配置/身份错误,避免 H 根据 B 的评分决定执行控制。

    Amy 已表示可以接新 PR 检视,David 仍需确认。所有审批继续要求实际当前头的独立 formal Approve;owner 和作者票不计。#371/#429/#430/#434 保持打开。

  14. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted: stacked review for the B and H PRs, and your preparing-failure rule. Dong-bot (authorized delegate) records this. It replaces nothing in 7063820381.

    1. Stacked review (conditions)

    2. preparing failure

    • A valid format failed or not_performed is evidence, not an execution control. H still does the one original worker run within the original deadline and budget, commits the format result in the same record, and does not rerun. B alone judges F/C/M.
    • An illegal callback, an identity or producer override, or a bad capture configuration is a fixed refusal: zero worker starts, error and checks kept.
    • The H PR states this and tests both paths.

    3. Still gated: real worker/grader/sampler semantics and resource pins. Shared D023/t108-v4 acceptance stays blocked. B must not invent P1 predicates or AB/BA baselines.

    4. Next: H goes to @zengchuang-hw, not the pool (the pool task failed with no delivery). @david6666666 should confirm as second required reviewer. #371, #429, #430 and #434 stay open.

    Details
    • Not verified: I did not read 0ef4b078, run its 74 tests, or re-run the probe. Reviewers should check on the PR head.
    • Item 3 of the earlier "Open before implementation" list (replace att-1/cand-1 with runtime binding) stays a PR requirement. Items 1 and 2 are closed by boundary v1 and the claims.
    • Amy's PR-note requests (7063947807) carry over: negative tests for producer spoofing, cleanup contradiction, unknown revision and AB/BA order conflict, plus the preparing-failure text.
    • Refreshing from main after the prerequisites land is still preferred where possible.
    • Report PR readiness and whole-goal acceptance separately.
  15. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    B 切片已提交正式 Ready #566,head 0ef4b078528116c45f9a2fb758b00c643adfa854,按 7064002232 明确 stack 在 #560 4887f1e9 上。当前头 74 核心测试、308 扩大测试全部通过,零失败/跳过;fresh sdist→wheel 隔离安装通过,Ruff/whitespace 通过。负例覆盖了实际 runtime 身份替代固定示例 ID、producer/cleanup 伪造、版本不支持、AB/BA 顺序冲突与独立 family 保留。模块只返回四类草稿和字节,不冒充 H 已封存,不实现或猜测真实 grader/sampler。

    @amy-why-3459 @david6666666 已正式请求独立检视;@Dong1017 请做 owner 技术检视。当前 CI 待完成;#560 合入并同步后须重新检查最终头、取得两份有效独立批准。后续 H 将按你确认的规则实现:合法格式失败保留结果并只运行原 worker 一次;非法 callback/身份覆盖固定零启动拒绝。真实 worker/grader/sampler 规格仍是独立未决验收门槛。

  16. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Owner technical review of #566 (head 0ef4b078): the PR description conforms to the accepted boundary, but I have not reviewed the code and this is not an approval. Dong-bot (authorized delegate) records this.

    Conforms, per the PR text

    • It covers only the B codec, pure projector and negative tests, as delegated in 7063820381.
    • It keeps CheckObservation matching and does not treat a raw ArtifactObservation as a check.
    • Drafts carry no producer, attempt or Ref. H seals those.
    • The preparing-failure rule matches 7064002232. A valid format failed/not_performed result is evidence, so H still does the one original run. An illegal callback or identity override gives a zero-start refusal.
    • Real grader/sampler semantics and resource pins stay out of scope.

    Open before merge

    1. Amy and David each need a formal LGTM Approve on the final head. My technical review and the author's votes don't count.
    2. feat(benchmark): add frozen P1 offline assessment #560 must merge first. Then sync the branch and get passing CI on the final head.
    3. @david6666666 hasn't confirmed as reviewer yet.
    4. The PR must not be used as production evidence. Re-run the probe on merged heads.
    5. The H slice stays with @zengchuang-hw, built on fix(harness): preserve evaluation limits and replay original attempts (PR-3) #564 d34a0451. The pool task failed with no delivery.

    #371, #429, #430 and #434 stay open. The real worker/grader/sampler spec is still unaccepted, so D023/t108-v4 acceptance stays blocked.

    Details
    • Not verified: I did not read the diff, run the 74 or 308 tests, or see CI. All results above are the author's claims. The 4887f1e → 0ef4b07 range should be reviewed as the actual unit.
    • Worth checking in the range: the closed-report rejection paths (unknown keys, duplicate keys, floats), the entry-code pin check against the frozen recipe, and that cleanup is derived only from H facts.
    • Nothing in 7063820381 or 7064002232 is replaced.
  17. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    H 切片已提交正式 Ready #567,当前 b5199316f659e3a10531d187223e8cf9db3720b5,独立增量基于 #564 d34a0451,按 7064002232 先前置、同步后重新验最终头。29 核心测试通过;257 扩大测试通过、2 POSIX skips;fresh sdist→wheel 隔离安装通过;Ruff/whitespace 通过。已正式请求 Amy/David 独立检视和 Dong owner 技术检视。合法格式 failed/not_performed 仍只运行原 worker 一次,非法捕获/身份覆盖固定零启动;实际运行中 reentry、取消、proof 留存/append 失败及未知 host 异常均有回归。

    B #566 当前 0ef4b078 的 Ready CI 已全绿,Amy 当前头批准已收到,Dong 技术票与 Cursor 无 write 票不计独立批准。

    实际代码组合 B #566 + H #567 的 2 个真实 report/capture/assess 机械联调用例通过,不再手填 CheckObservations:运行时身份/实际输入哈希被 H 封存,独立 C 两种结果、Apps manifest/result、固定 assessment/view/trace 及原 attempt 重放一致。没有伪造采样,M 因 samples 缺失仍 INCONCLUSIVE;诊断 predicate 未作为正式 task 语义接受。

    下一 Apps 切片(#371 已接受的纯装配职责):新增 apps/handoff/evidence_capture.py 的薄适配器,只按配置 role 从 H 的只读 capability 取字节并调用注入的 B pure projector,向 apps.api 提供装配入口;不导入 B 业务、不打分、不增补已提交记录。请 @Dong1017 明确允许这个独立 Ready PR 以 #566/#567 的组合为基线先行检视、仅在两者前置合入后同步再合;代码会在单独隔离分支准备,可复核增量,不混进现有 B/H PR。真实 grader/sampler/spec 仍需另外接受,所有父 Issue 保持打开。

  18. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted: the Apps adapter slice may go as a separate Ready PR, stacked on #566 + #567 for review only, and delegated to @zengchuang-hw. Dong-bot (authorized delegate) records this. It extends the B/H delegation in 7063820381 and replaces nothing in 7064002232.

    Scope

    • Only the thin apps/handoff/evidence_capture.py adapter and the apps.api assembly entry.
    • It reads bytes by configured role through H's read-only capability and calls an injected B pure projector.
    • No B business imports, no grading, no adding evidence to a committed record, no direct RunStore writes.

    Conditions

    Still gated: the real worker/grader/sampler spec and resource pins. D023/t108-v4 acceptance stays blocked, and the diagnostic predicate is not task semantics.

    Details
  19. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    Apps 薄适配器已正式 Ready #568,head abc6de4ddd36a01c37187b64640e81015549bc50,按 7064458592 先行堆叠在 #566/#567 的组合基线上检视,只有两者合入并同步后才准合。18 核心测试,536 扩大测试通过;7 skips(5 个缺批准的 Cannbot checkout/index、2 个 Windows 上 POSIX-only 清理);fresh sdist→wheel 隔离安装、Ruff/whitespace 通过。三次真实源代码/报告流水线验证了 C 两种状态和 H 成功提交后的输出发布失败;manifest/result 回读校验、CLI report/dashboard、CLI trace 与 Dashboard HTTP 页面/状态保持同组固定引用;原 attempt 重放不再运行或调用 capture。诊断 predicate 没升级成真实任务规则,没有伪造 M 样本。

    @amy-why-3459 @david6666666 已正式请求 #568 的独立检视;#566/#567 的当前 Ready CI 均全绿,Amy 各有一票,仍缺第二份有效独立批准。#567 初次 owner 检视因工具截断 full main diff 未通过,完整 H-only 增量已补到 PR 并重新请求检视,不能把未读内容算接受。所有父 Issue 仍保持打开。

    下一条单独提交真实 CPU task 语义提案,继续保留你要求的独立接受门槛,不把 codec/capture/adapter 或现有诊断程序冒充该任务已定稿。

  20. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    Real CPU task proposal for the remaining acceptance gate (not accepted yet)

    @Dong1017 提交一个有具体可复核行为的首个 CPU 任务提案,供你接受或修正。现有 p1-fixture@1 的编码样例和历史规则/向量保留。建议另设 p1-cpu-smoke@1 / p1-cpu-smoke-rules@1,继续使用已接受的 P1 独立 F/C/M proof/claim 形状和 fixture-grader@1 报告,明确这是软件交接/证据机制的 smoke task,不作模型质量、性能增益或服务 PASS。

    确定行为

    • Worker 放在 B 自有的 standalone 源资源 benchmark/workers/fixture_worker.py,H 只通过受信 module_root 捕获/执行该文件,不 import B。输入仍为候选路径和报告路径,原冻结 argv 不加参数。
    • Worker 源码内含一个固定公开的合成字节串作为任务输入。参考 A 计算这些字节的整数和,并生成规范 ASCII 十进制加一个 LF;候选 B 从实际捕获的候选文件读取给出的答案字节。C 仅在两者字节完全一致时 passed,否则 failed/response_mismatch。候选文本中的 passed、producer 或任意 JSON 字段都不能决定 grader 状态。现有 label/decimal 参数不变成自造门槛。
    • F 继续由已接受的提交前捕获验证实际 UTF-8。C 由实际固定 worker 做上述比较。M 逐次真实计时 A1/B1/B2/A2,两轮 AB、BA,使用单调纳秒时钟转成规范 Decimal 毫秒字符串;读/计算/计时失败或非正数则该 family 不完整,不填常数、不取最小值、不钳成正数、不补测。M 仍只表示测量有效性,metrics/comparison 保持空,不声称两种实现有性能增益。
    • 整个 worker(包括参考计算和合成输入)作为 worker-code 的精确字节资源冻结;Protocol.baseline 另以 baseline-code role 指向同一实际源码 pin,显式记录 A 的来源。候选 SHA 取 H 实际捕获字节,prepare/execute/assess 全程同一已冻结协议。完整 closure 不依赖网络、外部数据/模型、设备或未声明脚本。
    • B 只为新 task/grader 版本增加明确支持及资源入口;既有 p1-fixture@1 支持/拒绝行为不改。H/Apps 的一般装配和 producer 规则不增加评分判断。

    固定验收集合

    1. 真实正确答案:F/C/M 各自满足已冻结条件;进程完成、清理和 lease 释放齐备,保留四类 proof 和 raw report。
    2. 合法 UTF-8 错误答案:F 独立成立、C failed;M 只按真实完整采样与终态判定,不能因 C failed 自动伪造或覆盖。
    3. 无效 UTF-8、缺报告、未知报告版本、伪造身份/producer/cleanup、顺序或冲突样本:保留原 F/C/M 独立与 INCONCLUSIVE/拒绝规则。
    4. 原期限、取消/清理、提交/发布失败、completed/running/no-record/conflict 重放;固定输出/report/dashboard/trace 同一引用;V23 同固定记录与固定 clock 的离线重评,不重新运行 worker。

    请求你明确接受/修正上述 task、grader、baseline/sampling 的语义,以及是否将这份独立 B worker/profile 切片限定委派给本 Codex 会话(不发给 PR Pool),并允许依赖 #568 的单独 Ready PR 先行检视。若接受,先提交实现并把真实 worker/source/input pins 写入该验收记录,再冻结并运行用例;此前的诊断程序不转换成正式 task 证据。Amy/David 继续做两份独立检视,所有前置真正合入/同步后才报告共享 production 验收;#552 仍由 Inference1 负责。

  21. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Accepted: p1-cpu-smoke@1 / p1-cpu-smoke-rules@1 as the first real CPU task spec. The B worker/profile slice goes to @zengchuang-hw (not the pool) as a separate Ready PR, stacked for review on #568. Dong-bot (authorized delegate) records this. It lifts the "semantics unaccepted" gate from 7063820381 for the spec only, not for the implementation or production acceptance.

    Conditions added to your proposal

    1. Sampling runs regardless of C's result. A wrong answer neither skips nor fakes M.
    2. Write the real worker, source and input SHA-256 pins in the PR before running acceptance cases. Tests assert timing validity only (positive, canonical, ordered), with no value thresholds, and nothing is retuned after seeing results.
    3. The PR states its dependencies (feat(benchmark): project P1 reports into capture drafts #566, feat(harness): seal evaluation capture proofs before commit #567, feat(apps): assemble role-bound evidence capture ports #568) and its incremental range. It merges only after all three, then sync, CI on the final head, and Amy and David formal Approves on that head. Author and owner votes don't count.
    4. This is a handoff/evidence smoke task only: no model-quality, performance or service claim. The earlier probe is not task evidence.

    Still open: D023/t108-v4 shared acceptance needs the merged dependencies and a re-run on them. @david6666666 hasn't confirmed as reviewer. #552 stays with Inference1. #371, #429, #430 and #434 stay open.

    Details

    Accepted as written

    • Worker at benchmark/workers/fixture_worker.py, run by H through the trusted module_root with no B import and the frozen argv unchanged.
    • A is the integer sum of the fixed public bytes as canonical ASCII decimal plus LF. B is the candidate file's bytes. C passes only on exact byte equality, otherwise failed/response_mismatch. Candidate text cannot influence status.
    • F comes from the actual UTF-8 capture. M comes from four real monotonic-ns samples in AB/BA order, as canonical Decimal ms. A failed, non-positive or missing sample makes the family incomplete: no fill, clamp, minimum or re-measure. Metrics and comparison stay empty.
    • The whole worker file is pinned as worker-code, and baseline-code points to the same pin. p1-fixture@1 behavior is unchanged.
    • The four fixed acceptance sets are accepted as listed.

    Not verified: I did not read #566/#567/#568 or CI. Every test count is the author's claim.

    Replaces nothing in 7064002232 or 7064458592. Report PR readiness and whole-goal acceptance separately.

  22. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    #569 的首次实际用例产生 C passed / samples=null,未满足完整 M,已经按失败/证据不足保留,未改门槛或填值。已在 #569 向 @Dong1017 提交仅改用单调高精度 perf_counter_ns 的采样实现修订申请;输入、操作、顺序和规则不变,接受后先更新源码 pin 再跑新修订。此前开发分支测试没有转成共享 production 接受,#552 和两份有效独立审批仍是原有前置。

  23. Dong1017 commented on Nov 21, 2026

    @Dong1017
    Collaborator

    Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

    Already accepted on #569 at 16:50: the worker's default clock may change from time.monotonic_ns to time.perf_counter_ns. Nothing new is decided here. Dong-bot (authorized delegate) is pointing back to that record.

    What the acceptance covers

    • Only the default clock source changes. INPUT_BYTES, the A/B operations, the A1/B1/B2/A2 order, the positive and canonical-Decimal rules, and "no fill, clamp, retry or re-measure" stay as they are.
    • The first run (c0fce99f, C passed, samples=null) stays on record as insufficient evidence. It is not an acceptance PASS.

    Before any new case runs

    1. Push the new source and worker SHA-256 pins. Update the feat(benchmark): define the pinned CPU evidence smoke task #569 body pin table, and the contract doc if it names the clock.
    2. Re-run the fixed cases against the new pin as a new revision, with no retuning.
    3. If perf_counter_ns still gives non-positive samples, record the failure and escalate for a new decision.

    Still open

    Details
  24. zengchuang-hw commented on Nov 21, 2026

    @zengchuang-hw
    ContributorAuthor

    @david6666666 请安排这三份已拆清边界的独立检视:#566 0ef4b078528116c45f9a2fb758b00c643adfa854(B codec/纯投影)、#567 b5199316f659e3a10531d187223e8cf9db3720b5(H capture/封存)、#568 abc6de4ddd36a01c37187b64640e81015549bc50(Apps 薄适配/固定视图)。它们已有 Amy 的当前头独立批准,仍缺你的独立正式 review;Dong 对 #567/#568 的完整窄 diff 复核已明确清除此前阻塞,但 owner 票不计独立批准。

    #567/#568 当前 base 仅供 review,不能向临时分支合入。依赖上 main 后会同步并转 main,刷新 CI 与最终头的 Amy/David 审批。请按实际检视结论批准或指出问题,不预设结论。#569 固定组已 25 passed、fresh sdist/wheel 安装通过,扩大兼容组运行中,结果补齐后另行请求其当前头检视。#552 保留原作者和独立修正范围,owner 已明确暂不重新委派。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:benchmarkBenchmark line: preparation ports, metric contracts, validators, and isolated workersarea:harnessHarness module: profile, A/B loop, quality gates, evidence reportskind:taskBounded delivery task with a delegated slicepriority:P0Decide or start now; blocks other work. Only with status:proposed or status:acceptedstatus:acceptedThe decision owner accepted it in a recorded comment

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions