Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/apps-module-migrations.json
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@
],
"consumers": [
"docs/apps-output.md",
"tests/test_harness_prepared_binding.py",
"docs/architecture.md",
"docs/module-layout.md",
"docs/operations.md",
Expand Down
52 changes: 52 additions & 0 deletions docs/contracts/harness-prepared-binding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Harness preparation binding before registration

The Harness evaluation path checks declarations before registering an attempt.
`inferrsi.harness.prepared_binding.verify(spec, artifact_reader)` returns a
`FixedPreparation` projection or raises a Harness `PublicError`. The spec must
first pass `execution_service.validate`'s static shape checks; `Harness.execute`
performs both checks in that order. Ordinary command mode reads no preparation.

The projection's `read(ref, reader)` port obtains a binary stream through
`reader.open(ArtifactRef)`, closes it, bounds reads by the declared size plus one
byte, and independently checks size and SHA-256. Missing bytes or a missing
reader produce `ArtifactUnavailable`; changed bytes or noncanonical JSON
produce `ArtifactIntegrityError`. Unknown host programming errors propagate.
`project(bytes)` is pure: it parses the canonical v1 envelope and validates the
fields used by Harness. It does not read a protocol, resolve an entry or start a
process. A schema version other than v1 is explicitly unsupported.

Binding compares the semantic prepared ID, protocol digest, operation and scope
with the fixed preparation. Disagreement is `IdentityMismatch`. It also checks:

- Candidate kind and role, a frozen expected candidate ID/hash/format when
present, and the supported `captured-sha256@1` or `expected-sha256@1` identity
rule. An unknown rule/revision is `UnsupportedRuleVersion`. The expected rule
requires a declared expected input hash even without a frozen candidate.
- A non-null expected code digest on every evaluation step, matching the frozen
recipe. This is an expected declaration, not an observed entry-code hash.
- One worker step preserving runtime role, entry kind/name/revision, all output
declarations, and arguments with whole-argument `{input:role}` / `{output:role}`
expansion. Inputs may use caller-chosen materialized paths; arguments must
match those paths. Paths are relative to the attempt directory (`cwd="."`),
and input/output paths must not overlap. Required frozen capabilities cannot
be dropped; additional declared capabilities remain subject to admission.

Malformed or conflicting declarations are `InvalidExecutionSpec`. All of these
refusals precede `RunStore.claim`: neither attempt nor record is registered.
The public projection does not interpret candidate formats, grading rules,
evidence requirements or scores, and does not recompute Benchmark's semantic
prepared ID. The fixed ArtifactRef authenticates the preparation bytes; Apps
performs the separate archived-protocol binding through its canonical
`inferrsi.apps.handoff.bridge.verify_protocol` port.

The D023 conformance vector is kept separately under
`tests/fixtures/harness-prepared-binding/`. It adds the frozen code declaration
and a null step timeout without rewriting historical fixtures. Its canonical
bytes must agree between the Apps binder and Harness `request_digest`.
The vector is documentary; its historical code pin is not an available worker.

This slice keeps valid evaluation-mode execution explicitly unsupported.
Actual entry resolution and code-byte verification after registration (T37),
attempt deadlines/budget enforcement with a null step timeout, and evaluation
replay (T40) are later delivery slices. It does not establish production CPU
acceptance or completion of the governing Harness or Apps Issues.
1 change: 1 addition & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Effect classes used below:
| Rebuild scoped legacy evidence views | `python tools/rebuild_model_evidence.py <manifest> --archive <path> --output <new-directory>` | CPU-only; creates a new output directory | Authenticates original bytes and pinned projection/code; no extraction, network, archived-code execution or campaign qualification |
| Inspect evidence inventory / generate budget input | `python tools/validate_evidence.py [--budget-assets]` | Read-only | Offline layout/hash/schema checks; optional disposable #328 metadata view derived from campaign authority; no downloads |
| Execute one ordinary local CPU command with a fixed record | Python `inferrsi.harness.api.Harness.allocate` / `execute` / `read` | Experiment-executing / record-writing | One explicit RunStore root and one trusted local CPU target; ordinary command mode only; a record holds execution facts, so exit 0 is not a verdict; no device, model or network fallback |
| Verify a frozen preparation binding before registration | Python `inferrsi.harness.prepared_binding.verify` / `read` / `project` | Read-only / pure projection | Fixed ArtifactRef and read-only stream capability; no registration, entry resolution or execution; [binding contract](contracts/harness-prepared-binding.md) |
| Freeze a configured P1 preparation | Python `benchmark.preparation.PreparationService.prepare`; [production prepare](contracts/production-prepare.md) | Read-only resource access / pure encoding | Verifies supplied frozen worker/resource bytes and returns a preparation, protocol and role-keyed closure; Apps separately archives them before publishing a Ref |
| Visualize a recorded run or campaign | CLI `dashboard`; [dashboard](dashboard.md) | Report-generating / local read-only serving | Writes an offline HTML snapshot or serves an optional live view on loopback; does not run an agent or validate a claim |
| Capture a knowledge context | CLI `campaign capture-context`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Saves assembled context and resource hashes in a new `knowledge/<id>.json`. Supports ordinary questions and explicit IDs. The only accepted explicit `--relevance-method` value is `find-lexical-v1` |
Expand Down
84 changes: 76 additions & 8 deletions src/inferrsi/harness/execution_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@
from pathlib import Path

from inferrsi.contracts import json_v1
from inferrsi.harness import input_binding, local_process, run_store
from inferrsi.harness import input_binding, local_process, prepared_binding, run_store
from inferrsi.harness.errors import PublicError

# H02 single local-CPU execute in ordinary command mode (V05 null set). Evaluation mode,
# entry-code digests (T37) and generation chaining are later slices and are refused here.
# H02 single local-CPU execute in ordinary command mode (V05 null set). Evaluation-mode
# specs are checked before registration (V05/V28/V32 and their fixed preparation) and then
# refused: the T37 entry-code check and generation chaining are later slices.
ATTEMPT_FIELDS = (
"request_id", "run_id", "attempt_id", "owner", "target_request", "deadline_utc",
"resources", "budget", "retry_of", "retry_reason",
Expand All @@ -42,6 +43,10 @@
)
_OUTPUT_FIELDS = frozenset({"role", "path", "required", "media_type", "format_schema"})
_REF_FIELDS = frozenset({"location", "sha256", "size_bytes", "media_type"})
_PREPARED_REF_FIELDS = frozenset({"prepared_id", "schema_version", "artifact"})
_CANDIDATE_FIELDS = frozenset(
{"candidate_id", "kind", "input_role", "parent_id", "parent_digest", "context_refs"}
)
_DIMENSION_FIELDS = frozenset({"dimension", "unit", "limit", "reserve_estimate"})
_DEFINITION_FIELDS = frozenset({"name", "revision"})
_DECIMAL = re.compile(r"(?:0|-?[1-9][0-9]*|-?(?:0|[1-9][0-9]*)\.[0-9]*[1-9])", re.ASCII)
Expand Down Expand Up @@ -222,6 +227,9 @@ def _step_errors(steps: object, ordinary: bool) -> list[str]:
elif ordinary and digest is not None:
# V32-N1: ordinary command mode carries no expected entry digest.
errors.append(f"{where}.expected_code_sha256: must be null in ordinary command mode")
elif not ordinary and digest is None:
# V32-N2: null never opts an evaluation step out of the frozen code digest.
errors.append(f"{where}.expected_code_sha256: must be set in evaluation mode (V32)")
if not (isinstance(step["args"], list) and all(isinstance(a, str) for a in step["args"])):
errors.append(f"{where}.args: not a list of Text")
if step["cwd"] != "." and not _is_path(step["cwd"]):
Expand Down Expand Up @@ -265,6 +273,53 @@ def _output_errors(outputs: object, where: str) -> list[str]:
return errors


def _evaluation_errors(spec: Mapping) -> list[str]:
"""Static shape of the evaluation fields; the fixed preparation is read separately."""
errors: list[str] = []
ref = spec["prepared_ref"]
if not run_store._has_fields(ref, _PREPARED_REF_FIELDS):
errors.append("prepared_ref: fields must be exactly the PreparedRef fields")
else:
if not prepared_binding._is_semantic_id(ref["prepared_id"]):
errors.append("prepared_ref.prepared_id: not a SemanticId")
if type(ref["schema_version"]) is not int or ref["schema_version"] != 1:
errors.append("prepared_ref.schema_version: must be the integer 1")
if not _is_artifact_ref(ref["artifact"]):
errors.append("prepared_ref.artifact: not an ArtifactRef")
elif ref["artifact"]["media_type"] != prepared_binding.PREPARED_MEDIA_TYPE:
errors.append("prepared_ref.artifact.media_type: must be application/json")
if not prepared_binding._is_semantic_id(spec["protocol_digest"]):
errors.append("protocol_digest: not a SemanticId")
for name in ("operation", "validation_scope"):
if not run_store._is_id(spec[name]):
errors.append(f"{name}: not an Id")
candidate = spec["candidate"]
if not run_store._has_fields(candidate, _CANDIDATE_FIELDS):
errors.append("candidate: fields must be exactly the CandidateSubmission fields")
return errors
for name in ("candidate_id", "kind", "input_role"):
if not run_store._is_id(candidate[name]):
errors.append(f"candidate.{name}: not an Id")
# V02: parent id and digest are both null or both set.
if (candidate["parent_id"] is None) != (candidate["parent_digest"] is None):
errors.append("candidate.parent_id: parent_id and parent_digest must be paired")
elif candidate["parent_id"] is not None and not (
run_store._is_id(candidate["parent_id"]) and prepared_binding._is_sha256(
candidate["parent_digest"]
)
):
errors.append("candidate.parent_id: not an Id with a Sha256 parent_digest")
refs = candidate["context_refs"]
if not isinstance(refs, list) or not all(_is_artifact_ref(item) for item in refs):
errors.append("candidate.context_refs: not a list of ArtifactRef")
# V03: the candidate role names exactly one InputSpec.
inputs = spec["inputs"] if isinstance(spec["inputs"], list) else []
roles = [item["role"] for item in inputs if isinstance(item, Mapping) and "role" in item]
if roles.count(candidate["input_role"]) != 1:
errors.append("candidate.input_role: must name exactly one input role (V03)")
return errors


def _request_errors(spec: Mapping) -> list[str]:
"""Shape and resource/budget field errors beyond the AttemptContext rules."""
errors: list[str] = []
Expand Down Expand Up @@ -304,8 +359,12 @@ def _request_errors(spec: Mapping) -> list[str]:
return errors


def validate(spec: object, target: LocalCpuTarget) -> None:
"""Refuse a request before registration (D021 priority: version, fields, support)."""
def validate(spec: object, target: LocalCpuTarget, *, artifact_reader=None) -> None:
"""Refuse a request before registration (D021 priority: version, fields, support).

An evaluation-mode spec is also checked against its fixed preparation, read through
``artifact_reader`` (V05, V28, V32). Nothing here registers an attempt or writes a record.
"""
if isinstance(spec, Mapping):
version = spec.get("schema_version")
if type(version) is int and version != 1:
Expand All @@ -323,8 +382,11 @@ def validate(spec: object, target: LocalCpuTarget) -> None:
errors += run_store._attempt_errors({name: spec[name] for name in ATTEMPT_FIELDS})
errors += _request_errors(spec)
evaluation = [name for name in _EVALUATION_FIELDS if spec[name] is not None]
if evaluation and len(evaluation) != len(_EVALUATION_FIELDS):
complete = len(evaluation) == len(_EVALUATION_FIELDS)
if evaluation and not complete:
errors.append("prepared_ref: evaluation fields must be all null or all set (V05)")
elif complete:
errors += _evaluation_errors(spec)
errors += _input_errors(spec["inputs"])
errors += _step_errors(spec["steps"], ordinary=not evaluation)
try:
Expand All @@ -336,9 +398,15 @@ def validate(spec: object, target: LocalCpuTarget) -> None:
"InvalidExecutionSpec", stage="validate", message="illegal ExecutionSpec",
field_errors=errors,
)
if evaluation:
# Malformed or inconsistent bindings are refused here, before the RunStore sees the
# request. The reader is the read-only port; H does not write or register anything.
prepared_binding.verify(spec, artifact_reader)
unsupported = []
if evaluation:
unsupported.append("evaluation mode is not implemented in this slice")
# The T37 pre-start entry-code check runs after registration and is a later slice.
# Until it exists, no evaluation-mode candidate may start.
unsupported.append("evaluation-mode entry verification is not implemented in this slice")
if spec["generation_record_ref"] is not None:
unsupported.append("generation chaining is not implemented in this slice")
request = spec["target_request"]
Expand Down Expand Up @@ -920,7 +988,7 @@ def execute(
artifact_reader=None,
) -> dict:
"""H ``execute``: register, admit, bind, run and commit one ordinary command request."""
validate(spec, target)
validate(spec, target, artifact_reader=artifact_reader)
digest = run_store.request_digest(spec)
attempt = {name: copy.deepcopy(spec[name]) for name in ATTEMPT_FIELDS}
# Re-entry is decided by the store before any binding or start (V14 / D021 section 3).
Expand Down
Loading
Loading