Verify an engine-pinned public release

This is the successor to the frozen service 0.5.0 live verifier. It has its own runner and receipts. Older verifiers, published observations and received packages remain unchanged.

This runner supports separately authorised, version-specific public checks. See the shared publication record for retained deployments and observations. Its offline unit tests and in-process SDK checks are not public-service observations.

What the identities mean

A source version names an immutable OKF-DWP Git commit. An engine ID identifies the exact context-assembly implementation. A context ID identifies the selected context; the complete package also has a SHA-256 digest. All four matter when reproducing an old evidence link.

The verifier compares the public service with an exact local, committed service build. /health reports service, source and engine identities. It cannot prove which Worker bytes the hosting platform deployed. The operator must separately retain the hosting publication record for the expected Worker digest. The live receipt therefore always says deployed_worker_bytes_independently_verified: false.

Prepare a run without contacting the service

From services/ask-okf-mcp, install the locked dependencies and build the exact reviewed release using the existing service instructions. The verifier, its transitive local helpers and build inputs must already belong to the supplied comparison commit. Uncommitted comparator changes fail admission.

Use a new output directory whose parent already exists. Paths through symbolic links are rejected. Replace every value below with the release's recorded identity; these are placeholders, not claims about a deployed version.

node --experimental-strip-types scripts/verify-versioned-remote.ts \
  --origin https://ask-okf.crpage.chatgpt.site \
  --source-version EXACT_DWP_COMMIT \
  --expected-worker-sha256 EXACT_WORKER_SHA256 \
  --service-version EXACT_SERVICE_VERSION \
  --comparison-commit EXACT_EXPLORER_COMMIT \
  --dwp-root /path/to/okf-dwp \
  --out /path/to/new-offline-plan

Without --execute-public, this makes zero HTTP requests. It uses the real vendored source loader, verifies the source against immutable local DWP Git blobs, assembles each permitted source/engine pair and prepares the exact expected catalogue and evidence slices. It writes plan.json, the executed runner and its build receipt. It does not create a passing live observation.

The approved registry determines the pairs. Four historical sources support two frozen engines; the preceding combined source supports one of those engines; the new Evidence Connect source supports its separately pinned v3 engine. Six sources and three engines therefore give ten permitted pairs. An unsupported pair is never added merely to complete a rectangular matrix.

Only two explicit public questions are used: the existing care-home acceptance question and the unknown-term control xylophonicquasarteleportation. The unknown-term control is actually assembled for the current source and engine at 512 KiB; it is separately labelled and must return no selected records and an insufficient status. The care-home question preserves its original wording exactly, including its typographical error. The original 0.5.0 care-home case is replayed with its original source, budget and expected context ID and without an engine ID. Its complete canonical package must match the frozen 0.5.0 receipt. Historical origin remains unspecified; successful compatibility does not prove which engine originally produced the link.

Execute after the release is authorised

Repeat the command with a different, fresh output directory and add --execute-public. Do not reuse the planning directory. The runner repeats immutable admission and prepares all expected results before its first HTTP request. No public execution occurs merely by importing the runner or running its tests.

The planned HTTP upper bound is:

sum(catalogue pages + evidence slices for each permitted pair
    + catalogue pages + evidence slices for the original historical case
    + catalogue pages + evidence slices for the current unknown-term control)
+ 10 setup/discovery/closure/legacy-comparison requests
+ 4 negative controls
+ 1 incompatible-pair control, only when an incompatible approved pair exists

The allowance is conservative; the receipt records the actual requests. A plan that cannot fit 200 requests fails before contacting the service. It does not silently omit pairs, evidence slices or checks.

Boundary	Limit
Approved source/engine pairs	16
HTTP requests	Planned upper bound, at most 200
Time between request starts	At least 750 milliseconds
HTTP request body	32 KiB
HTTP response body	256 KiB
Total received HTTP body bytes	32 MiB
Individual HTTP request	60 seconds
HTTP run	10 minutes
Catalogue or individual evidence-value pages	128
Local immutable source files	512
Local immutable source bytes	64 MiB
Individual local input	8 MiB

Bodies are counted while streaming. Headers alone do not establish size. The run has no automatic retries or redirects and admits only the exact HTTPS origin, GET /health and the reviewed read-only MCP methods at POST /okf/mcp. No credentials or arbitrary questions are admitted. A failure stops further network requests. These bounds concern the verifier's HTTP requests, not the service's own source retrieval or its hosting platform's costs.

What a passing live observation proves

Health matches the approved source and engine catalogue and service version.
SDK v2 discovers the reviewed read-only tool schemas and reconstructs every allowed pair's full package from bounded slices. A new question defaults to the current engine; subsequent catalogue and evidence requests retain its explicit engine ID, source, context ID and budget.
SDK v1 discovers byte-equivalent canonical tool rows and returns the same current catalogue. Each SDK's protocol-era result envelope is validated separately: v2 declares the exact server identity, ttlMs: 0 and cacheScope: private; v1 has no such envelope fields. Input/output schemas, read-only annotations and all tool-level trust/authentication metadata remain checked, and complete tool rows are compared without dropping fields.
Every catalogue page, ordered record ID, evidence slice, selected-record provenance and complete canonical package matches the direct local reference.
The current and original historical cases also read a selected evidence record's full text and metadata, plus diagnostics, when evidence is selected.
Unknown engines, unknown sources, changed expected context IDs and unavailable historical contexts return no evidence. An incompatible approved pair is checked when the registry contains one; otherwise it is explicitly marked not applicable. Each negative response must match the exact error-only result prepared by the local adapter. Extra content, duplicate JSON keys, extra envelopes or SSE events and changed request IDs are rejected.

The verifier does not call full ask_okf: MCP duplicates values in structured and text output, which can make a 512 KiB package exceed a small response cap. It requests 16 KiB catalogue pages and 64 KiB evidence slices. Complete package reconstruction preserves the original budget, including 512 KiB contexts, while keeping individual deliveries bounded.

This is delivery and reproducibility evidence. It does not establish legal correctness, complete domain coverage, specialist acceptance, answer quality, ChatGPT connector acceptance or live Voice access.

Retained output

observation.json uses okf-versioned-remote-verification.v1 and is labelled actual-public-http only after the authorised run passes. It contains the reported health catalogue, comparator and source hashes, actual request count, per-response status/size/digest/time, package identities, provenance digests, negative-control outcomes and an artefact hash manifest. Received canonical packages are retained as gzip files; only the two fixed public questions appear in them. Transport telemetry retains hashes rather than raw response or question bodies. There is no model call or collection of anonymous users' questions.

failure.json records the bounded request telemetry and completed case summaries if admission or an actual run fails. It also records a fixed verification-stage label, the active prepared case identifier, and discovery envelope/tool-row hashes and counts when available. It does not retain raw error strings or arbitrary server metadata values. A partly completed failed attempt stays in its own directory. Creating a fresh attempt never overwrites a prior observation.

Offline controls

npm run check
node --experimental-strip-types --test test/versioned-remote.test.ts

The tests exercise exact CLI admission, the dynamic pair and request census, streaming limits, pacing, no-retry failure, symlink rejection, altered evidence and provenance, and both installed SDKs against the real in-process service. Injected clocks and service adapters in these tests are labelled offline test controls. They produce no actual-public-http receipt.

SDK envelope compatibility correction

The first public 0.6.0 attempt completed all 11 prepared evidence cases, then failed after SDK v1's successful tools/list response. Its failed receipt remains unchanged. An offline reproduction against the actual in-process service and the locked SDKs identified equal complete tool rows but different top-level result envelopes: v2 includes server/cache metadata that v1 omits. Comparing their whole result hashes incorrectly rejected this compatible discovery.

The verifier now validates both exact envelope contracts and compares the hash of the complete ordered tool array. It retains separate whole-result, envelope and tool-row digests for each SDK; tools_canonical_sha256 retains its existing meaning as the whole v2 result digest. No schema or tool metadata is normalised away. Unknown envelope fields, weakened schemas, changed annotations, changed trust/authentication metadata and even description-only differences between SDKs fail the controls. Fifteen offline controls pass, including both real SDKs. The subsequent successful 0.6.0 attempt is linked from the shared publication record. Later releases need their own exact build and observation.