P06 ยท Freeze a model and track evaluation exposure ================================================= This workflow accepts both supplied pixel and convolutional artifacts. It does not train, select a threshold, certify provenance or award project completion. Python 3.11+, standard library only. Keep the complete package in one directory. FIRST: select a candidate using development evidence, not the new evaluation. Write a reason and declare a per-image pet-IoU floor appropriate to your task. Example command (0.7 is a teaching choice, not a universal deployment standard): python3 perception-evaluation.py freeze --candidate conv-10.json --reason "Selected using the public development images; inspect every new source separately." --minimum-iou 0.7 --out frozen.json The protocol copies model parameters, threshold, preprocessing contract, gate, and exclusions for all eight known images. A digest identifies the exact JSON content. You can edit/refreeze a file and compute another digest; this is not a signature, timestamp proof, proctored assessment or authentication of training. Retain the selected file and explain any later revision as a new candidate. TRY THE WORKFLOW WITH AN AUTHORED FIXTURE (NOT A REAL-IMAGE TEST): python3 test-perception-evaluation.py python3 perception-evaluation.py evaluate --protocol frozen.json --batch perception-evaluation-example.json --ledger evaluation-history --out fixture-first.json python3 perception-evaluation.py evaluate --protocol frozen.json --batch perception-evaluation-example.json --ledger evaluation-history --out fixture-repeat.json The two-pixel example is authored numerical data with synthetic source metadata. It is not a licensed photograph, generalization evidence, or an independent test. The first output says first recorded use IN THIS LOCAL LEDGER; the second is repeat practice. A different candidate does not reset source exposure. Compare per-image counts/masks and the declared gate; a passing fixture proves no real perception ability. Output files must use new names. PREPARING NEW REAL-IMAGE DATA โ€” additional verification is required Prepare a separate JSON document with version p06-prepared-evaluation-v1. The example shows its structure. Its required descriptive fields are: attribution: original author/dataset credit license: applicable license/permission and any restrictions source: original source location and acquisition context preparation: exact conversion, resizing, annotation and tool/version details sourceGroupingReview: how photographs, duplicates and derivatives were grouped Each records entry contains: id, sourceGroup: the SAME stable source identity (one image per source group) sourceHash: lowercase SHA-256 of the original source-image file width, height: integers 1โ€“48 rgb: row-major, interleaved R/G/B integers, length 3*width*height, range 0โ€“255 trimap: row-major integers, length width*height; 1 pet, 2 background, 3 ignored For compatibility with training, convert originals to RGB, preserve aspect ratio with longest side 48 and rounded dimensions, resize RGB with BOX and the aligned trimap with NEAREST, as documented in the supplied subset. Keep original files, annotations, their actual hashes and license records for review. This script checks prepared-array shape/range; it does NOT decode originals, verify that claimed source hashes match files, validate licensing, or prove this transformation was applied correctly. Use the optional source preparation/replay tool described in P06-SOURCE-PREPARATION.txt to verify original bytes and transformations. Independent source/license review remains outstanding. Do not treat text metadata as verified provenance. Limits: at most 16 sources, one full prepared image per source group, at least one classified target per image, at most 1 MB of batch JSON. Exact known source IDs, claimed source hashes or prepared RGB bytes are rejected. Duplicates inside a batch are rejected too. Different encodings, crops, edits and related source identities can evade exact checks. Review near duplicates and shared source origins manually; never rename an old source to make it appear independent. Do not tune on these images after looking at outcomes and still call them new. EXPOSURE AND FAILURE BEHAVIOR Use the same ledger directory for every candidate in this study. It stores up to 256 exposure events, each containing the protocol/batch digests and source identities/hashes. It records an event BEFORE computing outcomes. A failed calculation or report write therefore still counts as exposure on retry. Malformed history fails closed rather than discarding earlier exposure. New filenames and an exclusive lock prevent accidental overwrite/concurrent runs. A process killed abruptly can leave .evaluation-lock. Confirm that no process is using this ledger before manually removing only that lock; retain all exposure-*.json files, including those from failed runs. A partially written history file requires recovery/review, not silently starting a clean history. Deleting, editing or replacing the directory can defeat local exposure memory; there is no account or authenticated cross-device history. Preserve/share the whole directory with the report. First local use is not proof of untouched data. INTERPRET THE RESULT The fixed gate requires EVERY image's pet IoU to meet the frozen floor. A missing pet union is undefined and does not pass. Pixel accuracy and ignored counts remain visible. One convenient batch of images cannot establish deployment readiness or population uncertainty. Record failure types, source limitations and what additional evidence would change the decision. Independently authored implementation, real-source preparation/evaluation review and learner pilots remain pending even if the local gate passes. OPTIONAL ORIGINAL-SOURCE REPLAY Add --source-manifest sources.json to evaluate after installing the pinned Pillow dependency. The evaluator re-decodes originals and compares the entire prepared batch before writing exposure or scores. A mismatch stops the run. The result's sourcePreparation field records whether this check was performed. Without the option, original files remain unverified. Even verified transforms do not authenticate licensing, independent source identity or prior exposure.