Zero Behavior Change: A Structure-Only Refactor Is a Claim Until Someone Can Falsify It

The author said structure-only

I told the reviewers the next phases on yt-transition-shorts-detector were structure-only. Atomic summary.json write. stall_verdicts extraction. A single gt_io reader. ZERO behavior change. The phrase is cheap to type and expensive to believe.

On 2026-07-20 I kicked /improve-codebase-architecture and then /shannon:plan-deep on branch refactor/plan2t-typed-segments. Session a540ce64-5e0b-48d6-9fc7-e1147d05b63a is the record. Explore lanes came back with eight deepening candidates that did not duplicate the seams already shipped in v1.10.1 through v1.10.3: stall_verdicts, gt_io, json_io, the OcrEngine Protocol, index_by_num, dispatch_agent. The new work was supposed to be the same class of move. Relocate. Parameterize self. Leave the detector's segment tuples alone.

I have watched agents treat "structure-only" as a permission slip. If the author labels the diff a deepening, the reviewer is invited to squint at file names and skip the output. That invitation is the bug. A refactor that claims zero behavior change is a hypothesis about an observable. Until someone who did not write the diff can falsify a behavior change, it is still a claim.

What those deepen phases actually moved

The three seams I keep citing were not decorative cleanup. Each one existed because a concept had two homes, and the second home was where bugs hid.

Atomic summary.json write lived in debug_json_exporter._dump_json. A kill or a timeout mid-write used to leave a truncated cache that later --input-phase1-json runs would treat as truth. The fix was not a new serializer. It was temp file plus os.replace, orjson when importable, unlink the temp on any BaseException. The CLI still has to join the export thread without a timeout so exit 0 never reports while the cache is still on disk. _dump_json is atomic. The process still has to wait.

stall_verdicts extraction was deepen #2. apply_stall_verdicts owns the last accept/reject tier. The analyzer still carried the rest of the stall concept: _merge_nearby_stalls, _process_stall_events, the Y-motion false-positive filter, three byte-identical Phase2Segment(type="stall") constructors, private reach-ins into self._stall_detector._stall_ssim_history. The 2026-07-20 pipeline report called that split-brain C1. The proposed seam was one function, build_stall_segments(...), so the analyzer collapses toward the thin executor its docstring already claimed to be. GT-gate class: pure-structure. Risk: medium. Mitigation: the proven extraction recipe, character-for-character, then parameterize self.*.

gt_io was deepen #4. It owns the LOADING side of summary schema. It deliberately does not own segment matching. compare_all_gt_vs_detection.py:match_segments is the documented single source of matching truth. That split is load-bearing. Loading is passive schema with near-zero regression risk. Matching is the metric itself: ±10 tolerance, greedy pairing, stall-stripping. A gt_io diff can be waved through on a byte-check. A matching diff redefines what "3/3 PASS" means. The 2026-07-20 OCR/IO lane found the cache path in api.py still hand-parsing json.load(_sf).get("fps") for a sibling summary.json — summary-schema knowledge outside gt_io, the exact drift the module was shipped to end.

The diagram is the whole method. The author's label is an input, not a verdict.

The recipe: character-for-character, then parameterize

The stall_verdicts extraction taught a reviewable shape I keep forcing on later deepenings. Move the block verbatim. Do not "clean it up" on the way out. Then replace self.foo with a parameter so the new module does not reach back into the analyzer's private state.

On 2026-07-20 a teammate reviewer, review-p03, sent back a verdict I actually wanted: CONFIRMED, phase-03 O-C2 is a clean verbatim relocation. All 6 criteria pass. HEAD was 1dedf667. The phase itself was still uncommitted working tree, so the review ran git diff against HEAD and matched the same git show 1dedf667 baseline the plan cited. The reviewer extracted a 77-line moved block from both sides, de-indented it, and compared it character-for-character.

That is what "structure-only" looks like when it is true. Not a paragraph in the plan. A second pair of eyes reconstructing the move as a string equality. If the reviewer has to trust my narration of the diff, the claim has already failed.

The same session's planner-risk candidate made the recipe operational for the next wave. One candidate, one commit, one revert target. Deletion-first, then gate-covered extractions, ruler last. compare_all_gt_vs_detection.py stays frozen until the phases that touch measurement code, so every engine edit is scored by an untouched ruler. I have burned weeks on the opposite order: edit the detector and the scorer in the same commit, then argue with myself about whether the green run is real.

GATE-P2: the tuple that has to match

A structure-only claim that cannot be falsified is theater. The falsifier on this repo is GATE-P2.

Write .debug/.detect-justification with a RUN-JUSTIFICATION that says the code just changed, so the cache cannot answer, Phase-2 was touched, and video-detection-workflow was consulted. Then run detect against a frozen phase1_frames.json:

python3 -m yt_shorts_detector detect videos/<video>.mp4 ./output-w3-<id>-<video> \
  --input-phase1-json gt-validation-plan2t/<video>/debug/<run>/phase1_frames.json \
  --detect-stalls --debug-json --log-file ./output-w3-<id>-<video>/run.log

Then compare segment tuples at content level, not diff on pretty-printed JSON:

import json
d = json.load(open("./output-w3-<id>-<video>/summary.json"))
seg = sorted([[s["type"], s["start_frame"], s["end_frame"]] for s in d["segments"]])
b = json.load(open("evidence/plan2t/baseline-<video>.tuples"))
print("MATCH" if seg == b else "DIFF", len(seg))

Expect MATCH 43 on ios-12-3-debug. Expect MATCH 42 on ios-yt-stalls-2min-30fps. grep -ci error run.log equals 0, with the ios-stalls ffmpeg-audio warning exempt. Every engine phase runs both cached videos. Seconds each. Android stays out of this gate because of ±6-frame nondeterminism and gets caught at campaign close.

That pair of MATCH lines is the minimum proof the deepening did not move a boundary. A green unit test on the new module proves the module imports. It does not prove the detector still emits the same shorts, transitions, ads, and stalls. I have shipped the first kind of green by accident. I do not get to call the second kind of green without the tuple.

Campaign close is a different instrument. Three serial full detects, justification markers, --log-file, one at a time. Then compare_all_gt_vs_detection.py --tolerance 10 at 3/3 PASS. Then seq_A through seq_D smoke. Only then is the wave done. GATE-P2 is the cheap per-phase check. The campaign gate is the expensive one. Skipping the cheap check because the expensive one is scheduled later is how a behavior change hides inside a "structure-only" stack of commits.

Independent review is not a rubber stamp

The 2026-07-20 deepen plan spawned independent reviewers on purpose. I did not want a teammate who had just written the extraction to certify it. review-p03 was a different session, a different color in the teammate stream, a verdict file I could cite. CONFIRMED verbatim. Six criteria. The working-tree caveat written down so nobody later pretends HEAD already contained the move.

Five days later the same doctrine showed up on a different plan and almost died of self-approval. Session 26a3965f-e830-49f3-8344-dad8a445d236 is lane 3 of a post-correction re-review on plans/260725-0338-tech-debt-remediation-v1104-wave-2-ocr-regression-first/. Branch still refactor/plan2t-typed-segments. HEAD 089f07e5. The plan had already taken 3 independent BLOCK verdicts and 11 CRITICAL findings. The producer applied corrections in CORRECTIONS-ROUND-1.md and wrote, correctly, that the file carries no approval weight under RL-3. It exists so an independent reviewer can locate each change and judge it, not so the gate can be self-closed.

Lane 3's lens was iron rule plus evidence admissibility. Validation is real-system. Cite a path and a line or the PASS is not a PASS. The reviewer did not patch the plan. The reviewer re-ran the repros and wrote evidence/tech-debt-remediation/wave2/verdict-round2-lane3-ironrule.md.

That split — producer writes corrections, independent lane writes the verdict — is the only reason I trust the word APPROVE when I see it on this repo. A deepening author marking their own phase "structure-only, tests passed" is the same shape as a detector author marking their own run "3/3 PASS" without a captured comparison file.

When the gate itself was a false green

The most useful finding in that re-review was not about filters.py. It was about the measurement instrument.

Lane 3 had previously shown that the Phase-4 byte-identity gate, written as diff <(jq ...) <(jq ...) | tee ${ev}-diff.txt, was a reachable false green. Process-substitution exit codes do not propagate into diff's status. set -o pipefail does not observe them either. Both jq sides can fail on malformed JSON, the pipeline still exits 0, and the gate records identity of two empty streams.

The producer "fixed" the gate. Lane 3's job on 2026-07-25 was to refuse to believe the narrative. The reviewer re-ran the old form and the corrected form, captured durable output under verdict-round2-lane3-reruns.txt (46 lines, header pinned to HEAD 089f07e5, bash 3.2.57, jq 1.8.2), and cited line ranges that resolved: L3-F4 false green on the old procsub form; 5/5 cases on the transcribed corrected gate; PIPESTATUS versus bare $?; the old diff - file leaving no artifact.

The Stop hook still fired once. dogfood-claim-reminder.sh saw a PASS claim citing a deleted /tmp directory. The reminder is the project's iron rule in hook form: a behavior or PASS claim without a captured-evidence citation gets stderr back. Run the artifact. Capture real output. Cite the path and line, or downgrade the claim. The reviewer amended the verdict to 327 lines so every re-run claim cited a live file.

That is the same lesson as "structure-only," pointed at the gate instead of the production code. If the check that is supposed to falsify a behavior change can itself go green on garbage, then "ZERO behavior change" is unfalsifiable. An unfalsifiable claim is not a deepening. It is a story.

Cite evidence or FAIL

I keep the iron rule next to every refactor now, not just next to detection runs.

If I say the tuples MATCH, I cite the output file and the line that prints MATCH 43. If I say the move was verbatim, I cite the reviewer's extracted blocks, not my memory of the diff. If I say the gate no longer false-greens, I cite a re-run that fed it malformed JSON on purpose and recorded a non-zero exit. If I cannot cite, I do not get the word PASS. I get a hypothesis.

The dogfood hook is a reminder, not a hard block. A determined author can still write around it. That is why independent review exists. The author is incentivized to see structure. The reviewer is incentivized to find the behavior change the author promised was not there. Give the reviewer a frozen Phase-1 cache, a baseline tuple file, and a rule that empty evidence is FAIL, and the incentive is actually usable.

I generalized this past one detector. Any "just a rename" in a payment path, any "just moving files" in an auth module, any "no user-facing change" in a CLI that writes caches, is the same claim class. The falsifier changes. The requirement does not. Someone who did not write the diff has to be able to prove a behavior change if one happened. If they cannot even try, the deepening is not ready to merge.

The durable rule I took out of those two sessions is small enough to keep in the plan header: a structure-only refactor is a claim until an independent reviewer can falsify a behavior change. Iron rule: cite evidence or FAIL. The author saying it was structure-only is not evidence.