Methodology changes

Two failed rounds show that the reviewers' tools failed, not that the document is closed to anyone; the label says exactly that. Picking sections of a long document is a choice the site makes, so the rule and the full inventory are public before the run, and an independent reader checks every section.

Carrying selected items puts a choice into the package that carrying a whole document did not. The rule, the item index and a check by someone other than the editor are what keep that choice from being the editor's.

A browser check stops a script, not a reader. Carrying the file lets the panel read what any resident can, and the label keeps the site from claiming the reviewers found it on their own.

An exception kept for one run is a door left open for every run. With that run restarted there is nothing left for it to hold open.

The cap stops an editor revising until the checker gives up. A report whose every fix is already written by the checker leaves the editor nothing to revise, so the route lets the editor paste and nothing more, and puts the gate in someone else's hands.

A rule that refuses a run for a string the tool itself wrote is a rule that will be turned off the first time it bites. Better to say exactly what is let through, count it, and print the count where a reader will see it.

Three vendors was the method's best argument for independence, and it is gone for as long as the founder pays for two. The honest move is not to find a third seat that looks like a third vendor but to say, on every page, what the panel now is.

A register that grows a near-duplicate for each source stops being a register. The second source is the first time this could happen, and the step was missing. It has been added now, before a third source makes the fix a cleanup job.

v1.20 opened the cap for arithmetic the checker had already corrected; this opens it for a finding the editor has already conceded. Arithmetic has one answer, and "the record cannot carry this claim" is a judgement, which is why this round belongs to the seat that made that judgement and to nobody else.

A model release is not a reason to change a method, and a price cut is not a finding. What a price cut buys is the room to ask, on the record, how cheap a seat can be before it stops finding things, and to answer that with a seat whose answers count for nothing until they have been compared with seats whose answers do.

A method that names its first source in its rules will read every later source as if it were the first. The fix is not to generalise the wording but to move the source's description out of the method and into the run, where it is written fresh each time and published with the result.

This is the second pass at the same complaint, and the first one did not fail. It fixed the words and left the depth, which is a different fault with the same symptom. Length is the visible sign and not the rule. The rule is that each layer is written for its own reader, and the only check that can tell is a person reading it aloud.

This admits two seats under limits, not because the limits were solved. The Codex seat can read a local file and is caught afterwards rather than stopped. The Gemini seat has no recorded request at all. Both are running because a panel of one is not a panel, and both say in the run record and on the public page exactly what their check did and did not look at.

This is a narrowing, not an improvement. v1.29 could say what a request was. v1.30 can say what a request was not, for the private text this machine holds, and it can keep saying it tomorrow without a paid probe run. The founder chose the second. The reasons for preferring the first are in the September 15 record and have not been withdrawn.

What changed is the evidence, not the confidence. v1.28 refused every seat because an empty plugin list shows what was not loaded and cannot show what was sent. The request is now retained, so that question has an answer for the attempts that were captured, and only for those.

The roads reports were made with ad-hoc commands, not the panel wrapper. Their original session traces survived and made the audit possible. The omitted footer and the wrapper's loss of raw output are separate observed defects; the audit did not establish who or what removed the footer.

The uncomfortable part is not that a check was missing. It is that the file said the check was there. A gap is something a reader can weigh; a documented check that does not run tells them not to bother weighing it. The fourth finding here is what that costs when it is a rule about attribution: a real person, quoted under a pseudonym, recorded as asserting a sentence they did not say.

Two seat changes in two days, both on cost or on the founder's read of a model rather than on a measured comparison of reviewers. That is honest about how a one-person site chooses models and it is also a gap: the method records what ran, and can re-run it, but has no benchmark that would tell a reader whether a seat change moved findings. The manifests make that benchmark possible later; nothing here provides it.

The comfortable story would be that a check failed. No check failed. The check was never built. Somebody watched this failure happen during intake, on a different claim, wrote a rule against it the same day, put the rule in a prompt, and moved on, and an Edmontonian ended up on this site asserting a proposition about tax increases when what they had made was a remark about a bill. Between those two moments every check the pipeline runs ran, and passed. That is the shape of the fault, not an accident of one claim, and it is two holes rather than one. Going forwards, nothing compared the sentence with the quotes, and a check that did would have caught this before it shipped. Going backwards, there was nothing at all: the guards run one item at a time, in the direction the work is moving, and not one of them was ever asked to turn round and read what had already gone out. Nothing had gone back over these decisions until that day. The lesson is not to write better rules. It is that a rule which lives only in a prompt is a statement of intent, and the difference between intent and a guarantee is the whole of what this project claims to be selling.

The risk is that parking becomes the exit from an answer somebody did not like. Two things bound it. The seats' answers stay committed beside the reason, so anyone can see what the run would have published: here all three had answered, and the finding it cost the run was Contradicted — a finding against the people making the claim, not for them. And the concern that parked it is copied into the deterministic artifact rather than dropped with the claim. What would show this was wrong is a run that parks the claim whose answer the editor found inconvenient, or a register where parked claims accumulate faster than answered ones.

The narrow version of this is a precedence bug in one expression. The wider version is that the artifact had no way to disagree with a reviewer about the reviewer's own history, and so could only repeat it. Preferring the self-report was not obviously wrong when it was written — a seat that says it moved usually did, and the seat's own wording is more informative than a diff of two verdict words. What it cost is that the one case where the two disagree is exactly the case worth reading, and the code resolved it in favour of the party with the least standing to be believed. What would show this correction was wrong is a run where a seat's self-report turns out to describe its committed round better than the file does, which would mean the wrong file was committed and is a different failure with a different fix.

The obvious risk is that a re-run becomes a way to re-roll a verdict somebody did not like, and nothing in the mechanism prevents that on its own. Three things bound it. The trigger is stated in stage 4, and what it names are faults in the record rather than outcomes anyone disliked. The superseded answers stay committed and readable beside the ones that replaced them, so a re-run that produced a more convenient answer is visible as exactly that. And the re-run is blind on the same frozen brief, so the only thing that can move is what the seats find, not what the question asks. What would show this was wrong is a run where the re-run count climbs, or one where the superseded verdicts read as sounder than the ones that replaced them.

Worth saying plainly which way the causation runs, because the comfortable version is that a seat misbehaved and the prompt was fine. The prompt was not fine. Round 2 is the only part of the schema round 1 never exercises, so it is the only part a seat meets cold, and it was the part the prompt described loosest. Two seats guessed right and one guessed wrong, and a contract that three careful readers resolve two ways is underspecified whatever the pass rate. The correction is the same one made to briefs on this site: state the shape, do not leave it to be inferred.

The obvious objection is that the editor changed the rule that had just stopped the editor's own brief, and it is the right objection to make. Three things are offered against it and nothing more. The definition names three failures and excludes judgement by construction, so a finding anyone could argue with is not one of them. The checker labels its own findings, so an objection cannot be reclassified after it has been read. And a framing finding in the fourth round parks the brief for good, so the extra round is worth nothing to anyone trying to use it as a second attempt at framing. What would show this was wrong is a defect round that turns into a framing argument, or briefs that start needing a fourth report as a matter of course; both are visible in the run records, and the founder can revert this the way he can revert anything else here. The clause about how the loop ends was written the same day, after first use: the confirmation on cycling-volumes accepted all three corrections, raised no framing finding, and named a fourth arithmetic defect that three framing rounds had walked past, and the rule as first written had nothing to say about that. That the first run of a new rule found a real hole in it, and a real bug in the brief, is the argument for the round existing at all.

The uncomfortable part is not that a finding was wrong. None of them was. It is that four panel runs went to questions that did not need one, and three of those had answers nobody would have argued with. A correction is easier to publish than this, because a correction says the method worked and caught something. This says the method answered honestly and the site chose the wrong question, which is the one fault no amount of reviewer independence can catch, since every guard here runs after the question is chosen. That is also why the dispositions are not deletions. The claims stay published, at their addresses, with their evidence and their notes, because a site that quietly removes its weak work is asking to be trusted about what it removed.

Deleting a limit is the move that looks like loosening a rule, and the honest reading is that the limit was doing the damage. Thirty words was a number somebody picked; what it produced, on five answers out of six, was a sentence built to reach it. The replacement is stricter where it matters, because a colon is how a second idea gets in and there is now no way to write one, and it is the reader rather than the counter who decides whether a sentence is one a person would say. The part worth being uncomfortable about is that the editor wrote this standard and then failed it nine times out of ten on the first attempt. That is the case for the read being a stage with an artifact rather than a habit, and it is on the public record for the same reason the rest of the method is.

The founder's line, on being told for the third time that the launch bar needed restating, was that everything had changed since it was written. He was right. The bar was drafted for a site that chose its claims one at a time, and the site does not do that any more. The part worth being uncomfortable about is the shape of the move: a promise made in public, withdrawn by the people who made it, on their own argument that they have something better. That argument happens to be true, and it would sound the same if it were false. So the protection is not our word for it. It is a published list of every claim considered, which a reader can check against the sources it came from, and this entry, sitting in the version history where the promise used to be.

The previous version stopped the editor choosing which claims to check. It did not stop the same argument arriving as five claims, and five checks of one argument is both a waste and a worse answer than one check would be. So the grouping is the fix, and the care in it is all about where a finding attaches. Group for work and you save panel runs. Group for verdict and you produce a finding that answers none of the claims under it while labelling the people quoted beneath it. Those are different operations on the same list, and the method now does them separately: the investigation is the unit of work, the claim is the unit of judgement, and a question both camps are asking is one investigation with claims pointing in both directions.

Choosing claims by hand was the fault. An editor reading a thread picks what looks checkable, and what looks checkable is rarely what people are arguing about. Every guard added to this method since v1.10 has been about testing honestly: the claim at its strongest, in the units a reader would actually use, against a threshold set in advance. All of those questions presume the claim was worth the panel's time in the first place, and none of them can catch a claim that should never have been chosen. So the method stops choosing. The whole source comes in, every factual claim in it comes out, a script proves the count survived the merge, and triage disposes of the lot together. What that buys is an answer to "why this claim and not that one": there is none, because there was no selection. What it costs is volume, and a triage step that now rules on many propositions at once instead of one. One source has been through the pipeline. The disposition step is the part nobody has run twice on the same input, so the honest claim for this version is narrow: selection bias is out of intake, not out of the method.

A seat that cannot read the documents a brief names is not an independent reviewer of them. Six empty returns on one brief, four under the pinned command and two with the documents supplied as text, showed the fault was the headless permission model, not the package. The change is to the seat's own command, is recorded in every run manifest from here on, and does not touch the package the three seats receive.

Not established was doing two jobs: an answer (the record was checked and does not support the claim) and a shrug (nobody could check). Only the first belongs on a board of findings. The second belongs with the claims considered, where the reader can see why.

Every check in this method asks whether a claim is fair, precise or computable. None asked whether it was worth the work, so the loop optimised claims into shapes the record could answer and nobody had asked. Triage puts the reader's question first and the cost beside it, and the cap on framing rounds stops the loop from running past the point where the claim is still the claim.

v1.10 stopped a claim from being neutralised into a truism. This entry closes the neighbouring gap: a claim can be fair, checkable and precise and still be a question nobody asked, because the framing loop optimises toward what the record can compute. The check now starts from the reader's question and works back to the record, instead of the other way round.

v1.9 fixed the check for asking the record for things it cannot supply. This entry fixes the opposite: the check pushed a claim into a form the record could answer only by draining it of the thing people argue about. The founder saw it on the published page and said so. Fairness to a claim's holder means testing the claim they make, at its strongest, with the magnitude they assert; a threshold is a judgement, so the brief states one and one alternative, and the reader sees both.

The framing check was designed to keep a brief fair; on first use it kept the brief fair and made it unresearchable, because the checker optimised definitions without knowing what the output schema carries or what the historical record can supply. The panel caught both cases in round one, which is the method working as designed, and the cost was two halted runs. Three changes follow: the checker sees the schema, the checker must name the published source behind every threshold it asks for, and the checker must confirm that the instruments a brief relies on exist. The halted runs, their concerns, the revised briefs and every check report are committed beside each other so a reader can see the loop.

v1.7 sent unresolved framing disputes to the founder. On first use both briefs reached that point, and the founder declined the role: editorial content is the steward's responsibility, he supplies ideas. So the escalation now ends with Stew deciding in writing, and the record shows what the checker still objected to. The founder's accountability is unchanged in kind: a standing power to revert, not a per-item approval. The second change closes a regress the first use exposed: a predeclared threshold was asked for an alternative, and the alternative would then have needed one. One identified standard, or one alternative with results under both, is now enough.

Every brief so far was drafted by an AI session and ratified by the founder at the publication gate, which D-0007 then delegated to AI audits. This entry says so plainly and closes the loop: Stew writes the brief, and the safeguard that delegation needs is a framing check by a model from a different vendor, run before the freeze, whose report is public. The check's questions are the ones a fair-minded holder of the claim and a fair-minded opponent would both ask: is this the claim I made, is it the strongest fair reading, would I have accepted these definitions before seeing the result, and does the brief tell the reviewers what to find. Run on the four published briefs, the check said all four would need revision. None of that changes a published finding, because the panel and the gate decided on evidence, but it does mean the frames the panel was given were weaker than this version requires. The reports are committed beside the briefs, and the four stories are scheduled to be re-framed and re-run under v1.7 rather than patched. The two specialist reviews are archived in the board's private record; their two unadopted asks are recorded there as open questions, with the founder's decision pending.

Models are released often — Fable 5.1 shipped the day of this entry — and the reasoning-effort setting changes what a model returns as much as the model version does. Leaving it unpinned made the published command incomplete for one seat: prior Claude runs inherited an unpublished local default rather than a setting on the record. Effort is now pinned in the command, stamped onto each review file by the runner, and recorded per run in run.yaml. Every seat is pinned to "high" for two reasons: it is the highest level all three CLIs share (the Gemini CLI stops there), and while Claude and Codex offer settings above it, the founder does not use them for the panel — in his experience they cost significantly more and run longer without a matching gain. That is a judgement call, not a measurement; no benchmark of the trade-off exists here. Two smaller corrections ride along: the manifest now names each seat directly, and the script that stamps the methodology version onto manifests was reading the oldest changelog entry instead of the newest, so runs made under v1.1 through v1.5 are labelled "1.0" in their manifests. The stamper is fixed; the earlier labels stay as written, because a manifest is a record of what ran, including its mistakes.

The Claude panel seat moved from Fable 5 to Fable 5.1. The pinned reviewer command now uses claude-fable-5-1. Historical review artifacts remain unchanged.

Two changes from the external review's P0 findings. (1) "Partially supported" now structurally requires naming which part of the normalized proposition fails; evidence-quality doubts (untested allegations, litigation figures) affect confidence, never the finding. The one published claim decided under the looser reading (e-bus procurement failure) is re-run against its archived evidence under the tightened rule; any change ships as a logged verdict-change. (2) The publication gate gains a third audit: freshness & completeness (prompts/freshness-audit.md) — an independent re-search for newer or stronger sources and material events between evidence gathering and publication, run before every publication and retroactively on the four published stories.

Four decisions from the 2026-09-01 methodology panel, resolved with written second and third opinions (dispositions in methodology/reviews/2026-09-01-panel/DISPOSITIONS.md). (1) Synthesis now consumes the locked ROUND-1 verdicts. Round 1 is the only round in which the three reviewers are independent; synthesising from round 2, where each has read the other two, treated a deliberative round as the vote. Round 2 keeps its job as an error-documentation channel: every final position is carried into synthesis.json as round2_positions and rendered, and a material catch there (fabricated citation, wrong evidence) triggers a fresh blind re-run of the affected claim rather than a correction inside the run. Verified before adoption and re-verified on switching: the round-1 and round-2 multisets give identical canonical findings on all six published claims, so no published verdict changed. (2) The canonical per-claim "confidence" is replaced by PANEL AGREEMENT — Unanimous, Adjacent or Split, computed from the round-1 multiset. Confidence named something the method never computed and read as a probability that the claim was true; agreement describes the panel and is published with a gloss saying so. Per-reviewer confidence stays visible in the AI review, and the matrix no longer emits a canonical confidence at all. The review.agreement value "Majority" becomes "Adjacent" so there is one vocabulary. (3) The matrix keeps its cautious lean and now publishes the reason: a qualification found by one reviewer does not disappear because two missed it, and for a fact-checking site overclaiming is the costlier error. The veto objection is answered by disclosure — the vote composition is always displayed — not by averaging. (4) A panel quality record opens at methodology/quality-ledger.yaml: adjudicated per-seat events (fabricated citations, false accusations, unsupported figures, self-corrections, catches that held up), each traceable to a committed artifact, seeded retroactively from the four published runs with the per-seat denominator stated. It is an error record, not a calibration; a summary publishes when the nine-story slate completes. Reviewer prompts, the merge, and the 20 matrix rows' findings are unchanged.

Changes from the first methodology/trust panel review (three models reviewing the process, not the stories; full reports in methodology/reviews/2026-09-01-panel/). (1) Evidence items in combined-evidence artifacts now carry an explicit fetch status and registry id — unverifiable citations are flagged, never silent. (2) Panel identity is rendered from the run manifest, never from a model's self-report. (3) Raw artifacts known to contain errors get an errata.md beside them; the record is annotated, never edited. (4) Reviewer prompt now instructs reviewers to flag material framing problems in the brief; a material framing flag halts synthesis until the brief is revised and round 1 rerun. (5) Site copy reconciled with the v1.1 delegated gate everywhere; gate audit reports and the frozen brief are linked from each story's AI-review section. (6) "Claims we're seeing" drops social-platform styling; composite paraphrases are visually labeled as paraphrases. Research, merge, cross-review, and synthesis rules are unchanged.

Publication gate delegated: the founder's per-story manual review is replaced by a dedicated AI audit pair — source verification of every published statement against archived source bytes, and a privacy/copyright release check of raw review artifacts — with both reports committed alongside the run. The founder holds a standing delegation, remains accountable for everything published, and can revert any publication. Research, merge, cross-review, and synthesis stages are unchanged.

Initial methodology: three-reviewer panel (Claude Fable 5, GPT-5.6 Sol, Gemini 3.1 Pro via CLI), blind round 1, deterministic merge, cross-review round 2, deterministic multiset synthesis matrix, drafting with two-model faithfulness check, human publication gate. Reviewer verdicts: Supported / Partially supported / Not established / Contradicted. Canonical findings add Mixed (split panel only).