A design was validated four independent ways.
Two of the four called the same function. Two more drew on the same nuclear data library and shared the same omission. A fifth check, added later, agreed with all of them.
The issue was a single array of numbers with no traceable source, sitting in the denominator of one integral. Every validation reproduced it faithfully, to three decimal places, for two months.
The correction that followed was itself wrong, in the same way, and an outside question was needed to expose it.
This document sets out the protocol that found both, why conventional cross-validation could not, and what a screen conducted under this protocol does and does not tell you.
Most technical review answers one question: do the results agree with each other? That is a consistency check. It is cheap, it feels rigorous, and it reliably catches implementation errors.
It cannot catch input errors. Every method downstream of a wrong input reproduces that input faithfully — and the more methods you run, the more confident everyone becomes.
Verification asks a different question: do the inputs agree with reality? The only operation that answers it is retrieval from a primary source. Nothing else counts. Not a textbook. Not a code comment. Not a well-regarded model. Not a colleague's recollection. Not a language model's output.
Agreement between two methods that share an assumption is not evidence. It is the same evidence, counted twice.
Each of the following was observed in a single project. Each is general.
| Failure mode | Mechanism |
|---|---|
| Shared function | Two "independent" methods call the same subroutine. An analytical model and a Monte Carlo agreed to three decimals because both invoked the same stopping-power routine. Agreement measured the routine's determinism, nothing else. |
| Shared data library | Two codes with different transport physics draw on the same evaluated nuclear data. Agreement tests the transport implementation and says nothing about the data. |
| Shared omission | Both models omitted thermal scattering treatment. Same missing physics, same direction of error, mutual confirmation. |
| Compensating errors | The most dangerous class. One model omitted a factor of ν — neutrons per fission — that another over-applied. The errors partially cancelled and produced apparent agreement between two quantities that were each wrong. Here agreement is not merely uninformative — it is actively misleading. |
| Unsourced constants | A number enters the code with no provenance, is documented as though sourced, and thereafter becomes "the value." Nobody re-derives it because it is already written down. The dangerous form is a false citation recorded beside a true one — in the same metadata field, in the same format, carrying the same apparent authority — and propagated into machine-readable output. An undocumented number invites scrutiny. A falsely documented one, sitting next to a correct one, does not. |
| Interpolation artifacts | Standard library interpolation clamps at the boundary of its data. Extrapolating below the lowest measured point returned the boundary value, producing an apparent optimum that was an artifact of the routine, not of the physics — and it looked attractive. |
| Unconstrained optimisation | Fixing an output and leaving an input free lets an optimiser trade expensive physical quantities for cheap unphysical ones. A container-count model with no current ceiling produced "optima" requiring beam currents forty times anything ever demonstrated. |
| Optimising the wrong quantity | A model can be internally perfect and answer a question that does not bind. The design in question was optimised for footprint when the binding constraint was beam power on target — a variable the model did not contain. |
| Mismatched quantities | The dominant failure in the case study, and the hardest to see. Two numbers are compared as though they measured the same thing when they did not — a fission count measured for one configuration against an identity computed for another; a total against a first flight; a weighting by atom fraction against one by reaction rate. The arithmetic on both sides is correct and the agreement can be close, which is why it survives review. Before comparing two numbers, establish that they are numbers of the same thing. |
| Circular closure | A residual is obtained by solving a balance equation, then substituted back into that equation, and the resulting identity is reported as agreement. It closes to arbitrary precision because it cannot do otherwise. The tell is that no independent measurement of the residual exists — and the check will nonetheless read as the strongest evidence in the document. |
| A test that cannot fail | A check is written, pre-registered, run and reported, and has no power to discriminate. A benchmark-similarity threshold was set at an overlap of 0.5 — but 0.56 was the subject system's own fast-fission fraction, so every fast-dominated benchmark passed automatically. The test measured nothing and read as coverage. A threshold must be shown to be capable of rejecting something before its satisfaction counts as evidence. |
| Retrospective mechanism | A result appears, and a plausible physical explanation is constructed for it from general principles — while a specific measurement already in the record contradicts that explanation. Three instances occurred in two days: a discrepancy attributed to a high-collision tail that did not exist; a claim that shield hydrogen supplied reactivity when removing it raised the eigenvalue by 1,399 pcm; and a density prediction wrong by 2.3× because the blanket was treated as a bare leaking system despite a reflector and shield worth 17,868 pcm already being tabulated. The explanation is the dangerous part, because it gets written down as a finding. |
MEASURED-CITED · experimental, with citation ·
MEASURED-OURS · from a run we executed ·
DERIVED · computed from tagged inputs ·
ESTIMATE · engineering judgement ·
UNVERIFIED · recalled or secondary.
Tags live in the code and print with every run, so they cannot be forgotten. An
UNVERIFIED number may not enter a decision. It is a hypothesis until
promoted.
Not from memory, not from a review article, not from an existing model. From NIST,
from the original measurement paper, from the design report. If a figure cannot be
traced, it is UNVERIFIED regardless of how long it has been in use or how
authoritative its apparent origin.
The test of a successful correction is that the corrected model reproduces an independent quantity it was not fitted to.
Retrieval before reconstruction. Where a validated artifact already exists, building your own inserts you into the error chain for no gain. Three attempts were spent reconstructing a reactor-physics benchmark from a paywalled specification before it was found that fifty-four validated models of the same benchmark family sat in a public repository, maintained by the code's own developers. Had the reconstruction been used, any disagreement with experiment could not have been separated from the author's transcription error.
The prediction and the decision rule are written down first — what result would confirm, what result would falsify, and what happens in each case. Pre-registration removes the possibility of rationalising an outcome after seeing it, which is the single most common failure in self-assessment.
Amended 7 August 2026 — abort criteria are separated from prediction falsifiers. An abort criterion may be derived only from a conservation law or identity, from reproduction of a prior measurement, or from availability of required data. Never from an expectation about the answer. A falsified prediction is a result: it is recorded, it updates the model of the physics, and the run continues.
The amendment was forced by two of its own violations — a decision threshold set on the eigenvalue when the objective function was the fission count, and an expected sign encoded as a halt condition, which duly halted a run on correct data. Since adoption, five identity or reproduction criteria have been written across five runs and two have fired on real defects, including a case where the pre-registration specified a perturbation at fixed heavy-metal loading and the implementation held fixed mass density instead. The code was corrected to match the document, not the reverse.
Before two methods may be said to agree, four questions are answered explicitly: do they share a subroutine? a data library? an omission? could their errors compensate? Any yes, and the agreement is discounted to a single observation.
Cost is almost always cheaper to check than physics. A cost screen needs arithmetic and a market price; a physics screen needs a model, a code, a data library and a week. Concepts are therefore screened economically before they are screened physically — including the limiting case in which the enabling technology is free.
A concept that fails with a free accelerator does not need a neutronics model.
Added on the evidence of the case study below. Three errors — including one in which a back-solved residual was presented as a closing balance — survived the author, the assisting analysis and a full write-up, and were found in under an hour by a reader given the record and instructed to be blunt.
Some errors are structurally invisible from inside. A calculation looks like a check until you trace where one of its inputs came from, and the person who produced the input is the least able to see it. External review is not a courtesy. It is the cheapest countermeasure in this protocol after Rule 5.
Most concepts terminate at step 3, in days rather than months, at a cost of arithmetic rather than simulation.
Claimed: net-positive electrical output, supported by four computational validations. Provisional patent, international PCT application, business plan, valuation study.
Found: two independent errors. An electronic stopping-power array of untraceable provenance, wrong by more than an order of magnitude against the NIST primary source, sitting in the denominator of the yield integral. And fissions per source neutron taken as k/(1−k) rather than k/[(1−k)·ν] — an overstatement by the mean number of neutrons per fission.
Corrected: the device does not produce net electrical power. The sign of the result was determined entirely by the unsourced array.
Verified how: the corrected model reproduces a NIST reference quantity to 0.6% — a quantity it was not fitted to, and the strongest form of confirmation this protocol recognises. A second check reported at the time, in which a Monte Carlo measurement of fissions per source neutron matched the corrected identity to 0.6%, has since been withdrawn. See below.
Then: a redesign of the neutron source — proton spallation in place of beam-target fusion — produced a genuinely net-positive architecture, which was nonetheless set aside at step 3 of the screening sequence: it failed the optimistic economic screen against a grid-electricity benchmark even with the enabling technology costed at zero.
Status of that closure, revised 8 August 2026. The screen that closed it measured capital cost per watt against wholesale electricity. That benchmark does not apply to a user for whom delivered energy is priced by logistics rather than by market — where the comparator is fuel convoyed to the point of use, not the grid. The spallation architecture is net-positive on energy and is therefore returned to conditional: open on physics, contingent on an application in which cost per watt is not the binding constraint. Its scale, of order tens of megawatts, is the open question rather than its energy balance.
Timeline: both findings reached in 48 hours, from primary reference data, before any external party raised a question and before any capital was committed.
On 4 August 2026 an external reviewer asked three technical questions, one of which required a calculation that had never been run — an inhomogeneous fixed-source calculation, without which the source-importance factor cannot be determined. Running it showed that the July correction contained two errors of its own.
The apparent 0.6% agreement on fissions per source neutron was a coincidence between two different geometries: a measurement taken from one configuration had been compared against an identity computed for another. The source importance derived from that pairing was wrong by a factor of nearly three. And the record's claim that the criticality eigenvalue had been confirmed by two independent methods could not have held under any pairing, because the driven-cascade quantity measures a product of two unknowns and determines neither alone.
Corrected, the device is a factor of 2.7 better than the July figures state and remains a factor of 2.0 short of break-even. The verdict did not change. The reasoning behind it did.
Diagnosing a factor-of-two disagreement between the two transport codes, which surfaced during the same work, identified a defect in one of them: its fission model emits an essentially energy-independent neutron multiplicity, reproducing the evaluated value at thermal energies and failing to scale it with incident energy. In a system operating near criticality, where a one per cent error in the eigenvalue becomes a twenty-five per cent error in power, a five per cent multiplicity deficit became a factor of two.
Why this belongs in the case study rather than being quietly fixed. The July audit applied Rule 2 to one error and not to the other. It retrieved the stopping-power data from a primary source, which worked, and it corrected the fission count by comparing a measurement against an identity — without asking whether the two described the same configuration. That is failure mode nine, committed by the protocol's own audit, and it took an outside question to expose it. A method that could not catch its own output would not be worth publishing.
Work continued under the protocol, one pre-registered run at a time, each bound to its predictions by the SHA-256 of the document written before it. Four claims standing in the record were overturned, three of them favourable to the design and one not.
Two terms that had been carried as engineering judgement were replaced by measurement. Nuclear-data and method bias was estimated at roughly 1,000 pcm; running every uranium-fuelled benchmark in a public criticality suite — fifty-four families, weighted by measured spectral overlap with the subject system rather than by the benchmarks' own category labels — gives 434 pcm. Manufacturing tolerance was estimated at 500 pcm and measures 428, of which 81% is fuel density alone. Both replacements were confirmations rather than corrections, which is the outcome an honest estimate should produce.
A binding constraint was then issued and treated as binding: a floor on the subcritical margin of 3,000 pcm in the cold, dry-shield, beginning-of-life state, derived from the measured terms and not from ambition, written down before any optimisation was allowed to see it. It happens to coincide with the 3% margin recited in the patent's own claims, which were drafted from the superseded figures.
And a lever the analysis expected to find does not exist. Neutron yield per unit beam energy was predicted to improve by roughly 30% at a better deuteron energy. Computed across 60 keV to 1 MeV against retrieved stopping-power data, the existing 180 keV design point is the optimum, to within a fraction of a per cent. Target pressure cancels out of the yield integral entirely and a solid target would be worse. The beam-target source is already maximal; there is no room there at all.
Where the concept stands. The device delivers 51% of the electrical power it consumes. Decomposing the energy balance into its four independent factors — source yield per unit beam energy, blanket energy multiplication, thermal conversion efficiency and accelerator efficiency — shows that beam current cancels, that the source factor is at its ceiling, and that the remaining route to unity requires the eigenvalue driven to its floor together with two plant efficiencies that are not the author's to verify. That path reaches approximately 1.03: a three per cent margin. It is not closed and it is not established. Work continues, and the outcome will be reported either way.
The full account — the arithmetic, the corrected analysis, the twenty-seven errors made during the audit and everything that followed it, and every remaining open item — is recorded in SFHB — Complete Project Record, version 2, 6 August 2026, and Technical Correction Notice No. 3 of the same date. The measurement state as of 8 August 2026, the forty-one errors, and the binding margin floor are recorded in version 3 of the same record.
Three features of that record are the reason this document exists. The first audit was self-initiated — nobody asked for it. The second was not, and the record says so: the errors of 4–6 August were found because a reviewer asked a question, and claiming otherwise would cost more than it gained. And the record is unrevised, including the parts that are unflattering to its author, because a record edited after the fact is worth less than one that was not.
| Deliverable | Content |
|---|---|
| Governing relation | The architecture reduced to its binding equation, with each term identified as physics-capped or engineering-capped |
| Provenance register | Every input tagged, with citations for those that carry them and an explicit list of those that do not |
| Optimistic economic screen | Capital cost per watt and levelised cost under best-case assumptions, including the free-enabling-technology limit |
| Sensitivity ranking | Which assumptions move the answer, ordered by magnitude, so diligence budget is spent where it changes the conclusion |
| Independence audit | Assessment of whether the subject's own validations are genuinely independent, applying the twelve failure modes of §2 |
| Reproducible code | The analysis as executable scripts with tagged inputs, so the client's own advisors can re-run and vary it |
| Finding | Go / no-go / conditional, with the arithmetic shown rather than asserted, and the conditions stated as testable propositions |
Engagement: fixed scope, fixed fee, two to three weeks. Deliverables are written to be handed to a third party for checking — which is the point.
Stated plainly, because a screen that oversells itself is the thing it exists to prevent.
One conclusion from the case study generalises beyond nuclear engineering, and it is the operating assumption of every screen conducted under this protocol.
Fluent, confident, internally consistent analysis is produced identically whether the underlying input is right or wrong. This is true of a well-built model, a respected consultancy, an experienced colleague, and an AI assistant. A collaborator who agrees with you, checks your arithmetic and reproduces your result has verified nothing if they are working from the same unverified number.
The protocol assumes this of itself. During the audit described in §5, the analysis produced eight errors of its own — an incorrect claim about a simulation cascade limit, a wrong stopping-power prediction, a mistaken accelerator gradient derived from confusing an energy with a length, an interpolation bug producing a false optimum, and an architectural proposal that did not survive its own arithmetic. All eight were caught by the same rules that caught the original error, and all eight are documented.
Two further days of work under the protocol added eleven more from the same source, bringing the assistant's own total to nineteen: two mis-specified halt conditions that forced the Rule 3 amendment, three retrospective mechanisms of the kind now recorded as failure mode eleven, a similarity threshold with no power to reject anything, a non-existent reference read off a screenshot and written into a pre-registration unverified, a special case reported as a general result, a sign convention inverted in a summary table, a date error in a pre-registration — which would have destroyed that document's only purpose — and a divergence between what a pre-registration specified and what its code did. Eight were caught by the protocol's own rules. Three were caught by the reader.
Across the audit, the screens that followed it, the second correction of 4–6 August and the measurement campaign of 6–8 August, the count now stands at forty-one, every one recorded. Nineteen of the forty-one are a single failure: two quantities treated as comparable when they were not the same quantity. That is why it was added to §2 as failure mode nine, and it is the most useful thing this project produced. Its persistence is the point — it recurred three more times in a fortnight, in a project actively watching for it, committed by people who could recite its definition.
Five of the forty-one were caught by no rule at all, but by a reader who had taken no part in producing them — which is why Rule 6 exists. Two of those five arrived in the last two days: a date on a pre-registration, and a framing in which a patent claim was being treated as a constraint on the physics when it is an amendable document and the physics is not.
The protocol is not a claim to be right. It is a method for finding out when you are not, before it becomes expensive.