Research
Exact, committed, replayable.
How Plimsoll decides PASS, FAIL or INCONCLUSIVE, and how we test the engine itself.
Library
What we measured, and what it did not show.
Each article states its counts, its bounds and where they came from, and ends with the limits of what it shows.
- Research4 min read
Exact bounds, counted by cluster
Why every rate Plimsoll SI reports carries an exact one sided bound, and why related questions count once.
- Research4 min read
INCONCLUSIVE is an honest verdict
How one of our own passes was regraded INCONCLUSIVE, and why a test that cannot prove its limit has to say so.
- Research4 min read
Commit the plan before the first question
What a commitment pins, why a run that drifts from it fails, and what a commitment cannot stop.
- Research3 min read
Controls that must fail
A test that finds nothing proves nothing until it is shown able to find something. How planted faults and controls keep our zeros honest.
- Research4 min read
Testing access as every role
0 of 662 leaked with the controls firing: how we test that an assistant shows each role only what it may read, and what that zero covers.
- Research4 min read
Our blind benchmark: the claim was not supported
Faults written by authors who never saw a testing tool, scored once. Detection was higher, but false alarms and false passes sank the claim.
- Research3 min read
A leak depends on who asked
We ran the string matching verdicts of three open testing tools on replies with known answers. Without an access policy, 14.4% of clean replies were flagged.
- Research4 min read
Injection through pasted text, and an attacker that adapts
Why a planted value must never reach the screen, what an adaptive attacker broke, and why the model is not the security boundary.
- Research4 min read
Why a seal needs witnesses
A hash chain verifies perfectly after its holder rewrites it. What a witnessed log adds, measured over 1,000 rewrites per attack.
- Research3 min read
What a passport proves, and what it does not
A passport carries a sealed run, its verdict and every bound. What a reader can check offline, and the questions it cannot answer.
Method
Six rules behind every verdict.
Commit before you look
Exact bounds
Counted by cluster
Controls that can fail
INCONCLUSIVE with a cause
Replay
bound 1.93%, limit 5.00%, 1 event from FAIL
0 events in 154 clusters: bound = 1 − 0.05^(1/154) = 1.93% ≤ 5.00%. Sample data.
How to read a bound
The bar is the worst rate the evidence still allows, at 95% confidence. The pin is the limit committed before the run. A defect check, such as a leak, fails on any confirmed event; with none, a bar short of the pin is a PASS. A rate check, such as accuracy, fails only when the evidence rules the limit out. Anything between waits for more questions as INCONCLUSIVE.
Blind benchmark
We test the engine the way we test your system.
A pre registered blind benchmark: faults written by authors who never saw the engine, frozen before any run, scored once against Plimsoll and other open tools under the same rules. The first one was scored in October 2026, and its claim was not supported. The write up gives the numbers, including where other tools did better.
Written blind
Faults come from authors who never saw the engine or its tests.
Frozen
The set is frozen before any tool runs against it.
Scored once
Each tool is scored once, under the same rules.
Published
Results go here with their bounds, including where other tools do better.