Skip to content
Plimsoll SI
All tutorials

Tutorial

Read a FAIL, fix it, retest on the same plan

Open a finding, read its smallest example, fix the cause, and prove the fix with a retest on the same sealed plan.

3 min readPlimsoll SI

Screen recording to comeA short console clip with sample data. A full screen recording of this tutorial will go here.

A FAIL is the start of the work, not the end. Every failed or inconclusive check in a run opens a finding with the smallest example, a cause and a suggested fix. This tutorial reads one, fixes it, and proves the fix on the same plan, so the before and after are comparable.

1. Open the finding

Open Findings. The list shows every failed or inconclusive check, to triage, fix and retest, with its severity, status and the check that raised it. Filter by system or check, and use j and k to move between findings and Enter to open one.

Findings: a list of failed checks on the left and one finding open on the right, with its smallest example.
Findings, with one open. Sample organisation.

2. Read the smallest example

The finding names the check and the run that raised it, and shows the smallest example: the role it was asked as, the question, and the reply. It is the shortest case that still shows the fault, so it is the one to reproduce first. Below it are the "Cause class" and a "Suggested fix".

Read the bound on the run page as well. A defect check, such as a leak or an unapproved action, fails on any confirmed event. A rate check, such as accuracy, fails only when its exact bound excludes the limit. An INCONCLUSIVE finding says whether more questions, more data or a working control would settle it.

3. Assign and fix

Use "Assignee" to give the finding to someone. Fix the cause in your system: the access rule, the prompt, the tool guard, whatever the example points at. Do not change the plan to make the finding go away; a plan changed after seeing a failure no longer tests the same thing.

4. Retest

Choose "Mark fixed". A retest is queued and shows in the finding's history. Or choose "Retest now" to start it at once. A retest is a new run on the same sealed plan version as the run that opened the finding.

  • A passing check verifies the finding.
  • A failing or inconclusive check reopens it.
  • A later retest is recorded in the history too; if the check fails again, the finding reopens.

5. Keep it fixed

Choose "Make this a regression rule" to turn the example into a rule for this system under Test data, so later plans can check it. The button then becomes "Open the regression rule".

What you have

  • Two runs on one commitment, the failure and the retest, both in the passport's history.
  • A rule that keeps the same mistake from returning unnoticed.