mnmx

Faithful tests

Automated feedback makes the stated behavioral contracts checkable and changes easy to verify. The test suite collectively provides credible evidence for the behavior users rely on, including state transitions, failure guarantees and meaningful regressions.

Usage

TypeScript
import { agent, goal } from '@mnmx/engine';
import { testsFaithful } from '@mnmx/engine/stdlib';

const owned = goal.with({
  owner: 'team:checkout',
  nextReview: { on: 'change' },
});

const reviewer = agent({
  id: 'checkout-reviewer',
  model: 'project-review-model',
  prompt: './reviewer.md',
});

export default owned(testsFaithful(reviewer));

Give your reviewer the behavioral contracts and the tests that exercise them. The rubric checks coverage of important boundaries and failures, the strength of assertions, and the clarity of scenarios.

Acceptance criteria

clauses
Taken together, tests cover the stated behavior and important refusals, boundaries and recovery guarantees. Different tests may cover different clauses of one entry, and one test may support several entries; traceability does not imply that each test proves the whole entry.
valid-refusals
A claimed refusal test reaches the condition it intends to exercise. An unrelated setup or validation failure is not evidence for that condition; suitable fixtures, controls or focused assertions make the reason clear.
causal-evidence
Assertions check meaningful observable outcomes and relevant state after failure. Evidence is chosen proportionately to risk; mutation testing and other fault injection can reveal gaps but are optional techniques, not required for every test or clause. Material coverage limitations are disclosed without claiming exhaustive proof.
clarity
A reader can identify the scenario, relevant starting state, action and promised outcome of a test, including what failure would mean for the contract. Tests protect observable behavior rather than mirroring implementation steps; types and other automated checks complement tests where they provide relevant evidence.

Review rubric

Read the bundled reviewer prompt
Judge whether understandable automated feedback makes behavioral contracts
checkable and meaningful changes easy to verify.

Trace important requirements to executed scenarios and assertions across the suite.
Several tests may cover an entry together, and one test may support several entries.
Do not demand that each tagged test cover every clause. Inspect critical success
paths, refusals, state preservation and recovery; choose depth proportionate to
impact and uncertainty rather than a fixed inventory or mutation quota.

A negative test must exercise its intended condition. An unrelated validation
failure, broken setup or timeout does not demonstrate the claimed refusal. Check
fixtures, assertions and controls as needed to establish what the test shows.
Mutation testing and fault injection may help, but neither is mandatory. A surviving
mutant can expose a coverage weakness; it is not by itself a bug in the unmodified
package or an automatic failure of the whole suite. Assess the affected contract,
other coverage and risk. Unit, integration, property and other test styles are valid.

For representative tests, identify the scenario, relevant starting state, action
and promised outcome. A failure should help a maintainer understand which contract
was broken. Assertions that only repeat implementation steps are weaker evidence
than checks of observable outcomes and preserved state. Types and other automated
checks may complement the suite; do not require a new tool or test style solely
for rubric compliance. Test readability matters when it affects diagnosis and
safe change, not because a particular naming or fixture convention is preferred.

Score 1–5:

1. Automated checks give little intelligible evidence of required behavior.
2. Important contracts or failure guarantees lack credible verification.
3. Core behavior is checkable but material evidence or scenario-clarity gaps remain.
4. Clear scenarios credibly verify important contracts and failure boundaries with limited gaps.
5. Understandable, risk-appropriate feedback makes meaningful behavior changes verifiable with no identified material gaps.

Cite the evidence that determines the score. Separate executed results, inspection
findings and untested claims. Report concrete coverage gaps and their significance,
not a completeness percentage or a claim of proof against every possible defect.
1
Automated checks give little intelligible evidence of required behavior
2
Important contracts or failure guarantees lack credible verification
3
Core behavior is checkable but material evidence or scenario-clarity gaps remain
4
Clear scenarios credibly verify important contracts and failure boundaries with limited gaps
5
Understandable, risk-appropriate feedback makes meaningful behavior changes verifiable with no identified material gaps