Skip to content
SkillModel-led2,736 tokens in use

derive-tests

Turns documented intent into a test coverage map: which rules have tests today, which are proposed, and which gaps are left, ranked by what crossing them exposes.

680 stars on the repository MIT Licensephuryn/product-engineering · ai-shipping/skills/derive-tests
claude plugin install ai-shipping@product-engineering
View source

Install command as written in the repository's README

Skill label

measured from its files

What installing derive-tests costs

32tokens idle
2,736tokens in use
2,914tokens, up to

Idle is what sits in context while the skill is installed. In use is its instructions, loaded when it runs. Up to adds the files its instructions link. Counts are approximate, made offline with cl100k_base.

Model dependence Model-led

Script-ledMixedModel-led

Mostly instructions, so how well it works depends on the model that follows them.

This describes the skill's structure, how much of it is scripts. It does not say how well it works.

Touches

Nothing detected: no shell, network, file writes, git or credentials in its files.

Declared tags come from the skill's frontmatter; detected ones from its scripts, its code blocks and the commands it names in inline code. This is detected, not audited.

How it scores

Quality

How well the skill is written, counted from its own files: its frontmatter, its steps, examples, stated output and constraints, its size and whether every file it ships is one it names.How we score it →
Excellent
  • Frontmatter8/15
  • Structure25/25
  • Examples12/15
  • Output15/15
  • Size15/15
  • Resources15/15

Security

Starts at 100 and loses points for each thing in the skill's files that a reader should look at first.How we score it →
Excellent

No findings.

Repository health

Pre-computed score based on development activity, maintenance, community, maturity, and trend momentum.How we score it →
Good
  • Development activity100
  • Maintenance100
  • Community44
  • Maturity4
  • Momentum40

derive-tests reads a system’s docs (flows, permissions, automation) and its existing tests, extracts the rules whose violation crosses a trust, data, money or tenant boundary, and writes a coverage map with a concrete test case for each one.

It separates coverage that exists today from coverage it proposes, so the map never claims a rule is pinned when nothing asserts it. A check mode compares an existing map with the docs and tests and reports where they disagree, without writing a file.

What You Get

  • A coverage map, written to the repo’s own coverage file or documentation/tests.md, with one row per rule: expected behavior including the deny case, evidence from docs and code, test type and status of existing, proposed or none
  • Concrete test cases grouped by flow, each with its name, arrange/act/assert intent and the negative case it must reject
  • A journeys section that maps each flows-doc journey to the end-to-end spec that exercises it, and flags journeys covered only by faked identity or a faked browser
  • A gaps list ranked by what crossing the boundary exposes, and a gate recommendation that follows the repo’s stated policy
  • A check-only mode that prints stale entries, journeys without end-to-end coverage and what was not checked, with no file written

Common Use Cases

  • Before a release, to see which documented permission and tenant rules have no test
  • After an audit finds a gap, to pin a regression test so the next AI edit cannot reopen it
  • Checking whether a hand-maintained coverage map still matches the docs and the test suite
  • Deciding which suites should gate merges when a repo has no stated policy

What Makes It Unique

It starts from documented intent and the code’s behavior is checked against it, so a rule nobody tests shows up as a gap and is never assumed covered. It marks a rule existing only when a repository test asserts it today, treats a pure helper as a complement to testing real enforcement, and defers to the repo’s own gate policy. The check mode lets a release verify a map kept by hand without overwriting it.

Under The Hood

The skill is instructions only, with one short receipts file of the cases behind its rules. The agent reads docs and test files in the repository, may fan out to subagents on large scopes, and writes one markdown map file. Check mode writes nothing and leaves the tests unrun. The skill calls no web services and ships no scripts. It points to the document-app skill when docs are thin and to security-audit-static for the audit itself.

Join founders building with open source

Opinionated takes, migration guides, cost-saving tips, and insights from the open source ecosystem.

Subscribe on SubstackJoin 750+ subscribers