Skip to content
SkillScript-led3,204 tokens in use

post-patch-validation

Check a security patch against a pinned base with exploit, variant, behavior, regression and suite checks, and hand the patch author reproducible failures.

8,319 stars on the repository CC-BY-SA-4.0trailofbits/skills · plugins/post-patch-validation/skills/post-patch-validation
/plugin marketplace add trailofbits/skills
View source

Install command as written in the repository's README

Skill label

measured from its files

What installing post-patch-validation costs

87tokens idle
3,204tokens in use
5,662tokens, up to

Idle is what sits in context while the skill is installed. In use is its instructions, loaded when it runs. Up to adds the files its instructions link. Counts are approximate, made offline with cl100k_base.

Model dependence Script-led

Script-ledMixedModel-led

Most of this skill is scripts, so it behaves much the same whichever model runs it.

This describes the skill's structure, how much of it is scripts. It does not say how well it works. Scripts: scripts/post_patch_validation.py.

Touches

shelldeclared and detectedwrites filesdeclared and detected

Declared tags come from the skill's frontmatter; detected ones from its scripts, its code blocks and the commands it names in inline code. This is detected, not audited.

How it scores

Quality

How well the skill is written, counted from its own files: its frontmatter, its steps, examples, stated output and constraints, its size and whether every file it ships is one it names.How we score it →
Good
  • Frontmatter12/15
  • Structure25/25
  • Examples9/15
  • Output15/15
  • Size15/15
  • Resources3/15

Security

Starts at 100 and loses points for each thing in the skill's files that a reader should look at first.How we score it →
Excellent
  • Runs shell commands−2

    does this on the machine, as the frontmatter declares

    SKILL.md, line 42
  • Writes files−4

    does this on the machine, as the frontmatter declares

    scripts/post_patch_validation.py, line 251

Repository health

Pre-computed score based on development activity, maintenance, community, maturity, and trend momentum.How we score it →
Good
  • Development activity100
  • Maintenance52
  • Community64
  • Maturity32
  • Momentum40

Post-patch-validation tests a security fix after it exists and before it is accepted or reported as fixed. The agent pins the vulnerable base and the patched input, scaffolds a plan, fills in checks, and a bundled Python runner executes them on both revisions in isolated Git worktrees.

The plan must hold at least one check of each kind: control, exploit, root-cause variant, behavior, regression, security and suite. The runner saves raw logs and writes a result and a report with findings and validation gaps, and passing checks still call for human review.

What You Get

  • A pinned plan.json scaffolded from base and patched refs or a patch file
  • A result.json with an assessment holding status, findings, gaps and a human review flag
  • A report.md plus raw stdout and stderr for each check
  • Archived scratch output and hashed helper files for review
  • Findings named by check ID and assertion, ready to return to the patch author

Common Use Cases

  • Validating an AI-generated security patch before human review
  • Looking for missed variants when a fix covers only one exploit path
  • Checking that a remediation commit keeps legitimate behavior byte-identical
  • Producing a fresh validation run after the patch author revises

What Makes It Unique

Exploit and variant checks must print a PPV_REACHED marker on stdout before the assertion and run side-blind, so a broken harness or a check that can see which revision it runs on cannot fake a pass. Commands are argv arrays and the environment is fixed and stripped, with credential-like and code-changing variables refused.

Under The Hood

The agent reads SKILL.md and an evidence model reference, and runs one bundled Python script through uv with scaffold, validate-plan and run commands. The runner creates locked Git worktrees for the pinned revisions, executes the plan’s argv commands under a minimal fixed environment, and writes logs, hashes and reports to an output folder. It runs the repository’s own code and tests with the caller’s privileges and is not a sandbox, so untrusted check code needs a container. Submodules come from local Git objects and it makes no network calls of its own. A Claude workflow in the plugin can drive it.

Join founders building with open source

Opinionated takes, migration guides, cost-saving tips, and insights from the open source ecosystem.

Subscribe on SubstackJoin 750+ subscribers