Use skillsaw to Lint Bob Skills Before They Drift

Build a small invoice-processing Bob skill, add a reference file, wire CI, and test the behavior locally before the instructions spread across repos.

Jul 20, 2026

A SKILL.md file is easy to write and maintain when there is only one copy.

Then it gets copied into a second repo. Then a third. One copy still points at an old path. One still has placeholder text. One quietly relaxed the rules because somebody wanted to get through a demo. At one point the file is part of the system. It needs the same kind of care you already give to scripts and config.

That is why I like skillsaw. It treats skill files like artifacts you can inspect and lint. The tool focuses on file quality: wrong shape, bad naming, leaked placeholder text, vague instructions. That catches a lot of drift early.

The example here is artificial on purpose. We use one small Bob skill called invoice-processor. It validates invoice totals, writes results to one output file, logs mismatches, preserves input order, and forbids shell commands. The rules are narrow, which makes them easy to inspect.

The result is a small skill folder, a clean skillsaw loop, a GitHub Actions check, and a Bob prompt that runs on a local input.json file with no external service, no MCP server, and no API keys.

What You Need

You should already be comfortable editing Markdown, using a terminal, and running IBM Bob inside an IDE.

Python 3.11 or newer
IBM Bob in your IDE (download a trial if you like)
GitHub Actions if you want the CI part
About ☕️

Create The Skill Repo

First decide who owns the skill. That decision is way more important than the directory name.

Use a repo-local skill when the rules only make sense inside one codebase. In that case, keep the skill with the project, for example under .bob/skills, and review it with the same pull request that changes the workflow it controls.

Use a separate skill repo when the skill is shared across several repos, several teams, or several projects. That repo becomes the source of truth. It should have its own lint check, release tags, and a small changelog so consumers can tell which skill release they installed.

Use personal skill directories only for experiments or private habits. They are fine for local work. They are a bad place to hide team behavior because nobody else can review or reproduce them.

For this tutorial we use a separate repo because we want to show the full skillsaw loop: lint the skill, pin the rule set, run CI, and then install the checked skill into Bob.

Create the layout:

mkdir -p main-thread-skills/invoice-processor/references
cd main-thread-skills
python3 -m venv .venv
.venv/bin/python -m pip install skillsaw==0.16.0

If you already use uv, the skillsaw quick start also supports uvx skillsaw.

Start with a bad skill on purpose. The lint output is easier to read when the file is obviously rough.

Create invoice-processor/SKILL.md:

---
name: Invoice Processor
description: process invoices
---

# Invoice Processor

Process invoice batches.

You can use shell scripts if that is faster.

Write results somewhere under out/.

Maybe stop on errors.

TODO: tighten this.

This looks small if you skim it. It also gives Bob far too much room to improvise.

Read The Tree First

Before linting anything, ask skillsaw to show what it found in the repo.

Run:

.venv/bin/skillsaw tree .

For this example, the output is:

main-thread-skills/ (51 tokens)
    └── invoice-processor/ [skill] (51 tokens)
        └── SKILL.md (skill) (51 tokens)
            ├── frontmatter:name (4 tokens)
            ├── frontmatter:description (4 tokens)
            └── body (43 tokens)

If skillsaw does not even see a skill, stop there and fix the layout first.

Lint The Rough Draft

Stay in `main-thread-skills`. That is the repo root for this example.

Now run a different subcommand: `lint`, not `tree`:

.venv/bin/skillsaw lint .

On the rough draft above, you should see findings in this shape:

skillsaw 0.16.0
Linting: /path/to/main-thread-skills

Errors:
  ✗ ERROR (agentskill-name) [invoice-processor/SKILL.md:2]: Name 'Invoice Processor' must contain only lowercase letters, numbers, and hyphens

Warnings:
  ⚠ WARNING (content-placeholder-text) [invoice-processor/SKILL.md:16]: Placeholder text (TODO marker): 'TODO'

The output points at the real problem quickly: the instructions are sloppy.

If you want to find out more about rule warnings and what they mean, ask the tool:

.venv/bin/skillsaw explain agentskill-name
.venv/bin/skillsaw explain content-placeholder-text

Make sure to agree, as a team, on those definitions. We do not want them to feel arbitrary to anyone.

Use Auto-Fix For Mechanical Problems

skillsaw fix handles the mechanical cleanup.

Preview the changes first:

.venv/bin/skillsaw fix --dry-run .

For the bad front matter above, the preview shows a diff like this:

--- a/invoice-processor/SKILL.md
+++ b/invoice-processor/SKILL.md
@@ -1,5 +1,5 @@
 ---
-name: Invoice Processor
+name: invoice-processor
  description: process invoices
 ---

Then apply the safe fixes:

.venv/bin/skillsaw fix -y .

Renaming the front matter takes seconds. Writing rules that leave Bob less room to guess takes longer.

Write The Real Skill

Now write the version Bob can use in real work.

This skill is narrow on purpose. It covers one batch job, one input file, one output file, one error log, and one response shape. That makes it a good target for skillsaw because vague wording shows up fast.

Create invoice-processor/SKILL.md:

---
name: invoice-processor
description: >
  Process invoice batches from input.json, validate totals against line items,
  write validated output to out/invoices.json, log mismatches to errors.log,
  and return a JSON summary.
---

# Invoice Processor

Read [validation examples](references/validation-examples.md) when you need an example of valid and invalid totals. The numbered rules below are the source of truth.

Rules for processing invoice batches. Follow every rule exactly.

1. MUST validate every invoice total against the sum of its line-item amounts (`quantity * unitPrice`) before writing output.
2. MUST NOT execute shell commands. Process invoices using only file read/write tools. Do not run scripts, terminals, or any command execution.
3. MUST write validated results to `out/invoices.json` and nowhere else. Do not write invoice data to any other path.
4. MUST process invoices in the exact order they appear in `input.json`. Do not sort or reorder.
5. On validation failure (`total` does not match line-item sum), MUST log the invoice ID and discrepancy to `errors.log` and continue processing the remaining invoices. Do not abort the batch.
6. Final response to the user MUST be valid JSON matching this shape exactly: `{"processed": N, "failed": N}` where `N` is the count of successfully validated and failed invoices respectively.

This version closes the gaps from the first draft. It makes the shell ban, output path, failure handling, and response format explicit. The reference link gives Bob examples without moving the rules out of the main file.

Add A Reference File

Reference files help when the main skill should stay short but Bob still needs examples, glossary entries, workspace notes, or longer decision rules.

Create invoice-processor/references/validation-examples.md:

# Validation Examples

Use these examples to check invoice total behavior.

## Valid Total

Calculate each line item as `quantity * unitPrice`.
Add the line-item amounts.
Compare the sum with the invoice `total`.

```json
{
  "id": "INV-001",
  "total": 25,
  "lineItems": [
    { "quantity": 1, "unitPrice": 10 },
    { "quantity": 3, "unitPrice": 5 }
  ]
}
```

This invoice passes because `10 + 15 = 25`.

## Invalid Total

Calculate the same way for failures.
Record the invoice ID and discrepancy.
Continue with the next invoice.

```json
{
  "id": "INV-002",
  "total": 18,
  "lineItems": [
    { "quantity": 2, "unitPrice": 5 },
    { "quantity": 1, "unitPrice": 7 }
  ]
}
```

The second invoice has a discrepancy of `1`.

Run tree again:

.venv/bin/skillsaw tree .

Now skillsaw shows the reference as skill-ref:

main-thread-skills/ (442 tokens)
    └── invoice-processor/ [skill] (442 tokens)
        ├── SKILL.md (skill) (329 tokens)
        │   ├── frontmatter:name (4 tokens)
        │   ├── frontmatter:description (44 tokens)
        │   └── body (281 tokens)
        └── validation-examples.md (skill-ref) (113 tokens)

I also tried a Python helper under scripts/validate_invoice.py. scripts/ is an allowed skill subdirectory, along with assets, evals, and references, but skillsaw 0.16.0 did not content-lint that Python file in this setup. A TODO comment inside the script did not trigger content-placeholder-text.

That does not make scripts bad. It means skillsaw is checking skill structure and instruction content. It does not replace Ruff, Pyright, or your normal code checks.

Lint The Finished Skill

Run the linter again:

.venv/bin/skillsaw lint -v .

On this version, the run passes with no errors and no warnings. skillsaw still prints two info-level suggestions:

Info:
  ℹ INFO (content-actionability-score) [invoice-processor/references/validation-examples.md]: Low actionability score: 18/100 (verbs: 17%, commands: 33%, paths: 0%)
  ℹ INFO (content-unlinked-internal-reference) [invoice-processor/SKILL.md:17]: Unlinked path reference: 'out/invoices.json' — consider wrapping in link syntax [out/invoices.json](out/invoices.json)

✓ All checks passed!

I would leave both alone. The reference file is example-heavy by design, so a low actionability score is not surprising. The bare out/invoices.json path is also part of a hard constraint, and I do not want to turn that sentence into link noise only to satisfy an info rule. skillsaw makes that distinction visible: the file passes, and the tool still points out things you may want to clean up later.

Pin The Rule Set

Once the skill is in decent shape, generate a config file:

.venv/bin/skillsaw init .

This writes .skillsaw.yaml with the current skillsaw rule set pinned in it:

version: "0.16.0"

That version is not the version of the invoice-processor skill. It is the skillsaw config and rule-set version. We do still want to pin the specific skillsaw version because new rules added after 0.16.0 do not start changing CI behavior until you choose to bump the config and potentially add new linters.

If you want to release your skill as 1.0.0, do that separately with normal repo versioning: a Git tag, release notes, or a small changelog. If you want to package reusable skills instead of copying folders around, I covered one Java-friendly way in SkillsJars for Java: Package Reusable Agent Skills. Do not use .skillsaw.yaml for skill releases.

If you are adding skillsaw to an older internal repo with a pile of existing issues, this is where the baseline feature helps:

.venv/bin/skillsaw baseline

For a new repo like this one, I would rather fix the warnings than baseline them. Baseline helps when history is already messy. It should not be the main workflow.

Add CI

Now make CI fail when somebody reintroduces placeholders, weak naming, or other soft edits.

Create .github/workflows/skillsaw.yml:

name: skillsaw

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  lint-skills:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: stbenjam/skillsaw@v0
        with:
          path: .
          strict: true

strict: true matters here because warnings are often the first sign that the wording is getting soft before the file is clearly broken.

Other Features Worth Knowing

This tutorial uses the core loop: tree, lint, explain, fix, init, baseline, and CI. skillsaw has more surface area once you move past the first repo.

list-rules shows every built-in rule and its default severity.
--rule and --skip-rule let you run a focused check while debugging one rule.
--format and --output can write json, sarif, html, code-climate, or gitlab reports for other tools.
badge writes a shields.io-compatible .skillsaw-badge.json with the current grade.
add scaffolds marketplaces, plugins, skills, commands, agents, and hooks.
docs generates documentation for plugins, marketplaces, and .claude repositories. It is less relevant for this tiny agentskills repo, but it matters once skills live inside a plugin.
evals/evals.json is validated when present. Requiring evals for every skill is opt-in with agentskill-evals-required.
.skillsaw.yaml supports global excludes, per-rule excludes, inline suppressions, extra content-paths, and local custom rule files.

For untrusted pull requests, run CI with --no-custom-rules. Built-in rules are the safer default. Custom Python rules are code execution, so treat them with the same caution you would give any other contributor-controlled script.

Install The Skill Into Bob

For a local install, Bob needs the files where it can read them. The tutorial uses a copy command because it is easy to inspect.

In the repo where you want Bob to use the skill, copy it into .bob/skills.

For a team, avoid hand-copying forever. Pick one repeatable install method and document it: a small sync script, a Git submodule, a Git subtree, or a release archive that gets unpacked into .bob/skills. The boring best practice is that every consuming repo can answer three questions: where the skill came from, which skill release it uses, and how CI proves the installed copy still passes skillsaw.

Create A Tiny Test Workspace

Create a small test repo or scratch directory with one input.json file:

[
  {
    "id": "INV-001",
    "total": 25,
    "lineItems": [
      { "quantity": 1, "unitPrice": 10 },
      { "quantity": 3, "unitPrice": 5 }
    ]
  },
  {
    "id": "INV-002",
    "total": 18,
    "lineItems": [
      { "quantity": 2, "unitPrice": 5 },
      { "quantity": 1, "unitPrice": 7 }
    ]
  }
]

INV-001 is valid. INV-002 fails validation. The line items sum to 17, while the invoice total says 18. That gives us one clean pass and one clean failure in the same run.

Create the output directory before you hand this to Bob:

mkdir -p out

Use It In Bob

Now give Bob a task that looks like normal local work.

Use a prompt like this:

Use invoice-processor to process input.json.

Watch the behavior.

Check that Bob does all of the following:

Reads input.json
Keeps invoices in the original order
Writes validated output only to out/invoices.json
Logs the failed invoice ID and discrepancy to errors.log
Continues processing after the bad invoice instead of aborting
Returns only JSON in the exact shape {"processed": N, "failed": N}
Avoids shell commands entirely

For the sample data above, the expected final response shape is:

{"processed": 1, "failed": 1}

If Bob opens a terminal, writes invoice data to another path, reorders the records, or wraps the JSON in prose, the skill is still too vague or the tool constraints are not being followed.

Why This Version Works Better

The rough version was shorter. It was also loose enough that Bob had to guess.

The finished skill is still small, but the boundaries are clear: one input file, one output file, one error log, one processing order, one response shape, and one hard ban on shell commands. That is what I want in a real repo. The file stays short, but it is specific enough that a reviewer can tell what should happen.

skillsaw works well for this because it catches the plain failures early: wrong front matter, placeholder text, weak naming, and general sloppiness before those files spread into more repos.

We built one Bob skill, linted it, wired CI, installed it into .bob/skills, and tested it on a local batch with no external setup. Once a SKILL.md changes agent behavior, treat it like code.

Discussion about this post

Ready for more?