I use agent skills throughout the production process behind The Main Thread, from tutorial structure and voice guidance to technical review and publication preparation. In this episode, I explain how those instructions developed from things I kept repeating to the model, and how I decide whether a skill helps me reach something I’m prepared to publish.
We start with the open Agent Skills format: what belongs in SKILL.md, how the description helps the agent select a procedure, and how references and scripts become available when the task needs them. Then we follow a pull request summary through the inputs it needs: the right comparison base, the relevant diff, the requirements, and evidence for any claimed test result. The skill has to explain what happens when one of those is missing. Access and permissions also depend on the host running the agent.
To create a skill, I start with a recurring task whose result I know how to assess. The examples and corrections help me separate this task’s facts from the procedure another task can reuse. Well, a clean file is only part of that. I discuss comparing the task with and without the skill, checking the agent’s actions, and counting the time someone spends reviewing and correcting the result.
Sharing the skill adds another responsibility. A colleague needs to understand the inputs, environment assumptions, expected result, and version they are using. I draw on the team memory idea from the Book of Bob, my internal IBM guidance, and discuss repository ownership, skillsaw checks, and distributing skill files through versioned Maven artifacts with SkillsJars.
I also talk openly about my use of AI across this production process. Drafting can become cheaper while checking the result takes time and understanding. I accept each reader and listener making their own judgment about that choice. The episode acknowledges the ethical and environmental implications and leaves them for a separate discussion.
The episode is narrated locally with Fish S2 Pro and my trained LoRA voice adapter.
Chapters
00:00 — Intro
00:12 — My publishing process and using AI openly
02:55 — What an agent skill contains
06:18 — Structure, voice, review, and publishing
09:25 — Designing a pull request summary skill
13:19 — Creating a skill from examples and corrections
16:35 — Evaluating the result and the review effort
19:27 — Sharing skills as team memory
23:39 — Outro
Links and references
Format and evaluation
Agent Skills specification — The open folder format,
SKILL.mdmetadata, optional files, and progressive disclosure.Evaluating skill output quality — Designing realistic cases and comparing results with and without a skill.
Related Main Thread articles
Agent Skills Need Guardrails, Not Just Prompts — Task procedures, supporting scripts, and checks around repeated agent work.
Use skillsaw to Lint Bob Skills Before They Drift — Structural checks, CI, and a separate local task to examine the skill’s behavior.
Stop Copying AI Skills: Version IBM Bob Instructions with Maven — The SkillsJars example discussed in the episode, packaging and consuming versioned skill files.
Research
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks — Version 4 of a benchmark studying skill effects across different tasks and agent configurations.
SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering? — Version 1 of a study comparing software engineering tasks with and without skills.
The studies use different tasks and setups. Their findings are a reason to evaluate a skill in your own process, rather than assume a reported gain will transfer.
