Only 20.9% of Agent Skills Follow Their Source: arXiv:2610.11169 Dates 2.2 Million Copies and Finds Stars Point the Wrong Way

2026-10-09 AI Security Research by al-ice.ai Editorial

We have covered malicious agent skills, scanners for agent skills, and skill-injection attacks. What nobody had measured is the thing that decides whether any of it matters: when a skill is fixed, does the fix reach the copies? arXiv:2610.11169, “Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub,” submitted 8 October 2026 by Fahd Seddik (University of British Columbia, Okanagan), answers it with a number: 20.9%.

That figure is the paper’s centre of gravity. Skills are SKILL.md files and scripts that Claude Code, Codex and Cursor execute with the permissions of their user, and developers distribute them by copying. Copying without a registry means no version to pin and no channel to push a fix through. Seddik reconstructs what the ecosystem lacks — a dated, directed graph of who copied what from whom — from the git history of every SKILL.md in the GitSkills dataset, covering 2,193,119 skill adoptions.

Why a snapshot cannot answer the question

Prior work, as the paper puts it, studies skills “in a snapshot, one skill or one pair of skills at a time.” A snapshot records which repositories hold a skill. It cannot tell you which held it first, which copied from which, or when. For clone detection that is adequate; for supply-chain reasoning it is useless, because every question that matters is about direction and time.

The reconstruction works from first-commit dates per SKILL.md path, bounded below by repository creation date, restricted to non-fork repositories with files first committed on or after 1 October 2025. A commit adding ten or more SKILL.md files counts as a copy event when a single earlier adopter held at least half of its existing lineages; that adopter is the source. The cascade of a single skill — its adopters as nodes, dated transmissions as edges — is a constellation. One example in the paper, web-design-guidelines, reached 972 adopters over 12 generations of copies.

The fix-propagation numbers

This is the section security teams should read twice.

Only 20.9% of skill copies follow a later edit of their source. The paper benchmarks this against ordinary copied code files, which adopt upstream security fixes in 47% to 84% of cases. Skills propagate fixes roughly half as well as copy-pasted source code — itself not a high bar.
Only 11.1% of changes to a clone group are consistent — meaning every repository holding one exact version moves to the same next version. The result holds at every observation window.
Only 4.0% of genealogies ever change consistently at all.
Consistency collapses across owners. A single owner raises the odds of a consistent change 3.32-fold, controlling for group size, platform and content. Within one owner’s repositories, updates roughly track; across owners, they do not.

The practical reading: patching a skill at its origin is close to a no-op for the population already running it. There is no dependency graph to walk, no version to compare, and no notification path. The fix lands in one repository and stays there.

A methodological finding compounds this. A snapshot understates the follow rate by 3.3-fold, because an updated copy is indistinguishable from a fresh copy of the new version. So the already-poor 20.9% is the corrected figure — snapshot-based studies would have reported something far worse, and been right by accident for the wrong reason.

Copies gain command execution without anyone editing them

The capability finding is the one that reframes the threat model. Modified copies gain command execution more often than they lose it — and that gain does not come from the edits of their owners, which add and remove capabilities about equally often. It comes from adopting another version of the skill.

So a skill’s privilege level drifts upward through the copy network without any individual developer deciding to grant it more power. The repository that re-copies a skill to get an improvement also inherits whatever capabilities that version accumulated. Nobody reviewed the change, because from the adopter’s perspective there was no change — they copied a skill, the same as last time.

Two further findings sharpen the review burden:

A third of copies between agent folders now cross platforms. Skills copied into the folders of other agents “often name tools that only Claude Code provides” — all 56 manually checked cases were correct. A skill written against one agent’s tool surface is being executed by another, with no adaptation.
Flagged skills spread less than chance would predict — reaching 28.6% of repositories against 41.1% under a uniform permutation null. High-risk skills are not the viral ones. The ecosystem is not primarily propagating malware; it is propagating ordinary skills that quietly acquire capabilities.

GitHub stars are the wrong signal, and the paper proves it

Stars are “nearly uninformative” about out-degree in the dated network, and the paper notes they are manipulable, citing the ICSE 2026 study of roughly six million suspected fake stars. Dating also corrects attribution: only 7 of the 20 repositories with the highest static out-degree remain in the top 20 once credited solely for skills they held first. The rest are late adopters that a snapshot mistakes for sources. Most unstarred top sources are ordinary projects, not curated catalogs.

The replacement signal is simple. Ranked before 1 April 2026, current copy out-degree identifies 4.0 times as many later transmissions as stars, with PageRank and skill count performing comparably.

The intervention result, with its ceiling stated

Seddik fits a conditional logit model of which earlier adopter a copy event chooses as its source, then evaluates it on a temporal split — ranking repositories using only the training period and measuring prevented high-risk adoptions in the held-out period. Auditing a repository removes its flagged skills, preventing its later high-risk adoptions and every copy descending from them.

Top 100 by the source-choice model: 14.9% of later high-risk adoptions prevented.
Top 100 by stars: 0.5%. A roughly thirtyfold difference.
The ordering holds at splits of 1 March and 1 May, under the lenient flag, and is reproduced independently in a calibrated simulation. A quarter of prevented adoptions are indirect copies.
A ranking chosen in hindsight prevents only about a quarter of them, because roughly half arrive as new skills. This is the honest ceiling: no repository-ranking strategy, however perfect, addresses more than a fraction.

One finding is unambiguously good news for defenders: reviewers have weeks rather than hours, because a new skill reaches few of its eventual adopters in its first week. Unlike a compromised npm package that propagates in hours — the tempo we documented in the GhostAction October burst — skill diffusion is slow enough for human review to matter, if anyone is reviewing.

Where the method is soft, per the author

The validation section is unusually forthcoming, and the central caveat deserves to be stated rather than buried. Against 1,534 folders whose skill installer recorded the copy source and date:

The reconstructed date lies within seven days of the record in 94.6% of cases — dating is solid.
The recorded source held the skill earlier in 99.1% of cases — direction is solid.
But the rule names that same source in only 26.7% of cases, because it credits the earlier adopter of the most lineages, which is usually a catalog.

The author draws the correct conclusion in one sentence: “Source-level measures therefore describe distribution rather than authorship.” The paper measures who a skill flows through, not who wrote it. For audit targeting that is the right quantity — you want the distribution chokepoints — but it is not an attribution claim, and the paper does not make one.

Two further limits. The risk flags “mark capability rather than malice,” with precision 98.3% and recall 80.9% against 200 labelled skills, which makes every risk count a lower bound. And the labelling of repository types and skill risks was performed by Claude Opus 5.5 and Claude Sonnet 5.5 through claude -p against a written codebook, with the two models agreeing at Cohen’s κ of 0.77 on repository types and 0.66–1.00 on skill questions; all 18 repository and 28 skill disagreements were resolved manually, and Opus reached κ = 0.91 against that reference before labelling the remaining 1,507 source repositories. The author flags the inflation himself — those values favour Opus, because the reference retains every answer the two models shared. An earlier manual labelling pass using only repository name, description and skill names agreed with Opus at κ = 0.41, which is what prompted adding README and file-tree context. Model-assisted labelling at this scale is defensible and disclosed; it is still model-assisted labelling, and the same caution we applied to AdvSim2Real’s judge-model robustness figures applies here.

What to do with this

Stop treating a skill fix as deployed. If you maintain a skill others copy, publishing a fix reaches roughly a fifth of copies. Notify downstream adopters directly, or accept that the vulnerable version is still running.
Inventory your copied skills and re-check them. A skill you copied months ago has not followed its source. Whatever was fixed upstream is still broken in your tree, and the paper says that is the normal case, not the exception.
Rank by copy out-degree, not stars. If you maintain an allowlist or review queue, the 4.0x and the 14.9%-vs-0.5% results both say the same thing: popularity is the wrong ordering. Network position is the right one.
Diff on re-copy, every time. Capability gain arrives through adopted versions, not local edits. Re-copying a skill to pick up an improvement is the moment command execution enters your tree. Treat it as a code change requiring review, because that is what it is.
Check that a skill names only the tools of the agent that loads it. A third of cross-folder copies cross platforms, and skills referencing Claude Code tools are running under other agents today.
Use the weeks you have. Slow diffusion is the one structural advantage defenders hold here. It is worth nothing if nobody reviews in that window.

The author’s own recommendation is the structural fix, and it is correct: platform vendors should distribute skills as versioned references rather than copies. Every finding above is downstream of the decision to ship instructions by duplication. Scanners like NVIDIA’s SkillSpector can tell you whether a skill is dangerous today; none of them can tell you whether the copy in your repository is the version that was fixed. That requires provenance the format does not currently have.

Verification note: we read the paper from the arXiv listing and the full HTML text of arXiv:2610.11169v1 (submitted 8 October 2026, cs.SE with cs.CR and cs.SI cross-lists, CC BY 4.0, single author Fahd Seddik, University of British Columbia Okanagan), and confirmed title, author, submission date, categories and abstract against the arXiv API metadata record. All figures quoted here — 2,193,119 adoptions, 20.9%, 11.1%, 4.0%, 3.32-fold, 3.3-fold, 28.6% vs 41.1%, 14.9% vs 0.5%, 4.0x, 94.6%, 99.1%, 26.7%, 98.3%/80.9%, the κ values, the 972-adopter and 12-generation cascade, the 56 manually checked cross-platform cases, and the 47–84% comparison figure the author cites from other work — are taken verbatim from that paper, as are the quoted sentences. This is a preprint: we found no peer-review record, no venue acceptance and no third-party replication at the time of writing, and we imply none. We did not run the released code, rebuild the copy network, or reproduce any measurement, and we did not independently verify the GitSkills dataset the reconstruction is built on.

Sources: