From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills
A new study from arXiv (2605.23899) systematically examines the full lifecycle of model-generated agent skills: experience generation, skill extraction, and skill consumption. The researchers built a utility-grounded evaluation framework covering five diverse agentic task domains. They found that model-generated skills are beneficial on average but exhibit non-trivial negative transfer, meaning they can sometimes harm performance. Notably, neither extractors nor target agents behave uniformly; a model can be a strong extractor yet a weak consumer, or vice versa. The study highlights the need for careful evaluation of skill extraction methods and their downstream impact.
Developers must evaluate both skill extraction and consumption to avoid negative transfer.