AI agents (systems that can reason, plan, and act) are becoming a popular paradigm for real-world AI applications. From coding assistants to personal health coaches, the industry is moving from one-shot question-answers to sustained, multi-step interactions. While researchers have long relied on established metrics to optimize the accuracy of traditional machine learning models, agents introduce a new layer of complexity. Unlike individual predictions, agents must navigate continuous, multi-step interactions where a single error can cascade throughout the workflow. This change is forcing us to look beyond standard precision and think about how these systems can actually be designed for optimal performance.
Experts often rely on heuristics such as the “more agents, the better” assumption, believing that adding specialized agents will consistently improve results. For example, “More Agents Is All You Need” reports that LLM performance scales with the number of agents, while a co-scaling study found that multi-agent collaboration “…often exceeds each individual through collective inference.”
In our new paper, “Towards a science of scaling agent systems,” we challenge this assumption. Through a large-scale, controlled evaluation of 180 agent configurations, we derive the first quantitative scaling principles for agent systems and reveal that “more agents” approaches often reach a plateau and can even degrade performance if not matched to the specific characteristics of the task.
