Paper: PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems
Listen to this article.
Problem
Large language model (LLM) agents are being deployed to tackle increasingly complex, real-world tasks. These tasks often involve interacting with numerous tools – think of navigating a retail environment and needing to use various APIs or functions to find products, manage orders, track shipments, etc. Existing benchmarks haven’t adequately tested these agents’ ability to effectively plan across long sequences of tool usage, especially when dealing with limited visibility into which tools are available and reliable at any given moment.




