RetailBench

Benchmark for long-horizon autonomous decision-making

RetailBench

RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments

A data-grounded simulator for evaluating whether tool-using LLM agents can operate a supermarket across pricing, replenishment, supplier choice, inventory aging, reviews, external events, and cash-flow constraints.

arXiv:2603.16453

Environment
hard_v2
Horizon
180 days
Evaluation
20 runs, 7 LLMs
RetailBench interaction framework connecting store state, agent tools, actions, and feedback.
RetailBench couples SKU assortment, suppliers, inventory, customer feedback, news, finance, and agent tools in a closed-loop retail operation setting.
7 LLM models
20 rollout runs
180 evaluation days
2/7 LLMs reach full horizon

Operational policy formation under delayed feedback.

RetailBench is not a one-shot web task or a short tool-use test. The agent repeatedly chooses what to inspect, what to buy, how much to stock, when to change prices, and how to react when outcomes arrive days later.

The current public result package uses a survival-first selection rule: maximize operating days, then final net worth, then total sales. A privileged quality-based heuristic is shown as an oracle-style reference, not as a fair LLM-agent baseline.

$24.4k

Best selected LLM net worth

GPT-5.5 is the top selected LLM by final net worth.

$131.5k

Heuristic reference net worth

Even full-horizon LLM runs remain far below the privileged reference policy.

164k

Top selected LLM sales

DeepSeek-V4-Pro leads selected LLMs by total unit sales.

Net worth trajectory

Solid lines show observed rollout days; dashed tails carry terminal net worth to the 180-day horizon for runs that stopped early.

Survival-first selected runs

Model Framework Days Final net worth Total sales Sold SKUs/day Return ratio