LogisticsBenchmark: a tool for benchmarking AGV roadmaps

In this paper a benchmark tool is presented that enables both design and evaluation of roadmaps for autonomous guided vehicles (AGVs). For designing different solutions for a given intralogistic problem, a GUI was developed based on Tiled software. For simulation and evaluation of the solutions, Flatland, a well known multi-agent path finding (MAPF) simulator is employed. The environment is upgraded for lifelong planning and execution. The evaluation of the generated solutions is based on the system throughput (number of executed tasks per unit of time), the computational complexity of planning (time for obtaining MAPF solution), failure rate, and metrics from graph theory. The use of the benchmark tool is demonstrated through an illustrative example and a series of studies, where experts were involved in designing roadmaps for industrial cases. To provide a common ground for future research, the benchmark cases as well as the tool are open sourced.