Abstract: This report presents an overview of the state-of-the-art methods and models for planning for teams of embodied agents. Due to the nature of the real world, this means we focus on multi-agentplanning in stochastic, partially observable systems. In particular we focus on decentralized partially observable Markov decision processes (Dec-POMDPs), partially observable stochastic games (POSGs) and related models. Regarding such models, we review complexity results and recently proposed methods for finding (approximate) solutions.
Abstract: The Dec-POMDP is a model for multi-agentplanning under uncertainty that has received increasingly more attention over the recent years. In this work we propose a new heuristic QBG that can be used in various algorithms for Dec-POMDPs and describe differences and similarities with QMDP and QPOMDP. An experimental evaluation shows that, at the price of some computation, QBG gives a consistently tighter upper bound to the maximum value obtainable.
Abstract: Passenger railway operations are based on an extensive planning process for generating the timetable, the rolling stock circulation, and the crew duties for train drivers and conductors. In particular, crew scheduling is a complex process.
After the planning process has been completed, the plans are carried out in the real-time operations. Preferably, the plans are carried out as scheduled. However, in case of delays of trains or large disruptions of the railway system, the timetable, the rolling stock circulation and the crew duties may not be feasible anymore and must be rescheduled.
This paper presents a method based on multi-agent techniques to solve the train driver rescheduling problem in case of a large disruption. It assumes that the timetable and the rolling stock have been rescheduled already based on an incident scenario. In the crew rescheduling model, each train driver is represented by a driver-agent. A driver-agent whose duty has become infeasible by the disruption starts a recursive task exchange process with the other driver-agents in order to solve this infeasibility. The task exchange process is supported by a route-analyzer-agent, which determines whether a proposed task exchange is feasible, conditionally feasible, or not feasible. The task exchange process is guided by several cost parameters, and the aim is to find a feasible set of duties at minimal total cost.
The train driver rescheduling method was tested on several realistic disruption instances of Netherlands Railways (NS), the main operator of passenger trains in the Netherlands. In general the rescheduling method finds an appropriate set of rescheduled duties in a short amount of time. This research was carried out in close cooperation by NS and the D-CIS Lab.