Abstract: This report presents an overview of the state-of-the-art methods and models for planning for teams of embodied agents. Due to the nature of the real world, this means we focus on multi-agent planning in stochastic, partially observable systems. In particular we focus on decentralized partially observable Markov decision processes (Dec-POMDPs), partially observable stochastic games (POSGs) and related models. Regarding such models, we review complexity results and recently proposed methods for finding (approximate) solutions.
Abstract: This document was written for two purposes. First to elaborate on the design of this research project which is dealt with in the first chapter. The second purpose was to gain some insight in the literature on agent technology which is dealt with in the second chapter.
The first chapter gives an overview of the project. It starts with a statement on the overall project goals and a discussion of some of the issues that arise in crisis situations which we will have to take into account during the project. We then continue with a problem statement which is a more specific description of our research and this results in a set of research questions. Finally an approach is described of how we intend to find answers to these research questions.
The second chapter of this deliverable consists of a literature review on agent technology. This literature review intends to provide some background in agent technology literature and it discusses the following topics: What are intelligent agents? What architectures exist to design agents and how can we compare them? How can knowledge be represented in an agent and how can the agent use this knowledge for reasoning? What are the different aspects in communication between agents.
Abstract: Embodied cognition, or the notion that cognitive processes develop from goal-directed interactions between organisms and their environment has stressed the automaticity of perceptual and motor resonance mechanisms in other cognitive domains like language. The present paper starts with reviewing abundant empirical evidence for automatic resonance mechanisms between action and language and examples of other cognitive domains such as number processing. Special attention is given here to social implications of embodied cognition. Then some more recent evidence indicating the importance of the action context on the interaction between action and language is reviewed. Finally, a theoretical notion about how automatic and selective mechanisms can be incorporated in an embodied cognitive perspective is provided.
Abstract: This deliverable concerns an internal report. A literature review and work plan for cognitive control and decision making is described. This literature overview with additional research ideas and the description of several experiments is outlined as follows: an introduction is given on how cognitive control is being defined in literature. Then, brain areas related to cognitive control are being discussed, since research on cognitive control focuses more and more on the neurological substrates of this phenomenon. Additionally, several tasks, as often used in research on cognitive control, are discussed, as well as some alternative tasks. Furthermore, decision making processes and decision biases as well as some tasks to investigate the decision making process are being described. In a separate section, the influences of several human factors on cognitive control as well as personality differences in cognitive control are depicted. Consecutively, the role of cognitive control in moral dilemmas are discussed. Then, some physiological measures for cognitive control are pointed out and some general research ideas are proposed for investigating the role of cognitive control in decision making. Finally, the set-up of several experiments are being described.
Abstract: Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, economics. Many tasks arising in these domains require that the agents learn behaviors online. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. However, due to different viewpoints on central issues, such as the formal statement of the learning goal, a large number of different methods and approaches have been introduced. In this paper we aim to present an integrated survey of the field. First, the issue of the multi-agent learning goal is discussed, after which a representative selection of algorithms is reviewed. Finally, open issues are identified and future research directions are outlined.
Abstract: Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, etc. Although the individual agents can be programmed in advance, many tasks require that they learn behaviors online. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. This paper gives a survey of multi-agent reinforcement learning, starting with a review of the different viewpoints on the learning goal, which is a central issue in the field. Two generic goals are distinguished: stability of the learning dynamics, and adaptation to the other agents' dynamic behavior . The focus on one of these goals, or a combination of both, leads to a categorization of the methods and approaches in the field. The challenges and benefits of multi-agent reinforcement learning are outlined along with open issues and future research directions.
Abstract: Gas distribution models can provide comprehensive information about a large
number of gas concentration measurements, highlighting, for example, areas of unusual
gas accumulation. They can also help to locate gas sources and to plan where
future measurements should be carried out. Current physical modeling methods,
however, are computationally expensive and not applicable for real world scenarios
with real-time and high resolution demands. This chapter reviews kernel methods
that statistically model gas distribution. Gas measurements are treated as random
variables, and the gas distribution is predicted at unseen locations either using a
kernel density estimation or a kernel regression approach. The resulting statistical apmodels
do not make strong assumptions about the functional form of the gas distribution,
such as the number or locations of gas sources, for example. The major
focus of this chapter is on two-dimensional models that provide estimates for the
means and predictive variances of the distribution. Furthermore, three extensions
to the presented kernel density estimation algorithm are described, which allow to
include wind information, to extend the model to three dimensions, and to reflect
time-dependent changes of the random process that generates the gas distribution
measurements. All methods are discussed based on experimental validation using
real sensor data.