Abstract: Multi-agent systems are rapidly finding applications
in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must instead discover a solution on their own, using learning. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multi-agent reinforcement learning (MARL). A central issue in the field is the formal statement of the multi-agent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents' learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim - either explicitly or implicitly - at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where MARL techniques have been applied. Finally, an outlook for the field is provided.
Abstract: Noticeable developments have lately been achieved on
designing automated multimodal smart processes to
increase security in every-day life of people. As these
developments continue, proper infrastructures and
methodologies for the aggregation of various demands that
will inevitably arise, such as the huge amount of data and
computation, become more important. In this research, we
introduce a multimodal framework with support for an
automatic surveillance application. The novelty of the
attempt resides in the modalities to underpin data
manipulation as a natural process but still keeping the
overall performance at high levels. At the application
level, the typical complexity behind the emerging
distributed multimodal systems is reduced in a transparent
manner through multimodal frameworks that handle data
on different abstraction levels and efficiently
accommodate constituent technologies. The proposed
specifications includes the use of shared memory spaces
(XML data Spaces) and smart document-centered content-
based data querying mechanisms (XQuery formal language
). We also report on the use of this framework in an
application on aggression detection in train compartments.
Abstract: We propose a formal framework based on finite state process algebra for modeling service coordination in agent systems. The model is inspired by WS-coordination specification. Several participant agents can coordinate their actions towards reaching a semantically consistent outcome in a distributed activity. The model defines coordinator and participant agents and their generic interaction protocols and allows integration of application specific protocols by means of protocol handlers. The approach is demonstrated by a sample model showing how coordination is achieved between a set of agents involved in contracting activities.
Abstract: This paper introduces an information theoretic approach to verification of causal models in modular Bayesian fusion systems. We assume distributed fusion systems which are gradually extended by adding new modules, each having a limited domain knowledge captured in local Bayesian networks. However, since dierent modules originate from different, independent design processes important dependencies between the variables in different modules might not correctly be captured in the distributed fusion system. This could have a significant impact on the fusion quality. The introduced
method supports discovery of significant dependencies which are ignored in the distributed fusion system.
Abstract: This paper appeared at the ISCRAM 2010  and presents an intelligent system facilitating better in-
formed decision making under severe uncertainty, as often found in emergency management. The
construction of decision-relevant scenarios, being plausible descriptions of a situation and its future
development, is used as a rationale for collecting, organizing, filtering and processing information for
decision making. The development of scenarios is geared to assessing decision alternatives, thus avoiding
time-consuming analysis and processing of irrelevant information.
The scenarios are constructed in a distributed setting allowing for a flexible adaptation of reasoning (prin-
ciples and processes) to the problem at hand and the information available. Each decision can be founded
on a coherent set of scenarios, which was constructed using the best expertise available within a limited
timeframe. Our theoretical framework is demonstrated in a distributed decision support system by
orchestrating both automated systems and human experts into workflows tailored to each specific problem.
Abstract: Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, etc. Learning approaches to multi-agent control, many of them based on reinforcement learning (RL), are investigated in complex domains such as teams of mobile robots. However, the application of decentralized RL to low-level control tasks is not as intensively studied. In this paper, we investigate centralized and decentralized RL, emphasizing the challenges and potential advantages of the latter. These are then illustrated on an example: learning to control a two-link rigid manipulator. Some open issues and future research directions in decentralized RL are outlined.
Abstract: This paper describes the sample implementation of a distributed goaloriented reasoning engine for multi-agent systems. The paper summarizes part of the design and programming issues that we addressed for providing the initial prototype with customization and self-configuration facilities.
Abstract: We introduce Distributed perception networks (DPNs), a distributed architecture for efficient and reliable fusion of large quantities of heterogeneous and noisy information. DPNs consist of agents, processing nodes with limited fusion capabilities, which cooperate and can autonomously form arbitrarily large distributed classifiers. DPNs are based on causal models, which often facilitate analysis, design and maintenance of complex information fusion systems. This is possible because
observations obtained from different information sources often result from causal processes which in turn can be modeled with relatively simple, yet mathematically rigorous and compact probabilistic causal models. Such models, in turn, facilitate
decentralized world modeling and information fusion.
Abstract: When we talk about improving Information Systems for Crisis Response and Management, we must consider at least two sorts of problems to solve: the processing of human provided information (perception and knowledge) in combination with other sensor information and the human processing of sensor based information (for situation awareness and decision making). As an illustration, two examples of research work that is done at the D-CIS lab in Delft is presented here. The first example is about Distributed Perception Networks in disaster management and the second example is about cognitive performance factors in decision making (in crisis situations).
Abstract: The increasing complexity of our world demands new perspectives on the role of technology in human decision making. We need new technology to cope with the increasingly complex and information-rich nature of our modern society. This is particularly true for critical environments such as crisis management and traffic management, where humans need to engage in close collaborations with artificial systems to observe and understand the situation and respond in a sensible way. The book Interactive Collaborative Information Systems addresses techniques that support humans in situations in which complex information handling is required and that facilitate distributed decision-making. The theme integrates research from information technology, artificial intelligence and human sciences to obtain a multidisciplinary foundation from which innovative actor-agent systems for critical environments can emerge. It emphasizes the importance of building actor-agent communities: close collaborations between human and artificial actors that highlight their complementary capabilities in situations where task distribution is flexible and adaptive. This book focuses on the employment of innovative agent technology, advanced machine learning techniques, and cognition-based interface technology for the use in collaborative decision support systems.
Abstract: Advances in network technologies enable distributedsystems, operating in complex physical environments, to co-ordinate their activities over larger areas within shorter time intervals. Some envisioned application domains for such systems are defence, crisis management, traffic management and public safety. In these systems humans and machines will, in close interaction, be adaptive to a changing environment. Various architecture models are proposed for such Networked Adaptive Interactive Hybrid Systems (NAIHS) from different research areas like (networked) sensor fusion, command and control, artificial intelligence, robotics and human machine interaction. In this paper an architecture model is proposed that seeks to combine their merits. The NAIHS model focuses on the ‘hybrid mind’ that is layered in several dimensions
defining specific functional components and their
interactions. Subsequently, the interaction between the human and artificial part of the system is discussed.
Abstract: Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, economics. Many tasks arising in these domains require that the agents learn behaviors online. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. However, due to different viewpoints on central issues, such as the formal statement of the learning goal, a large number of different methods and approaches have been introduced. In this paper we aim to present an integrated survey of the field. First, the issue of the multi-agent learning goal is discussed, after which a representative selection of algorithms is reviewed. Finally, open issues are identified and future research directions are outlined.
Abstract: Self-healing relies on correct diagnosis of system malfunctioning. \linebreak This paper presents a use-case based approach to self-diagnosis. Both a static and a dynamic model of a managed-system are distinguished with explicit functional, implementational, and operational knowledge of specific use-cases. This knowledge is used to define sensors to detect and localise anomalies at the same three levels, providing the input needed to perform informed diagnosis. The models presented can be used to automatically instrument existing distributed legacy systems.
Abstract: Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, etc. Although the individual agents can be programmed in advance, many tasks require that they learn behaviors online. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. This paper gives a survey of multi-agent reinforcement learning, starting with a review of the different viewpoints on the learning goal, which is a central issue in the field. Two generic goals are distinguished: stability of the learning dynamics, and adaptation to the other agents' dynamic behavior . The focus on one of these goals, or a combination of both, leads to a categorization of the methods and approaches in the field. The challenges and benefits of multi-agent reinforcement learning are outlined along with open issues and future research directions.
Abstract: This report is the first of four documents, which describe the distributed tasking architecture that
has been developed by Thales Research and Technology (UK) Ltd. This architecture has been
developed to test generic tasking and re-tasking policies as part of the Tasking and Re-tasking
sub-project for the Interactive Collaborative Information Systems (ICIS) programme. This report offers: 1) An overview of the entire architecture, focusing on those components that are common
to both distributed and centralised tasking processes
2) Information on the Java services required to run the tasking architecture.
Abstract: This report is the second of four documents, which describe the distributed tasking
architecture that has been developed by Thales Research and Technology (UK) Ltd.
This architecture has been developed to test generic tasking and re-tasking policies as
part of the Tasking and Re-tasking sub-project for the Interactive Collaborative
Information Systems (ICIS) programme. This second document focuses on how objects and interfaces have been implemented
to enable centralised tasking algorithms processes to be utilised in the tasking
architecture. Centralised tasking is a traditional, top down approach to tasking whereby
a centralised actor or agent has access to all information about tasks and resources and
is responsible for making all task allocations.
Abstract: Research on multi-agent systems frequently involves experiments with agents, including situations where humans engage in interactions with agents. Consequently, the ﬁeld of experimental (human) sciences becomes more and more relevant. This paper clariﬁes how things can and often do go wrong in distributed AI experiments. We show the ﬂaws in methodological design in existing literature (both with and without humans) and work out an example involving human test-subjects to introduce the
fundamental issues of experimental design. Furthermore, we provide researchers with an approach to improve their experimental design. We wish to stimulate researchers to conduct better experiments – which will beneﬁt us all.
Abstract: Self-monitoring of autonomic distributedsystems requires knowledge of the
states and events of many different parts of a system. One of the main
challenges is to determine which information is most crucial for analysis of
a system's behaviour, and when. This paper proposes a model-based approach
to self-monitoring for which structural and behavioural models of a system
are described at different levels: application, subsystem, component and
class level. In this approach, a system's behaviour is monitored in the
context of a hierarchy of use-cases related to these levels. The structural
and behavioural models are used to automatically instrument an existing
distributed system. The proposed architecture of a self-monitoring engine is
described as is the implementation. The models have been specified in the
Ontology Web Language (OWL) and the self-monitoring (as a part of our
self-management framework) has been implemented in Java. The scenario used
to illustrate the approach is that of authentication for a simplified
version of a distributed portal application.
Abstract: Automated support for management of complex distributed object-oriented systems is a challenge: self-management the goal. A self-management system needs to reason about the behaviour of the distributed entities in a system, and act when necessary. The knowledge needed is multi-leveled: different levels of concepts and rules need to be represented. This paper explores the requirements that hold for representing this knowledge in self-managed distributed object-oriented systems, and explores the potential of Semantic Web technology in this context. A model for self-management knowledge and a simplified version of a real-life use case are used to illustrate the potential.