Abstract: One of the first requirements for building multi-agent systems with complex and dynamic structures is to have agents that are able to operate in such organizations. Being able to adopt different organizational roles is one of the key requirements for an agent in order to have this ability. Another requirement for the agent we intend to build is the ability to operate in a dynamic environment. This means the agent has to be able to construct a plan for the task it is about to perform and while performing the task, the agent has to be able to evaluate whether the plan is still valid. When changes in the environment have caused the plan to become invalid, the agent needs to be able to generate a new and valid plan for the task. The agent architecture that is described in this document is a step towards an agent that meets these requirements of operating in a dynamic organization and dynamic environment.
Abstract: Our work addresses the problem of autonomous concept formation from a design point of view, providing an initial answer to the question: What are the design features of an architecture supporting the acquisition of different types of concepts by an autonomous agent?
Autonomous agents, that is systems capable of interacting independently with their environment in the pursuit of their own goals, will provide the framework in which we study the problem of autonomous concept formation. Humans and most animals may in this sense also be regarded as autonomous agents, but our concern will be with artiﬁcial autonomous agents. A detailed survey and discussion of the many issues surrounding the notion of ‘artiﬁcial agency’ is beyond the scope of this thesis and a good overview can be found in [Wooldridge and Jennings, 1995]. Instead we will focus on how artiﬁcial agents could be endowed with representational and modelling capabilities.
The ability to form concepts is an important and recognised cognitive ability, thought to play an essential role in related abilities such as categorisation, language understanding, object identiﬁcation and recognition, reasoning, all of which can be seen as different aspects of intelligence. Concepts and categories are studied within cognitive science, where scientists are concerned with human conceptual abilities and mental representations of categories, but they have been addressed also in the rather different domain of machine learning and classiﬁcatory data analysis, where the focus is on the development of algorithms for clustering problems and induction problems [Mechelen et al., 1993]. The two ﬁelds are well distinct and only recently have started to interact, but even though the importance of concepts have been recognised, the nature of concepts is controversial, in the sense that there is no commonly agreed theory of concepts, and it is still far from obvious which representational means are most suited to capture the many cognitive functions that concepts are involved in.
Among the goals of this thesis there is the attempt to bring together different lines of argumentation that have emerged within philosophy, cognitive science and AI, in order to establish a solid foundation for further research into the representation and acquisition of concepts by autonomous agents. Thus, our results and conclusions will often be stated in terms of new insights and ideas, rather than resulting in new algorithms or formal methods.
Our focus will be on affordance concepts — discussed in detail in Chapter 4 — and our main contributions will be:
* An argument showing that concepts should be thought of as belonging to different kinds, where the differences among these kinds are to be captured in terms of architecture features supporting their acquisition.
* A description (and partial implementation) of a minimal architecture (the Innate Adaptive Behaviour architecture – IAB architecture for short) supporting the acquisition of affordance concepts; the IAB architecture is actually a proposal for a sustaining mechanism, in the sense of [Margolis, 1999], for affordances, and makes clear the necessity of a minimal structure for the representation of affordances.
When addressing concept formation in AI, what can be called the ‘system level’ is often overlooked, which means that concepts and categories are rarely studied from the point of view of a system, autonomous and complete, that might need such constructs and can acquire them only by means of interactions with its environment, under the constraints of its cognitive architecture. Also within psychology, the focus is usually on structural aspects of concepts rather than on developmental issues [Smith and Medin, 1981]. Our approach – an architecture-based approach – is an attempt (i) to show that a system level perspective on concept formation is indeed possible and worth exploring, and (ii) to provide an initial, maybe simple, but concrete example of the insights that can be gained from such an approach. Since the methodology that we propose to study concept formation is a general one, and can be applied also to other types of concepts, we decided to mention broadly ‘autonomous concept formation’ rather than ‘autonomous affordance-concepts formation’ in the title of the thesis.
Abstract: Reinforcement learning (RL) is a widely used learning paradigm for adaptive agents. There exist several convergent and consistent
RL algorithms which have been intensively studied. In their original form,
these algorithms require that the environment states and agent actions
take values in a relatively small discrete set. Fuzzy representations for
approximate, model-free RL have been proposed in the literature for the
more difficult case where the state-action space is continuous. In this
work, we propose a fuzzy approximation architecture similar to those
previously used for Q-learning, but we combine it with the model-based
Q-value iteration algorithm. We prove that the resulting algorithm converges. We also give a modified, asynchronous variant of the algorithm
that converges at least as fast as the original version. An illustrative
simulation example is provided.
Abstract: This document was written for two purposes. First to elaborate on the design of this research project which is dealt with in the first chapter. The second purpose was to gain some insight in the literature on agent technology which is dealt with in the second chapter.
The first chapter gives an overview of the project. It starts with a statement on the overall project goals and a discussion of some of the issues that arise in crisis situations which we will have to take into account during the project. We then continue with a problem statement which is a more specific description of our research and this results in a set of research questions. Finally an approach is described of how we intend to find answers to these research questions.
The second chapter of this deliverable consists of a literature review on agent technology. This literature review intends to provide some background in agent technology literature and it discusses the following topics: What are intelligent agents? What architectures exist to design agents and how can we compare them? How can knowledge be represented in an agent and how can the agent use this knowledge for reasoning? What are the different aspects in communication between agents.
Abstract: We introduce Distributed perception networks (DPNs), a distributed architecture for efficient and reliable fusion of large quantities of heterogeneous and noisy information. DPNs consist of agents, processing nodes with limited fusion capabilities, which cooperate and can autonomously form arbitrarily large distributed classifiers. DPNs are based on causal models, which often facilitate analysis, design and maintenance of complex information fusion systems. This is possible because
observations obtained from different information sources often result from causal processes which in turn can be modeled with relatively simple, yet mathematically rigorous and compact probabilistic causal models. Such models, in turn, facilitate
decentralized world modeling and information fusion.
Abstract: An architecture for dynamic
reconfiguration of complex services, in which the enactment is
automated, and the matching of services is not limited to a
pre-determined set of matchers and repositories, is presented. The
proposed architecture consists of three, previously developed,
components: the CoWS template-based reconfiguration service, the
Knoogle MatchMaker service, and the Triana workflow enactment
engine. This architecture has the following innovative aspects: 1)
automated adaptation of complex services, which is more flexible
than existing approaches based on replacing failing instances of
services within a workflow, 2) use of heterogeneous components that
may be both local and distributed, and 3) dynamic selection of
matchers and repositories.
Abstract: Advances in network technologies enable distributed systems, operating in complex physical environments, to co-ordinate their activities over larger areas within shorter time intervals. Some envisioned application domains for such systems are defence, crisis management, traffic management and public safety. In these systems humans and machines will, in close interaction, be adaptive to a changing environment. Various architecture models are proposed for such Networked Adaptive Interactive Hybrid Systems (NAIHS) from different research areas like (networked) sensor fusion, command and control, artificial intelligence, robotics and human machine interaction. In this paper an architecture model is proposed that seeks to combine their merits. The NAIHS model focuses on the ‘hybrid mind’ that is layered in several dimensions
defining specific functional components and their
interactions. Subsequently, the interaction between the human and artificial part of the system is discussed.
Abstract: The main goal of this thesis is to provide a first overview of the current architectures the most able to design a cognitive agent. The notion of cognitive agent is in line with the Actor-Agent Community (AAC) project of D-CIS Lab (second part of this thesis). This project aims to design a prototype of an artificial system with cognitive capabilities (the cognitive agent) capable to interact with humans within a team (the Actor-Agent Community).
Abstract: The proliferation of small mobile devices and wireless networks has resulted in an increasing demand to support the applications found in wired environments on mobile devices. In real time replication systems, such as collaborative systems, this trend gives some new problems to address. The properties of wireless networks are low bandwidth and high latency, which change dynamically over time. The risk of the network getting congested is therefore high with the result that the user will not receive the important information in time. Consequently there is a need to develop algorithms and methods for adaptive work environments and adaptive data distribution, to minimise the traffic load. An architecture based on multi- and mobile-agents is proposed as a solution. Personalized behaviour is included in a flexible and extensible system. A prototype of the architecture has been implemented in a crisis environment and was used for an evaluation. It is assumed that each individual in the field is equipped with a PDA that can communicate with other PDA's in the surrounding and remote servers. Users can report about their environment using a personalized iconic language. Each user is supervised by a personal agent. In case of emergency users are routed outside a dangerous area using a personalised dynamic routing system, called PIRA "Personal Intelligent Routing Assistant". The system and results of testing will be presented in this paper.
Abstract: Management in crisis response requires continuous adaptation, for crisis situations are highly dynamic. Crisis response in general involves multiple parties, each with their own autonomy and capabilities, leading to differentiations in structure, goals and strategies, and constraints for cooperation. A crisis management system needs to support distributed and continuous adaptation on different levels of organisation, in a reliable fashion, ensuring at least some minimal level of service for every defined task. This paper presents an architecture of a generic reflective autonomic management system (GRAM). The GRAM system tackles this real-time configuration challenge by the combination of a template-based configuration system (COWS) and a workflow-based configuration system (SMDS). The first proof-of-concept GRAM system shows a reliable and predictive performance in changing environments. Future work includes extending the current realisation and validating its performance in more realistic settings.
Abstract: (Extended abstract)
Management in crisis response requires continuous adaptation, for crisis situations are highly dynamic. Crisis response in general involves multiple parties, each with their own autonomy and capabilities, leading to differentiations in structure, goals and strategies, and constraints for cooperation. A crisis management system needs to support distributed and continuous adaptation on different levels of organisation, in a reliable fashion, ensuring at least some minimal level of service for every defined task. This paper presents an architecture of a generic reflective autonomic management system (GRAM). The GRAM system tackles this real-time configuration challenge by the combination of a template-based configuration system (COWS) and a workflow-based configuration system (SMDS). The first proof-of-concept GRAM system shows a reliable and predictive performance in changing environments. Future work includes extending the current realisation and validating its performance in more realistic settings.
Abstract: This paper proposes a reinforcement learning architecture containing multiple "experts", each of which is a specialist in a different region in the overall state space. The central idea is that the different experts use qualitatively different (but sufficiently Markov) state representations, each of
which captures different information regarding the true underlying world state, and which for that reason is suitable for a different part of the state space. The experts themselves learn to switch to another state representation (other expert) by having switching actions. Value functions can be learned using standard reinforcement learning algorithms. Experiments in a small, proof-of-principle experiment as well as a larger, more realistic experiment illustrate the validity of this approach.
Abstract: This report is the first of four documents, which describe the distributed tasking architecture that
has been developed by Thales Research and Technology (UK) Ltd. This architecture has been
developed to test generic tasking and re-tasking policies as part of the Tasking and Re-tasking
sub-project for the Interactive Collaborative Information Systems (ICIS) programme. This report offers: 1) An overview of the entire architecture, focusing on those components that are common
to both distributed and centralised tasking processes
2) Information on the Java services required to run the tasking architecture.
Abstract: This report is the second of four documents, which describe the distributed tasking
architecture that has been developed by Thales Research and Technology (UK) Ltd.
This architecture has been developed to test generic tasking and re-tasking policies as
part of the Tasking and Re-tasking sub-project for the Interactive Collaborative
Information Systems (ICIS) programme. This second document focuses on how objects and interfaces have been implemented
to enable centralised tasking algorithms processes to be utilised in the tasking
architecture. Centralised tasking is a traditional, top down approach to tasking whereby
a centralised actor or agent has access to all information about tasks and resources and
is responsible for making all task allocations.
Abstract: This paper introduces the MultiAgent Decision Process software toolbox, an open source C++ library for decision-theoretic planning under uncertainty in multiagent systems. It provides support for several multiagent models, such as POSGs, Dec-POMDPs and MMDPs. The toolbox aims to reduce development time for planning algorithms and to provide a benchmarking platform by providing a number of commonly used problem descriptions. It features a parser for a text-based ﬁle format for discrete Dec-POMDPs, shared functionality for planning algorithms, as well as the implementation of several Dec-POMDP planners. We describe design goals and architecture of the toolbox, and provide an overview of its functionality, illustrated by some usage examples. Finally we report on current and future work.
Abstract: Self-monitoring of autonomic distributed systems requires knowledge of the
states and events of many different parts of a system. One of the main
challenges is to determine which information is most crucial for analysis of
a system's behaviour, and when. This paper proposes a model-based approach
to self-monitoring for which structural and behavioural models of a system
are described at different levels: application, subsystem, component and
class level. In this approach, a system's behaviour is monitored in the
context of a hierarchy of use-cases related to these levels. The structural
and behavioural models are used to automatically instrument an existing
distributed system. The proposed architecture of a self-monitoring engine is
described as is the implementation. The models have been specified in the
Ontology Web Language (OWL) and the self-monitoring (as a part of our
self-management framework) has been implemented in Java. The scenario used
to illustrate the approach is that of authentication for a simplified
version of a distributed portal application.