Abstract: Multimodal applications stand for the missing chain to overcome the limitations of classical multimedia processing tools currently used. Therefore data fusion is seen as a very active research field and is also set to grow in importance in the coming years. At Delft University of Technology there is a project running on the development of a software workbench with native capabilities for signal and information processing and for fusion of data acquired from hardware equipments such as microphones and video cameras. A first prototype of the system has been already developed. At the moment, one additional project aims to develop an automatic surveillance system by using only the resources of the workbench.
Abstract: Multimodal applications stand for the missing chain to
overcome the limitations of classical multimedia processing tools currently used. Therefore data fusion is seen as a very active research field and is also set to grow in importance in the coming years. At Delft University of Technology there is a project running on the development of a software workbench with native capabilities for signal and information processing and for fusion of data acquired from hardware equipments such as microphones and video cameras. A first prototype of the system has been already developed. At the moment, one additional project aims to develop an automatic surveillance system by using only the resources of the workbench.
Abstract: This paper introduces an information theoretic approach to verification of causal models in modular Bayesian fusion systems. We assume distributed fusion systems which are gradually extended by adding new modules, each having a limited domain knowledge captured in local Bayesian networks. However, since dierent modules originate from different, independent design processes important dependencies between the variables in different modules might not correctly be captured in the distributed fusion system. This could have a significant impact on the fusion quality. The introduced
method supports discovery of significant dependencies which are ignored in the distributed fusion system.
Abstract: In this paper, two different methods for informationfusion are compared with respect to communication cost. These are the lambda-pi and the junction-tree approach as the probability computing methods in Bayesian networks. The analysis is done within the scope of large distributed networks of computing nodes. The result of this comparison enables us to make astatement about the most appropriate method for reasoning in distributed Bayesian networks. Each node in the network is considered an intelligent agent in a multi-agentsystem.
Abstract: The current audio-only speech recognition still lacks the expected robustness when the Signal to Noise Ratio (SNR) decreases. The video information is not affected by noise which makes it an ideal candidate for data fusion for speech recognition benefit. In the paper  the authors have shown that most of the techniques used for extraction of static visual features result in equivalent features or at least the most informative features exhibit this property. We argue that one of the main problems of existing methods is that the resulting features contain no information about the motion of the speaker's lips. Therefore, in this paper we will analyze the importance of motion detection for speech recognition. For this we will first present the Lip Geometry Estimation(LGE) method for static feature extraction. This method combines an appearance based approach with a statistical based approach for extracting the shape of the mouth. The method was introduced in  and explored in detail in . Further more, we introduce a second method based on a novel approach that captures the relevant motion information with respect to speech recognition by performing optical flow analysis on the contour of the speaker's mouth. For completion, a middle way approach is also analyzed. This third method considers recovering the motion information by computing the first derivatives of the static visual features. All methods were tested and compared on a continuous speech recognizer for Dutch. The evaluation of these methods is done under different noise conditions. We show that the audio-video recognition based on the true motion features, namely obtained by performing optical flow analysis, outperforms the other settings in low SNR conditions.
Abstract: We introduce Distributed perception networks (DPNs), a distributed architecture for efficient and reliable fusion of large quantities of heterogeneous and noisy information. DPNs consist of agents, processing nodes with limited fusion capabilities, which cooperate and can autonomously form arbitrarily large distributed classifiers. DPNs are based on causal models, which often facilitate analysis, design and maintenance of complex informationfusion systems. This is possible because
observations obtained from different information sources often result from causal processes which in turn can be modeled with relatively simple, yet mathematically rigorous and compact probabilistic causal models. Such models, in turn, facilitate
decentralized world modeling and informationfusion.
Abstract: French coastguard missions have become increasingly varied implying new challenges such as the reduction of the decision cycle and the expansion of available information. Thus, it involves new needs for enhanced decision support. An efficient situation awareness system has to quickly detect and identify suspicious boats. The efficiency of such a system relies on a reliable sensor fusion since a coastguard uses sensors to achieve his mission. We present an innovative approach based on multi-agent negotiation to fuse classifiers, benefiting from the efficiency of existing classification tools and from the flexibility and reliability of a multi-agent system to exploit distributed data across dispersed sources. We developed a first prototype using a basic negotiation protocol in order to validate the feasibility and the interest of our approach. The results obtained are promising and encourage us to continue on this way.