Abstract: Reinforcement learning (RL) is a widely used
paradigm for learning control. Computing exact RL solutions is
generally only possible when process states and control actions
take values in a small discrete set. In practice, approximate
algorithms are necessary. In this paper, we propose an approximate, model-based Q-iteration algorithm that relies on
a fuzzy partition of the state space, and a discretization of
the action space. Using assumptions on the continuity of the
dynamics and of the reward function, we show that the resulting
algorithm is consistent, i.e., that the optimal solution is obtained
asymptotically as the approximation accuracy increases. An
experimental study indicates that a continuous reward function
is also important for a predictable improvement in performance
as the approximation accuracy increases.
Abstract: Ant Colony Optimization (ACO) has proven to be a very powerful optimization heuristic for Combinatorial Optimization Problems. While being very successful for various NP-complete optimization problems, ACO is not trivially applicable to control problems. In this paper a novel ACO algorithm is introduced for the automated design of optimal control policies for continuous-state dynamic systems. The so called Fuzzy ACO algorithm integrates the multi-agent optimization heuristic of ACO with a fuzzy partitioning of the state space of the system. A simulated control problem is presented to demonstrate the functioning of the proposed algorithm.
Abstract: Reinforcement learning (RL) is a learning control paradigm that provides well-understood algorithms with good convergence and consistency properties. Unfortunately, these algorithms require that process states and control actions take only discrete values. Approximate solutions using fuzzy representations have been proposed in the literature for the case when the states and possibly the actions are continuous. However, the link between these mainly heuristic solutions and the larger body of work on approximate RL, including convergence results, has not been made explicit. In this paper, we propose a fuzzy approximation structure for the Q-value iteration algorithm, and show that the resulting algorithm is convergent. The proof is based on an extension of previous results in approximate RL. We then propose a modified, serial version of the algorithm that is guaranteed to converge at least as fast as the original algorithm. An illustrative simulation example is also provided.
Abstract: Traffic state estimation is a prerequisite for traffic surveillance and control. For macroscopic traffic flow models several estimation methods have been investigated, including extended and unscented Kalman filters and particle filters. In
this paper we propose a fuzzy observer for the continuous time version of the macroscopic traffic flow model METANET. In order to design the observer, we first derive a dynamic Takagi-Sugeno fuzzy model that exactly represents the traffic model of a segment of a highway stretch. The fuzzy observer is designed based on the fuzzy model and applied to the traffic model. The simulation results are promising for the future development
of fuzzy observers for a highway stretch or a whole traffic
Abstract: A large class of nonlinear systems can be well
approximated by Takagi-Sugeno fuzzy models, for which methods and algorithms have been developed to analyze their stability and to design observers and controllers. However, results obtained for Takagi-Sugeno fuzzy models are in general not directly applicable to the original nonlinear system. In this paper, we investigate what conclusions can be drawn when an observer-based controller is designed for an approximate model and then applied to the original nonlinear system. In particular, we consider the case when the scheduling vector depends on the states that have to be estimated, and in the membership functions of the observer estimated scheduling vectors are used. The results are illustrated throughout the
paper using simulation examples.