Tractable Dual Optimal Stochastic Model Predictive Control: An Example in Healthcare

Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate problems, which however come at a loss of the optimal probing nature of the control signals. While probing can be superimposed in some approaches, this is done sub-optimally. In this paper, we examine approximation of the system dynamics by a Partially Observable Markov Decision Process with its own Finite Horizon Stochastic Optimal Control Problem, which can be solved for an optimal control policy, implemented in receding horizon fashion. This procedure enables maintaining probing in the control actions. We further discuss a numerical example in healthcare decision making, highlighting the duality in stochastic optimal receding horizon control.Comment: 6 pages, 3 figures, submitted for publication in Proc. 1st IEEE Conference on Control Technology and Application

The use of fake algebraic Riccati equations for co-channel demodulation

Copyright © 2003 IEEEThis paper describes a method for nonlinear filtering based on an adaptive observer, which guarantees the local stability of the linearized error system. A fake algebraic Riccati equation is employed in the calculation of the filter gain. The design procedure attempts to produce a stable filter at the expense of optimality. This contrasts with the extended Kalman filter (EKF), which attempts to preserve optimality via its linearization procedure, at the expense of stability. A passivity approach is applied to deduce stability conditions for the filter error system. The performance is compared with an EKF for a co-channel frequency demodulation application.Einicke, G.A.; White, L.B.; Bitmead, R.R

Stable state and signal estimation in a network context

Power grid, communications, computer and product reticulation networks are frequently layered or subdivided by design. The layering divides responsibilities and can be driven by operational, commercial, regulatory and privacy concerns. From a control context, a layer, or part of a layer, in a network isolates the authority to manage, i.e. control, a dynamic system with connections into unknown parts of the network. The topology of these connections is fully prescribed but the interconnecting signals, currents in the case of power grids and bandwidths in communications, are largely unavailable, through lack of sensing and even prohibition. Accordingly, one is driven to simultaneous input and state estimation methods. We study a class of algorithms for this joint task, which has the unfortunate issue of inverting a subsystem, which if it has unstable transmission zeros leads to an unstable and unimplementable estimator. Two modifications to the algorithm to ameliorate this problem were recently proposed involving replacing the troublesome subsystem with its outer factor from its inner-outer factorization or using a high-variance white signal model for the unknown inputs. Here, we establish the connections between the original estimation problem for state and input signal and the estimates from the algorithm applied solely to the outer factor. It is demonstrated that the state of the outer factor and that of the original system asymptotically coincide and that the estimate of the input signal to the outer factor has asymptotically stationary second-order statistics which are in one-to-one correspondence with those of the input signal to the original system, when this signal is itself stationary. Thus, the simultaneous input and state estimation algorithm applied just to the outer factor yields an unbiased state estimate for control and the statistics of the interface signals.Comment: 12 pages, 1 figur

Adaptive Optimal Control The Thinking Man's GPC

Exploring connections between adaptive control theory and practice, this book treats the techniques of linear quadratic optimal control and estimation (Kalman filtering), recursive identification, linear systems theory and robust arguments

A Kalman-filtering derivation of simultaneous input and state estimation

Simultaneous input and state estimation algorithms are studied as particular limits of Kalman filtering problems. This admits interpretation of the algorithm properties and critical analysis of their claims to being partly model-free and to providing unbiased estimates. A disturbance model, white noise of unbounded variance, is provided and the bias feature is shown to be a geometric projection property rather than probabilistic in nature. As a consequence of this analysis, the algorithm is connected, in the stationary case, to Algebraic Riccati equation computations for the gains, estimate covariances and filter frequency response.acceptedVersion© 2019. This is the authors’ accepted and refereed manuscript to the article. Locked until 9.7.2021 due to copyright restrictions. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0

Simultaneous input & state estimation, singular filtering and stability

Input estimation is a signal processing technique associated with deconvolution of measured signals after filtering through a known dynamic system. Kitanidis and others extended this to the simultaneous estimation of the input signal and the state of the intervening system. This is normally posed as a special least-squares estimation problem with unbiasedness. The approach has application in signal analysis and in control. Despite the connection to optimal estimation, the standard algorithms are not necessarily stable, leading to a number of recent papers which present sufficient conditions for stability. In this paper we complete these stability results in two ways in the time-invariant case: for the square case, where the number of measurements equals the number of unknown inputs, we establish exactly the location of the algorithm poles; for the non-square case, we show that the best sufficient conditions are also necessary. We then draw on our previous results interpreting these algorithms, when stable, as singular Kalman filters to advocate a direct, guaranteed stable implementation via Kalman filtering. This has the advantage of clarity and flexibility in addition to stability. En route, we decipher the existing algorithms in terms of system inversion and successive singular filtering. The stability results are extended to the time-varying case directly to recover the earlier sufficient conditions for stability via the Riccati difference equation.acceptedVersio