Tractable Dual Optimal Stochastic Model Predictive Control: An Example in Healthcare

Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving this problem leads to an optimal probing nature of the resulting control law, called dual control, which trades off benefits of exploration and exploitation. In practice, intractability of Stochastic Model Predictive Control is typically overcome by replacement of the underlying Stochastic Optimal Control problem by more amenable approximate surrogate problems, which however come at a loss of the optimal probing nature of the control signals. While probing can be superimposed in some approaches, this is done sub-optimally. In this paper, we examine approximation of the system dynamics by a Partially Observable Markov Decision Process with its own Finite Horizon Stochastic Optimal Control Problem, which can be solved for an optimal control policy, implemented in receding horizon fashion. This procedure enables maintaining probing in the control actions. We further discuss a numerical example in healthcare decision making, highlighting the duality in stochastic optimal receding horizon control.Comment: 6 pages, 3 figures, submitted for publication in Proc. 1st IEEE Conference on Control Technology and Application

A weakly stable algorithm for general Toeplitz systems

We show that a fast algorithm for the QR factorization of a Toeplitz or Hankel matrix A is weakly stable in the sense that R^T.R is close to A^T.A. Thus, when the algorithm is used to solve the semi-normal equations R^T.Rx = A^Tb, we obtain a weakly stable method for the solution of a nonsingular Toeplitz or Hankel linear system Ax = b. The algorithm also applies to the solution of the full-rank Toeplitz or Hankel least squares problem.Comment: 17 pages. An old Technical Report with postscript added. For further details, see http://wwwmaths.anu.edu.au/~brent/pub/pub143.htm

System Identification for Limit Cycling Systems: A Case Study for Combustion Instabilities

This paper presents a case study in system identification for limit cycling systems. The focus of the paper is on (a) the use of model structure derived from physcal considerations and (b) the use of algorithms for the identification of component subsystems of this model structure. The physical process used in this case study is that of a reduced order model for combustion instabilities for lean premixed systems. The identification techniques applied in this paper are the use of linear system identification tools (prediction error methods), time delay estimation (based on Kalman filter harmonic estimation methods) and qualitative validation of model properties using harmonic balance and describing function methods. The novelty of the paper, apart from its practical application, is that closed loop limit cycle data is used together with a priori process structural knowledge to identify both linear dynamic forward and nonlinear feedback paths. Future work will address the refinement of the process presented in this paper, the use of alternative algorithms and also the use of control approachs for the validated model structure obtained from this paper

Particle Model Predictive Control: Tractable Stochastic Nonlinear Output-Feedback MPC

We combine conditional state density construction with an extension of the Scenario Approach for stochastic Model Predictive Control to nonlinear systems to yield a novel particle-based formulation of stochastic nonlinear output-feedback Model Predictive Control. Conditional densities given noisy measurement data are propagated via the Particle Filter as an approximate implementation of the Bayesian Filter. This enables a particle-based representation of the conditional state density, or information state, which naturally merges with scenario generation from the current system state. This approach attempts to address the computational tractability questions of general nonlinear stochastic optimal control. The Particle Filter and the Scenario Approach are shown to be fully compatible and -- based on the time- and measurement-update stages of the Particle Filter -- incorporated into the optimization over future control sequences. A numerical example is presented and examined for the dependence of solution and computational burden on the sampling configurations of the densities, scenario generation and the optimization horizon.Comment: 6 pages, 5 figures, to appear in proc. 20th IFAC World Congres