Necessary First-Order and Second-Order Optimality Conditions in Discrete-Time Stochastic Systems

Abstract

In this paper, first-order and second-order necessary conditions for optimality for discrete-time stochastic optimal control problems governed by discrete-time Ito equations are established. A new discrete-time backward stochastic equation and discrete-time backward stochastic matrix equation are introduced. Based on the discrete-time backward stochastic Ito equation, a discrete-time stochastic maximum principle for the stochastic discrete optimal control problems is obtained. Moreover, using the discrete-time backward stochastic matrix equation the second-order necessary conditions for optimality are obtained.