In this
work, for a control consumption-investment process with the discounted reward
optimization criteria, a numerical estimate of the stability index is made.
Using explicit formulas for the optimal stationary policies and for the value
functions, the stability index is explicitly calculated and through statistical
techniques its asymptotic behavior is investigated (using numerical
experiments) when the discount coefficient approaches 1. The results obtained define
the conditions under which an approximate optimal stationary policy can be used
to control the original process.
References
[1]
Dynkin, E.B. and Yushkevich, A.A. (1979) Controlled Markov Processes. Springer-Verlag, New York.
[2]
Hernandez-Lerma, O. (1989) Adaptive Markov Control Process. Vol. 79, Springer-Verlang, New York. https://doi.org/10.1007/978-1-4419-8714-3
[3]
Gordienko, E.I. (1992) An Estimate of the Stability of Optimal Control of Certain Stochastic and Deterministic Systems. Journal of Soviet Mathematics, 59, 891-899. https://doi.org/10.1007/BF01099115
[4]
Gordienko, E.I. and Salem, F.S. (1998) Robustness Inequalities for Markov Control Processes with Unbounded Cost. Systems & Control Letters, 33, 125-130. https://doi.org/10.1016/S0167-6911(97)00077-7
[5]
Gordienko, E.I. and Yushkevich, A.A. (2003) Stability Estimates in the Problem of Average Optimal Switching of a Markov Chain. Mathematical Methods of Operations Research, 57, 345-365. https://doi.org/10.1007/s001860200258
[6]
Gordienko, E.I., Lemus-Rodriguez, E. and Montes-de-Oca, R. (2008) Discounted Cost Optimality Problem: Stability with Respect to Weak Metrics. Mathematical Methods of Operations Research, 68, 77-96. https://doi.org/10.1007/s00186-007-0171-z
[7]
Gordienko, E., Martínez, J. and Ruiz de Chávez, J. (2015) Stability Estimation of Transient Markov Decision Processes. In: Mena, R.H., Pardo, J.C., Rivero, V. and Bravo, G.U., Eds., XI Symposium on Probability and Stochastic Processes, Mexico, 18-22 November 2013, 157-176. https://doi.org/10.1007/978-3-319-13984-5_8
[8]
Martínez-Sánchez, J.E. (2020) Stability Estimation for Markov Control Processes with Discounted Cost. Applied Mathematics, 11, 491-509. https://doi.org/10.4236/am.2020.116036
[9]
Gordienko, E.I. and Salem-Silva, F. (2000). Estimates of Stability of Markov Control Processes with Unbounded Costs. Kybernetika, 36, 195-210.
[10]
Montes-de-Oca, R. and Salem-Silva, F. (2005) Estimates for Perturbations of Average Markov Decision Process with a Minimal State and Upper Bounded by Stochastically Ordered Markov Chains. Kybernetika, 41,757-772.
[11]
Martinez, J. and Zaitzeva, E. (2015) Note on Stability Estimation in Average Markov Control Processes. Kybernetika, 51, 629-638. http://doi.org/10.14736/kyb-2015-4-0629
[12]
Rachev, S.T. (1991) Probability Metrics and the Stability of Stochastic Models. Wiley, Chichester.
[13]
Hernandez-Lerma, O. and Lasserre, J. (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York. https://doi.org/10.1007/978-1-4612-0729-0
[14]
Hernandez-Lerma, O. and Lasserre, J.B. (1999) Further Topics on Discrete-Time Markov Control Processes. Springer, New York. https://doi.org/10.1007/978-1-4612-0561-6
[15]
Van Nunen, J.A. and Wessels, J. (1978) Note—A Note on Dynamic Programming with Unbounded Rewards. Management Science, 24, 485-586. https://doi.org/10.1287/mnsc.24.5.576
[16]
Carlton, D. and Perloff, J. (2005) Modern Industrial Organization. Pearson, Addison Wesley, Boston.
[17]
Viscusi, W., Harrington, J. and Vernon, J. (2005) Economics of Regulation and Antitrust. The MIT Press, Cambridge.
[18]
Kmenta, J. (1971) Elements of Econometrics. 2nd Edition. Macmillan Publishing Company, New York.
[19]
Cobb, C.W. and Douglas, P.H. (1928) A Theory of Production. American Economic Review, 18, 139-165.
