By Professor Qiying Hu, Professor Wuyi Yue (auth.)
Markov selection methods (MDPs), also referred to as stochastic dynamic programming, have been first studied within the Sixties. MDPs can be utilized to version and clear up dynamic decision-making difficulties which are multi-period and ensue in stochastic conditions. There are 3 easy branches in MDPs: discrete-time MDPs, continuous-time MDPs and semi-Markov selection approaches. ranging from those 3 branches, many generalized MDPs versions were utilized to numerous sensible difficulties. those versions comprise partly observable MDPs, adaptive MDPs, MDPs in stochastic environments, and MDPs with a number of pursuits, constraints or obscure parameters.
Markov determination strategies With Their Applications examines MDPs and their functions within the optimum regulate of discrete occasion structures (DESs), optimum substitute, and optimum allocations in sequential on-line auctions. The booklet offers 4 major subject matters which are used to check optimum keep watch over problems:
*a new method for MDPs with discounted overall present criterion;
*transformation of continuous-time MDPs and semi-Markov determination strategies right into a discrete-time MDPs version, thereby simplifying the appliance of MDPs;
*MDPs in stochastic environments, which drastically extends the world the place MDPs should be applied;
*applications of MDPs in optimum keep watch over of discrete occasion structures, optimum substitute, and optimum allocation in sequential on-line auctions.
This ebook is meant for researchers, mathematicians, complex graduate scholars, and engineers who're attracted to optimum regulate, operation study, communications, production, economics, and digital commerce.