Modeling, Analysis, Design, and Control of Stochastic Systems

V.G. Kulkarni, University of North Carolina

Readership: This book is meant to be used as a textbook in a junior or senior level undergraduate course in stochastic models. Students are expected to be undergraduate students in engineering, operations research, computer science, mathematics, statistics, business administration, public policy, or any other discipline with a mathematical core.  Students are expected to be familiar with elementary matrix operations (additions, multiplications, solving systems of linear equations; but not eigenvalues, eigenvectors), first-year calculus (derivatives and integrals of simple functions; but not differential equations), and probability (which is reviewed in Chapters I to 4 of this book).

As the title suggests, this book addresses four aspects of using stochastic methodology to study real systems.

1. Modeling. The first step is to understand how a real system operates, and what is the purpose of studying it. This enables us to make assumptions to create a model that is simple yet sufficiently true to the real system so that the answers provided by the model will have some credibility. In this book this step is emphasized repeatedly with the use of a large number of real life modeling examples.
2. Analysis. 'Me second step is to do a careful analysis of the model and compute the answers. To facilitate this step the book develops special classes of stochastic processes in Chapters 5, 6, and 7: discrete-time Markov chains, continuous time Markov chains, renewal processes, cumulative processes, semi-Markov processes, etc. For each of these classes, we develop tools to compute the transient distributions, limiting distributions, cost evaluations, first passage times, etc. These tools generally involve matrix computations, and can be done easily in any matrix oriented language, e.g., MATLAB. Chapter 9 applies these tools to queueing systems.
3. Design. In practice, a system is described by a small number of parameters, and we are interested in setting the values of these parameters so as to optimize the performance of the system. This is called "designing" a system. The perfon-nance of the system can be computed as a function of the system parameters using the tools developed here. Then the appropriate parameter values can be determined to minimize or maximize this function. This is illustrated by several examples in Chapter 9.
4. Control. In some applications, the system can be controlled dynamically. Thus instead of finding optimal parameters as in the design aspect, the aim here is to find an optimal operating policy. Chapter 10 show§ how this can be done using linear programming.