Contents
 

1. Probability

1.1.   Probability Model
1.2.    Sample Space
1.3.    Events
1.4.    Probability of Events
1.5.    Conditional Probability
1.6.    Law of Total Probability
1.7.    Bayes' Rule
1.8.    Independence
1.9.    Problems

2.1.    Random Variables
2.2.    Cumulative Distribution Function
2.3.    Discrete Random Variables
2.4.    Common Discrete Random Variables
2.5.    Continuous Random Variables
2.6.    Common Continuous Random Variables
2.7.    Functions of Random Variables
2.8.    Expectation of a Discrete Random Variable
2.9.    Expectation of a Continuous Random Variable
2.10.   Expectation of a Function of a Random Variable
2.11.   Reference Tables
2.12.   Problems

3.1.    Multivariate Random Variables
3.2.    Multivariate Discrete Random Variables
3.3.    Multivariate Continuous Random Variables
3.4.    Marginal Distributions
3.5.    Independence
3.6.    Sums of Random Variables
3.7.    Expectations
3.8.    Problems

4.1 Introduction
4.2 Conditional Probability Mass Function
4.3 Conditional Probability Density Function
4.4 Computing Probabilities by Conditioning
4.5 Conditional Expectations
4.6 Computing Expectations by Conditioning
4.7 Problems

5.1. What Is a Stochastic Process?
5.2. Discrete-Time Markov Chains
5.3. Examples Of Markov Models
5.4. Transient Distributions
5.5. Occupancy Times
5.6. Limiting Behavior
5.7. Cost Models
       5.7.1. Expected Total Cost Over a Finite Horizon
       5.7.2. Long-Run Expected Cost per Unit Time
5.8.  First Passage Times
5.9.  Problems

6.1. Continuous-Time Stochastic Processes
6.2. Continuous-Time Markov Chains
6.3. Exponential Random Variables
6.4. Examples of CTMCS: I
6.5. Poisson Processes
6.6. Examples of CTMCS: 11
6-7. Transient Analysis: Uniformization
6.8. Occupancy Times
6.9.  Limiting Behavior
6.10.Cost Models
        6.10.1. Expected Total Cost
        6.10.2. Long-Run Cost Rates
6.11. First Passage Times
        Appendix A: Proof Of Theorem 6.4
        Appendix B: Uniformization Algorithm to Compute P(t)
        Appendix C: Uniformization Algorithm to Compute M(T)
 6.12.  Problems

7.1. Introduction
7.2.    Renewal Processes
7.3.    Cumulative Processes
            7.4.    Semi-Markov Processes: Examples
7.5. Semi-Markov Processes: Long-Terrn Analysis
          7.5.1. Mean lnter-Visit Times
          7.5.2. Occupancy Distributions
          7.5.3. Long-Run Cost Rates
7.6. Problems

8.1.    Queueing Systems
8.2.    Single-Station Queues: General Results
8.3. Birth and Death Queues with Finite Capacity
           8.3.1.   M/ M / 1 / K  Queue
           8.3.2.   M / M / s / K   Queue
           8.3.3    M / M / K / K  Queue
8.4. Birth and Death Queues with Infinite Capacity
            8.4.1.     M / M / 1   Queue
            8.4.2.     M / M / s   Queue
            8.4.3.     M / M / 1 Queue
8.5.     M / G / 1  Queue
8.6.     G / M / 1  Queue
8.7.     Networks of Queues
            8.7.1. Jackson Networks
            8.7.2. Stability
            8.7.3. Limiting Behavior
8.8.    Problems

9.1.    Introduction
9.2.    Optimal Order Quantity
9.3.    Optimal Leasing of Phone Lines
9.4.    Optimal Number of Tellers
9.5.    Optimal Replacement
9.6.    Optimal Server Allocation
9.7.    Problems

10.1.  Introduction
10.2.  Discrete-Time Markov Decision Processes: DTMDPs
10.3.  Optimal Policies for DTMDPs
10.4.  Optimal Inventory Control
10.5.  Semi-Markov Decision Processes: SMDPs
10.6.  Optimal Policies for SMDPs
10.7.  Optimal Machine Operation
10.8.  Problems