Modeling and Analysis of Stochastic Systems
Contents
1. Introduction
- What in the World Is a Stochastic Process?
- How to Characterize a Stochastic Process
- What Does One Do with a Stochastic Process? The Grand Plan
2. Discrete-Time Markov Chains: Transient Behavior
- Definitions and Examples
- Markov Chain Models in Other Fields
- Marginal Distributions
- Computation of Matrix Powers
3. Discrete-Time Markov Chains: Limiting Behavior
- Introduction
- Classification of States
- Recurrence and Transience
- Determining Recurrence and Transience: Infinite DTMC
- Limiting Behavior of Irreducible Markov Chains
- Computation: Irreducible Markov Chains with Infinite State Space
- Computation: Finite State Irreducible Markov Chains
- Limiting Behavior of Reducible Markov Chains
- DTMCs with Costs/Rewards
- Reversibility
- Stochastic Ordering for DTMCs
4. Discrete-Time Markov Chains: First Passage Times
- Definitions
- Probability Distribution of T
- Absorption Probabilities
- Generating Function and Moments of T
5. Poisson Processes
- Exponential Distribution
- Poisson Process: Definitions
- Event Times in a Poisson Process
- Superposition and Splitting of Poisson Processes
- Nonhomogeneous Poisson Process
- Compound Poisson Process
6. Continuous-Time Markov Chains
- Definitions and Sample Path Properties
- Examples
- Properties of the Transition Matrix
- Computation of P(t): Finite State Space
- Computation of P(t): Infinite State Space
- Limiting Behavior of CTMCs
- First Passage Times and Phase-Type Distributions
- CTMCs with Costs/Rewards
- Reversibility
- Stochastic Ordering for CTMCs
7. Applications of Markov Chains to Queueing Theory
- Introduction
- Properties of General Queueing Systems
- Birth and Death Queues
- Open Queueing Networks
- Closed Queueing Networks
- M/G/1 and G/M/1 Queues
- M/G/1/1 Retrial Queue
- M/G/oo Queue
8. Renewal Processes
- Introduction
- Properties of N(t)
- The Renewal Function
- Renewal-Type Equations
- Key Renewal Theorem
- Recurrence Times
- Delayed Renewal Processes
- Alternating Renewal Processes
- Renewal Processes with Costs/Rewards
- Regenerative Processes
9. Markov Renewal Processes
- Definitions and Examples
- Elementary Properties of Markov Renewal Processes
- Markov Renewal Functions
- Generalized Markov Renewal Equations
- Semi-Markov Processes
- Limiting Behavior of Semi-Markov Processes
- Markov Regenerative Processes
- Applications to Queues
A. Appendices
- Probability of Events
- Random Variables
- Bivariate Random Variables
- Multivariate Random Variables
- Computing Expectations
- Transforms
- Modes of Convergence
- Stochastic Ordering