An Introduction to Stochastic Processes.

INTRODUCTION Overview / Introduction / Discrete Random Variables and Generating Functions / Continuous Random Variables and Laplace Transforms / Some Mathematical Background / Problems / Bibliographic Notes / References / Appendix 2. POISSON PROCESSES Overview / Introduction / Properties of Poisson Processes / Nonhomogeneous Poisson Processes / Filtered Poisson Processes / Two-Dimensional and Marked Poisson Processes / Poisson Arrivals See Time Averages (PASTA) / Problems / Bibliographic Notes / References / Appendix 3. RENEWAL PROCESSES Overview / Introduction / Renewal-Type Equations / Excess Life, Current Life, and Total Life / Renewal Reward Processes / Limiting Theorems, Stationary and Transient Renewal Processes / Regenerative Processes / Discrete Renewal Processes / Problems / Bibliographic Notes / References / Appendix 4. DISCRETE-TIME MARKOV CHAINS Overview / Introduction / Classification of States / Ergodic and Periodic Markov Chains / Absorbing Markov Chains / Markov Reward Processes / Reversible Discrete-Time Markov Chains / Problems / Bibliographic Notes / References / Appendix 5. CONTINUOUS-TIME MARKOV CHAINS Overview / Introduction / The Kolmogorov Differential Equations / The Limiting Probabilities / Absorbing Continuous-Time Markov Chains / Phase-Type Distributions / Uniformization / Continuous-Time Markov Reward Processes / Reversible Continuous-Time Markov Chains / Problems / Bibliographic Notes / References / Appendix 6. MARKOV RENEWAL AND SEMI-REGENERATIVE PROCESSES Overview / Introduction / Markov Renewal Functions and Equations / Semi-Markov Processes and Related Reward Processes / Semi-Regenerative Processes / Problems / Bibliographic Notes / References / Appendix 7. BROWNIAN MOTION AND OTHER DIFFUSION PROCESSES Overview / Introduction / Diffusion Processes / Itos Calculus and Stochastic Differential Equations / Multidimensional Itos Lemma / Control of Systems of Stochastic Differential Equations / Problems / Bibliographic Notes / References / Appendix / APPENDIX: GETTING STARTED WITH MATLAB / INDEX