Introduction to Stochastic Processes Using R

This textbook presents some basic stochastic processes, mainly Markov processes. It begins with a brief introduction to the framework of stochastic processes followed by the thorough discussion on Markov chains, which is the simplest and the most important class of stochastic processes. The book then elaborates the theory of Markov chains in detail including classification of states, the first passage distribution, the concept of periodicity and the limiting behaviour of a Markov chain in terms of associated stationary and long run distributions. The book first illustrates the theory for some typical Markov chains, such as random walk, gambler's ruin problem, Ehrenfest model and Bienayme-Galton-Watson branching process; and then extends the discussion when time parameter is continuous. It presents some important examples of a continuous time Markov chain, which include Poisson process, birth process, death process, birth and death processes and their variations. These processes play a fundamental role in the theory and applications in queuing and inventory models, population growth, epidemiology and engineering systems. The book studies in detail the Poisson process, which is the most frequently applied stochastic process in a variety of fields, with its extension to a renewal process.