Stochastic Modeling and Analysis of Telecoms Networks

Chapter 7Markov Process

 

 

 

The Markovian modeling of a dynamic system often leads to a Markov chain, for which the sojourn time in each state becomes random. In many cases, this description is insufficient to establish interesting mathematical properties. In that purpose, we introduce the formalism of Markov jump processes, with their semi-groups and infinitesimal generators.

To go one step further and, in particular, to prove several crucial results such as PASTA or its avatars, we need to see a Markov process as the solution of a martingale problem. In this chapter, we review these different characterizations, and show that they are in fact equivalent.

Throughout this chapter, E denotes a state space that is at the most countable, and equipped with the discrete topology. We refer the reader to the definitions and notations of Appendix A.1.

7.1. Preliminaries

We start by stating two technical Lemmas on the exponential distribution, which will be useful in the following.

LEMMA 7.1.– Let U and V be the two independent random variables, of respective distributions ε (λ) and ε (μ), where λ,μ > 0. Then,

Proof.

(i) The density of the random couple (U, V) is given for all (u, v) by

By denoting the subset A = {(u, v) ∈ R²; u ≤ v}, we can write

(ii) It suffices to see that for any