Assignment 3

Assignment 3 Solutions

1. Consider a time-homogeneous discrete-time Markov chain with transition probability matrix and states where:

Is it an ergodic Markov chain?

Answer

An ergodic Markov chain must be both irreducible and aperiodic. For a finite chain, irreducibility is the key condition, because finite irreducible chains are automatically positive recurrent, and if they are also aperiodic then they are ergodic.

Here the chain is not irreducible. The reason is visible directly from the matrix:

• State is absorbing, since .
• State is also absorbing, since .
• States and communicate with each other and cannot reach , , or .
• State can move to , , or stay at , but no other state can reach from the states .

So the state space splits into multiple communicating classes, and the chain is not irreducible. Therefore it is not ergodic.

2. Consider a time-homogeneous discrete-time Markov chain with transition probability matrix and states where:

1. Find the equivalence classes of states of the above Markov chain.
2. Classify the states of the Markov chain as positive recurrent, null recurrent or transient.

Answer

The communicating classes are read off from the transition structure.

States and communicate with each other:

Neither of these states can reach , and cannot be reached from or . Hence one equivalence class is

State can move to , , or , but once the chain leaves state and enters the class , it can never return to . So is in its own class:

States and communicate with each other and do not connect to the first three states, so they form another class:

Therefore the equivalence classes are

Now classify the states.

The classes and are closed communicating classes, because once the chain enters either class, it cannot leave it. Since the chain is finite and each closed communicating class is finite and irreducible, all states in these classes are positive recurrent.

So states are positive recurrent.

State is transient. It can move to states or , and after that it cannot be reached again. Since there is a nonzero probability of never returning to , state is not recurrent.

There are no null recurrent states in a finite Markov chain, so the full classification is:

3. If is a Poisson process and , then prove that

Answer

Let and define the increments

Because is a Poisson process:

1. ,
2. ,
3. and are independent.

Also,

We want

Using the definition of conditional probability,

Now compute the numerator using independence:

Since ,

Therefore

After cancellation,

Hence

This shows that, conditional on , the number of events occurring by time is binomial with parameters and .

4. Suppose that customers arrive at a jewellery shop according to a Poisson process with an intensity of 15 persons per hour.

(a) What is the probability that no customer has arrived in a 4-minute period?

(b) What is the probability that more than 4 customers have arrived in an 8-minute interval?

Answer

The arrival rate is per hour, so per minute the rate is

For a Poisson process, the number of arrivals in time is Poisson with mean .

(a) No customer in 4 minutes

For minutes, the mean number of arrivals is

So

Thus the probability is

(b) More than 4 customers in 8 minutes

For minutes, the mean number of arrivals is

Hence , and

So

Expanding the sum,

Therefore

and numerically

5. Suppose that a system fails at the cumulative effect of 3 shocks. If shocks occur in accordance with a Poisson process with parameter 1 per week, then find the probability that the system lifetime is greater than five weeks.

Answer

Let be the number of shocks by time weeks. Since shocks occur according to a Poisson process with rate per week,

The system fails at the third shock, so its lifetime is greater than five weeks exactly when fewer than 3 shocks have occurred by time 5. In other words,

Therefore

Using the Poisson formula,

So

Hence

6. Suppose in an exhibition, the visitors are coming in a Poisson process with 15 persons per hour. In a randomly chosen hour:

(a) What is the probability of exactly 5 visitors arriving in the first 15 minutes?

(b) Given that exactly 5 visitors arrived in the first 15 minutes, what is the probability that all 5 arrived in the last 7 minutes out of these 15 minutes?

Answer

The arrival process is Poisson with rate 15 per hour, so the number of visitors in a time interval of length hours is Poisson with mean .

(a) Exactly 5 visitors in the first 15 minutes

Fifteen minutes is of an hour, so the mean number of arrivals in the first 15 minutes is

Therefore,

So the required probability is

(b) All 5 arrivals occur in the last 7 minutes

Conditioned on there being exactly 5 arrivals in the 15-minute interval, the 5 arrival times are distributed like the order statistics of 5 independent uniform random variables on that interval.

So each arrival is equally likely to fall anywhere in the 15-minute interval, and the probability that a given arrival lies in the last 7 minutes is

Since the 5 conditional arrival times are uniformly spread over the interval, the probability that all 5 lie in the last 7 minutes is

Hence,

7. Suppose that people of Ukraine immigrate to Poland at a Poisson rate of 3 per day.

(a) What is the expected time until the 15th immigrant arrives?

(b) What is the probability that the elapsed time between the 14th and the 15th arrival exceeds one day?

Answer

Let the immigration process be a Poisson process with rate per day.

(a) Expected time until the 15th arrival

For a Poisson process, the waiting time until the th arrival has mean

Therefore, the expected time until the 15th immigrant arrives is

So the answer is

(b) Probability that the time between the 14th and 15th arrival exceeds one day

The time between consecutive arrivals in a Poisson process is exponential with rate 3.

So if is the time between the 14th and 15th arrivals, then

Hence

Therefore,

8. Find the differential equation of the pure birth process. If the process starts with individuals, find the mean of the number present at time .

Answer

For a pure birth process with birth rate from state equal to , the forward equations for the state probabilities are

and for ,

This is the standard differential-difference system for a pure birth process.

For the usual linear pure birth process, also called the Yule process, the birth rate is proportional to the current population:

If , then the mean satisfies

with initial condition when .

Solving the differential equation gives

Thus the mean number of individuals at time is

9. Write down the differential-difference equations for the linear growth process with immigration having

and

(a) Show that satisfies the differential equation

(b) Find the solution for from the above differential equation with the initial condition when .

(c) What is the limit of as for ?

Answer

Let

For a birth-death process with immigration, the differential-difference equations are

for , with the usual boundary convention that terms involving negative indices are omitted.

Substituting and , we get

To find the mean, multiply the forward equation by and sum over all . Using standard index shifts, the terms combine to give

where .

(b) Solve for

We solve the linear differential equation

with .

If , the solution is

Equivalently,

(c) Limit when

If , then , so as .

Therefore,

So the limiting mean is