Answered: Consider a Markov chain Xn with state… | bartleby

Linear Algebra: A Modern Introduction

Linear Algebra: A Modern Introduction

4th Edition

ISBN:9781285463247

Author:David Poole

Publisher:David Poole

Chapter3: Matrices

Section3.7: Applications

Problem 16EQ

See similar textbooks

Related questions

Question

Please do not give solution in image format thanku

**Can choose more than one

Consider a Markov chain Xn with state space S and
probability transition function p. For x E S, which of
the following are equivalent to the statement that a is
transient:
P({T <∞ for all k ≥ 1}) = 0
Ex [N] <∞ where N is the number of times that Xn
visits x
Σ₁ p(n) (x,x) < 0

Expert Solution

Step by step

Solved in 3 steps with 8 images

SEE SOLUTION Check out a sample Q&A here

Blurred answer

Similar questions

Explain how you can determine the steady state matrix X of an absorbing Markov chain by inspection.
Exercise 2.7. Consider an unfair coin with probability p of heads. (a) The coin has been tossed (a deterministic number) n times. Denote by X the number of heads among the n tosses and by Y the number of tails. Show that X and Y are dependent. Poisson(A). (b) The coin has been tossed (a random number) N times where N - Denote by X the number of heads among the N tosses and by Y the number of tails. Show that X and Y are independent.
4. Let X be a Markov chain with state space S = {1, 2, 3) and transition matrix (² where 0 < p < 1. Prove that P = 0 Ph = P P P 1-P 0 0 P 1-p ain a2n a3n aln a2n a3n where ain + wa2n + w²a3n = (1 − p + pw)", w being a complex cube root of 1. a3n a2n ain
= 7. Let X be a continuous-time Markov chain with transition probabilities pij (t) and define F; = inf{t > T₁: X(t) =i} where T₁ is the time of the first jump of X. Show that, if gii #0, then P(F; <∞ | X (0) = i) = 1 if and only if i is persistent.
Consider the following very simple random walk model. Let the starting position Xo = 0 be deterministic. At each timet you fip a fair coin and add 1 to X,-1 if you see heads H, else add 0 if you see tails T. Suppose the first 9 coin flips result in the sequence HTTTHHTTT. Compute the first 10 terms of the time series {X}0 Find the expected position E [X10] at time t = 10. 3.5 Find the expected position E [X20] at time t = 20. 8.5 Find the variance of the position X 10 at time t = 10. 0.25 Find the variance of the position X20 at time t = 20. 0.25
Consider a time-homogeneous markov chain (Xt: t = 0,1, 2, ...) with states (1,2,3}. what is P[X1 a, X4 = d | XO = i0]?
2. Let {Xt: t = 0, 1, 2,...} be a discrete-time Markov chain. Prove that given X₂ = i, Xn+1 is independent of {Xo, X₁,..., Xn-1}. In other words, prove that P(Xn+1 = J, Xo = 20, X₁ = 1,..., Xn-1 = in-1|Xn = i) = P(Xn+1 =jXn = i) P(Xo = io, X1 = 1,..., Xn-1 = in-1 Xn = i) for any j, io, i1,..., in-1,i in the state space.
2. ) Recall that the simple random walk is just a Markov chain on the integers where we move from j to j+1 with probability, and from j to j-1 with probability: . Find 1 P(Xn+1 =j+1| Xn = j) = 2 1 P(Xn+1 =j-1 | Xn = j) = ²/ P(Xn+1 = m | Xn = j) = 0 ifm #j+1 or j - 1. (a) P(Xn+1 = 2 | X, = 0) (i.e. the probability of going from 0 to 2 in one-step) (b) P(X+1 = 2 | X,,= 1) (i.e. the probability of going from 1 to 2 in one-step) (c) P(Xn+2= 2 | X, = 2) (i.e. the probability of going from 2 to 2 in two-steps)
Let X be a Markov chain containing an absorbing states with which all other states i communicate, in the sense that pis (n) > 0 for some n = n(i). Show that all states other than s are transient.
A Markov chain is stationary if Select one: a. EP =1 for i=1,2,.s b.Pis not equal to zero. c.For any i, j from 1,2,.n 0
5. Let X be a continuous-time Markov chain with generator G satisfying gi = -8ii > 0 for all i. Let HA = inf{t 20: X(t) = A} be the hitting time of the set A of states, and letn; = P(HA <∞ | X (0) = j) be the chance of ever reaching A from j. By using properties of the jump chain, which you may assume to be well behaved, show that Σk 8jknk = 0 for j & A.
Consider a Markov chain with state space S = {1, 2,...} and transition probability function P(1, 2) = P(2, 3) = 1, P(x, x + 1) = ½ and P(x, 3) = ½ for all x ≥ 3 in S. Find the limit of P" (4,7) as n tends to infinity.

SEE MORE QUESTIONS

Recommended textbooks for you

Linear Algebra: A Modern Introduction

Linear Algebra: A Modern Introduction

Algebra

ISBN:

9781285463247

Author:

David Poole

Publisher:

Cengage Learning

Elementary Linear Algebra (MindTap Course List)

Elementary Linear Algebra (MindTap Course List)

Algebra

ISBN:

9781305658004

Author:

Ron Larson

Publisher:

Cengage Learning

Linear Algebra: A Modern Introduction

Linear Algebra: A Modern Introduction

Algebra

ISBN:

9781285463247

Author:

David Poole

Publisher:

Cengage Learning

Elementary Linear Algebra (MindTap Course List)

Elementary Linear Algebra (MindTap Course List)

Algebra

ISBN:

9781305658004

Author:

Ron Larson

Publisher:

Cengage Learning

SEE MORE TEXTBOOKS