10170

# A Two-State, Discrete-Time Markov Chain

Consider a system that is always in one of two states, 1 or 2. Every time a clock ticks, the system updates itself according to a 2×2 matrix of transition probabilities, the entry of which gives the probability that the system moves from state to state at any clock tick. A two-state Markov chain is a system like this, in which the next state depends only on the current state and not on previous states. Powers of the transition matrix approach a matrix with constant columns as the power increases. The number to which entries in the column converge is the asymptotic fraction of time the system spends in state .
The state of the system at the time step given by the time slider is the colored circle. Transition probabilities are shown on the left diagram and can be changed using the new transition matrix slider. State 1 is colored yellow for "sunny" and state 2 is colored gray for "not sunny" in deference to the classic two-state Markov chain example. The number of visits to each state over the number of time steps given by the time slider is illustrated by the histogram. Powers of the transition matrix are shown at the bottom.

### PERMANENT CITATION

 Share: Embed Interactive Demonstration New! Just copy and paste this snippet of JavaScript code into your website or blog to put the live Demonstration on your site. More details » Download Demonstration as CDF » Download Author Code »(preview ») Files require Wolfram CDF Player or Mathematica.

#### Related Topics

 RELATED RESOURCES
 The #1 tool for creating Demonstrations and anything technical. Explore anything with the first computational knowledge engine. The web's most extensive mathematics resource. An app for every course—right in the palm of your hand. Read our views on math,science, and technology. The format that makes Demonstrations (and any information) easy to share and interact with. Programs & resources for educators, schools & students. Join the initiative for modernizing math education. Walk through homework problems one step at a time, with hints to help along the way. Unlimited random practice problems and answers with built-in Step-by-step solutions. Practice online or make a printable study sheet. Knowledge-based programming for everyone.