Markov expectation: How many stones will the hero cost? [closed]

Question

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 8 years ago.

Improve this question

In a game, a hero has 100% probability to go from level 0 to level 1.

When at level 1, he has 1/3 probability to go to level 2, 1/3 probability to level 0, 1/3 probability to stay at level 1.

When at level 2, he has 1/9 probability to win, 4/9 probability to go to level 1, 4/9 probability to stay at level 2.

If each step (upgrade/downgrade/stay at the same level) costs one stone, how many stones will the hero pay to win averagely?

Answer 1

You could introduce 3 random variables A, B, C as follows.

A: expected number of stones from level 0 to success
B: expected number of stones from level 1 to success
C: expected number of stones from level 2 to success

Then you can formulate the following equations:

A = 1 + B
B = 1 + 1/3 * A + 1/3 * B + 1/3 * C
C = 1 + 1/9 * 0 + 4/9 * B + 4/9 * C

Then, you can solve for A, B and C.