The Counterfactual Store does resemble a Markov chain at the conceptual level. Both deal with states and the possible transitions between them. But a Markov chain is mainly a mathematical model for probabilistic transitions: given that I’m in state A, what’s the probability the next state is B, C or D? And the classic Markov property says the next-state probability depends on the current state, not the full history.

The Counterfactual Store is deterministic branching, not probabilistic transition. If the current application state is A, you create a hypothetical A′, change something, propagate the consequences, inspect the result and throw it away without touching A. No probability needs to be involved anywhere.