Are Markov chains memoryless?

So in the phrase, "the structure of the cell," the model can use previous context like blood and mitochondria to know the cell most likely refers to biology rather than a prison cell. And it uses that to tune its prediction. But as large language models become more widespread, one concern is that the text they produce ends up on the internet and that becomes training data for future models. - When you start doing that, the game is very soon over.

You come, in this case, to us, a very dull, stable state, it just says the same thing over and over and over again forever. The language models are vulnerable to this process. - And any system like this where we have a feedback loop, will become hard to model using Markov chains. Take global warming, for instance, as we increase the amount of carbon dioxide in the air, the average temperature of the Earth increases.

But as the temperature increases, the atmosphere can hold more water vapor, which is an incredibly powerful greenhouse gas. And with more water vapor, the temperature increases further allowing for even more water vapor. So you get this positive feedback loop, which makes it hard to predict what's going to happen next. So there are some systems where Markov chains don't work, but for many other dependent systems, they offer a way of doing probability.

- But what's fascinating is that all these systems have extremely long histories. I mean, you could trace back all the letters in a text, trace back all the interactions of what a neutron did, or trace back the weather for weeks. But the beautiful thing Markov and others found is that for many of these systems you can ignore almost all of that. You can just look at the current state and forget about the rest, that makes these systems memoryless.

And it's this memoryless property that makes Markov chains so powerful because it's what allows you to take these extremely complex systems and simplify them a lot to still make meaningful predictions. - [Derek] As one paper put it, "Problem-solving is often a matter of cooking up an appropriate Markov chain." - It's kind of ridiculous to me that this basic fact of mathematics would come out of a fight like that, which, you know, really had nothing to do with it. But all the evidence suggests that it really was this determination to show up Nekrasov that led Markov to do the work.