Lenny's Podcast · Sunday, October 18, 2026
Emmanuel Ameisen reveals that although language models are trained as next-token predictors, they actually plan ahead over many words. By analyzing intermediate results during complex tasks like writing poetry, researchers found that models formulate plans that are accessed and reused by future predicted words without needing to state the plan aloud.
“we find that models will like in their intermediate results, sometimes formulate a plan.”
“And so even though they're outputting one word at a time, just like I am right now, they can of course, have a plan that they follow over many, many, many, many words.”