Training Data · Wednesday, July 29, 2026
Jerry Tworek and Rohan Anil, founders of Core Automation, believe the current AI paradigm is limited by its reliance on transformers. They argue that while transformers have been highly effective, the next frontier in AI requires fundamentally new architectures that can better handle real-world complexities and enable more continuous learning. Their new company, Core Automation, aims to explore these novel architectures.
“the first step to replacing transformers is appreciating deeply how far they were able to carry us”
“I think at this moment what the bottleneck is to better models and to smarter systems is the architecture itself it is this moment to revisit the train we've been riding for the last six years of trying to add more and more parameters to essentially two of the same operation which is MOE and attention”
“We want to be automating tasks and why those things not yet automated why is not codex doing everything for me and this is the question of like the research where we want to go”