Machine Learning Street Talk · Wednesday, September 23, 2026
Hutter explains that deep learning has historically struggled with tabular data due to its inherent complexities like heterogeneous features, outliers, and missing values. Previous attempts like Google's TabNet (2019) failed to generalize well, unlike the new approach using in-context learning which allows transfer across different datasets.
“And, uh, I think that's one of the reasons that deep learning took so long to, to actually do well for it.”
“There's been countless attempts at at for deep learning for tabular data. Uh, like 2019, TabNet by by Google was was really hyped, uh, thousands of citations, uh, yeah, the the new thing for tabular data and it just doesn't work, it doesn't generalize to new data sets.”
“But what you can do and where you can transfer is on the levels of these patterns, um, detecting the patterns, how the different features interact, etc, detecting causality potentially. And on that level, um, using in context learning, you can actually transfer across different data sets. And that's that's why, um, yeah, with in context learning, we finally saw this breakthrough for tabular data.”