AI Models Use Geometric Representations for Concepts

“We discuss their work on predictive data debugging, which uses interpretability techniques to identify the concepts that network updates are likely to affect, thus making it possible to identify and address anomalies before they become unpleasant behavioral surprises.”

“We'd go deep on their series of papers on the intricate and often quite beautiful geometries that large language models use to represent advanced concepts, including how we should understand this as an evolution of the linear representation hypothesis, how they're using this new deeper understanding to improve model steering, and how they've identified spatial representations of such advanced concepts as the periodic table, the tree of life.”