finance
Conversations with the best investors and business leaders in the world. We explore their ideas, methods, and stories to help you better invest your time and money. Hear stock market and boardroom insights you can't find anywhere else. If you're a professional investor, CEO, entrepreneur, or business strategist, this is for you. Explore all our episodes and learn more at https://www.colossus.com
Gavin Newberry and Rob Locken, founders of Etched, discuss the initial skepticism they faced as young founders trying to break into the semiconductor industry. They highlight how industry veterans were initially dismissive of their potential to build superior AI chips, a sentiment that has since shifted.
Etched's founders explain their philosophy of designing AI chips for specific use cases, contrasting it with the general-purpose approach of the broader semiconductor industry. They leverage the understanding that AI data centers do not operate in freezing temperatures to optimize chip performance.
Mark Ross, a seasoned semiconductor expert and former CTO of Cypress Semiconductor, was an early supporter of Etched. Initially skeptical, he was convinced by a functional simulation and later became Etched's full-time CTO as he witnessed the development progress.
Etched is not just building a chip, but a complete inference solution including the chip, power delivery, boards, interconnects, and production. They focus on optimizing both the pre-fill and decode stages of AI inference, differentiating their approach from traditional GPU architectures.
Etched's founders explain their innovative 'low-voltage inference' approach, which allows their chips to operate at significantly lower voltages than GPUs. This is achieved by re-evaluating constraints and creating new power delivery mechanisms, potentially leading to more efficient AI chips.
For the decode stage of AI inference, Etched emphasizes the critical role of memory bandwidth across the entire cluster, not just on a single chip. They have developed interconnects that provide higher bandwidth and lower latency, enabling more effective use of memory and improving time per token.
Etched's chip architecture is fundamentally different from older designs, as it's built for modern AI workloads like those powering ChatGPT. This involves rethinking flop organization, voltage domains, power planes, packaging, and board design to achieve superior performance.