ai
Join us as we train our neural nets on the theme of the century: AI. Sonya Huang, Pat Grady and more Sequoia Capital partners host conversations with leading AI builders and researchers to ask critical questions and develop a deeper understanding of the evolving technologies—and their implications for technology, business and society. The content of this podcast does not constitute investment advice, an offer to provide investment advisory services, or an offer to sell or solicitation of an offer to buy an interest in any investment fund.
Paragon is developing AI technology aimed at improving city and community safety while also protecting individual privacy. The company's approach focuses on building foundational infrastructure for modern cities, enabling them to preserve individuality and foster community connection. Their goal is to help cities become "awesome" by providing objective safety and a sense of security.
Nick and Ben discuss the concept of forward-deployed engineering, emphasizing the importance of psychologically owning or co-owning a customer's problem to achieve desired outcomes. This involves immersing oneself in the customer's context, understanding their challenges at a human level, and delivering solutions effectively, even if it means setting aside one's own expertise.
Nick explains that Silicon Valley often misses the mark when it comes to forward-deployed engineering by underestimating the complexity of institutional context. He points out that engineers need to understand not just data, but also the human elements and 30 years of history within an organization to effectively implement solutions.
Nick Noone shares his formative experience at Palantir, where he learned the value of forward-deployed engineering. This experience directly influenced the founding of Paragon, a company that aims to leverage technology to impact cities and communities positively.
Chai Discovery is working to make the drug discovery process more like engineering, aiming to move beyond trial-and-error. Co-founder Matt explained that they want to enable users to specify the desired molecule and have an AI materialize it, rather than screening millions of possibilities. He added that this approach could industrialize the process, similar to how LLMs have impacted code generation.
The field of protein folding and design has significantly evolved with the advent of deep learning, particularly marked by breakthroughs like AlphaFold 2 in 2020. Initially focused on predicting protein structure from amino acid sequences, the field has advanced to designing sequences that fold into specific structures, moving closer to drug design. This evolution was further accelerated by diffusion models, enabling simultaneous generation of protein structures and sequences.
Chai Discovery was founded in 2024 after observing significant progress in protein folding and design, particularly the potential to design antibodies computationally, a task previously considered too difficult. Co-founder Josh noted that while many early protein design efforts focused on 'mini proteins,' their focus on antibodies aligns with the broader needs of the drug industry. They believed that if antibody structures could be predicted, designing them would become feasible.
Diffusion models have been a key breakthrough for generative AI in biology, enabling the generation of reasonable-looking protein structures and sequences. Unlike previous methods like variational autoencoders, diffusion models allow the model more time to iteratively refine outputs, making small improvements step-by-step. This approach has proven highly effective for biological problems, allowing for the generation of protein sequences that fold into desired structures with specific properties.
Chai Discovery emphasizes a pragmatic approach to building a team that combines diverse expertise, including AI researchers, chemists, biologists, and product engineers. Co-founder Josh highlighted his background in AI from OpenAI, focusing on the question of whether AI could learn DNA and protein languages like it learned human languages. The company believes this interdisciplinary synergy is crucial for advancing AI in drug design.
Chai Discovery has significantly improved the success rate of AI-driven antibody design, increasing it from 0.1% to 15% with their Chai 2 model. This leap in accuracy allows for more robust data collection and analysis of molecular properties. The company focuses on end-to-end generative design, taking an idea from concept to a testable molecule in the lab, differentiating them from competitors who may only assist in finding existing molecules.
Pharmaceutical partners have shown a surprisingly quick adoption rate for Chai Discovery's AI drug design technology, integrating it into their workflows faster than anticipated. The company has focused on building the entire technology stack, from data and models to compute and evaluation infrastructure, enabling them to deploy functional models and form strong partnerships. This integrated approach allows them to deliver tangible value to drug discovery efforts.
Looking ahead, Chai Discovery's ultimate goal is to see the first molecules designed through their platform enter clinical trials, driving patient impact. The company faces the challenge of scaling operations to serve more customers by growing their team and infrastructure. They are excited by the prospect of AI transforming the world and are actively seeking talented individuals to join their passionate team.
Jerry Tworek and Rohan Anil, founders of Core Automation, believe the current AI paradigm is limited by its reliance on transformers. They argue that while transformers have been highly effective, the next frontier in AI requires fundamentally new architectures that can better handle real-world complexities and enable more continuous learning. Their new company, Core Automation, aims to explore these novel architectures.
Rohan Anil reflects on his past belief that scaling reinforcement learning (RL) would be the direct path to AGI. Despite significant advancements and scaling of RL models, he notes that real-world tasks remained largely unsolved. Anil attributes this to a disconnect between benchmark performance and real-world applicability, suggesting that current models do not adequately learn from real-world distributions.
Rohan Anil outlines the limitations of current methods for AI learning at test time, specifically in-context learning and fine-tuning. In-context learning, while not suffering from catastrophic forgetting, is not scalable. Fine-tuning, on the other hand, is prone to catastrophic forgetting and is data-inefficient. Anil suggests that new algorithms are needed to enable models to learn effectively over longer horizons.
Jerry Tworek explains that transformers have been economically valuable because their training cost is lower than the revenue they generate, unlike LLMs which might not offer the same economic advantage. He highlights that OpenAI's success was partly due to a contrarian bet on scaling algorithms with more compute, rather than solely focusing on algorithmic efficiency, a strategy that was met with criticism.
Anthropic's platform team has two north stars: enabling internal teams to ship AGI-pilled products quickly and reliably, and providing external builders with tools to work with Claude. Angela Jiang stated their external goal is to give any builder the tools to create whatever they want with Claude, bringing the platform close to businesses by integrating with hyperscalers like AWS and Google.
Anthropic's platform team strives for consistency in offering the same primitives to both internal and external builders. Katelyn Lesse explained this approach is due to the rapid evolution of AI form factors, shifting from chat to agents and beyond. By providing a robust platform with flexible tools, they aim to enable experimentation and naturally discover new use cases.
Anthropic's platform has evolved significantly from its initial state as just the messages API. Katelyn Lesse highlighted that they've introduced standards like MCP and developer tooling, but the core development has been towards creating higher-order abstractions. These abstractions help customers build agentic work out-of-the-box by addressing challenges like infrastructure management for sandboxes and context window management for harnesses.
Anthropic's platform adoption varies by customer type, with AI-native startups preferring primitives for low-level experimentation. More traditional enterprises and startups focused on unique user value tend to opt for higher-order, packaged offerings. Angela Jiang noted that the choice depends on whether the customer's primary goal is optimizing specific AI aspects or integrating AI into broader workflows.
Katelyn Lesse outlined Anthropic's abstraction layers, starting with knowledge, then execution, and finally coordination. At the coordination layer, they are developing 'strategies' or meta-harnesses that allow tokens to be given different jobs, such as advising versus executing. This layered approach aims to enable the composition of strategies that ladder together for end-to-end execution and knowledge utilization.
Zipline co-founder Keller Rinaudo rejects the 'drone company' label, stating that customers are more interested in the automated logistics system and the experience of fast, cost-effective delivery. He explained that Zipline aims to be an automated logistics system for Earth, approximating tele-transportation.
Zipline initially faced significant regulatory hurdles in the US, with investors deeming their autonomous logistics vision 'illegal.' This led the company to pivot and launch in Rwanda in 2016, delivering blood transfusions, which allowed them to work with the government to establish legal operations.
Zipline's initial launch in Rwanda was a 'total disaster,' serving only one of 21 contracted hospitals for nine months. The company learned that focusing on the 'cool vehicle' was a mistake, and the core solution lay in building robust infrastructure, as emphasized by Eric, who noted the constant all-nighters and weekends spent fixing issues.
Zipline has expanded its operations to be 24/7, 365 days a year, serving 5,000 hospitals and health facilities across eight countries. This growth signifies its evolution from a startup to the largest commercial autonomous system on Earth.
Zipline's autonomous delivery system has demonstrated significant impact, including a 51% reduction in maternal mortality in a University of Pennsylvania study and an estimated 10-12,000 lives saved annually across its various use cases. The company is expanding its efforts through a partnership with the US State Department.
Zipline has secured a $550 million partnership with the US State Department to expand its life-saving services, a move dubbed 'commercial diplomacy.' The initiative aims to accelerate economic development in partner countries by promoting the adoption of US AI and robotics technology.
Zipline has had to engineer its navigation systems to be robust against solar weather events, such as solar flares, which can impact GPS signals. This was an unexpected challenge that required deep expertise in designing systems resilient to space weather conditions.
Rumors suggest that research firm SemiAnalysis has surpassed $100 million in annual revenue, according to a discussion on the 'Training Data' podcast. The company, known for its deep dives into semiconductor technology and supply chains, has also been rumored to be considering launching its own venture fund due to high demand for affiliation.
Dylan Patel, founder of SemiAnalysis, shared anecdotes from his childhood spent in his parents' family hotel and gas station business. He humorously described how he trained an early 'neural network' by observing customers to predict their cigarette choices, a formative experience that blended technology and business acumen.
Dylan Patel's fascination with hardware began in childhood after his Xbox 360 suffered the 'Red Ring of Death' issue. He successfully repaired it by shortening a temperature sensor, sparking an interest that led him to explore computer hardware forums and build PCs by age 12.
Dylan Patel founded SemiAnalysis in early 2020, a period marked by personal loss, including the death of his grandmother, and professional setbacks where his work was not properly credited. He found solace and focus by traveling across the US in a campervan, dedicating himself to writing blog posts about semiconductors, which eventually grew into his research firm.
Dylan Patel has been living a nomadic lifestyle since mid-2020, traveling to over 40 conferences annually to gather insights for SemiAnalysis. He described his approach of engaging with experts at these events, noting that many older professionals in the semiconductor industry are eager to share their knowledge with younger, enthusiastic individuals.