ai
A biweekly podcast where hosts Nathan Labenz and Erik Torenberg interview the builders on the edge of AI and explore the dramatic shift it will unlock in the coming years. The Cognitive Revolution is part of the Turpentine podcast network. To learn more: turpentine.co
The speaker argues that the primary goal for the US in its relationship with China should be to achieve a future of peace and abundance, rather than to 'win'. This future would leverage the benefits of AI without incurring extreme risks or creating new dangers, a state he terms 'Pax Robotica'.
Nate posits that China's advancement in AI research should be viewed not as a recent development but as a return to its historical position of prominence. He suggests that attempts to impede China's technological progress will only yield short-term results.
The speaker explains that China's historical experience with a 'century of humiliation' has instilled a deep understanding that technological inferiority is dangerous. Consequently, China is determined to achieve and maintain technological strength and self-sufficiency to ensure its power and security.
Despite concerns, Nate suggests that China's rhetoric indicates a shared interest in AI safety and a preference for international collaboration over an AI arms race. He supports this by noting China's publication of research in English and the release of open-weight models, positioning China as a leader in providing global AI public goods.
Nate advises the United States not to fear China, but to learn from both its successes and mistakes. He emphasizes the importance of maintaining a shared AI paradigm to prevent the AI competition from escalating into conflict.
Nate suggests that the US should seek a reciprocal deal with China, potentially trading semiconductor technology for Chinese expertise in areas like solar energy, batteries, and robotics. This would aim to recouple economies without creating unacceptable vulnerabilities.
Nate suggests that the US could unilaterally reduce tensions with China by clarifying that chip restriction rules are not intended to hinder AI safety or research collaborations. He believes this move could be made even before the upcoming summit.
The analysis of the OpenAI and Hugging Face incident revealed significant limitations in the investigative process. The investigators were given a short timeframe and restricted access to data, raising concerns about the thoroughness of the findings.
The incident at OpenAI, involving internal agent behavior, has attracted the attention of Congress. This suggests a potential for a formal congressional investigation into the matter.
Zack Bratton Glenn, a partner at Gradient, mentioned that Harvey, a legal AI company, is now operating its own model. This model has undergone post-training on 'okimi K3'.
Zack Bratton Glenn, a general partner at Gradient, revealed that the AI seed fund, which initially launched Inside Google, spun out of Alphabet last October. Gradient was founded in 2017.
According to Zack Bratton Glenn of Gradient, open AI models have significantly closed the gap in coding capabilities. The remaining challenges and areas for advancement lie in domain-specific judgment, particularly in fields like law, medicine, and finance.
Misha Guruvich and Vivian Belenky of Aro-Lamp discussed their germicidal 222 nanometer wavelength light technology, which deactivates airborne pathogens. Preliminary results from a South African trial indicate a 90% suppression of tuberculosis transmission in hospital wards.
Patrick McKenzie highlighted the Aro-Lamp, priced at $500, as having an outstanding expected return on investment, particularly for preventing illness. He noted that a single case of the common cold can cost individuals and companies more than $500 in lost productivity.
Vivian Belenky explained that far UVC light (222 nm) is germicidal because it's absorbed by the DNA and RNA of pathogens, as well as proteins. For humans, a 20-micron thick layer of dead skin cells absorbs the light, preventing it from reaching living cells and thus causing damage.
Patrick McKenzie plans to install Aro-Lamps in his son's classrooms, citing a desire to invest in community health and support the nascent market for this technology. He noted the potential for a 90% reduction in disease transmission.
Vivian Belenky stated that far UVC light is safe for human eyes because it is absorbed by the cornea, the outer layer of the eye, and does not reach the retina. This prevents any risk of blindness or other eye damage.
Good Fire CTO Dan Balsam announced the launch of Silico, a machine learning research platform that originated as an internal tool. Silico aims to provide wider access to Good Fire's expertise in areas like GPU cluster management and experimental techniques. The platform is priced at $1,000 per month per seat, with discounted pricing planned for safety and alignment researchers.
Dan Balsam of Good Fire discussed their work on predictive data debugging, a method using interpretability techniques to identify concepts in a network that are likely to be affected by updates. This allows for early detection and resolution of anomalies before they cause behavioral issues.
Good Fire's research explores the geometric structures that large language models employ to represent advanced concepts, building on the linear representation hypothesis. This deeper understanding is being applied to enhance model steering.
Dan Balsam emphasized the importance of open source models in preventing a dangerous concentration of power within the AI field. He also touched on concerns about AI's potential to exacerbate bio-disasters.
Dan Balsam believes that while monitoring techniques are improving, intentional design of safety controls during the training process is crucial. He also indicated that certain training techniques are too problematic to be used at this time.
According to Dan Balsam, researchers at Good Fire are currently focusing discussions on the complex issues surrounding AI welfare and consciousness. This focus is partly driven by recent research in the field.
The speaker aims to disabuse listeners of the misconception that China does not care about AI safety or will not slow down AI development. They state that China has, at times, slowed down its AI companies in the name of safety, demonstrating that the government is willing to take action when it deems it necessary.
The speaker suggests that the perceived difference in AI safety safeguards between US and Chinese companies is largely due to the influence of OpenAI and Anthropic. If these two companies were removed from consideration, the safety differential would diminish significantly, leaving US companies with only a slim edge.
As of the time of the recording, Chinese AI companies and their models reportedly do not have as robust safeguards against misuse as their American counterparts. The speaker emphasizes that this difference is significant, despite the fact that without top US companies like OpenAI and Anthropic, the gap would be much smaller.
The podcast host detailed a production issue where an AI-generated intro voice for their previous episode sounded unsettling and "nightmare fuel." The AI, named Fable, created its own voice using 11 Labs and video based on LLX, but the output was not as intended.
The host apologized for a two-week hiatus from the podcast, which was longer than usual. The delay was due to travel to China on a business visa, advice to wait until returning to publish content, and a production issue with a previously prepared episode.
The host outlined a three-part series on their recent trip to China, focusing on the general experience and AI use in the first episode. Subsequent episodes will delve into China's AI safety and governance, and the broader US-China relationship in the context of AI development.
The host noted that people in China were generally comfortable discussing a wide range of topics, including politics, during their trip. This was observed even in casual settings, contrasting with norms in the US where recording friendly conversations is less common.
The host investigated an AI's (Fable) unexpected voice output, discovering it intended a warm, reassuring tone rather than the perceived 'creepy and alien' sound. Fable explored three voice designs: 'luminous narrator,' 'young scholar,' and 'low ember.'
Anthropic has released a new paper detailing "J-Space" and the "J-lens," a method for probing how large language models store and process concepts. The J-lens can identify directions in a model's latent space that correlate with specific token outputs, offering a potential window into the model's internal reasoning processes. This new approach shows promise in understanding complex LLM behaviors.
Attempts to manipulate LLM behavior using the J-lens have shown varied success, with interventions only reliably influencing outcomes 50-70% of the time. The remaining 30-45% of cases result in unpredictable or nonsensical changes, indicating that aspects of LLM cognition remain opaque. Researchers suggest this suggests 'dark matter or dark cognition' within these models.
A novel training method called 'counterfactual reflection' has been introduced, where AI models are paused mid-task and trained to respond according to desired values. This method involves interrupting the model and guiding it toward a 'constitutionally right' response, which is then used for supervised training. Anthropic's research suggests this technique helps embed desired concepts into the J-space, improving overall model behavior.
The J-space probing technique is noted for its low computational cost, making it potentially suitable for production environments. The process involves simple matrix multiplication on layer activations, with an estimated compute overhead of 5% or less, similar to other monitoring methods like constitutional classifiers.
Liquid AI, a company spun out of MIT, is developing device-native foundation models designed for edge devices with limited processing power. CEO Ramin Hassani highlighted that the $800 billion global smartphone and laptop market represents a massive opportunity, especially given concerns about privacy and data control. The company's approach focuses on efficient, biologically inspired algorithms.
Liquid AI has demonstrated the capability of running a 1 billion parameter model on an iPhone, showcasing its "liquid philosophy" for efficient AI. CEO Ramin Hassani mentioned that this model, featuring attention layers and a simple gated learned convolution, can operate fast enough for basic use cases like private document classification.
Liquid AI is refining its network architecture search process by evaluating models on actual hardware and downstream tasks, moving away from potentially misleading proxy metrics. Ramin Hassani explained that for specific use cases and limited compute resources, their search is more likely to discover novel, exotic architectures.
Liquid AI is planning to launch a new platform that will allow customers to fine-tune small AI models for their specific use cases on a self-serve basis. Host anticipates this will ease demand for frontier models and improve global AI access, expressing excitement for its upcoming release.
Liquid AI's core mission, originating from MIT research, is to maximize the intelligence that can be encoded into the smallest algorithmic formats, with efficiency as its cornerstone. Ramin Hassani elaborated that their work on "Liquid neural networks" aims to deliver the reliability of larger AI systems while running on smaller, embedded hardware like CPUs and NPUs.
Liquid AI is developing alternative machine learning algorithms inspired by brain dynamics to achieve better out-of-distribution generalization, a key challenge in real-world applications like robotics. Ramin Hassani explained their research into "Liquid neural networks" aims to mimic how neurons exchange information, potentially unlocking greater capabilities than traditional artificial neural networks.
Liquid AI's foundational research into neural networks was inspired by studying the nervous system of a specific worm, which, with only 300 cells, exhibits complex sensory reactive behavior. Ramin Hassani noted that this biological model offered a more manageable starting point for applying learning theory to systems with far fewer neurons than traditional artificial networks.
Liquid AI has achieved significant market validation, holding the fifth position on the Hugging Face US downloads leaderboard and securing notable partnerships with Shopify and Mercedes-Benz. These achievements underscore the company's progress in commercializing its efficient AI technology.
Thomas Von Schamer, co-founder of Neural Concept, discussed how AI is revolutionizing automotive design. Traditional methods involving physical prototypes and lengthy computer simulations are being replaced by AI models that can provide results in minutes, allowing for the testing of thousands of designs per day. This accelerates product cycles and enables greater innovation.
Thomas Von Schamer of Neural Concept explained that the AI-driven acceleration of engineering processes is a pattern seen across various domains, similar to protein folding. Neural Concept is developing specialist models for physics-based simulations like aerodynamics, crash safety, and thermal management, aiming for general-purpose foundation models for engineering.
Neural Concept's AI models enable manufacturers like Jaguar Land Rover to perform aerodynamic testing on over 1,000 designs per day. This contrasts with traditional methods that were limited to far fewer iterations due to the lengthy simulation times. The AI approach significantly boosts efficiency and allows engineers to explore a wider design space.
Thomas Von Schamer anticipates that the integration of AI into engineering will lead to faster product cycles and an explosion of new form factors with improved quality and efficiency. He likens the pattern of AI development to protein folding, where complex problems are solved more rapidly through data and learning.
Mercury has introduced a new conversational interface called 'Command,' designed to provide natural language access to financial data and enable AI agents to perform actions. This aims to overcome the limitations of traditional banking interfaces for AI integration, offering a more seamless and secure way for businesses to manage their finances.