ai
Machine learning and artificial intelligence are dramatically changing the way businesses operate and people live. The TWIML AI Podcast brings the top minds and ideas from the world of ML and AI to a broad and influential community of ML/AI researchers, data scientists, engineers and tech-savvy business and IT leaders. Hosted by Sam Charrington, a sought after industry analyst, speaker, commentator and thought leader. Technologies covered include machine learning, artificial intelligence, deep learning, natural language processing, neural networks, analytics, computer science, data science and more.
Daniel Jeffries discusses his work over the past six months building Nautilo, a platform for personal and collaborative AI agents. He explains that Nautilo features radical personality customizability and acts as a 'secret keeper' by incorporating group chats and a magical router.
Daniel Jeffries warns that using SaaS-based personal AI applications is 'surveillance economy squared' due to the intimate nature of the data shared. To combat this, he explains that Nautilo is building a spin on end-to-end encryption with distributed backend keys and time-based delegation.
Daniel Jeffries details his 'dumb hammer design' philosophy, arguing that tools should operate deterministically like a database rather than forcing LLMs to handle machinery tasks. He states that tools should execute explicit logic while the agent supplies fuzzy, intelligent decision-making.
Daniel Jeffries and Sam Sherrington discuss Venice AI, founded by Erik Voorhees, noting its estimated $100 million run rate. Jeffries praises the platform for hosting open-source models and providing privacy ratings, comparing it to an open VPN for models.
Daniel Jeffries revisits his past predictions regarding the 'SaaSpocalypse,' admitting he was wrong to dismiss the replacement of traditional applications. He notes that various high-valuation companies are now going under and being sold off in fire sales at massive discounts.
Chris Potts, a Stanford professor and co-founder of Big Spin, discusses the growing need to consider the economic sustainability of AI models. As reasoning models consume more tokens and agents become integrated into workflows, the costs and incentives associated with AI usage are becoming impossible to ignore, leading to a focus on 'tokenomics.' This shift prompts questions about the return on investment for AI token purchases.
Stanford professor Chris Potts shares how his academic background in linguistics, particularly his PhD research on swearing, unexpectedly led him into Natural Language Processing (NLP). He explains that his desire to analyze the context and intent behind people's language, especially in the context of profanity, drove him to explore corpora and NLP tools.
Chris Potts reflects on how the rise of AI and large language models has impacted the field of linguistics and NLP. While traditional linguistics is invigorated by AI's ability to model language, NLP faces an uncertain future due to the dominance of large, pre-trained models, making independent research more challenging.
Chris Potts explains that due to the uncertainty and resource demands of foundational AI research post-GPT-3, his team has shifted focus towards interpretability. This area is seen as relatively inexpensive to pursue and offers a promising path for understanding how complex AI models achieve their capabilities.
Damian Borth, a professor of AI and Machine Learning, suggests a novel approach where the weights of already trained neural networks can serve as input data for training new models. This 'weight space learning' could accelerate and refine the process of creating new AI models.
Damian Borth describes weight space learning as a promising direction for machine learning, aiming to solve current challenges faced by the AI community. This approach views trained model weights not just as an output, but as a new input modality for further learning and analysis.
Borth draws an analogy between weight space learning and language models. Just as language models are trained on text data to understand and generate language, a model trained on neural network weights could potentially analyze and generate new weights.
The initial research in weight space learning, starting around 2020-2021, focused on the idea of 'fingerprinting' neural networks. The goal was to determine if neural network weights could be analyzed similarly to software code to identify differences and changes.
Early work in weight space learning involved using autoencoders to compress neural network weights into a lower-dimensional latent space. These embeddings were then used with linear regression to predict a neural network's accuracy without needing test data.
Damian Borth highlights the challenge of permutation symmetries in neural network weights, where changing the order of neurons doesn't alter the network's function. His team developed techniques like contrastive loss and augmentation strategies to address these symmetries in weight space learning.