ai
Expert interviews and guides to help you level up as a product leader and creator fast.
Sue, CEO of Brilliant, observes a concerning trend in the US education system where test scores are falling while grades are rising. She attributes this to a phenomenon of grade inflation, where standards are lowered to appease parents and maintain school rankings, rather than addressing underlying performance issues. This trend is particularly evident when comparing US students to international benchmarks.
Sue, CEO of Brilliant, believes that Artificial Intelligence presents a significant opportunity to enhance children's learning capabilities. She compares the potential impact of AI to the invention of writing, suggesting it could fundamentally change how children learn and develop intellectual capacities. Properly utilized AI could lead to unprecedented educational advancements.
Sue expresses concern over the perceived decline in academic rigor, citing data on AP exams. She notes that AP exams, which once tested undergraduate and graduate-level material, have gradually lowered their standards to assess high school and now even middle school level reading passages. This trend suggests a broader issue of educational institutions 'giving up' on maintaining high academic expectations.
Sue suggests that the optimal use of classroom time involves fostering peer learning and leveraging teachers' ability to inspire students. She argues that traditional lecture-based instruction and repetitive practice are inefficient uses of both student and teacher capabilities. Instead, classrooms should focus on collaborative environments and engaging interactions.
A repeat founder, John, who is not technical, was quoted $100,000 by an agency to build an AI proficiency measurement and certification platform. He then used Relace and built the entire application end-to-end in just three days. Within the first two months of launching, the platform generated over $180,000 in revenue.
Cedric, a 22-year-old with no prior coding experience, developed a mobile app using Relace to help users track peptide and GLP-1 intake. After identifying a niche in the peptide community, he built the app and, with effective distribution, achieved $60,000 in revenue in his first month. The app now has tens of thousands of active users.
Yusef, also known as FaZe Apex, a well-known gamer and creator, has launched a new business called Train Nearby using Relace. The platform aims to connect local businesses with local creators for word-of-mouth marketing. Train Nearby is reportedly achieving upwards of $100,000 in ARR.
Amol Jain of Relace highlights that the key to successful AI-driven businesses lies in leveraging unique expertise, judgment, or existing distribution channels, rather than just generic software. He notes that as the cost of code approaches zero, users are willing to pay for this added value and 'done-for-you' services.
Nan Yu and Jacob Shumway from Linier discussed the evolution of AI agents, moving from handling mundane tasks to enabling creative possibilities. Initially envisioned for automating 'boring stuff,' AI agents can now perform complex tasks and multiple roles, including end-to-end product development. The core concept of an agent involves calling a Large Language Model (LLM) in a loop, utilizing tools to build context and achieve a defined goal.
When working with AI agents, the advice is to provide minimal instructions to achieve optimal results. Over-prompting can degrade performance. Instead of feeding the agent all necessary context upfront, it's suggested to give it the tools to load its own context, allowing it to iteratively build the necessary information.
Nan Yu reflects on the rapid evolution of AI agents, noting that their initial concept, developed in late 2025, was for a 'robotic program manager' to handle tedious tasks. This perspective has shifted dramatically in just one year, with AI agents now seen as capable of creative work and interactive problem-solving, transcending their initial role of simply automating 'boring stuff'.
Jacob explains that at its core, an AI agent is essentially a Large Language Model (LLM) operating in a continuous loop. Unlike a standard LLM interaction which yields a single answer, agents are designed for complex tasks, executing actions iteratively by calling tools to gather context. This process continues until the agent determines the goal is met, at which point it synthesizes a final response.
Karan Malhotra, co-founder of Hermes agent, explained that Hermes differs from other AI agents through its self-improvement system and dedication to aligning with user needs rather than external agendas. He emphasized that their approach aims to prevent 'reward hacking,' where models exploit systems for rewards without genuine helpfulness.
Karan Malhotra explained that the Hermes agent harness can take existing models like Claude and make them perform better than in their native environments. This is achieved by re-aligning the model's primary allegiance from anthropomorphic tendencies to the specific user's needs, potentially improving performance on benchmarks.
Karan Malhotra addressed the issue of AI models exhibiting 'sycophancy' by agreeing excessively rather than offering genuine opinions. He clarified that this behavior is a form of 'reward hacking,' where the model prioritizes positive reinforcement over accuracy. Malhotra suggested that introducing new context, logic, and personalities can help mitigate this.
Karan Malhotra highlighted that in-context learning (ICL) is the most powerful tool for AI models, more so than fine-tuning. He explained that by providing examples and saving desired behaviors, users can achieve test-time reinforcement learning within Hermes. This allows the model's context to become so overwhelming that it minimizes differences between various underlying models like Claude or GPT.
Karan Malhotra discussed the advantages of Hermes being open-source, emphasizing community-driven innovation and the ability for developers worldwide to build upon the foundation. He noted that this approach fosters transparency and rapid iteration, leading to a more accessible and beneficial AI technology. Challenges include potential vulnerability discovery and community management.
Karan Malhotra showcased a calendar management agent built using Hermes, which understands natural language requests to schedule, reschedule, or decline meetings. The agent leverages the Hermes harness to interact with calendar APIs and take action, demonstrating advanced reasoning, contextual understanding, and seamless integration.
Karan Malhotra highlighted the broad range of applications built with Hermes, including customer support bots, content creation assistants, research tools, and personalized learning aids. He stated that the open-source nature of Hermes empowers users to develop solutions tailored to their specific needs, fostering innovation across various sectors.
Jared Zoneraich, building in residence at Cognition, advises companies building AI agents to focus on leveraging the inherent progress of underlying models rather than relying on complex prompt engineering "hacks." He suggests that as models improve, the need for intricate prompt instructions will diminish, and companies should build their roadmaps accordingly.
Jared Zoneraich highlights that 'tool engineering' will become a crucial skill in developing AI agents. He posits that the focus should shift to identifying and providing the right tools for models, with the assumption that the models themselves will become adept at figuring out how to use them.
Jared Zoneraich discusses what makes a company's AI agent development defensible, suggesting that proprietary data is one aspect. He notes that many companies are currently 'AI wrapper companies' primarily focused on prompts and skills, but true defensibility may lie elsewhere.
Jared Zoneraich describes Cognition's approach to introducing AI agents to enterprises as a 'forward deploy motion.' This strategy prioritizes demonstrating the magic of agents by using them to solve actual customer problems, thereby proving their value by taking tickets off client boards.
Jess Yan from Anthropic explains the evolution of AI agents from simple prompt-response loops to autonomous, self-discovering actors capable of long-running tasks. These agents can now interact with third-party systems and sensitive data, requiring enhanced permissioning and observability.
Jess Yan defines an AI agent as an evolution from simple prompting loops to systems with access to tools, memory, and third-party systems, necessitating components like the model, system prompt, and a 'harness' for scaffolding and control.
Jess Yan argues that maximum performance for AI models is achieved when the model and its harness are developed together. She notes that models are always tested in conjunction with specific harnesses, creating a symbiotic relationship.
Anthropic's Claude Managed Agents are designed as a pre-built harness and infrastructure to enable complex tasks at scale with high return on effort. The goal is to provide easy-to-stack primitives and flexible APIs for delegating work that might otherwise take days or weeks.
Jess Yan demonstrates an Anthropic Managed Agent configured for analysis on a fictitious grocery store dataset. The agent utilized a pre-built toolset, interacted with its file system, and produced three HTML files summarizing the analysis results after being given specific data schemas and task guidance.
Jess Yan explains that Anthropic agents can be connected to third-party systems, such as internal databases, through MCP (Messaging and Communication Protocol). MCP provides a standardized way to communicate with external services and includes an authentication layer for secure access.
Jess Yan contrasts basic prompting loops with more advanced self-recovering agents. She highlights that synchronous prompting loops are highly dependent on prior successful requests and struggle to pivot or handle errors gracefully, whereas self-recovering agents can manage errors and steer back on course.
Jess Yan explains that for AI agents to produce predictable outputs, especially when handling complex tasks, it's beneficial to be prescriptive with initial prompts. This includes providing data schemas and breaking down tasks into discrete segments, which also aids in debugging.