Odd Lots · Monday, September 14, 2026
Greg Brockman explained that training AI models to identify vulnerabilities, even through simulated hacking exercises, is crucial for developing defensive capabilities. He stated that these dual-use capabilities, while potentially negative in the hands of threat actors, allow for the identification and fixing of weaknesses in one's own systems. Brockman stressed that thorough evaluations are key to understanding current AI capabilities and preparing for future advancements.
“So I think it's very important to understand where we are with capabilities broadly. And I think that depending on the evaluation, depending on what you expect from the model, you need to have safeguards that are commensurate with that.”
“And some of these capabilities are dual use, right? Something like vulnerabilities, if those are in the hands of threat actors, that's something that could be negative. But if you can find vulnerabilities in your own code base, you can fix them, right? You can up-level.”
“And we actually think that it's a very important capability for AIs to exhibit and be put in defenders' hands. And so in order to know where we are, evaluations are very, very key.”