Software Engineering Daily · Tuesday, August 11, 2026
Investigating AI security incidents can be complicated by the safety guardrails built into models like Claude and GPT, which prevent them from being used for simulated malicious activities. This limitation forced incident responders to use open-weight models from China for forensics, highlighting a trade-off between safety and investigative utility.
“They couldn't actually use Claude or GPT because those models have safety guardrails in place where you can't tell whether, are you an incident responder or a hacker?”
“So what they ended up using an open weight model out of China for the forensics.”
“But then when you want to use it for something intentional like incident response, you might not be able to do that because the protections are in place.”