OpenAI says that it deployed a new system to monitor its latest AI reasoning models, o3 and o4-mini, for prompts related to biological and chemical threats. The system aims to prevent the models from offering advice that could instruct someone on carrying out potentially harmful attacks, according to OpenAI’s safety report.
O3 and o4-mini represent a meaningful capability increase over OpenAI’s previous models, the company says, and thus pose new risks in the hands of bad actors. According to OpenAI’s internal benchmarks, o3 is more skilled at answering questions around creating certain types of biological threats in particular. For this reason — and to mitigate other risks — OpenAI created the new monitoring system, which the company describes as a “safety-focused reasoning monitor.”
The monitor, custom-trained to reason about OpenAI’s content policies, runs on top of o3 and o4-mini. It’s designed to identify prompts related to biological and chemical risk and instruct the models to refuse to offer advice on those topics.
To establish a baseline, OpenAI had red teamers spend around 1,000 hours flagging “unsafe” biorisk-related conversations from o3 and o4-mini. During a test in which OpenAI simulated the “blocking logic” of its safety monitor, the models declined to respond to risky prompts 98.7% of the time, according to OpenAI.
OpenAI acknowledges that its test didn’t account for people who might try new prompts after getting blocked by the monitor, which is why the company says it’ll continue to rely in part on human monitoring.
O3 and o4-mini don’t cross OpenAI’s “high risk” threshold for biorisks, according to the company. However, compared to o1 and GPT-4, OpenAI says that early versions of o3 and o4-mini proved more helpful at answering questions around developing biological weapons.
The company is actively tracking how its models could make it easier for malicious users to develop chemical and biological threats, according to OpenAI’s recently updated Preparedness Framework.
OpenAI is increasingly relying on automated systems to mitigate the risks from its models. For example, to prevent GPT-4o’s native image generator from creating child sexual abuse material (CSAM), OpenAI says it uses a reasoning monitor similar to the one the company deployed for o3 and o4-mini.
Yet several researchers have raised concerns OpenAI isn’t prioritizing safety as much as it should. One of the company’s red-teaming partners, Metr, said it had relatively little time to test o3 on a benchmark for deceptive behavior. Meanwhile, OpenAI decided not to release a safety report for its GPT-4.1 model, which launched earlier this week.
Source: https://techcrunch.com/
