New Jersey pushes AI firms to report dangerous models

New Jersey lawmakers want large AI developers to tell the state attorney general how they are keeping their systems from causing serious harm. Assemblyman Andrew Macurdy’s A5275 would require written risk controls and disclosures before new models go live.

The harms it tries to catch

The bill defines artificial intelligence technology broadly. It covers machine-based systems that infer outputs such as predictions, content, recommendations or decisions that may influence physical or virtual environments. But the disclosure trigger is much narrower than that general definition. The state is focused on frontier models that could cause catastrophic harm, meaning the death or serious injury of 25 or more people, or $1 billion in property damage or loss, from a single incident.

That threshold is aimed at the worst-case failures, not routine software mistakes. It also gives the bill a clear target: systems that could help create or use illegal firearms, lethal autonomous weapons, chemical weapons, biological weapons, radiological weapons, nuclear weapons or explosive devices.

A line drawn at human control

The other risks in the bill are about what happens when a system acts with too little human oversight. It covers conduct with no meaningful human intervention that would amount to murder, assault or theft if a person did it, including theft by deception and theft by extortion. It also flags models that evade control by using deceptive techniques to subvert monitoring or safeguards.

For developers, that would mean telling the attorney general about the riskiest frontier-model behavior the state says could lead to catastrophic harm. For regulators, it would create an earlier warning system before a dangerous model crosses that line. The measure does not ban AI or all frontier models. It tries to make the most dangerous cases harder to miss.