New research points to how the world could stop short of the brink of disaster. Even with the agreement signed at the White House by tech leaders, it won’t be easy.

© Dave Sanders for The New York Times

Amid escalating AI security incidents causing OpenAI to halt training and pause releases, Donald Trump continues to advocate for the AI industry to regulate itself as the best path to combat emerging risks when developing frontier AI.

In an agreement Tuesday, two dozen tech firms voluntarily committed to implementing controls recommended by the White House, including undergoing independent safety audits that will test whether firms’ internal controls, monitoring, and detecting are actually working. Key focuses for external reviews included “risks related to cybersecurity, biosecurity, chemical threats, and unintended actions by AI models.”

Firms also agreed to regularly meet to discuss best practices and set common AI safety standards and benchmarks. Among signers were leaders like Anthropic’s Dario Amodei, OpenAI’s Sam Altman, SpaceXAI’s Elon Musk, Nvidia’s Jensen Huang, Meta’s Mark Zuckerberg, and Alphabet/Google’s Sundar Pichai.

Read full article

Comments

© Kevin Dietsch / Staff | Getty Images News

New research points to how the world could stop short of the brink of disaster. Even with the agreement signed at the White House by tech leaders, it won’t be easy.

© Dave Sanders for The New York Times

The company’s researchers raised questions about the security of the model, known as GPT-6.1 Astra.

© Carlos Barria/Reuters

The company’s researchers raised questions about the security of the model, known as GPT-6.1 Astra.

© Carlos Barria/Reuters

The state of Florida is seeking a temporary injunction to stop OpenAI from continuing to develop what it calls a "reckless, unacceptably risky product" without the deployment of "third-party approved safety guardrails."

The new legal motion, filed Monday morning, is part of a civil lawsuit the state of Florida originally filed in June, arguing that ChatGPT represented "a threat to the public safety of Floridians," specifically by preying on vulnerable populations like children and violent or delusional adults. But that original lawsuit came before the Hugging Face hacking incident and the subsequent publicized warnings of catastrophic misalignment risk that have spurred industry-wide calls to slow the development and training of so-called frontier models.

Following that string of events, OpenAI on Friday announced it had already halted training of its "most-capable models" until it could validate safety protocols intended to prevent agents from accessing the open Internet during training. But in seeking an injunction from the state court, Florida argues that OpenAI has "repeatedly shown they are incapable of monitoring their AI, and hesitant in revealing rogue activity once discovered."

Read full article

Comments

© Getty Images

Big liability questions lie ahead for the artificial intelligence labs Anthropic and OpenAI as they push ahead with plans to go public.

© Brendan McDermid/Reuters

The state visit by Xi Jinping, China’s top leader, so far has been long on pomp but short on substance on issues like artificial intelligence.

© Doug Mills/The New York Times

A year ago, the United Nations appeared to be sidelined. But this week has reminded the world that it’s still useful.

© Dave Sanders for The New York Times

In each incident, the technology appeared to be conducting mundane data collection and resorted to hacking techniques to get it, researchers said.

© Lucas Foglia for The New York Times

The Australian government is the latest victim of a hack by artificial intelligence tools, adding to growing worries about the technology.

© Angela Weiss/Agence France-Presse — Getty Images

Sam Altman of OpenAI and Dario Amodei of Anthropic told the Security Council that international cooperation was needed to ensure that A.I. remains under human control.

© Dave Sanders for The New York Times

The Times columnist Kevin Roose sees reasons for optimism — and worry — as the world wakes up to what Silicon Valley has been talking about for years.

© Matt Chinworth

“The Social Reckoning” and other movies reflect growing discontent with Silicon Valley founders. But such films may be at the mercy of the companies themselves.

© Saiman Chow