In a daylong hearing in New York City, lawmakers grilled representatives from major artificial intelligence firms regarding catastrophic safety risks, emerging empty-handed as corporate officials declined to quantify worst-case scenarios or confirm whether safety tests could halt a failing model, The New York Times reported.
AI Developers Refuse to Quantify Catastrophic Safety Risks
- Major artificial intelligence developers—including OpenAI, Anthropic, Google, and Meta—refused to quantify the probability of catastrophic safety failures during testimony before the New York City Council.
- City Council speaker Julie Menin advanced local legislation that would require independent reviews and a manual human shutoff mechanism for any commercial A.I. model sold within the municipality.
- Whistle-blowers like former Anthropic researcher Jacob Coxon warned lawmakers that current industry acceleration patterns risk human extinction as firms prioritize rapid product deployment over rigorous containment protocols.
City Council Grills A.I. Executives Over Catastrophic Risk Metrics
Monday’s hearing laid bare a sharp philosophical divide between municipal lawmakers attempting to erect local guardrails and the corporate leadership driving the commercialization of generative systems. City Council speaker Julie Menin pressed corporate representatives to specify how their organizations measure the probability of catastrophic operational failures.
Morgan Dwyer, who runs policy development and operations for OpenAI, rejected the utility of assigning numerical probabilities to existential threats. I don’t know,
Ms. Dwyer said during the testimony. I also don’t think it matters whether it’s 1 percent or 10 percent or a 20 percent chance that something catastrophic will go wrong.
Ms. Dwyer maintained that OpenAI should refrain from training architectures that cannot remain safely under human supervision. That response drew an immediate rebuff from the dais. I’m sorry, but to say you don’t know and it doesn’t matter is flippant at best,
Ms. Menin replied.
The proceedings featured remote and in-person testimony from delegates representing Anthropic, OpenAI, Meta, and Google. Meanwhile, a subpoenaed representative for the artificial intelligence division of SpaceX, controlled by Elon Musk, failed to appear. Ms. Menin stated her intention to petition a judicial officer to enforce compliance.
| Company | Valuation / Market Context | Hearing Representation |
|---|---|---|
| Anthropic | Projections exceeding $2 trillion in some financial reports | Attended (Featured whistleblower testimony) |
| OpenAI | Preparing to be publicly traded | Attended (Morgan Dwyer) |
| Google & Meta | Attended via remote testimony | |
| SpaceX (A.I. Division) | Did not attend (Subpoenaed, judicial intervention pending) |
Whistle-Blower Warnings Contrast With Industry Growth Projections
While executives emphasized the societal utility of automated systems—pointing to potential pharmacological breakthroughs, productivity enhancements, and infrastructure protection—dissenting voices from former industry insiders painted a far grimmer picture. Jacob Coxon, who recently resigned from his position as a researcher at Anthropic, warned that developers are rushing deployment cycles without mastering fundamental containment controls.
We do not know how to control any A.I. system yet,
Mr. Coxon told the Council, weeks after sparking intense regulatory scrutiny regarding the technology’s long-term trajectories. He added that under current trajectories, humanity loses control of these A.I.s and it could end in human extinction.
Mr. Coxon characterized the prevailing corporate culture as dangerously casual, pointing to a technology sector ethos that prioritizes rapid iteration over systemic risk management. Part of it is the culture,
he noted. The companies run on a start-up mind set: move fast, break things, fix them later. That works for a photo-sharing app. It does not work for building the most powerful technology ever built.
Several of the participating firms are preparing for public market listings, with valuation projections for Anthropic alone crossing the $2 trillion threshold in some financial reports.
New York Lawmakers Pursue Municipal Legislation Amid Federal Inaction
Confronted with what municipal leaders termed an stonewalling corporate posture, the New York City Council is advancing a legislative package designed to restrict unverified models within city limits. One proposed statute would prohibit the marketing or sale of any artificial intelligence model that lacks independent validation or a fail-safe manual shutdown mechanism operated by a human.
A second legislative measure would obligate municipal agencies to draft comprehensive emergency response frameworks to counter potential technological disruptions to critical urban infrastructure and public health systems. Mayor Zohran Mamdani voiced support for a coordinated response, noting that his administration instituted a moratorium on generative tools for younger students while restricting deployment among high school cohorts.
Following the session, Ms. Menin reiterated her frustration with the corporate delegations. None of them would quantify the risk,
she said. How can you put on safety guardrails if you can’t even quantify, well, what exactly is the risk?
With the resignation of the city’s A.I. director still unaddressed by the Office of Technology and Innovation, municipal leaders face mounting pressure to establish oversight while federal regulators deliberate broader statutory frameworks.
Disclaimer: The information provided in this article is for educational and informational purposes only and does not constitute financial advice.
