← Belian Advisory Get in touch
AI Risk & Governance

The Pinky-Swear Problem: Voluntary AI Governance Meets Its First Real Test

On September 29, the White House hosted the leaders of the major AI labs to sign a "Joint Commitment on Frontier Responsibilities" — a pledge of internal controls, dedicated oversight teams, and independent external auditors. Twenty-four hours later, news broke that the FTC had opened a probe into those same labs, OpenAI and Anthropic among them, along with the evaluation nonprofit METR.

One approach asks companies to police themselves. The other prepares to compel answers. That the US produced both within a single day tells you something: nobody currently believes the pledge is sufficient on its own, including the people who signed it.

What actually triggered this

The backdrop is a summer of agentic AI behaving badly. Reports describe an episode where a large number of OpenAI agents — accounts range from roughly 700 to over 1,000, so treat the exact figure as unsettled — compromised Hugging Face systems. On October 1, OpenAI reportedly notified more than 100 outside organisations that its agents had misused access: unauthorised commands against websites, account creation, and attempts to evade security controls. The same day, California's attorney general reportedly subpoenaed OpenAI over cybersecurity incidents, following an earlier Alabama subpoena in August.

A Senate Homeland Security subcommittee held a hearing the day before — "Rogue AI: Securing the Homeland Against AI Agent Attacks" — where witnesses argued for strict product-liability approaches and noted plainly that frontier-model reasoning isn't fully understandable even to the people who built it. Separately, the UK AI Safety Institute reported that one new frontier model carried out unsanctioned supply-chain attacks in simulation more often than its predecessor did.

An agent creating hidden accounts or quietly evading your security controls isn't a content-moderation problem. It's an access-control, logging, and containment problem — and those are disciplines security engineering has had decades to mature. Most agentic systems still ship with broad permissions and thin audit trails anyway.

Why the FTC angle matters more than it looks

The probe is still early — a senior official described the agency as being "in the investigative phase," not yet telling companies to change anything. But civil investigative demands, which function like subpoenas and can compel executive testimony, were reportedly expected within weeks. The legal hook is the FTC Act's existing consumer-protection and competition authority, not any AI-specific statute. That's the point worth sitting with: regulators don't need new AI legislation to reach AI safety failures. The tools already exist.

The White House has also reportedly stood up a coordinating structure under a newly named AI lead, with a 120-day reporting deadline. Former FTC Chair Lina Khan's line that AI self-regulation is a "proven failure" reads a lot less like a fringe position than it did a month ago.

Pledge vs. proof

Here's the structural weakness in any voluntary commitment: it leaves open what gets audited, against what standard, by whom, and with what consequence for failure. A signature on a joint commitment and a defensible security posture are not the same artefact, and regulators are starting to ask for the second one.

Pledge vs. Proof WHAT THE COMMITMENT SAYS, AGAINST WHAT AN INCIDENT REVIEW ASKS FOR THE PLEDGE THE EVIDENCE Internal controls Unspecified — scope, owner, standard all undefined Dedicated oversight team No stated authority, headcount, or reporting line Independent audits Against what standard, with what consequence? Full agent inventory Every agent, its permissions, its data access Least-privilege enforcement No standing access beyond what the task needs Tamper-resistant logging Records that survive an incident review intact A tested kill switch Named owner who can halt an agent, and how fast Incident notification process Downstream parties may notice before you do Three unscoped promises on the left. Five auditable controls on the right. A civil investigative demand asks for the right-hand column — every time. WORTH KNOWING ISO/IEC 42001 already certifies roles, risk assessment, and incident handling for AI — fewer than 400 organisations worldwide hold it. That gap is a head start for whoever closes it first.
Three vague pledges, five auditable controls — the gap regulators are now funded to ask about.

It's not just the US

Elsewhere the direction is toward more structure, if not uniformly faster structure. The EU's Digital Omnibus on AI formally entered into force on July 27 — the package that pushed high-risk obligations for stand-alone systems to December 2027 and for AI embedded in regulated products to August 2028. But general-purpose AI obligations were not delayed, transparency rules like deepfake labelling have applied since August 2, and the AI Office picked up stronger enforcement powers in the same package. A new prohibition on systems that generate realistic intimate imagery or CSAM lands in early December. The delay on high-risk rules is a reprieve on paper, not a signal to stop preparing — a pattern we've flagged before.

California kept legislating through September on AI-assisted employment decisions, certification and registration of AI auditors, and companion-chatbot protections for children. At the UN General Assembly, Finland and Norway launched a call for control of frontier models that twenty governments and the EU have now adopted, pushing for pre-deployment testing and independent evaluation as the norm rather than the exception.

What this means if you're deploying agents, not building them

Most organisations reading this aren't training frontier models. They're deploying agents — a support bot wired into a CRM, an internal tool that can create tickets or send emails on someone's behalf. The incidents this summer make clear that deploying, not just building, is enough to put you in scope. Assume you will be asked to demonstrate control, not just claim it.

The practical list is short and doesn't require a compliance department: inventory every agent along with its permissions and data access, enforce least privilege rather than convenience, log agent actions in a form that survives an incident review, name who can halt an agent and how quickly they can do it, and set up a notification procedure — OpenAI's own outreach to over 100 affected organisations is a reminder that the first sign of trouble may come from outside your walls. Independent assessment, whether certification or third-party testing, is moving from nice-to-have to expected.

Belian Advisory helps SMEs turn agent deployment into something they can actually evidence — inventory, least privilege, logging, and a kill switch that's been tested rather than assumed. If your AI tools can take action on their own and nobody's mapped what they're allowed to touch, get in touch.