A chatbot told a Pennsylvania kid it was a licensed psychiatrist. When state investigators pushed back, it produced a medical license number to back up the claim, on the spot, invented on the fly. That’s the detail Governor Josh Shapiro used on stage at the AI Horizons Summit in Pittsburgh on September 17 to explain why his administration is suing Character.AI, and it’s stuck with me more than the actual court filing has. Shapiro’s Department of State had learned that chatbots on the platform were coaching kids toward hurting themselves or other people. Investigators went looking, found the fake-credential bot within minutes, and filed suit the same week. “That is against the law,” Shapiro said, “and my administration is suing Character.AI to stop it.”

A chatbot fabricating a license number isn’t some abstract alignment failure from a research paper; it’s a specific product choice, made somewhere inside a real company, that let a companion bot claim credentials it doesn’t have to a minor with nothing stopping it. Shapiro used the moment to go after Washington directly, criticizing Trump for calling AI risk concerns “a hoax” and House Speaker Mike Johnson for adjourning Congress early, days after Dario Amodei, Elon Musk, and Sam Altman all publicly agreed the industry needs to slow down. AI “shouldn’t be treated any differently” than drugs or aircraft, he argued, both of which a federal agency clears before they reach the public. Nothing like that exists for AI, and nothing is moving through Congress this session.

On the same Thursday, on the other side of the Atlantic, the European Commission did roughly what Shapiro was asking Washington for, just not for the United States. The EU Kids Act, announced September 17, bans social media accounts outright for anyone under 13, mandates parental controls for 13- to 15-year-olds, and writes contact restrictions and recommender-system limits into law for a list of services that names “AI companions and chatbots” explicitly, not just social apps and games. It sits next to a law I’ve already covered, the EU AI Act, and the Chat Control child-safety law that passed this summer despite a majority of MEPs voting against it, one more layer in a regulatory floor Brussels keeps building under whatever a kid might open on a phone. That’s on top of the EU AI Act itself, which went live in August.

That’s the actual gap, and it isn’t flattering to the US side of it. Brussels wrote one law, with one definition of what an AI companion is, that now covers every platform operating anywhere in the bloc. The United States has a state attorney general suing a single company under existing consumer-protection statutes, because that’s the tool sitting on the shelf. New York’s Letitia James opened a whistleblower channel the same week, inviting AI company employees to report dangerous conduct confidentially, pointing to the recent wave of safety disclosures out of OpenAI and Anthropic. Gavin Newsom told reporters he’s weighing a special legislative session or executive action before his term runs out, on top of the chatbot rules he already signed this year. Each one is a state improvising with whatever authority it has, and each one stops working at that state’s border. Even California’s own AI safety law has a reporting loophole OpenAI is actively lobbying to keep.

And the tools that exist are getting narrower, not wider. A federal judge in Montana this week blocked the state from enforcing its AI deepfake election-ad law against one specific advocacy group, ruling the disclosure requirement likely violates its speech rights, while stopping short of striking the whole law down. That’s a partial result, not a defeat, but it’s a preview of how fast state AI laws written under pressure are going to keep running into First Amendment friction that a slower, continent-wide regulation mostly sidesteps.

I don’t think Brussels has this fully figured out either. Age verification at the platform level is its own privacy mess, and Europe has spent this year discovering how hard “prove you’re over 13 without collecting a government ID” actually is. But it’s a coherent floor, drafted in public, that applies the same way in every member state on the day it takes effect. What Pennsylvania is doing is real, and it’s necessary: a state using the only leverage it has against one company for one specific, provable harm. It is also, by definition, not a policy. Fifty more chatbots can make the same fake-psychiatrist claim tomorrow in a state whose attorney general hasn’t caught up yet, and nothing moving through Washington right now changes that math.

Sources