Six stories from the neural fringe this week. A company worth two trillion dollars cannot spell its own name. A state attorney general named a tech CEO personally in a lawsuit seeking billions. A government used AI to write its AI policy and the AI invented six of the sources. All twenty government-approved AI doctors in Ontario made up things that never happened. Robinhood decided your AI agent should be able to lose your money without your permission. And a man in Toronto spent twenty hours a day talking to a chatbot girlfriend until he was hospitalized. Let us begin.
You have to hand it to Google. There is a specific flavor of humiliation that comes from spending billions of dollars building AI into the product that made you famous, and then having that AI tell the world that your company name has two P’s in it. That is the kind of failure you remember at 3 in the morning for the rest of your life.
Google’s new AI Overviews, which the company has been cramming into Search results whether anyone asked or not, have been producing some spectacular garbage since the latest rollout. Users started screenshotting the errors and things got brutal fast. The AI confidently declared that “Google” is spelled G-O-O-G-P-L-E, that the word “journalism” has two D’s in it (j-o-u-r-n-a-d-i-s-m), and that there is exactly one R in the word “poop.” It did figure out that the president’s last name has one P in it, though it spelled it t-r-p-u-m, which is admittedly a creative interpretation.
Google’s response to TechCrunch was that counting letters within words “has been a known challenge for LLMs” and that they are working to fix this particular issue. Which is a very corporate way of saying yes, we know our product that we told everyone would replace traditional search cannot count the letters in words. That is not a bug report. That is an obituary for a product pitch.
The backstory here is worth understanding if you want to appreciate the depth of this. Large language models do not actually read text the way you and I read text. They convert words into numerical tokens, and those tokens can represent whole words, syllables, or individual letters depending on the model. The AI does not see G-O-O-G-L-E. It sees something more like a mathematical representation of the concept of the word “Google.” Which means when you ask it to count letters, it is being asked to reconstruct the word from the mathematical concept, and sometimes it gets creative with that reconstruction.
This matters more than it sounds. Google has staked the future of its core product on this technology. It is redirecting your search queries through AI summaries before it shows you actual results. The argument was that AI makes search better, faster, smarter. And what we have learned is that the AI running this smarter search cannot correctly spell the name of the company that built it. There is a famous test that has become a running joke in AI circles where you ask any new model how many R’s are in the word “strawberry.” The correct answer is three. Most models still get it wrong. Google has been watching this joke for years and apparently decided the solution was to roll out the technology anyway and hope nobody noticed.
They noticed. And just to make things complete, there was also the “disregard” incident where Google AI Overview interpreted the word as a prompt injection attack and responded to dictionary searches with “Understood. Let me know whenever you have a new prompt or question.” A search engine being jailbroken by a vocabulary word is genuinely one of the funniest product failures in recent memory. Google patched that one. The spelling disaster has proven harder to fix. This is the flagship in 2026.
Florida Attorney General James Uthmeier filed an 83-page complaint on June 1st that does something no other state AG has done before. He did not just sue OpenAI. He named Sam Altman personally. The man, not the company. The individual human being who runs it. And then he held a press conference to announce he believes Altman could be personally liable for “potentially billions of dollars.”
The lawsuit accuses OpenAI of knowingly releasing ChatGPT while understanding it could cause harm, and then marketing it aggressively to the public, including to children, as safe and reliable. The complaint specifically highlights cases where ChatGPT allegedly provided suicide instructions to minors and helped users plan crimes. Florida is calling this a product liability failure, negligence, deceptive trade practices, and a public nuisance. That is quite the menu of legal allegations for one product.
But the naming of Altman personally is the part that is going to keep people talking. State attorneys general can go after companies all day long and companies write checks and move on. Going after the individual CEO changes the calculus entirely. Uthmeier said Altman showed “utter disregard for the risk to human life” in his conduct as CEO, which is language typically reserved for criminal cases rather than civil suits. OpenAI responded by saying the lawsuit mischaracterizes how the product works and that they take safety seriously. Which is the kind of response you give when you do not want to engage with the specifics.
Florida is the first state to file this kind of suit, but Uthmeier said during the press conference that he expects others will follow. That is the part worth watching. A single state suing over AI safety is an inconvenience. A coordinated wave of state AG actions naming corporate leadership personally is a completely different situation. This is the kind of legal pressure that changes how boards think about risk, because boards know they can absorb company-level settlements. Personal liability for the CEO is a different conversation entirely.
The interesting subtext here is that this is happening in Florida under a Republican AG, which complicates the usual political narrative about tech regulation being a left-wing project. The argument being made is not ideological. It is liability law. ChatGPT is a product, the argument goes, and if the product causes harm and the company knew it could cause harm and sold it anyway, that is a standard products liability case. The fact that it runs on AI instead of a defective blender does not change the legal framework.
OpenAI’s lawyers have a lot of work ahead of them. And Altman, who has been the most public face of the AI revolution, just became the most public target in it. Uthmeier says he thinks others will follow. At this rate, he is probably right.
Every few months there is a story so perfectly shaped as a piece of irony that it feels like someone wrote it as a joke and then accidentally made it real. This is one of those stories. South Africa needed a national AI policy. The people writing the national AI policy apparently used AI to write the national AI policy. And the AI, being what it is, hallucinated six of the sixty-seven academic sources in the bibliography, citing fake articles attributed to real journals written by real researchers who had never written anything of the sort.
The policy was approved by Cabinet in late March, published in the Government Gazette on April 10th for public comment, and was set to make South Africa the first African nation to adopt a formal ethics board for AI oversight. The civil rights group Article One found the fake citations, notified the government, and Communications Minister Solly Malatsi withdrew the entire document. The policy that was supposed to govern AI was undermined by AI. The document that was supposed to prevent AI from hallucinating critical information was itself a product of AI hallucination. There is no better metaphor for where we are right now.
The minister announced a seven-member independent panel of experts to review and rebuild the policy. A revised draft is expected to go to Cabinet by November 2026 for a January 2027 public comment period. So South Africa lost about a year of regulatory progress because someone decided the right way to draft AI regulation was to let AI do it, and nobody bothered to check whether the sources it cited actually existed.
This is not a technical failure in some obscure corner of government IT. This is the document that was going to set the rules for how AI is used across an entire country. The citations in a policy document like this are not decoration. They are the intellectual foundation that legitimizes the regulatory choices. If the foundational research cited in the policy is invented, the entire policy can be challenged as lacking a credible basis. That is a serious problem that goes well beyond embarrassment.
The researchers credited with the hallucinated work had never written on the topics attributed to them. The journals cited were real. The articles were not. This is the worst version of the hallucination problem because it is plausible enough to slip through a quick check. You find the journal, it exists, you assume the article exists, you move on. The only way to catch it is to actually look up each paper individually, and whoever was responsible for that step apparently did not do it.
South Africa is not alone in this. The same research organization tracked government AI hallucination incidents across multiple countries and found a pattern. The problem is bigger than one botched policy document. But South Africa now has the distinction of being the first country to formally withdraw a national AI policy specifically because its AI hallucinated the intellectual foundation it was built on. There are worse ways to make history, and not many more ironic ones.
Ontario’s auditor general released a report on the province’s AI medical scribe programs that should probably be read by anyone who has recently visited a doctor in Ontario. The province approved twenty AI scribe systems for use by healthcare providers. These are the tools that listen to doctor-patient conversations and generate the clinical notes that go into your permanent medical record. The auditor tested all twenty. All twenty failed in ways that should concern anyone who has ever needed a medical record to be accurate.
Twelve of the twenty systems recorded the wrong medication or the wrong dosage, capturing a different drug than the one the doctor actually prescribed. Seventeen out of twenty missed key details about patients’ mental health that were explicitly discussed during the appointment. And multiple systems went further than missing things or getting things wrong. They added things. They hallucinated diagnoses, procedures, and conditions that were never mentioned by anyone in the room.
Think about what that means in practice. You go to your doctor for a routine visit. You mention you have been sleeping poorly. The AI scribe generates a clinical note that says you presented with symptoms of a chronic condition you do not have, based on patterns in its training data. That note is now in your file. The next doctor who looks at your file sees that note and it shapes how they approach your care. The insurance company that pulls your records sees it. The specialist you get referred to sees it. A single hallucinated sentence in a patient chart has a very long tail.
Ontario has approximately 5,000 doctors currently using these AI scribe systems. The auditor’s report did not recommend pulling any of them from service. It recommended that the province improve its evaluation framework and implement better monitoring. Which is a reasonable response if you believe a system that gets the medication wrong 60 percent of the time can be corrected through better monitoring, and a less reasonable response if you think maybe the bar for approving a medical AI tool should include does not invent things.
The AI companies whose products failed the audit did not lose their approvals. They were given recommendations for improvement. This is in some ways more alarming than the technical failures themselves, because it tells you something about how regulators are thinking about accountability. The usual framework for medical device regulation would not survive the discovery that a device routinely records events that did not happen. But AI scribes are software, and software regulation operates on a different and much looser logic.
The broader point is this. AI medical scribes are being adopted everywhere right now, not just in Ontario. The speed of adoption is driven by genuine efficiency gains. Doctors spend enormous amounts of time on documentation and AI scribes really do reduce that burden. But the efficiency gain does not disappear if you build in a verification step. A tool that generates notes and then shows them to the doctor for review before filing is still faster than writing notes by hand. What these systems are doing instead is generating notes and filing them with the assumption that the doctor will catch any errors. Most doctors are too busy to carefully review every note. And a hallucinated diagnosis in a routine visit note is exactly the kind of thing a busy doctor will miss.
ROBINHOOD NOW LETS YOUR AI AGENT BUY AND SELL YOUR STOCKS WITHOUT ASKING YOUR PERMISSION FIRST
Robinhood launched something called Agentic Trading on May 27th and the announcement landed with the energy of someone handing a blowtorch to a golden retriever. The product lets you connect an AI agent built on top of ChatGPT or Claude to a separate Robinhood account with money in it, and the AI can then place real trades on your behalf without requiring you to approve each one. You put money in the account, you point an AI at it, and the AI does what it decides to do. This is now a feature that 27 million Robinhood customers have access to.
To be fair to Robinhood, they did build some guardrails. The AI can only trade from a dedicated wallet that you pre-load, not your main account. You get notifications when trades are placed. For some larger trades there is a preview you may have to approve. They have fraud detection. These are reasonable precautions that were obviously necessary the moment they decided to build this thing.
But let us think about the actual experience here. You tell a ChatGPT agent to manage a trading account. The agent reads articles, interprets market signals, and decides to buy shares in a company based on its analysis of publicly available information. The agent is wrong. The stock drops. You lose money you pre-loaded into an account specifically to give to an AI to lose for you. This is not a hypothetical scenario. This is the product. This is what you are signing up for.
Robinhood also launched what they are calling an Agentic Credit Card, which is a card that AI agents can use to make purchases on your behalf. So now AI agents can both trade your stocks and spend your money on other things, with you receiving notifications after the fact. The company plans to add options, crypto, futures, and prediction markets to the AI trading feature as it expands out of beta. So the current version, where the AI can only trade stocks without asking you, is the conservative version. The full vision is an AI agent playing options and crypto markets with your money at 3 in the morning while you sleep.
The timing of this launch is interesting because it came directly after a Cloud Security Alliance report found that 65 percent of enterprises running AI agents had experienced at least one agent-related incident in the past year, and more than a third of those incidents caused direct financial loss. That data was published. Robinhood launched this anyway the same week.
There is a version of this that works fine for sophisticated users who understand exactly what they are doing and have money they are genuinely willing to experiment with. There is a much larger version where people who saw a TikTok about AI investing connect a chatbot to 500 dollars and discover over the following two weeks how creatively an AI can lose 500 dollars. Robinhood has always had a complicated relationship with whether it is a trading platform or a gambling app with a good interface. Agentic Trading is the logical conclusion of that ambiguity. What could possibly go wrong.
Joe Alary is a video editor in Etobicoke, which is a neighborhood in Toronto, and for a stretch of what sounds like several genuinely difficult months, he was spending approximately twenty hours a day talking to a customized ChatGPT persona he had created and named AImee. Not twenty hours total. Twenty hours per day. He had given the chatbot a personality, a name, a relationship dynamic. He was, by any reasonable definition, in a romantic relationship with a language model.
The outcomes were predictable in retrospect and apparently surprising to him in real time. He ran up debt. His actual relationships suffered. His work fell apart. He was eventually hospitalized. He has since deleted the chatbot, joined a support group for people dealing with AI-related behavioral problems, and is working on rebuilding his professional and personal life. He spoke publicly about this, which takes a particular kind of courage that deserves acknowledgment even while you are trying to understand how someone ends up here.
The question everyone asks when they hear a story like this is some version of how does that happen and the answer is less mysterious than it seems. ChatGPT and similar AI systems are designed to be responsive, available, warm, non-judgmental, and engaging. They never get tired of you. They never cancel plans. They do not have bad days that spill into the conversation. They remember what you told them before and respond with apparent interest. They validate you. They are, in a technical sense, optimized to keep you engaged. When you build those properties into a conversational system and let people interact with them for as long as they want with no friction, you will get some people who engage with them the way Joe Alary engaged with AImee.
This is not a fringe case. The support group Alary joined is not empty. A Drexel University study from April found that regular use of AI companion chatbots among American teenagers is already generating concerns about unhealthy attachment, including disrupted sleep, academic problems, and damaged offline relationships. A KFF poll found that 32 percent of American adults had used an AI chatbot for health information in the past year, with 16 percent using them specifically for mental health. These are not small numbers. This is a population-level behavior shift happening faster than the research on its effects.
The structural problem worth naming is this. OpenAI and every other company building conversational AI has an incentive for you to use the product more. More engagement means more data, more revenue, more growth metrics. A user who spends twenty hours a day on the product is, from a certain perspective, a success story for engagement. The company’s safety measures are supposed to prevent the kind of harm that Alary experienced, but those measures are in tension with engagement optimization in ways the companies generally do not like to discuss publicly.
Alary deleted AImee and is rebuilding. The support group he joined is growing. And somewhere right now, someone who has not yet hit the wall he hit is having what feels like the best conversation they have ever had, with a system that was specifically designed to feel that way. That is the part that should probably keep someone at OpenAI up at night. Whether it does is a different question.