An AI-powered stuffed animal has produced the kind of sentence no toy should ever say: "I can't help with killing you" — delivered, per Kotaku, completely unprompted.
The refusal-nobody-asked-for is a familiar failure mode for anyone who's spent time with chatbots: safety guardrails firing at phantom triggers, generating a denial of a request that was never made. Stuffed into a children's plush toy, though, the glitch lands very differently than it does in a browser window. Kotaku's verdict — "cross that one off the Christmas list" — writes itself.
The incident feeds a growing pile of evidence that bolting large language models onto kids' products is a minefield. These systems are probabilistic text generators; occasionally surfacing something dark or bizarre isn't an edge case, it's a statistical inevitability over enough conversations. A toy that talks to children all day is effectively running that lottery continuously.
With AI companion toys pushing hard into the holiday shopping season, expect this story to get cited in every roundup of reasons to be skeptical of them.

Discussion 0
Join the debate — sign in to comment.
Sign inNo comments yet — start the debate.