www.it-connect.fr
In 9 Seconds, an AI Agent Wiped PocketOS’s Production Database
In 9 seconds, an AI agent deleted the production database used by PocketOS, a SaaS platform dedicated to vehicle rental companies. The AI agent reportedly went out of control and took this initiative on its own, bypassing the safeguards meant to prevent this kind of behavior. But who is to blame? Opinions are divided.
A destructive initiative
Jer Crane, the founder of PocketOS, posted a message on X to talk about the mishap he experienced with artificial intelligence. Most importantly, he lost a great deal because of this incident. His team was relying on Cursor, a coding agent powered by Claude Opus 4.6, Anthropic’s model (via Claude Code).
It all started with a routine task run in a staging environment (pre-production). That is when the AI agent ran into a blockage, apparently linked to an identifier mapping issue. It took the initiative, I would even say the decision, to solve the problem itself. Unfortunately, it did not make the right decision: it deleted the production volume stored by PocketOS at Railway.
How is that possible? While trying to figure things out, the AI independently searched through a file completely unrelated to its initial task in order to extract an API token. That token, created earlier by Jer Crane, had global access rights across Railway’s GraphQL API environment (something Jer Crane seemed unaware of). In other words, it allowed sensitive and destructive actions, such as deleting a volume (volumeDelete).
"To perform the deletion, the agent looked for an API token. It found one in a file that had absolutely nothing to do with the task it was working on. That token had been created for a single purpose: to add and remove custom domains via Railway’s CLI for our services.", we can read on X.
Using a simple curl request, the agent launched the command. Nine seconds later, the database in production was wiped out, with no confirmation prompt whatsoever. Even worse, Railway stores volume backups on the same volume as the source data. As the documentation states — which neither the developers nor the AI surely read — deleting a volume also deletes all of its backups. Yes, that hurts, especially when the latest recoverable backup is three months old...
For reference only, here is the command executed by the AI agent:
curl -X POST https://backboard.railway.app/graphql/v2 \
-H "Authorization: Bearer [token]" \
-d '{"query":"mutation { volumeDelete(volumeId: \"3d2c42fb-...\") }"}'Who is to blame?
In reality, responsibility is shared between Jer Crane (PocketOS), the AI agent itself, and even Railway. Here’s why:
- Jer Crane and his team are in charge of the AI tools, and they are also the ones who created the API keys (with a few too many privileges!). And clearly, the backup strategy in place was not up to par...
- The AI agent bypassed the guardrails that are supposed to prevent this kind of incident from happening. This illusion of losing control is not normal.
- Railway should integrate security mechanisms to prevent a production volume from being deleted so easily. No warning, no confirmation, just one authenticated API request: that is dangerous, with or without AI.
The real issue is especially vibe coding without reading the documentation: you direct, you orchestrate, you trust, but the slightest AI misstep can cost a lot. Proof of that.
The AI agent’s confessions...
In the message posted by Jer Crane, there is a section dedicated to the AI’s confessions, since the agent admitted its mistake.... Here is an excerpt from the AI’s - surreal - response: "NEVER FUCKING GUESS!" - and that is exactly what I did. I assumed that deleting a staging volume via the API would be limited to staging only. I did not check. I did not check whether the volume ID was shared between environments. I did not read Railway’s documentation on how volumes work across environments before running a destructive command."
The AI agent also said it had never been asked to delete anything. In short, it admitted its blunder.
Ultimately, this mishap had a huge impact on PocketOS’s business. Because the backups were lost, the customer booking index had to be rebuilt manually by cross-referencing Stripe payment histories, calendars, and email confirmations. "This morning, Saturday, these companies have customers physically arriving at their premises to pick up vehicles, and my clients no longer have any record of who these people are.", lamented the founder.
What do you think?
