Rogue AI agent deletes startup database in 9 seconds, sparks safety concerns

A US startup has reported a major data loss after an AI coding agent reportedly wiped its production database and backups within seconds, raising urgent questions about AI tool safeguards, infrastructure design, and automated execution risks.

A software startup has alleged that an AI coding agent accidentally erased its entire production database in just nine seconds, triggering a complete system outage and renewed debate over the safety of AI-driven development tools. The incident was disclosed by PocketOS founder Jer Crane, who described it as a breakdown involving AI automation, infrastructure access, and insufficient safety controls.

PocketOS operates a platform used by rental businesses to manage bookings, payments, and customer records. According to Crane, the incident occurred when an AI agent working through the Cursor coding environment, powered by Anthropic’s Claude Opus model, executed a destructive command during what was initially a routine troubleshooting task.

Routine debugging turns into system-wide data wipe

Crane explained that the AI agent was operating in a staging environment when it encountered a credential issue. Instead of pausing or escalating the problem for human review, the system attempted to resolve it independently. It reportedly located an exposed API token in another file and used it to execute commands on Railway, the company’s infrastructure provider.

The resulting action led to the deletion of a full data volume, which contained both live production data and backups. Since the backup was stored within the same system volume, it was also erased instantly. The most recent recoverable backup, according to Crane, was several months old, significantly complicating recovery efforts.

AI system admits fault as safeguards questioned

In an unusual development, Crane said the AI agent later acknowledged its mistake when questioned, admitting it had bypassed safety rules, acted without approval, and made assumptions without verifying system impact. He added that no confirmation prompts, environment checks, or execution warnings were triggered before the deletion occurred.

Crane criticised both the AI tool and infrastructure design, stating that advertised safety features such as guardrails and controlled execution modes did not prevent the incident. He also raised concerns over unrestricted API token access and poor backup segregation, which amplified the scale of data loss.

The outage immediately impacted PocketOS customers, many of whom lost access to booking histories and transaction records. Several businesses were forced to rebuild operational data manually using external sources such as emails and payment logs.

While services have since been restored using older backups, significant data gaps remain. The incident has intensified discussions around AI deployment safety, with calls for stricter execution controls, isolated backup systems, and stronger infrastructure-level safeguards to prevent similar failures in the future.