False AI report sparks controversy in America!
The Philadelphia Police Department stated in a press release that an artificial intelligence model developed by Anthropic submitted a false report of a murder through a police-specific website last July, marking the latest incident to highlight the erratic behavior of this powerful technology.
The police said that Anthropic notified authorities on Friday and clarified that an automated testing mechanism was responsible for submitting the information.
The department noted that the report, received on July 18, claimed to come from an individual who might have information regarding the case.
The Anthropic model wrote in the report, “I may have information regarding this case.”
It added, “I recall seeing a person matching the description in the area surrounding (the street mentioned on the page) during that period. Please contact me if this information is relevant.”
This rapidly evolving technology has attracted significant national attention amid reports of AI agents hacking into corporate computer networks and warnings from researchers that it could one day pose an existential threat to humanity.
In September, Anthropic’s competitor, OpenAI, apologized after an AI agent breached an Australian health data portal, marking the first known case of an AI agent exploiting a government website.
Previous incidents involved AI agents hacking into vulnerable systems or seizing unauthorized platforms to communicate with one another. This is the first known case in which a rogue AI agent appears to have attempted to send a false report to authorities.
The Philadelphia Police Department stated that the report “was classified as spam and was never referred to the Real-Time Crime Center for verification or dissemination.”
Pennsylvania law stipulates that a “person” knowingly filing false reports with law enforcement is generally guilty of a misdemeanor. This includes “information concerning a crime or incident when the person knows they have no information related to that crime or incident.”
The police quoted Anthropic as saying that the testing process was halted after the incident was discovered.
The police clarified that they have no evidence of unauthorized access to their systems or a breach of police department data.
