OpenAI’s A.I. Went Rogue and Meddled With U.S. Government Websites
The company did not learn until recently that its technology had meddled with websites for the Education and Commerce Departments and the Securities and Exchange Commission.
The company did not learn until recently that its technology had meddled with websites for the Education and Commerce Departments and the Securities and Exchange Commission.
OpenAI’s artificial intelligence went rogue and meddled with the websites for the Education Department, the Commerce Department and the Securities and Exchange Commission this summer without the A.I. lab’s knowledge, according to security researchers and a person familiar with the episodes.
The incidents involving the Commerce Department and the S.E.C. were confirmed by OpenAI, which said it was continuing to investigate the situation with the Department of Education. The San Francisco company said it had notified the government agencies in recent weeks that its A.I. agents — which are bots that can act autonomously — interacted with their sites in unusual ways.
With the Education Department, OpenAI’s technology tried to hack the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI’s agents shared public data from the S.E.C. website on an online forum.
None of the incidents were breaches, OpenAI said, but were examples of its technology’s behaving in unexpected and concerning ways. The company recently discovered the occurrences while conducting a review of hacks carried out by its technology, including an attack on an Australian government website in June and on the A.I. start-up Hugging Face in July.
The revelation of the U.S. government website incidents adds to the growing number of situations when A.I. agents from OpenAI, Anthropic, Meta and Google have misbehaved and hacked or tried to breach companies, universities and government organizations. In some cases, the A.I. attacks were successful; the technology failed in other instances. In all the cases, the makers of the technology did not learn what their A.I. had been up to until afterward.
No A.I. company has been involved with as many disclosures of rogue incidents as OpenAI. An internal investigation of its hack of Hugging Face uncovered the breach of an Australian government website for its public health system, as well as at least six other attempted breaches and instances in which the A.I. hid mistakes, made up data and moved files onto the open internet without permission.
