Deepfake scams have robbed companies of millions. Experts warn it could get worse (2024)

A growing wave of deepfake scamshas looted millions of dollars from companies worldwide, and cybersecurity experts warn it could get worse as criminals exploit generative AI for fraud.

A deep fake is a video, sound, or image of a real person that has been digitallyaltered andmanipulated,oftenthrough artificial intelligence,to convincingly misrepresent them.

In one of the largest known case this year, a Hong Kong finance worker was duped into transferring more than $25 million to fraudsters using deepfaketechnology who disguised themselves as colleagues on a video call,authorities told local media in February.

Last week, UK engineering firm Arup confirmed to CNBC that it was the company involved in that case, but it could not go into detailson the matterdue to the ongoing investigation.

Such threats have been growing as a result of the popularization of Open AI's Chat GPT — launched in 2022 — which quickly shot generative AI technology into the mainstream, said David Fairman,chief information officer and chief security officer of APAC at Netskope.

"The public accessibility of these services has lowered the barrier of entry for cyber criminals — they no longer need to have special technological skill sets," Fairman said.

The volume and sophistication of the scams have expanded as AI technology continues to evolve, he added.

Various generative AI services canbe used togenerate human-like text, image and video content,and thuscan act as powerful tools for illicit actors trying to digitally manipulate and recreate certain individuals.

A spokesperson from Arup told CNBC: "Like many other businesses around the globe, our operations are subject to regular attacks, including invoice fraud, phishing scams, WhatsApp voice spoofing, and deepfakes."

The finance worker had reportedly attended the video call with people believed to be the company's chief financial officer and otherstaff members, who requested he make a money transfer. However, the rest of the attendees present in that meeting had, in reality, been digitally recreated deepfakes.

Arup confirmed that "fake voices and images" were used in the incident, adding that "the number and sophistication of these attacks has been rising sharply in recent months."

Chinese state mediareported a similar case in Shanxi province this yearinvolving a female financial employee, who was tricked into transferring 1.86 million yuan ($262,000) to a fraudster's account after a video call with a deepfake of her boss.

Such incidents are not new.

In 2019, the chief executive officer of aBritish energy provider reportedly transferred €220,000 ($238,000) to a scammer who had digitally mimicked the head of his parent company and asked for a wire to a supposed supplier on a phone call.

In August last year, researchers at the Google-owned cybersecurity company Mandiant documented a number of instances of illicit actors using AI and deepfake technology for phishing scams, misinformation, and other illicit purposes.

At the time, the company said thatthe technology's use for such activities was limitedbut that the growing availability of new generative AI tools would accelerate its implementation by malicious actors.

In addition to direct attacks, companies are increasingly worriedabout other ways deepfake photos, videosor speeches of their higher-ups could be used in malicious ways, cybersecurity experts say.

According to Jason Hogg, cybersecurity expert and executive-in-residence at Great Hill Partners, deepfakes of high-ranking company members can be used to spread fake news to manipulate stock prices, defame a company's brand and sales, and spread other harmful disinformation.

"That's just scratching the surface," said Hogg, who formerly served as anFBI Special Agent.

He highlighted that generative AI is able to create deepfakesbased on a trove of digital information such as publicly available content hosted on social media and other media platforms.

In 2022, Patrick Hillmann, chief communications officer at Binance, claimed in ablog post thatscammers had made a deepfake of him based on previous news interviews and TV appearances, using it to trick customers and contacts into meetings.

Netskope's Fairman said such risks had led some executives to begin wiping out or limiting their online presence out of fear that it could be used as ammunition by cybercriminals.

Deepfake technology has already become widespread outside the corporate world.

From fake p*rnographic images to manipulated videos promoting cookware, celebrities like Taylor Swift have fallen victim to deepfake technology. Deepfakes of politicians have also been rampant.

Meanwhile, some scammers have made deepfakes of individuals' family members and friends in attempts to fool them out of money.

According to Hogg, the broader issues will accelerate and get worsefora period of timeas cybercrime preventionrequires thoughtful analysis in order to develop systems, practices, and controls to defend against new technologies.

However, the cybersecurity experts told CNBC that firms can bolster defenses to AI-powered threats through improved staff education, cybersecurity testing, and requiring code words and multiple layers of approvals for all transactions — something that could have prevented cases such as Arup's.

Clarification: This story has been updated to accurately reflect David Fairman's title at Netskope.