Deepfake video of Taylor Swift speaking Mandarin sparks discussion over AI in China

In a video posted on Chinese social media platform Weibo, the 33-year-old singer can be seen flaunting her Mandarin-speaking skills in what looks to be a talk show. - PHOTO: SCREENGRAB FROM WEIBO

AMERICAN pop star Taylor Swift got the attention of her Chinese fans this week when clips of her speaking fluent Mandarin went viral on Chinese social media.

In a video posted on Chinese social media platform Weibo, the 33-year-old singer can be seen flaunting her Mandarin-speaking skills in what looks to be a talk show.

“Recently, I’ve been to many places, like Italy, France and Japan,” Swift said in Mandarin with a slight American twang. The video that was shared on Oct 21 and has since gained over 6 million views.

In another video, Swift talked about songs that get “left behind”, and how she wished that people could hear these tracks.

The twist? Swift owes her Mandarin-speaking prowess to artificial intelligence (AI).

The deepfake videos were generated by an AI tool from HeyGen, a Chinese start-up, reported news outlet The China Project.

The lip movements were synced to her Mandarin that sounded just like Swift’s real voice, making the videos so realistic that it led Chinese netizens to express their amazement at the technology, with comments like “This is awesome!”. Others asked for Mandarin versions of their favourite show.

But it also triggered discussion over the potential pitfalls that will come as AI gets more advanced, like scams and the loss of jobs.

“The scariest thing is that if someone uses his technology to make fake news, because the AI is able to manipulate the voice and movement of the mouth, people will easily believe it,” said one netizen.

Another Weibo user wrote that it would be “terrible if (this AI technology) is used for fraud”, as criminals can use it to ask dupe victims of their money.

HeyGen, the company behind the tool that made the videos, was co-founded in November 2020 by Mr Joshua Xu and Mr Wayne Liang.

The firm’s AI-powered video generator allows users to create text-to-speech videos in over 300 voices in more than 40 languages, using over 100 AI avatars with different ethnicities, ages, poses and clothes.

Technology like this has sparked concern over the criminal applications of AI. Earlier this year, a scammer in northern China had used deepfake technology to convince a man to transfer money to a supposed friend

The scammer had used AI-powered face-swapping technology to impersonate a friend of the victim during a video call. The man ended up transferring him some 4.3 million yuan (S$804,370).

In January, China put in force new rules that require businesses offering deepfake services to obtain the real identities of their users. It also requires deepfake content to be appropriately tagged to avoid confusion. - The Straits Times/ANN