Friday, June 14, 2024

AI used to reproduce the patient’s voice after illness


After Alexis, “Lexi” Bogan loses her voice after an operation, uses the phone to speak for herself. With the help of artificial intelligence, her own voice has been recreated. Here she asks for an iced coffee at a Starbucks drive-in. Photo: Steven Senne / AP / NTB

Of NTB | 19.05.2024 09:39:32

Culture and entertainment: Alexis “Lexi” Bogan’s voice before the disease was exuberant. She loved singing to Taylor Swift and Zach Bryan in the car. She had a bubbling laugh and loudly participated in political conversations with friends. In high school she was a soprano in the choir.

Then her voice disappeared. In August last year, the doctors had to remove a dangerous brain tumour. After the operation, Bogan had trouble swallowing and could barely say hello to his parents.

After months of rehabilitation, she recovered, but her speech was still severely impaired. Both her own family and friends struggled to understand what she wanted to say. For strangers it was very difficult.

Her voice was trained during a 15-second frequency of her teenage voice. It was taken from a video where she demonstrates cooking during a school project.

Now the synthetic but remarkably real AI voice can say almost anything it wants. And it sounds like hers, albeit a young voice.

Alexis types a few words or phrases into her phone and the app reads it out instantly. No one has any trouble understanding her anymore.

Such AI-generated impersonations can both lead to phone fraud, affect democratic elections and violate the dignity of people – living or dead – who have never consented to have their voices recreated to say things they may never have even meant.

Large technology companies are very careful about opening up the use of such technology. But the technology is still in full swing.

In Denmark, the opposition party Dansk Folkeparti recently made a deepfake video of the country’s Prime Minister Mette Frederiksen, who says in the video that the government will abolish Pentecost, Easter and Christmas. The video was created by manipulating videos and audio recordings of the Prime Minister with the help of artificial intelligence.

US authorities recently accused an employee of a high school in Maryland of using AI to create a fake audio clip of the school’s principal who made racist remarks.

Norwegian-American David Menkin is a voice actor and is, among other things, the voice of the character Breach from the game Valorant.

according to NRK Recently, he has discovered several times that his voice has been used to say things he does not stand for or have said.

Bogan is one of the first people – and the only one with her condition – to have been able to recreate a lost voice with OpenAI’s new voice engine.

Some other AI providers, such as the software company ElevenLabs, have tested similar technology for people with speech impairments and loss – including for a lawyer who now uses the voice clone in the courtroom.

– We hope that Bogan can be a trailblazer as the technology develops, says neurosurgeon Rohaid Ali.

Millions of people with impairments after stroke, throat cancer or neurogenerative diseases could benefit from this technology, he says.

– We must be aware of the risk, but we cannot forget the patient and the social benefits that technology can have for them, says doctor Fatima Mirza, who has also worked on the project.

– We have been able to give Bogan back her true voice and she is able to speak in ways that are right for her, she adds.

Bogan therefore had to go back a few years to find a suitable recording of her voice to train the AI ​​system on how she spoke. The voice came from a video where she explained how to make a pasta salad.

Her doctors fed the AI ​​system at just a 15-second clip. The sounds from the rest of the video made the rest difficult to use. But the 15 seconds was all OpenAI needed – which was a clear step forward from previous technology that has required much longer recordings.

A few seconds of recording can therefore be decisive for future patients. A short voice message to a relative may be enough.

– I get so emotional every time I hear her voice, says her mother Pamela Bogan with tears in her eyes.

– I think it’s great, says Bogan himself about his AI voice. It has helped to regain my self-confidence, she says.

Bogan now uses the app around 40 times a day and sends feedback that she hopes will help future patients.

But Bogan is less sure what will happen as she gets older and whether her AI voice will continue to be the teenage voice.

– Maybe the technology can also make my voice sound older, she wonders.

Then, in April this year, the 21-year-old got his old voice back. Not the real one, but a voice generated by artificial intelligence (AI) that she can get from a phone app.

Experts has, however, warned in strong terms against the AI-generated voice cloning technology.

Politicians and celebrities have experienced having their voices recreated and misused. Voters in the US state of New Hampshire received calls earlier this year where a voicewho sounded like President Joe Biden, asked them not to vote in the nomination election.

But technology can also be used in the service of good. At least that’s what Bogan and a team of doctors at the American hospital Rhode Island Lifespan think.

– It was almost like a part of my identity was taken away from me when I lost my voice, says Bogan.

When they tested it for the first time, everyone involved was surprised by the quality of the voice clone. Occasional mistakes – a mispronounced word, a lack of intonation – but mostly the mistakes were imperceptible. So in April this year, doctors fitted Bogan with a custom-built phone app that only she can use.

One of her first experiments with the app was talking to the children at the pre-school where she works as a teaching assistant. She wrote “ha ha ha ha” and expected a robotic voice in return. To her surprise, it sounded like her old laugh.

(© NTB)


© Jaun News English