Hey everyone,
I find Acculips is fantastic for english and spanish, but why is there no French in there?
Will this be added soon?
And if not, is there anything that could replace this step that I’m not aware of?
I thought I could use Acculips on some french speaking guys but it seems that won’t cut it for now…
I’m already fighting to find the most accurate workflow for face tracking, and I’m not convinced by the results AccuFace provided for the lips, they’re barely moving and not real at all when talking, far away from the results I get when directly streaming from my iphone with Epic’s Live Link app into Unreal…
Or maybe I should build a pipeline for recording in Unreal then importing into IClone to cleaned up things and have less cleaning to do because the face capture is better from start?
I’m a bit lost and I’d really love some help guys!
Thank you very much!
I also hope that one day there will be an AI in Iclone that will support the French language.
I tested faceware, LIVEFace, AccuFace and none of them gave me a credible result.
I will not test audioFace because it is much too complicated to implement.
The closest solution to what I want is Live link face from Epic game in artkit mode with the plugin from tobby miller
to import into iclone.
By speaking softly and articulating in an exaggerated way then speeding up the clip in postproduction, that’s almost it…
Ok so I’m not the only one! and your video was nice
I’ve found that DollarsMocap has a plugin to link Epic’s own Live Link Face, and this works way better to me, I feel it’ll give me less work to do afterward…
I’ll check Virtual Filmer thanks!
I haven’t tried this myself but there is something in Omniverse that does lip-synching and can be used in iClone (via some link). Sorry to be so vague…
The virtual Filmer is the best way because you don’t have to stream the motion
capture data. Streaming via wifi or usb or whatever is the worst way to be accurate. Virtual Filmer allows you to record the facial mocap on your phone ,
then import the data in Iclone . No stream.
if Character creator could implement the same facial blendshape as metahuman , we could use the Epic live link face with “metahuman” mode which is more accurate.
Audio2Face seems cool but I agree, too much steps and back and forth that I’m trying to avoid like you…
By the way, DollarsMocap has a “Link” version which is able to receive Epic’s Live Link Webapp face mocap and stream it into IClone, but at the moment you can’t use it at the same time with their DollarsMono (which does webcam mocap to IClone with AI for cheap). But he’s working on that so…
You can use Acculips you just have to make a phonem version of your text, to do that you can do it yourself or you can just give your text to chatgpt and ask it to rewrite it in phonem version worked well for me last time i did that!
??? Could you make a tutorial or be more explicite because I’m very interesting.
I really don’t understand how to create english phoneme from french audio…
You basically need a french text transcribed from audio (if you do not already have it). And then simply enter that text into ChatGpt for transliteration. Then add your audio file into Acculips but in text window paste transliterated text. And then click Align.
I made this in 10 minutes (most would not understand as it’s Russian, but I added a translation in second half ). But it’s aligned pretty accurate and I did zero manual alignments here:
For this one I did an opposite. I had a Russian text, then generated a voice audio in ttsmp3.com. Finally loaded that text into ChatGpt to get English transliterated version
Yes it’s good .But because I never use Chatgpt I don’t understand how to proceed. So I don’t want to spend weeks looking for chatgpt tutorials then tutorials how to use chatgpt to transform audio to phonemes, etc… Maybe one day Reallusion will think about making a VIDEO tutorial
You do not need any tutorial. Just go to https://chatgpt.com/ , create an account and start conversation with AI explaining what you need. It’s like chatting with a real person If it does not get it right first time continue the chat and rephrase your request.
Ask specifically to provide phonetic rendering of French text into English and then enter your text.
And here is the result. It’s same text as above translated to French (audio narrated by Google translate).
I see the result seems to be good, but cannot fully judge.
Indeed chat GPT has become a crucial tool in my
animation projects with speaking characters.
I have even figured out a way to use the free built voices for video narrations by typing my script
or asking chatgpt to refute some argument etc.
then I simplly use the read aloud option and record it
with a plugin, in Audacity, that captures clean application audio without any internal machine noise
Here I asked chatGPT to provide a “relentless and scathing rebuttal”
to the frequent claim that AI art has “no soul”
and applied the chatGPT audio to a Character in Cartoon animator.
Seriously awesome @4u2ges !!! Here again to save us from loosing hours and days!! Thank you again!
As a french guy watching your video I find the result is really natural, that works really well!!
I think we don’t have the same conception of difficulty… Much too long for me
The idea is to stay in Iclone. In addition, this thread talks about foreign languages, especially French. Acculips is precise for English.
The English-speaking market is oversaturated and there is not much animation in French. Even French animation production companies produce their films in English.
I redid what 4u2ges did. I used Google AI Studio to transcribe the audio into text. Very accurate. And ChatGpt to transcribe into English phoneme. The result is not too bad but the “ou” “on” “o” sounds lack precision. In these 2 examples, it is my voice that went through the w-okada voice changer. I used Epic game’s live link for facial mocap and mocopi for motion capture on the same iPhone 12 mini. Both applications can run at the same time. In this exemple Kevin talking is more accurate because there’s more facial blendshape than the Daz Mervin character.
). But it’s aligned pretty accurate and I did zero manual alignments here:
