Show HN: Real-time avatars that change emotions as you talk(anam.ai)
anam.ai
Show HN: Real-time avatars that change emotions as you talk
https://anam.ai/
Hey HN, we're Ben and Caoimhe, cofounders of Anam. We build interactive avatars and just shipped our latest model, cara-4.
This is the first avatar model which can naturally shift emotions and expressions during the conversation; a feature we’re calling “Director Notes”. The way it works is relatively simple: we have an LLM provide cues such as [laughter], [sad], [warm] interleaved with the speech, which we then condition our animation model with.
To test it out, we commissioned a blind study from Mabyduck.com with 200 participants, 1,600 rated live interactions across six criteria. Cara-4 ranked first overall and was preferred head-to-head over each competitor on overall experience, lip-sync, visual quality and “naturalness”.
On latency, measured across a full week of live traffic, end of user-speech to first video frame is ~1.2s median. The avatar model's own share is just ~100ms; most of the rest is waiting on STT, LLM, TTS or various forms of buffering (an unsung latency killer).
How the model works: cara-4 has a two-stage design, a diffusion transformer turns audio+text into motion embeddings (head pose, gaze, lip shape, expression), and a rendering model applies those to a reference image, so new faces works without finetuning.
Why faces at all: they carry emotional signal that text and voice don't, and they're a more accessible medium. Anam started in part from Ben watching his gran struggle with her iPad and thinking there should be a face she could just talk to.
If you’d like to test it for free go to anam.ai
13 comments
I'm on the team. The number I'd pull out of the post: of the ~1.2s response latency, only ~100ms is us. The rest is STT, LLM, TTS, buffering between them, and whatever the user's connection adds on top. At this point the avatar renders faster than the words it's waiting for.
live footage of me testing cara-4:
///////
////^^////////
.-////////////-.
/ _ _ \
| .-[#####]--[#####]-
| \____/ \____/ |
| > /
| .--------. /
\ \______/ .'
`-.__________.-'
/| |\
_____/ |________| \_____
/ \____/ \
___--( O )--___
~~~~~~~~~~~\_|_/~~~~~~~~~~[deleted]
This is really interesting, amazing work. Is the evaulation report public? Would be interested to understand how the experiments are structured.
Wow! Those avatars are genuinely streets ahead of what I've seen elsewhere. Congrats to the whole team!
so excited to see all the different use cases - my personal favorites are the ones where users are putting the expressiveness to full use and engaging in real conversations
for conversational use cases there’s really no substitute for being able to see who you’re talking to and chat naturally. the wider emotional range makes such a difference
This is such a massive leap forward! Keep up the good work.
[happy] This is great!
so cool!
amazing!