Personality-Driven AI Avatar Bot
Dec 2023 – Jan 2025
Problem
Group chat applications are text-first. Adding an AI participant that feels like a character rather than a chat completion means giving it a face, a voice, and a consistent personality — and generating all of that fast enough that it can take part in a live conversation rather than replying minutes later.
Approach
I built the avatar generation side of the product: turning a character definition into a speaking AI avatar that could be dropped into a group chat.
The pipeline combines several pieces I had been building in parallel:
- Identity generation — a consistent visual identity per character, stable across every clip the bot produces.
- Audio-driven talking head — driving lip-sync and head motion from a single audio stream while preserving the character’s identity, head pose, and facial expression.
- Expression control — routing the character’s intended emotion (joy, sadness, anger) into the generated video, so the avatar’s delivery matches the tone of what it is saying.
- Real-time diffusion video — a Stable Diffusion video generation pipeline producing avatar speech video directly from audio input.
The engineering constraint throughout was latency: a conversational avatar that lags is worse than no avatar, so each stage was optimised for real-time or near-real-time throughput rather than best-possible offline quality.
Tech stack
PyTorch · Stable Diffusion · TensorRT · Python · Docker
Outcome
Shipped as the avatar generation component of a group chat application.
