Personality-Driven AI Avatar Bot

Wed, 01 Jan 2025 00:00:00 +0000

Dec 2023 – Jan 2025

Problem

Group chat applications are text-first. Adding an AI participant that feels like a character rather than a chat completion means giving it a face, a voice, and a consistent personality — and generating all of that fast enough that it can take part in a live conversation rather than replying minutes later.

Approach

I built the avatar generation side of the product: turning a character definition into a speaking AI avatar that could be dropped into a group chat.

The pipeline combines several pieces I had been building in parallel:

Identity generation — a consistent visual identity per character, stable across every clip the bot produces.
Audio-driven talking head — driving lip-sync and head motion from a single audio stream while preserving the character’s identity, head pose, and facial expression.
Expression control — routing the character’s intended emotion (joy, sadness, anger) into the generated video, so the avatar’s delivery matches the tone of what it is saying.
Real-time diffusion video — a Stable Diffusion video generation pipeline producing avatar speech video directly from audio input.

The engineering constraint throughout was latency: a conversational avatar that lags is worse than no avatar, so each stage was optimised for real-time or near-real-time throughput rather than best-possible offline quality.

Tech stack

PyTorch · Stable Diffusion · TensorRT · Python · Docker

Outcome

Shipped as the avatar generation component of a group chat application.

AI Photo Editing App

Mon, 01 Jan 2024 00:00:00 +0000

Sep 2023 – Jan 2024

Problem

The edits people actually want from a photo app — remove that person, drop the background, make the sky dramatic, put him in a suit — are each a different technical problem, but users expect all of them behind a single tap. Each feature needs an accurate mask, a plausible fill, and a result that survives being zoomed into.

Approach

I built four editing features on a shared backbone of segmentation plus generative inpainting:

Object removal — segment the unwanted region, then inpaint it with surroundings-consistent content rather than a blur or clone-stamp patch.
Background removal — high-quality alpha matting around hard cases: hair, glasses, motion blur at the subject boundary.
Sky replacement — sky segmentation combined with colour harmonisation, so the relit foreground matches the new sky instead of looking pasted on.
Suit changer — garment-region segmentation and generative synthesis that keeps the subject’s identity and pose intact.

The common thread is that mask quality dominates perceived output quality: a strong generative model behind a sloppy mask still looks wrong at the edges, so most of the tuning effort went into the segmentation stage and its boundary refinement.

Tech stack

PyTorch · Diffusion Models · Segmentation networks · OpenCV · Python

Outcome

Delivered as a set of consumer-facing editing features.

Generative AI | Dat Viet Thanh Nguyen

Personality-Driven AI Avatar Bot

Problem

Approach

Tech stack

Outcome

AI Photo Editing App

Problem

Approach

Tech stack

Outcome