AI Photo Editing App

Mon, 01 Jan 2024 00:00:00 +0000

Sep 2023 – Jan 2024

Problem

The edits people actually want from a photo app — remove that person, drop the background, make the sky dramatic, put him in a suit — are each a different technical problem, but users expect all of them behind a single tap. Each feature needs an accurate mask, a plausible fill, and a result that survives being zoomed into.

Approach

I built four editing features on a shared backbone of segmentation plus generative inpainting:

Object removal — segment the unwanted region, then inpaint it with surroundings-consistent content rather than a blur or clone-stamp patch.
Background removal — high-quality alpha matting around hard cases: hair, glasses, motion blur at the subject boundary.
Sky replacement — sky segmentation combined with colour harmonisation, so the relit foreground matches the new sky instead of looking pasted on.
Suit changer — garment-region segmentation and generative synthesis that keeps the subject’s identity and pose intact.

The common thread is that mask quality dominates perceived output quality: a strong generative model behind a sloppy mask still looks wrong at the edges, so most of the tuning effort went into the segmentation stage and its boundary refinement.

Tech stack

PyTorch · Diffusion Models · Segmentation networks · OpenCV · Python

Outcome

Delivered as a set of consumer-facing editing features.

Real-Time Portrait Segmentation on Mobile

Wed, 01 Dec 2021 00:00:00 +0000

Aug 2021 – Dec 2021

Problem

Portrait effects — background blur, replacement, relighting — need a person mask computed every frame, on a phone, inside the camera’s frame budget, without draining the battery. A model that is accurate but too slow is useless here, and so is a fast model that flickers between frames or cuts through someone’s hair.

Approach

Two halves, and the data half mattered as much as the model:

The network. A deliberately small segmentation architecture designed against the mobile inference budget rather than trimmed down afterwards. Design decisions were driven by what the target runtime could actually execute efficiently, not just by parameter count — an operation that is cheap on paper is not cheap if the on-device runtime falls back to CPU for it.

The data pipeline. I built the full pipeline: collection, annotation, quality control, and augmentation. Boundary quality is where portrait segmentation visibly succeeds or fails, so the annotation standard around hair and edges drove most of the final result.

The model was quantized and deployed through the mobile inference stack used for VinSmart devices, alongside the other lightweight face analysis networks (FaceID, Masked FaceID, camera effects) our team shipped.

Tech stack

PyTorch · TFLite · SNPE · NCNN · INT8 quantization · Python · C++

Outcome

A portrait segmentation network running in real time on target mobile hardware.

Segmentation | Dat Viet Thanh Nguyen

AI Photo Editing App

Problem

Approach

Tech stack

Outcome

Real-Time Portrait Segmentation on Mobile

Problem

Approach

Tech stack

Outcome