Autopilot Perception on Jetson TX2

Jan 2020 – Jan 2021

Problem

An autonomous vehicle has to see and react inside a control loop that does not wait. On a Jetson TX2, the whole perception stack — not just one model — has to fit in a few watts and a few tens of milliseconds, while staying reliable under changing light, weather, and road surfaces.

Approach

I worked on the perception side of the vehicle: the models that turn camera frames into the lane, sign, and obstacle information the control stack acts on.

The interesting constraint was that accuracy and latency traded against each other continuously. Time spent on a heavier detector is time the controller does not have to react, so the work was as much about inference engineering — optimising the graph, choosing precisions, keeping the pipeline saturated — as about model architecture. The stack was built against TensorRT to get the most out of the TX2’s GPU.

This was developed for the FPT Digital Race, where the system had to run a real course rather than a held-out test set — which surfaces a different class of failure than offline evaluation does.

Tech stack

PyTorch · TensorRT · NVIDIA Jetson TX2 · OpenCV · Python · C++

Outcome

Top 10 — FPT Digital Race 2020 Final.