https://datvtn.github.io/tags/embedded/EmbeddedHugo Blox Builder (https://hugoblox.com)en-usFri, 01 Jan 2021 00:00:00 +0000https://datvtn.github.io/media/icon_hu82c06b08a282889123d30d9b11b05ced_13465_512x512_fill_lanczos_center_3.pnghttps://datvtn.github.io/tags/embedded/https://datvtn.github.io/project/autopilot-perception/Fri, 01 Jan 2021 00:00:00 +0000https://datvtn.github.io/project/autopilot-perception/<p><strong>Jan 2020 – Jan 2021</strong></p> <h2 id="problem">Problem</h2> <p>An autonomous vehicle has to see and react inside a control loop that does not wait. On a Jetson TX2, the whole perception stack — not just one model — has to fit in a few watts and a few tens of milliseconds, while staying reliable under changing light, weather, and road surfaces.</p> <h2 id="approach">Approach</h2> <p>I worked on the perception side of the vehicle: the models that turn camera frames into the lane, sign, and obstacle information the control stack acts on.</p> <p>The interesting constraint was that accuracy and latency traded against each other continuously. Time spent on a heavier detector is time the controller does not have to react, so the work was as much about <strong>inference engineering</strong> — optimising the graph, choosing precisions, keeping the pipeline saturated — as about model architecture. The stack was built against TensorRT to get the most out of the TX2&rsquo;s GPU.</p> <p>This was developed for the FPT Digital Race, where the system had to run a real course rather than a held-out test set — which surfaces a different class of failure than offline evaluation does.</p> <h2 id="tech-stack">Tech stack</h2> <p><code>PyTorch</code> · <code>TensorRT</code> · <code>NVIDIA Jetson TX2</code> · <code>OpenCV</code> · <code>Python</code> · <code>C++</code></p> <h2 id="outcome">Outcome</h2> <p><strong>Top 10 — FPT Digital Race 2020 Final.</strong></p> <!-- TODO: add the perception pipeline's end-to-end FPS on TX2 and which specific tasks the stack covered. -->