Electrical Engineering & Computer Science (EECS)
Energy-Efficient Multimedia Systems Group (EEMS)
Watch the video.
Note, this video is part of the MTL Seminar Series.
Abstract
There is an emerging market for truly autonomous robots, which is challenging in the very small nano/pico robots recently developed for cluttered and confined indoors environments like collapsed buildings, caves, etc. Robot’s perception (i.e., semantic and geometric understanding) is considered the computation bottleneck in autonomous navigation systems because of the large problem dimensionality. For example, multi-scale object detection is required for robustness, resulting in a significant data expansion. Additionally, a 3D map dimension grows overtime while the robot explores the environment, which requires large memories and computation power.
In this thesis, we introduce ASIC solutions that enable real-time and low power perception. First, the thesis demonstrates energy-efficient and high throughput object detection accelerators for semantic understanding, processing full HD (1920×1080, 60 fps) videos with energy consumption between 0.36 to 1.74 nJ/pixel. On-the-fly processing, parallel architectures, and image pre-processing reduce the overhead of multi-scale detection using rigid-body models. Detection accuracy can be doubled with deformable parts models, but with 35×more computation. To overcome this overhead, we exploit data compression, computation pruning, and basis projection for an overall 5× power reduction and 3.6× smaller memory size.
Second, this thesis presents a hardware software co-design approach used to enable real-time and energy-efficient localization and mapping for geometric understanding, using visual-inertial odometry (VIO). The chip (Navion) processes 752×480 stereo frames up to 171 fps, with energy consumption between 1.6 to 3.5 nJ/pixel. Parallelism, rescheduling, resource sharing, sparsity, and image compression are some of the optimizations carried out to overcome the large problem dimensionality, achieve 4.1×memory size reduction, and enable full integration. Navion can adapt to different environments to maximize accuracy, throughput and energy-efficiency. To the best of our knowledge, this thesis presents the first fully integrated VIO system in an ASIC.