“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics
•
0 likes•4 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/07/how-to-run-audio-and-vision-ai-algorithms-at-ultra-low-power-a-presentation-from-synaptics/ Deepak Mital, Senior Director of Architectures at Synaptics, presents the “How to Run Audio and Vision AI Algorithms at Ultra-low Power” tutorial at the May 2024 Embedded Vision Summit. Running AI algorithms on battery-powered, low-cost devices requires a different approach to designing hardware and software. The power requirements are stringent at standby, but the device needs to be able to awaken quickly when an event is detected. The device needs to “pseudo” wake up, determine if the event needs attention, and then either go back to standby or become active to classify the event. This multistage wake-up process and the associated intelligence requires tight orchestration of hardware and software. Apart from runtime software, the AI models must be highly optimized to fit and run on the constrained device. To show how this can be done, Mital presents a solution that combines hardware, software and AI models to enable running audio and video AI algorithms at ultra-low power.
Report
Share
More Related Content
Similar to “How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics
Similar to “How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics
- 1. How to Run Audio and Vision AI Algorithms at Ultra-Low Power Presenter: Deepak Mital Sr. Director, Architecture Synaptics Incorporated
- 2. • Many IoT applications do not require “continuous maximum” compute • Continuous monitoring results in battery drain • Examples: • Security camera: Turn on main processing for actual detection only when confirmed necessary • Human presence detection (HPD) and identification to turn device on: Run HPD detection and identification algorithm only when detected “potential” presence • Predictive maintenance: Enable advanced detection only when initial metrics are met • Shoplift prevention: Enable detailed analytics only when “potential” threat detected Problem statement 2 © 2024 Synaptics Inc
- 3. • Multistage hardware: Capable of running Audio and Video AI algorithms • Highly efficient AI models with different KPIs for each stage • Tight orchestration of software to invoke each stage Solution 3 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 4. • Ultra-low power: Microwatts hardware, always on • Sound detection • Image change detection • Critical model requirements are for very few false negatives • False negatives will render device unresponsive Solution – Stage 1 4 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 5. • Mid- to low power – 10s of microwatts hardware, activated by stage 1 via software • AI algorithms (example): • Wake-word detection • Human presence detection • Critical model requirements are for very few false negatives and false positives • False negatives will render device unresponsive • False positives will increase power consumption Solution – Stage 2 5 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 6. • High performance, activated by Stage 2 via software • AI algorithms (example): • Person identification • Object detection • Critical model requirements are for very high performance at low power • Slow run times will increase power consumption Solution – Stage 3 6 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision Ai pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 7. • Different requirements for AI models at each stage • Need AI models optimized for different KPIs: accuracy, performance, and size • NAS-based model generation architecture where the models are purpose built for the constrained silicon • Primary factors affecting inference KPI • Model architecture design • Model quantization • Approach: Jointly optimize model architecture and quantization under memory constraints AI models 7 © 2024 Synaptics Inc
- 8. • Resolution – [28x28 – 32x32] • Kernel size – [3x3, 5x5, 7x7] • Depth – [2, 3, 4] • Width (channel expansion factor) – [2, 3, 4] • Mixed-precision quantization parameters – [4 bit, 6 bit, 8 bit] Multi-precision NAS search range for classification 8 © 2024 Synaptics Inc
- 9. CIFAR-10 classification – Mixed vs 8- or 4-bit precision 9 © 2024 Synaptics Inc
- 10. CIFAR-10 classification comparison 10 © 2024 Synaptics Inc
- 11. • Resolution – [320x240 – 640x480] • Kernel size – [3x3, 5x5, 7x7] • Depth – [2, 3, 4] • Width (channel expansion factor) – [2, 3, 4] • Mixed-precision quantization parameters – [4 bit, 6 bit, 8 bit] Object detection dataset 11 © 2024 Synaptics Inc
- 12. COCO person detection – Mixed vs 8- or 4-bit precision 12 © 2024 Synaptics Inc
- 13. COCO person detection comparison 13 © 2024 Synaptics Inc
- 14. • Model development stage KPI: • COCO Instance Mask mAP: 0.636 • Latency: 92.19 ms • Resolution: 480x640 (VGA) • Weights: 1.57 M parameters • Model run on hardware: • Inference time: 96 ms • Total frame time: 120 ms Segmentation run on Stage 3 14 © 2024 Synaptics Inc
- 15. • Building full applications running at ultra-low power requires high levels of integration of hardware and software • Multiple levels of processing is needed to wake up silicon components as needed • Stage 2 and Stage 3 come out of deep sleep based on results from previous stage • The low-power orchestration demands tight software integration • Each stage requires AI models with different KPIs on accuracy, model size, and speed • Need to have NAS-based model generation/training software to enable the complete solution • Solution enables battery-powered devices that are AI capable and can run for many months/years Summary 15 © 2024 Synaptics Inc
- 16. Resources 16 Synaptics Astra embedded processors https://www.synaptics.com/products/embedded-processors Synaptics Astra evaluation Kit https://synacsm.atlassian.net/servicedesk/customer/portal/543/grou p/563/create/6387 Synaptics Astra software https://github.com/synaptics-astra © 2024 Synaptics Inc