Embedded AI applications at the edge, including video analytics and enhancements, necessitate stringent data security measures to protect personal information and comply with legal and DRM requirements, despite often operating on encrypted video data. The session explores challenges in secure data processing and proposes solutions through secure inferencing frameworks, emphasizing efficient and protected handling of sensitive information in AI-driven video applications.

Neural network models for embedded applications are traditionally designed and optimized in two separate stages—architecture selection and quantization—which limits efficiency. We introduce a framework that unifies these stages using the Once-For-All approach, enhancing model performance and efficiency on target hardware by incorporating mixed-precision quantization and optimizing both architecture and quantization precision together, achieving faster and more memory-efficient models compared to conventional methods.

To address the high resource demand of object detection on embedded devices, we replace CNN layers with RNN layers in object detection models, significantly reducing parameter count and power consumption. This novel approach, applied to a modified YOLOx5 model, maintains accuracy on the COCO dataset while offering lower latency, demonstrating RNN's efficiency over CNN for image-based inference.