Reproduce “Towards smart and adaptive agents for active sensing on edge devices” on OpenResearch

About this paper

Devendra Vyas, Nikola Pižurica, Nikola Milović, Igor Jovančević, Miguel de Prado, Tim Verbelen

Researchers combined deep learning-based perception with active inference planning to develop a smart, adaptive agent for active sensing on edge devices. The integrated system achieves real-time operation on an Nvidia Jetson Orin NX, with total memory usage as low as 304 MB and perception at 45 FPS, enabling dynamic control of IoT cameras.

Key insights

• Active inference provides a principled, lightweight framework for real-time planning and adaptation on edge devices, effectively complementing deep learning's strengths in feature extraction and object detection.
• Strategic optimization of deep learning models with edge-specific inference engines (e.g., TensorRT with FP16 precision on GPU) is crucial for achieving high throughput and efficient memory usage on constrained hardware.
• The combined deep learning and active inference approach enables a unified perception-action loop, allowing agents to actively seek information and dynamically adjust behavior in response to environmental uncertainty.

Summary by alphaXiv.