Reproduce “Adaptive Neural Networks for Intelligent Data-Driven Development” on OpenResearch

About this paper

Youssef Shoeb, Azarm Nowzad, Hanno Gottschalk

Continental AG and Technische Universität Berlin researchers developed an adaptive neural network framework to efficiently integrate previously unknown objects into perception systems, crucial for autonomous driving. Their "smart data loop" combines dynamic Out-of-Distribution detection with retrieval-based data augmentation and a parameter-efficient network extension, achieving comparable performance to state-of-the-art methods while significantly reducing parameter count and mitigating catastrophic forgetting.

Key insights

• Freezing the backbone and adding lightweight, class-specific "adaptive heads" trained with a contrastive loss effectively mitigates catastrophic forgetting while enabling incremental learning.
• Utilizing generative Gaussian Mixture Models (GMMs) for classification provides probabilistic class membership, offering more flexibility for integrating new categories compared to softmax.
• Dynamic OoD detection based on likelihood ratios, which updates to account for incrementally learned classes, allows the system to continuously identify true novelties in evolving environments.

Summary by alphaXiv.