Reproduce “Disentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zahra Rahiminasab, Michael Yuhas, Arvind Easwaran
Researchers at Nanyang Technological University introduced a Disentangled Distilled Encoder (DDE) framework, utilizing constrained optimization and Rademacher guarantees to compress Out-of-Distribution (OOD) reasoners for resource-constrained Cyber-Physical Systems. The method achieved up to a 65% reduction in model size and a 59% decrease in inference time on an NVIDIA Jetson Nano, while maintaining OOD reasoning accuracy.
Key insights
- Disentanglement in latent spaces can be effectively preserved during model compression through a principled constrained optimization approach, even with weak supervision via 'match-pairing' on generative factors.
- Rigorous theoretical analysis using Rademacher complexity provides strong confidence in the generalization capabilities of compressed, disentangled models for OOD reasoning.
- Combining knowledge distillation with disentanglement-preserving constraints enables the deployment of interpretable OOD reasoners on resource-constrained edge devices, bridging a gap for robust AI in safety-critical systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.