Reproduce “Batch Normalization with Enhanced Linear Transformation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuhui Xu, Lingxi Xie, Cihang Xie, Jieru Mei, Siyuan Qiao, Wei Shen, Hongkai Xiong, Alan Yuille

Researchers from Johns Hopkins University, Shanghai Jiao Tong University, and UC Santa Cruz introduced BNET, a modification to Batch Normalization's linear transformation that integrates local spatial context via depth-wise convolution. This enhancement yields consistent performance improvements of up to 4.3% in action recognition and 2.0% in object detection across diverse computer vision tasks and network architectures with negligible computational overhead.

Key insights

  • The linear transformation step of Batch Normalization, typically a simple per-channel scaling and shifting, presents an overlooked opportunity for improvement.
  • Introducing local spatial contextual information during this post-normalization recovery step significantly enhances the model's ability to learn complex representations.
  • Effective normalization layers necessitate a balance between rigid normalization for stability and sufficient flexibility in feature recovery for expressive power.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.