Reproduce “Feature Coding in the Era of Large Models: Dataset, Test Conditions, and Benchmark” on OpenResearch

About this paper

Changsheng Gao, Yifan Ma, Qiaoxi Chen, Yenan Xu, Dong Liu, Weisi Lin

Researchers from Nanyang Technological University and the University of Science and Technology of China introduce Large Model Feature Coding (LaMoFC) by providing the first comprehensive dataset, unified test conditions, and initial benchmarks. Their analysis demonstrates that VTM generally outperforms a Hyperprior baseline in rate-accuracy performance, but reveals the inadequacy of Mean Squared Error (MSE) for measuring semantic distortion in these features.

Key insights

• Large model features exhibit distinct distribution and redundancy characteristics compared to traditional CNN features, necessitating tailored coding strategies.
• Feature Mean Squared Error (MSE) is largely inadequate for characterizing semantic distortion, especially for textual and non-image-like features, often showing low correlation with task accuracy.
• Learning-based feature codecs (like the Hyperprior baseline) show poor generalizability across different tasks, indicating a need for more versatile and robust designs.

Summary by alphaXiv.