Reproduce “A Hierarchical Grocery Store Image Dataset with Visual and Semantic Labels” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marcus Klasson, Cheng Zhang, Hedvig Kjellström
Researchers from KTH Royal Institute of Technology and Microsoft Research developed and released a hierarchical grocery store image dataset featuring 5125 natural images, 81 fine-grained classes, and multimodal labels including iconic images and product descriptions. This dataset facilitates the development of robust assistive technologies for visually impaired individuals by enabling accurate product recognition in complex, real-world shopping environments, demonstrating enhanced classification accuracy through multimodal learning and improved model interpretability.
Key insights
- Leveraging multimodal information, specifically iconic images, significantly enhances the quality of learned latent representations in multi-view VAE-CCA models, leading to improved classification accuracy.
- Fine-tuning pretrained CNNs on the dataset's real-world images is crucial for achieving high recognition performance, outperforming methods based on off-the-shelf features.
- The VAE-CCA model offers visual interpretability, capable of decoding noisy natural images into recognizable iconic product images, indicating robustness to clutter and background noise.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.