Reproduce “From Pixels to Posts: Retrieval-Augmented Fashion Captioning and Hashtag Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Moazzam Umer Gondal, Hamad Ul Qudous, Daniya Siddiqui, Asma Ahmad Farhan
A retrieval-augmented framework for fashion captioning and hashtag generation leverages multi-garment detection and retrieval-based attribute inference to create factually grounded and stylistically diverse social media content. Focusing on South Asian apparel, the system employs large language models to produce descriptions that are more contextual and human-like compared to end-to-end supervised approaches, achieving robust attribute coverage in hashtags.
Key insights
- Decomposing the generation task into detection, retrieval, and LLM-based composition effectively enhances factual grounding and reduces hallucination compared to monolithic end-to-end models.
- Retrieval augmentation provides a mechanism for domain adaptation and generalization, enabling the system to handle new or low-resource categories without extensive fine-tuning.
- While generative models may yield lower lexical overlap scores against reference texts, they can produce more contextualized, human-like, and stylistically appropriate narratives essential for social media engagement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.