Reproduce “Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen-An Li, Cheng-Han Chiang, Hung-yi Lee
IRIS-Embed adapts audio-language models for speech retrieval by using text instructions to guide query embeddings. It combines representations from different audio-encoder layers according to the instruction and trains embeddings to rank documents that satisfy its requirements. On INSPIRE, Qwen2.5-Omni-3B reaches 62.92% macro-average Recall@50, but specialized baselines perform better on some individual tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.