Reproduce “GigaChat Audio: Time-aware Large Audio Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, Fyodor Minkin
SaluteDevices researchers developed GigaChat Audio, a time-aware audio Large Language Model, capable of processing up to 120-minute audio recordings and generating responses with explicit, accurate timestamps. The model achieved robust temporal grounding by integrating periodic temporal anchors and training on diverse audio lengths, outperforming existing multimodal LLMs in long-form temporal tasks.
Key insights
- Explicit, periodic temporal anchors (inter-timings) are essential for achieving stable and accurate temporal grounding in long-form audio recordings.
- Training on a diverse mixture of audio durations is critical for robust generalization across varying input lengths, preventing asymmetric performance degradation on either short or long recordings.
- The frequency of temporal anchors presents a trade-off between grounding accuracy and computational overhead, while canonical timestamp formats like hh:mm:ss prove most effective.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.