Reproduce “ESQA: Event Sequences Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Irina Abdullaeva, Andrei Filatov, Mikhail Orlov, Ivan Karpukhin, Viacheslav Vasilev, Denis Dimitrov, Andrey Kuznetsov, Ivan Kireev and 1 more
ESQA adapts large language models for event sequence question answering, developing methods for structured data encoding, long sequence processing, and temporal feature integration. The model achieved competitive performance across various fintech tasks, including superior accuracy in predicting next-event categorical and temporal attributes.
Key insights
- Diverse event sequence tasks can be effectively unified and solved by adapting LLMs through a natural language question-answering paradigm.
- A multi-stage embedding and encoding pipeline, leveraging pre-trained vision/audio model components (Whisper) and specialized connectors (Q-Former), can efficiently bridge structured event data with LLM architectures.
- Parameter-Efficient Fine-Tuning (LoRA) enables the adaptation of large, pre-trained LLMs to complex event sequence domains with minimal trainable parameters, preserving general linguistic knowledge while specializing for new tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.