Reproduce “FINSKILLOPS: A Self-Evolving Multi-Agent System for SEC Filing QA” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanzhang Ma, Zhenghan Tai, Hanwei Wu, Sizhe Guan, Jianliang Lei, Hailin He, Chaolong Jiang, Jijun Chi and 20 more
FINSKILLOPS treats post-deployment SEC-filing question answering as controlled behavioral maintenance: it diagnoses recurring errors, writes scoped natural-language skills with false-trigger guards, and tests them against targeted, protected, and negative-control cases before promotion. On a 114-question enhanced in-house benchmark, W-Corr. was 4.55 versus 3.70 without skill evolution. This judge-based score does not independently verify evidence support.
Key insights
- Self-improvement is framed as lifecycle governance: diagnosing failures and writing a correction is insufficient unless the system also controls applicability, tests side effects, and records or reverses updates.
- The strongest benefits come from specialized, narrowly scoped repairs—especially fiscal-period alignment—rather than broad instructions to improve accuracy.
- Skill libraries need consolidation as well as growth: in the operational study, 33 proposals produced only six promotions, and the final active registry shrank from 421 to 347 tokens.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.