Reproduce “Inducing Task Models from Computer-Use Traces” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yucheng Jiang, Zora Zhiruo Wang, Ruishi Chen, Diyi Yang
TMI turns computer activity into task models pairing objective hierarchies with trace-grounded control flow. It interprets events, groups activities by latent task, then reconciles objective and procedure models. On synthetically shuffled single-task sessions, task assignment reached an adjusted Rand index of 0.974. Step descriptions had a 74.9% judged pass rate; transfer accuracy was 18.57% on held-out instances within task families.
Key insights
- Decomposing raw events into semantic actions and then into goal-oriented activities provides a necessary bridge from low-level interactions to higher-level user intent.
- Inducing distinct hierarchical objective models (the "why") and structured procedure models (the "how") separately, then reconciling them, is essential for a complete and robust task representation.
- Naturalistic, multi-threaded computer usage can be effectively disentangled into distinct latent tasks, even when activities are interleaved, without requiring predefined task boundaries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.