Reproduce “Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Youxing LI
PixelTriage selects which retrieved images a multimodal memory assistant should send as pixels, using dialogue, notes, and thumbnails rather than full-resolution images. It predicts which pixels add information missing from text proxies, then delivers selected images alongside proxies. On M³Exam, one-image delivery scored 75.73%, versus 75.08% for opening all images, using 23% of visual tokens; the difference was not statistically significant.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.