Reproduce “Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anjali Khurana, Hari Subramonyam, Parmit K Chilana

Research from Simon Fraser University and Stanford University investigated how end-users interact with LLM-based assistants for software help, revealing that even enhanced models providing more accurate guidance do not significantly improve user task completion or accuracy due to users' overtrust and difficulties mapping textual instructions to visual interfaces.

Key insights

  • Users exhibit 'blind faith' in LLM outputs, perceiving even hallucinated or incorrect information as credible due to coherent, human-like presentation, particularly problematic for novice users.
  • A significant disconnect exists between improved LLM output accuracy and actual user task performance, indicating that better LLM responses alone do not guarantee successful task completion.
  • Users often default to search engine-like, keyword-based prompting and tend to ignore explicit in-context prompt guidelines, struggling to understand their impact on LLM output quality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.