Reproduce “When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Parth Darshan, Abhishek Divekar
Researchers from IIT Jodhpur and Amazon empirically investigate multi-objective prompt optimization for Large Language Model (LLM) judges, identifying "optimization-time gradient dilution" and "inference-time instruction interference" as two distinct failure modes that hinder effectiveness. Their study shows multi-objective textual gradient optimization frequently degrades performance or stagnates, resulting in a 59% reduction in gradient specificity during combined gradient calls and causing a 0.085 Spearman ρ degradation when individually optimal instructions are integrated.
Key insights
- Multi-objective textual gradient optimization for LLM judges is significantly impeded by two distinct failure modes: optimization-time gradient dilution and inference-time instruction interference.
- Gradient dilution occurs when the gradient LLM processes multiple tasks jointly, struggling to provide task-specific feedback and leading to overly generic suggestions.
- Inference-time instruction interference arises when individually optimal, often verbose, per-task instructions are combined, causing disproportionate attention to longer rubrics during the LLM judge's inference.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.