Reproduce “Continuous Object State Recognition for Cooking Robots Using Pre-Trained Vision-Language Models and Black-box Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kento Kawaharazuka, Naoaki Kanazawa, Yoshiki Obinata, Kei Okada, Masayuki Inaba
Researchers from The University of Tokyo developed a method for continuous food state recognition in cooking robots by leveraging pre-trained Vision-Language Models (VLMs) and optimizing text prompt weights using a genetic algorithm. The approach improved accuracy and stability in detecting continuous changes like water boiling and butter melting, culminating in a successful robotic cooking demonstration.
Key insights
- Black-box optimization of VLM text prompt weights significantly enhances the accuracy and stability of continuous state recognition compared to uniform or single-prompt approaches.
- Pre-trained VLMs can effectively interpret subtle, continuous visual cues of food transformation through natural language descriptions, reducing the need for extensive task-specific training.
- The sigmoid function provides a robust and generalizable mathematical representation for diverse patterns of continuous state changes observed in cooking processes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.