Reproduce “Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junwei Hu, Weicheng Zheng, Yihan Liu, Yan Liu
A study from Tongji University investigates methods to optimize token consumption in Large Language Models (LLMs) for code reasoning, particularly with Chain of Thought (CoT) processes. It demonstrates that strategic code refactoring and specific prompt engineering techniques can substantially reduce token usage and inference time, often improving generated code quality.
Key insights
- Intrinsic code characteristics directly influence LLM resource usage; smelly code consistently leads to significantly higher token consumption than clean code during inference.
- External optimization strategies, such as code refactoring and prompt engineering, can effectively mitigate token inflation without requiring modifications to the LLM's internal architecture.
- The type of code smell matters, with structural and expression-related complexities imposing a greater token burden on LLMs, offering guidance for targeted refactoring efforts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.