Reproduce “LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Felipe Tommaselli, Francisco Affonso, Arthur Pompeu, Gianluca Capezzuto, Arun Narenthiran Sivakumar, Girish Chowdhary, Marcelo Becker

LeCropFollow, a visual navigation framework developed by researchers at the University of São Paulo and the University of Illinois Urbana-Champaign, enhances agricultural robot autonomy by using latent space planning with semantic heatmaps and Model-Based Reinforcement Learning. It achieves superior robustness in unstructured crop fields, particularly in plantation gaps, and demonstrates successful zero-shot sim-to-real transfer.

Key insights

  • Preserving the full semantic context and uncertainty encoded in high-dimensional visual heatmaps, rather than reducing them to explicit geometric keypoints, is vital for robust navigation in unstructured agricultural environments.
  • Model-Based Reinforcement Learning (MBRL) with a learned world model, which plans trajectories over a multi-step horizon in a latent space, is critical for enabling anticipatory behavior and overcoming environmental discontinuities like plantation gaps.
  • Zero-shot sim-to-real transfer can be effectively achieved by training the learned world model and policy in a perceptually "harder" simulated environment, exposing the system to noisier and more diffuse visual activations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.