Reproduce “Using Neural Networks for Data Cleaning in Weather Datasets” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jack R. P. Hanslope, Laurence Aitchison
The authors test a network mitigating mismatches between ERA5 conditions and IBTrACS storm locations in North Indian cyclones. The U-Net maps 850 hPa winds to grid-cell probabilities, training on track labels. In a blinded test, one author preferred its location in 46 of 59 test and 49 of 64 training preferences; this does not verify storm centers or generalization.
Key insights
- Neural networks can exhibit an implicit "denoising effect" when trained on labels known to be inconsistent with their corresponding input features, learning a more accurate underlying representation.
- The U-Net model learned to infer storm locations that were more consistent with the visual characteristics of wind fields than the original, imperfect IBTrACS labels.
- The network improved upon the provided labels even on the training set, indicating a generalization to a more ideal target rather than simply overfitting to the noisy training data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.