Reproduce “Forward and Inverse models in HCI:Physical simulation and deep learning for inferring 3D finger pose” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Roderick Murray-Smith, John H. Williamson, Andrew Ramsay, Francesco Tonolini, Simon Rogers, Antoine Loriette
University of Glasgow researchers developed a framework combining physics-based forward models with deep learning-driven inverse models for robust 3D finger pose inference from capacitive sensor data. This approach achieved state-of-the-art accuracy, inferring 3D position within 0.21 cm (x,y) and 0.13 cm (z), and pose within 7.8° (pitch) and 7.2° (yaw) on mobile devices up to 5cm above the screen.
Key insights
- A formal forward and inverse modeling framework offers a more rigorous and predictable approach to HCI sensor interpretation compared to traditional ad-hoc methods.
- Intelligently fusing multi-fidelity data sources, particularly leveraging computationally accelerated physics simulations, significantly mitigates the challenge of obtaining extensive paired ground-truth data for complex inverse problems.
- Probabilistic models like CVAEs can effectively handle the inherent uncertainty of ill-posed inverse problems in HCI, providing not just point estimates but also measures of confidence in their predictions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.