Reproduce “Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddings” on OpenResearch

About this paper

Theodor Westny, David Axelsson, Björn Olofsson, Erik Frisk

A framework is introduced that learns latent dataset-level representations to quantify transferability between trajectory prediction datasets using KL divergence. This latent dissimilarity metric strongly correlates with cross-dataset model performance, showing a Spearman's rank correlation of 0.811 for zero-shot transfer and 0.729 for fine-tuning degradation.

Key insights

• Latent dataset embeddings effectively capture complex behavioral and structural patterns, revealing nuanced relationships and asymmetries in transferability that go beyond simple statistical features.
• The KL divergence between latent dataset distributions provides a robust, predictive measure of actual cross-dataset model performance, guiding the selection of optimal pretraining sources and anticipating catastrophic forgetting.
• Analysis of 24 diverse datasets uncovered unexpected cross-domain transfer potential (e.g., between certain drone-acquired and instrumented-vehicle datasets), suggesting underutilized data sources for improved model generalization.

Summary by alphaXiv.