Reproduce “Embedding-based Recommender System for Job to Candidate Matching on Scale” on OpenResearch

About this paper

Jing Zhao, Jingya Wang, Madhav Sigdel, Bopeng Zhang, Phuong Hoang, Mengshu Liu, Mohammed Korayem

CareerBuilder developed a two-stage embedding-based recommender system for job-to-candidate matching, achieving a 104% increase in online click-through rate and an 18% improvement in offline nDCG compared to their baseline. This system uses a fused-embedding strategy combining deep learning, knowledge graphs, and geolocation, scaled with Approximate Nearest Neighbor search.

Key insights

• Domain-specific deep learning models, like their CNN-based DLEM trained on job data, generate superior embeddings for job classification compared to general pre-trained models like distilBERT.
• Combining diverse embedding types (raw text, structured semantic entities, and geographical location) into a single "fused-embedding" creates a richer, more accurate representation for complex job and candidate profiles.
• A two-stage architecture effectively balances high recall in the initial retrieval phase with refined precision and contextual awareness during reranking, crucial for industrial-scale recommender systems.

Summary by alphaXiv.