Nonparametric Click Modeling Using Dirichlet Process Mixture Model for Information Retrieval

Click models are essential for comprehending user search behavior and enhancing ranking algorithms; nevertheless, current methodologies face challenges due to the variability of user interaction patterns across different query settings. Current techniques depend on static mixture components that might not accurately represent the diverse behaviors of users across various contexts and demographics. This study presents a Dirichlet Process Mixture Model for click modeling that autonomously adjusts model complexity to accommodate diverse user behavior patterns without necessitating predetermined assumptions on the quantity of user behavior clusters. This approach develops an efficient inference algorithm that alternates between Bayesian cluster assignment and neural network training, enabling scalable learning for large-scale click prediction applications. The theoretical foundation builds upon established Dirichlet Process theory while extending it to neural click modeling, providing convergence guarantees for the proposed inference system. A comprehensive comparative evaluation of click modeling approaches is conducted, comparing proposed method to five established baseline techniques across nine distinct click model configurations. The experimental results indicate that the Dirichlet Process Mixture Model consistently excels existing baselines across various evaluation metrics and demonstrates a particular aptitude for rating quality metrics when contrasted with the baseline averages. Experimental validation on real-world data shows that Dirichlet Process Mixture Model achieves substantial improvements over existing methods, with a 75.5% relative improvement in Mean Average Precision (0.6138 vs. 0.3496) and 48.7% improvement in Precision@1 (0.6539 vs. 0.4490).