Expert CF: Solving Data Matrix Sparsity and Computation Complexity Problems
DOI:
https://doi.org/10.14738/tmlai.62.4337Keywords:
Recommendation, Collaborative Filtering, Artificial Intelligence, Experts, Data Matrix CompletionAbstract
Collaborative Filtering (CF) is widely used to provide recommendations in ecommerce systems. CF works on a large data set by constructing an item-user matrix through association analyses among items and similarity analyses among users. However, CF suffers from data sparsity and computation complexity. This paper introduces a concept of “experts” to overcome the two identified problems. An expert is an artificially created user, who represents a cluster of users in terms of behavior and taste. The construction of experts can be done off-line through data filtering and classification. In actual recommendation, when data are spars, a number of experts can be added as existing users to predicate shopping habit for a particular user. The mechanism of “off-line expert’s construction” and “on-line expert’s addition” in recommendation not only to overcome the data sparsity but also improving on scalability by reduce computation in recommendation phase.References
(1) J.B. Schafer, J.A. Konstan, and J. Reidl, “E-Commerce Recommendation Applications,” Data Mining and Knowledge Discovery, Kluwer Academic, 2001, pp. 115-153.
(2) Greg Linden, Brent Smith, and Jeremy York，“Amazon.com Recommendations Item-to-Item Collaborative Filtering” IEEE INTERNET COMPUTING
(3) Gediminas Adomavicius, Alexander Tuzhilin. (2005). Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING. 17, 734-749.
(4) F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, R. E. Gruber. (2006). Bigtable: a distributed storage system for structured data. Proceedings of OSDI, Berkeley, CA, USA. USENIX Association. 205-218.
(5) WY Chen，Y Song，H Bai，CJ Lin，EY Chang. (2011). Parallel spectral clustering in distributed systems. IEEE Transactions on Software Engineering. 33(3), 568-586.
(6) G. Linden, B. Smith, J. York. (2003). Amazon.com Recommendations: Item-to-Item Collaborative Filtering, IEEE Internet Computing, Jan./Feb.
(7) Nathan Nan Liu, Xiangrui Meng, Chao Liu, Qiang Yang. (2011). Wisdom of the Better Few: Cold Start Recommendation via Representative based Rating Elicitation. Proceedings of the 5th ACM Recommender Systems Conference.
(8) Daqiang Zhang, Jiannong Cao, Jingyu Zhou, Minyi Guo and Vaskar Raychoudhury. (2009). An Efficient Collaborative Filtering Approach Using Smoothing and Fusing. 2009 International Conference on Parallel Processing. 558-565.
(9) X Amatriain，N Lathia，JM Pujol，H Kwak and N Oliver. (2009). The Wisdom of the Few A Collaborative Filtering Approach Based on Expert Opinions from the Web. International Acm Sigir Conference on Research & Development in Information Retrieval. 532-539.
(10) Gangmin Li, Minghuang Chi and Gautam Pal. (2017). Expert CF: Sparse data matrix completion with artificial experts. International Conference on Recent Advancements in Computing, IoT and Computer Engineering Technology (CICET’17). Taipei, Taiwan October 23-25, 2017.
