Collaborative filtering

lastmod 2025-10-16created 2024-04-2417 edits

Collaborative filtering is a method of automatically predicting users' interests based on taste information gathered from many users. The fundamental assumption of the collaborative filtering approach is the premise that users' past tendencies will hold in the future as well. For example, collaborative filtering for music, or a recommendation system, predicts a user's taste in music using a partial list of that user's preferences (likes, dislikes). A distinguishing feature of this system is that it uses information collected from many users rather than being limited to one particular user's information. This sets it apart from simply producing an average rating of interest for each item based on the number of votes. In other words, it is a technique that identifies customers with similar patterns of preference and interest, based on customers' preferences and expressions of interest. It is used to provide services that cross-recommend products not yet purchased among customers with similar tastes, or that recommend related products according to the tastes or lifestyle of a classified customer.

Projects

• Metarank - Real-time personalization service

Memory-based

Recommends items based on similarity derived from the User-Item Matrix.

User-based Filtering

Recommending by comparing preferences (ratings of items) or purchase histories between users is called User-based CF.

Various methods can be used to compute the similarity between two vectors, but usually Korean: 코사인 유사도 (cosine similarity) or Pearson similarity is used.

Item-based Filtering

The opposite of User-based: recommends by comparing the lists of users between items.

Model-based

• (Machine learning) A complete summary of recommendation algorithm basics - Collaborative filtering , Matrix Factorization, SVD, Factorization machines
• Deep Learning-based recommender systems
• (Survey & Code) Deep Learning based Recommender Systems
• Kakao AI recommendations: on the criteria for choosing a collaborative filtering model – tech.kakao.com

Model-based CF means training a model, such as a machine learning or deep learning model, on User-Item interactions.

It can learn relationships between items, as in market basket analysis (Association rule), or form groups of similar items or users through Clustering.

• Matrix Factorization
• SVD
• Factorization machines
• Bayesian Network
• Decision Tree
• MLP-based Factorization
  • Neural Collaborative Filtering (NCF)
  • Generalized Matrix Factorization (GMF)
• RNN-based Model
  • Session-based Recommendations with Recurrent Neural Networks
  • Neural Attentive Session-based Recommendation
  • AE-based Model
• Alternating Least Squares (ALS; Korean: 교대 최소 제곱법)
  • A technique for optimizing the objective function that trains the Latent Factors of users and items alternately, one at a time.
  • A method that learns the optimal Latent Factors by repeating the process of holding the item matrix constant while training the user matrix, then holding the user matrix constant while training the item matrix.
• SGD
• KNN
• Bayesian Personalized Ranking (BPR)
  • A model that probabilistically models users' preferences for items; it analyzes the items a user prefers by categorizing them step by step (positive items, negative items).
  • When categorizing items it uses only whether or not the user gave implicit feedback, so some information can be lost, but it performs better than earlier techniques.
• Point-wise
  • A method that considers only one item at a time in the loss function (Loss Function); it computes a Score using just one item for one user and optimizes by comparing it with the Label Score.
  • It has the drawback of training items as independent entities, ignoring the order relationships between them, and only sorting the results, but for the same reason it is an intuitive model.
• Pair-wise
  • A method that considers 2 items at a time in the loss function (Loss Function).
  • Because it considers 1 positive item and 1 negative item, the dataset must contain three fields, {user, positive item, negative item}, and Sampling techniques are used to deal with the resulting data duplication.
• nDCG (normalized Discounted Cumulative Gain)
  • An evaluation metric widely used for ranking recommendation in recommender systems; especially useful in domains where the top of the ranking list is clearly more important than the lower part.
• Latent Factor
  • A model that tries to represent the hidden features in user and item data using latent dimensions (Factor); the Matrix Factorization approach is the typical example.
• Autoencoder
  • An unsupervised learning model that can be trained on the given data alone, with no need to build separate label data for the training data; it consists of two neural networks, the Encoder and the Decoder.
  • The Encoder keeps only the information in the input data judged important for learning and removes the data judged unnecessary, producing data smaller than the original input,
  • and the Decoder restores it to the original input image. The smaller data produced by the Encoder is called the Latent Vector.
  • In other words, the way an Autoencoder learns and generates data is similar to drawing a composite sketch after hearing a one-sentence description.
• Generalization
  • Refers to a model reaching a state where it does not overfit (Overfitting) to specific data and can be applied to all kinds of data.

Improving recommender systems and search in the age of LLMs

See Recommender system#Improving recommender systems and search in the age of LLMs.

Content-based filtering vs Collaborative filtering

See Recommender system#Content-based filtering vs Collaborative filtering.

Implementing Matrix Factorization with Pytorch

• Implementing collaborative filtering (Matrix Factorization) with Pytorch - Dohee’s ML Lab

MLP based Recommender Systems Architecture

• MLP based Recommender Systems Architecture (2022)

• DIAG: A Deep Interaction-Attribute-Generation model for user-generated item recommendation
• A Ranking Recommendation Algorithm Based on Dynamic User Preference
• A Recommendation System Framework to Generalize AutoRec and Neural Collaborative Filtering
• Deep Collaborative Recommendation Algorithm Based on Attention Mechanism
• DeepSet: Deep Learning-based Recommendation with Setwise Preference
• Dual attention recommendation algorithm based on item attributes
• IntegrateCF: Integrating explicit and implicit feedback based on deep learning collaborative filtering algorithm
• Neural Attention Networks for Recommendation with Auxiliary Data
• Neural Matrix Factorization Recommendation for User Preference Prediction Based on Explicit and Implicit Feedback
• POI Recommendation Method of Neural Matrix Factorization Integrating Auxiliary Attribute Information
• Simultaneous Learning of the Inputs and Parameters in Neural Collaborative Filtering
• SDNN: Symmetric deep neural networks with lateral connections for recommender systems
• BRUCE: Bundle Recommendation Using Contextualized item Embeddings
• Context and Attribute-Aware Sequential Recommendation via Cross-Attention
• Effective and Efficient Training for Sequential Recommendation using Recency Sampling
• Modeling User Repeat Consumption Behavior for Online Novel Recommendation
• Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)

See also

• Recommender system (Korean: 추천 시스템)
• Content-based filtering (Korean: 컨텐츠 기반 필터링)
• Collaborative filtering (Korean: 협업 필터링)
• Hybrid recommender systems
• Learning to rank (machine-learned ranking; MLR; Korean: 순위 학습)
• Similarity (Korean: 유사도)

Favorite site

• Collaborative filtering - Wikipedia
• Collaborative Filtering Recommendation System
• Recommender system basics - Collaborative Filtering - ①
• 03. Collaborative filtering-based recommender systems - KNN
• 03. Collaborative filtering-based recommender systems - SGD
• 03. Collaborative filtering-based recommender systems - ALS - Alternating Least Squares (ALS)