EnsVAE: Ensemble Variational Autoencoders for Recommendations

Proceedings of the 13th International Conference on Web Search and Data Mining

Recent research has shown the advantages of using autoencoders based on deep neural networks for collaborative filtering. In particular, the recently proposed Mult-VAE model, which used the multinomial likelihood variational autoencoders, has shown excellent results for top-N recommendations. In this work, we propose the Recommender VAE (RecVAE) model that originates from our research on regularization techniques for variational autoencoders. RecVAE introduces several novel ideas to improve Mult-VAE, including a novel composite prior distribution for the latent codes, a new approach to setting the β hyperparameter for the β-VAE framework, and a new approach to training based on alternating updates. In experimental evaluation, we show that RecVAE significantly outperforms previously proposed autoencoder-based models, including Mult-VAE and RaCT, across classical collaborative filtering datasets, and present a detailed ablation study to assess our new developments. Code and models are available at https://github.com/ilya-shenbin/RecVAE. CCS CONCEPTS • Computing methodologies → Latent variable models; Learning from implicit feedback; Regularization; Neural networks; • Information systems → Learning to rank.

Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining - WSDM '19, 2019

Variational autoencoders were proven successful in domains such as computer vision and speech processing. Their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention in the current literature. In this work, we propose a model which extends variational autoencoders by exploiting the rich information present in the past preference history. We introduce a recurrent version of the VAE, where instead of passing a subset of the whole history regardless of temporal dependencies, we rather pass the consumption sequence subset through a recurrent neural network. At each time-step of the RNN, the sequence is fed through a series of fully-connected layers, the output of which models the probability distribution of the most likely future preferences. We show that handling temporal information is crucial for improving the accuracy of the VAE: In fact, our model beats the current state-of-the-art by valuable margins because of its ability to capture temporal dependencies among the user-consumption sequence using the recurrent encoder still keeping the fundamentals of variational autoencoders intact.

Lecture Notes in Computer Science, 2020

In this paper, we propose a Conditioned Variational Autoencoder (C-VAE) for constrained top-N item recommendation where the recommended items must satisfy a given condition. The proposed model architecture is similar to a standard VAE in which the condition vector is fed into the encoder. The constrained ranking is learned during training thanks to a new reconstruction loss that takes the input condition into account. We show that our model generalizes the state-of-the-art Mult-VAE collaborative filtering model. Moreover, we provide insights on what C-VAE learns in the latent space, providing a human-friendly interpretation. Experimental results underline the potential of C-VAE in providing accurate recommendations under constraints. Finally, the performed analyses suggest that C-VAE can be used in other recommendation scenarios, such as context-aware recommendation.

Collaborative Filtering aims at exploiting the feedback of users to provide personalised recommendations. Such algorithms look for latent variables in a large sparse matrix of ratings. They can be enhanced by adding side information to tackle the well-known cold start problem. While Neural Networks have tremendous success in image and speech recognition, they have received less attention in Collaborative Filtering. This is all the more surprising that Neural Networks are able to discover latent variables in large and heterogeneous datasets. In this paper, we introduce a Collaborative Filtering Neural network architecture aka CFN which computes a nonlinear Matrix Factorization from sparse rating inputs and side information. We show experimentally on the MovieLens and Douban dataset that CFN outperforms the state of the art and benefits from side information. We provide an implementation of the algorithm as a reusable plugin for Torch, a popular Neural Network framework.

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021

Variational Autoencoders (VAEs) have shown to be effective for recommender systems with implicit feedback (e.g., browsing history, purchasing patterns, etc.). However, a little attention is given to ensembles of VAEs, that can learn user and item representations jointly. We introduce Joint Variational Autoencoder (JoVA), an ensemble of two VAEs, which jointly learns both user and item representations to predict user preferences. This design allows JoVA to capture user-user and item-item correlations simultaneously. We also introduce JoVA-Hinge, a JoVA's extension with a hinge-based pairwise loss function, to further specialize it in recommendation with implicit feedback. Our extensive experiments on four realworld datasets demonstrate that JoVA-Hinge outperforms a broad set of state-of-the-art methods under a variety of commonly-used metrics. Our empirical results also illustrate the effectiveness of JoVA-Hinge for handling users with limited training data. CCS CONCEPTS • Information systems → Collaborative filtering; Learning to rank.

Research Square (Research Square), 2024

Recommendation systems are crucial in boosting companies' revenues by implementing various strategies to engage customers and encourage them to invest in products or services. Businesses constantly desire to enhance these systems through different approaches. One effective method involves using hybrid recommendation systems, known for their ability to create high-performance models. We introduce a hybrid recommendation system that leverages two types of recommendation systems: first, a novel deep learning-based recommendation system that utilizes users' and items' content data, and second, a traditional recommendation system that employs users' past behaviour data. We introduce a novel deep learning-based recommendation system called Convolutional Autoencoder Recommendation System (CAERS). It uses a Convolutional Autoencoder (CAE) to capture high-order meaningful relationships between users' and items' content information and decode them to predict ratings. Subsequently, we design a traditional model-based collaborative filtering recommendation system (CF) that leverages users' past behaviour data, utilizing Singular Value Decomposition (SVD). Finally, in the last step, we combine the two method's predictions with linear regression. We determine the optimal weight for each prediction generated by the collaborative filtering and the deep learning-based recommendation system. Our main objective is to introduce a hybrid model called CAERS-CF that leverages the strengths of the two mentioned approaches. For experimental purposes, we utilize two movie datasets to showcase the performance of CAERS-CF. Our model outperforms each constituent model individually and other state-ofthe-art deep learning or hybrid models. Specifically, the hybrid CAERS-CF model exhibits improvements across various metrics, highlighting the effectiveness of our approach in enhancing recommendation systems.

Proficient Recommender Systems heavily rely on Matrix Factorization (MF) techniques. MF aims at reconstructing a matrix of ratings from an incomplete and noisy initial matrix; this prediction is then used to build the actual recommendation. Simultaneously, Neural Networks (NN) met tremendous successes in the last decades but few attempts have been made to perform recommendation with autoencoders. In this paper, we gather the best practice from the literature to achieve this goal. We first highlight the link between these autoencoder based approaches and MF. Then, we refine the training approach of autoencoders to handle incomplete data. Second, we design an end-to-end system which handles external information. Finally, we empirically evaluate these approaches on the MovieLens and Douban dataset.

The Indonesian Journal of Electrical Engineering and Computer Science (IJEECS), 2023

In recent years, information overload has become a phenomenon where it makes people difficult to filter relevant information. To address issues such as high-dimensional data, cold start, and data sparsity, semi-autoencoder is one of the unsupervised deep learning methods used in the recommendation systems. It is particularly useful for reducing data dimensions, capturing latent representations, and flexibly reconstructing various parts of input data. In this article, we propose an improved hybrid semi-stacked autoencoder for itemfeatures of recommendation system (iHSARS) framework. This method aims to show better performance of the hybrid collaborative recommendation via semi-autoencoder (HRSA) technique. Two novel elements for iHSARS's architecture have been introduced. The first element is an increase sources of side information of the input layer, while the second element is the number of hidden layers has been expanded. To verify the improvement of the model, MovieLens-100K and MovieLens-1M datasets have been applied to the model. The comparison between the proposed model and different state-of-the-art methods has been carried using mean absolute error (MAE) and root mean square error (RMSE) metrics. The experiments demonstrate that our framework improved the efficiency of the recommendation system better than others.

Applied Sciences

Visual representation of user and item relations is an important issue in recommender systems. This is a big data task that helps to understand the underlying structure of the information, and it can be used by company managers and technical staff. Current collaborative filtering machine learning models are designed to improve prediction accuracy, not to provide suitable visual representations of data. This paper proposes a deep learning model specifically designed to display the existing relations among users, items, and both users and items. Making use of representative datasets, we show that by setting small embedding sizes of users and items, the recommender system accuracy remains nearly unchanged; it opens the door to the use of bidimensional and three-dimensional representations of users and items. The proposed neural model incorporates variational embedding stages to “unpack” (extend) embedding representations, which facilitates identifying individual samples. It also replac...

Computers, Materials & Continua

The era of the Internet of things (IoT) has marked a continued exploration of applications and services that can make people's lives more convenient than ever before. However, the exploration of IoT services also means that people face unprecedented difficulties in spontaneously selecting the most appropriate services. Thus, there is a paramount need for a recommendation system that can help improve the experience of the users of IoT services to ensure the best quality of service. Most of the existing techniques-including collaborative filtering (CF), which is most widely adopted when building recommendation systems-suffer from rating sparsity and cold-start problems, preventing them from providing high quality recommendations. Inspired by the great success of deep learning in a wide range of fields, this work introduces a deep-learning-enabled autoencoder architecture to overcome the setbacks of CF recommendations. The proposed deep learning model is designed as a hybrid architecture with three key networks, namely autoencoder (AE), multilayered perceptron (MLP), and generalized matrix factorization (GMF). The model employs two AE networks to learn deep latent feature representations of users and items respectively and in parallel. Next, MLP and GMF networks are employed to model the linear and non-linear user-item interactions respectively with the extracted latent user and item features. Finally, the rating prediction is performed based on the idea of ensemble learning by fusing the output of the GMF and MLP networks. We conducted extensive experiments on two benchmark datasets, MoiveLens100K and MovieLens1M, using four standard evaluation metrics. Ablation experiments were conducted to confirm the validity of the proposed model and the contribution of each of its components in achieving better recommendation performance. Comparative analyses were also carried out to demonstrate the potential of the proposed model in gaining better accuracy than the existing CF methods with resistance to rating sparsity and cold-start problems.