Project Overview: A Deep Learning Approach to Research Paper

Recommendation

Objective: To build and validate a system that recommends relevant research papers to a user based on
the title and abstract of a source paper.

Core Components:

1. Paper Classification: Training a model to accurately predict the subject area of a research paper.
2. Paper Retrieval: Developing a recommendation engine to find and rank the most relevant papers.
3. Validation: Implementing a robust framework to evaluate the performance of both the
classification and retrieval systems.

Primary Resources:

Dataset: Arxiv Paper Abstracts on Kaggle

Video Reference: Deep Learning Project Idea: Research Paper Recommendation System

Part 1: Paper Classification

A. Basic Information

Goal: To build a robust model that can automatically classify a research paper into its correct subject
area (e.g., "Computer Science," "Physics," "Mathematics") using only its abstract.

Importance:

System Feature: This model provides a key feature for the end-user by telling them the predicted
subject of their input paper.
Crucial for Validation: As you correctly identified, a reliable classifier is essential for the
automated quantitative validation of our recommendation system. We will use this model to check
if the recommended papers are on the same topic as the input paper.

B. Methodology

1. Data Preparation & Preprocessing:

Source: Utilize the [Link] file from the Kaggle

dataset.
Feature Extraction: Extract the abstract (as the input feature) and categories (as
the target label) for each paper.

Page 1 of 4
 Data Cleaning: The abstracts will require significant cleaning: remove newline characters,
special characters, and extra spaces.
Label Simplification: The categories field contains multiple specific tags (e.g., [Link] ,
[Link] ). We will need to map these to broader parent categories (e.g., Computer
Science ) to create a manageable set of labels for classification.

2. Text Vectorization:

Convert the cleaned text abstracts into numerical vectors that a machine learning model
can understand.
Technique: We will use the TF-IDF (Term Frequency-Inverse Document Frequency)
method. This technique effectively represents the importance of a word in a document
relative to its frequency across all documents.

3. Model Training:

Model Choice: A Multilayer Perceptron (MLP) is an excellent choice for this text
classification task. As shown in the reference video, it is a powerful deep learning model
capable of learning complex patterns.
Architecture: We will build a sequential model with dense layers, using ReLU activation
functions for hidden layers and a Sigmoid (for multi-label) or Softmax (for single-label)
activation for the output layer. Dropout layers will be included to prevent overfitting.
Training Process: The model will be trained on the vectorized abstracts and their
corresponding category labels. We will split the data into training, validation, and test sets
to monitor performance and prevent overfitting.

Part 2: Paper Retrieval (Recommendation)

A. Basic Information

Goal: To develop a system that takes the abstract of a user's paper as input and returns a ranked list of
the top 5 most similar or relevant papers from our dataset.

Core Idea: The "relevance" of a paper is determined by the similarity of its abstract to the user's input
abstract. If two abstracts discuss similar concepts using similar language, they are considered relevant.

B. Methodology

1. Corpus Creation:

The entire collection of abstracts from the Kaggle dataset will form our "corpus"—the pool of
documents from which we will retrieve recommendations.

Page 2 of 4
 2. Similarity Measurement:

Vectorization: Just like in the classification task, all abstracts in the corpus must be
converted into numerical vectors. We will use the same TF-IDF vectorization technique to
ensure consistency.
Similarity Metric: We will use Cosine Similarity. This metric calculates the cosine of the
angle between two vectors. It is highly effective at determining how similar two documents
are, regardless of their length. A score of 1 means they are identical, and 0 means they are
completely different.

3. Retrieval Process:

A user provides an abstract.

This input abstract is cleaned and vectorized using our pre-trained TF-IDF vectorizer.
We calculate the Cosine Similarity score between the input vector and the vector of every
single paper in our corpus.
The papers are then ranked in descending order based on their similarity score.
The top 5 papers are returned to the user as the final recommendation.

Part 3: Validation

A. Basic Information

Goal: To rigorously measure the performance of both the classification model and the retrieval system
to understand their effectiveness and identify areas for improvement.

B. Methodology for Validation

1. Validating the Classification Model:

Method: We will use the held-out test set (data the model has never seen). We will compare
the model's predicted subject areas against the true labels from the dataset.
Metrics:
Accuracy: Overall percentage of correct predictions.
Precision, Recall, and F1-Score: A more detailed look at the performance for each
subject category, which is crucial for understanding class-specific performance.
Confusion Matrix: A visual tool to see which categories the model is confusing with
each other.

2. Validating the Retrieval System (Your Proposed Method):

This uses the validated classifier to create an automated evaluation pipeline for the
recommender.

Page 3 of 4
Method: Topical Consistency Evaluation
1. Take each paper from our test set as an input to the recommender.
2. Generate the top 5 recommended papers.
3. For each of the 5 recommendations, use our validated classifier to predict its subject
area.
4. A recommendation is marked as a "Hit" if its predicted subject area matches the true
subject area of the input paper.
Primary Metric:

Mean Precision@5: We will calculate (Number of Hits) / 5 for each input

paper and then average this score across the entire test set. This single number will
give us a strong, quantifiable measure of our recommender's ability to provide
topically consistent results.

Mean Reciprocal Rank (MRR): This metric evaluates how high up the first correct
recommendation appears in the list. It answers: "On average, how quickly does a user
find the first relevant item?" A score of 1 is perfect (the first item is always a hit); a
score of 0.5 means the first hit is, on average, at position 2.

Success Rate@K (or Hit Rate@K): A simpler metric that measures the percentage of
all queries for which at least one relevant item was found in the top K
recommendations. It answers: "What perc

Page 4 of 4