Spam emails can be both annoying and dangerous, often attempting to fool individuals into revealing sensitive information or falling victim to fraud. With the increasing amount of spam emails circulating online, users need an automated approach to detecting such messages.
For this research project, we aim to answer the following question: Can we develop a model to detect spam emails and determine which keywords most likely indicate spam content?
To explore this question, we use a dataset compiled from the CEAS, Nazario, Nigerian Fraud, and SpamAssassin corpora. In total, the dataset includes approximately 82,500 emails, with 42,891 labeled as spam and 39,595 labeled as non-spam. Each data point contains the body of the email, subject line, sender, receiver, date sent, embedded URL links, and a label indicating whether it is spam or not.
EDA Tab This section includes data cleaning and preprocessing, where we normalize and tokenize the text, and prepare the data for modeling. It also features initial summary statistics and data visualizations.
Visuals Tab Here, we present interactive visualizations to explore patterns in word frequency, TF-IDF scores, bi-grams, timestamps, and vocabulary diversity. These help us understand the linguistic differences between spam and non-spam messages.
Modeling Tab We use the cleaned data and metadata to train various machine learning models including Naive Bayes, Logistic Regression, and MLP (Neural Network). We tune hyper-parameters and compare model performance on a test set using accuracy metrics and confusion matrices.
Results Tab We interpret the models by identifying the most predictive keywords for spam and non-spam, and test the final best model on example email inputs.
Using this dataset, our goals are to:
This project aims to improve user safety by utilizing data science tools for automatic spam detection.