Supervised Learning
Uploaded by
M SridharSupervised Learning
Uploaded by
M SridharSupervised Learning
Supervised Learning
Supervised Learning is a type of machine learning where the model is trained using labeled
data (input with correct output).
The main focus is on two key tasks:
→
Classification predicting categories or classes (e.g., spam or not spam).
→
Regression predicting continuous values (e.g., house price, temperature).
It works by mapping input features to output labels through a learning process.
Plays a major role in AI applications such as:
Medical diagnosis
Fraud detection
Speech and image recognition
Business forecasting
Classification
Definition: Classification is a supervised learning task where the goal is to assign an input
into one of several predefined categories (classes).
Nature of Output: The output is categorical (e.g., Yes/No, Pass/Fail, Fraud/Not Fraud).
Purpose: Helps in decision-making by grouping data into meaningful classes.
Process:
1. Collect labeled training data.
2. Train a model to identify class patterns.
3. Use the model to classify new, unseen data.
Examples:
→
Email filtering Spam / Not Spam
→
Medical diagnosis Disease / No Disease
→
Image recognition Dog / Cat / Human
Classification Model
Definition: A classification model is a mathematical or computational representation that learns from
labeled data to predict the correct class for new inputs.
Input & Output:
→
Input Features (X1, X2, …, Xn)
→
Output Class label (Y)
Training Process:
Feed the model with training data.
Learn patterns and relationships between features and labels.
Adjust model parameters to minimize errors.
Prediction: For any unseen input, the model assigns it to the most probable class.
Evaluation: Performance is measured using metrics such as accuracy, precision, recall, and F1-score.
Example:
Input: Age, income, browsing history
Output: Likelihood of buying a product (Yes/No)
Classification – Learning Steps
Step 1: Data Collection
Gather a labeled dataset containing input features and their correct class labels.
Step 2: Data Preprocessing
Clean missing values, remove noise, and normalize data for better accuracy.
Step 3: Train-Test Split
Divide data into training set (to build the model) and testing set (to evaluate performance).
Step 4: Choose Algorithm
Select a suitable classification algorithm (e.g., Decision Tree, kNN, SVM).
Step 5: Model Training
Train the model using the training dataset to learn input-output relationships.
Step 6: Model Evaluation
Test the model with unseen data and calculate metrics (accuracy, precision, recall).
Step 7: Hyperparameter Tuning
Adjust parameters (like learning rate, depth, neighbors) to improve performance.
Step 8: Deployment & Prediction
Deploy the trained model to classify new real-world data.
Common Classification Algorithms
1. k-Nearest Neighbors (kNN) 4. Support Vector Machine (SVM)
Classifies a sample based on the majority Finds the best separating hyperplane
class of its nearest neighbors. between classes.
Simple and effective for small datasets. Effective for high-dimensional data.
2. Decision Tree 5. Naïve Bayes
Represents decisions in a tree-like structure Based on Bayes’ theorem with
of rules. independence assumptions.
Easy to interpret and visualize. Works well for text classification (e.g., spam
filtering).
3. Random Forest
An ensemble method combining multiple 6. Neural Networks
decision trees. Mimics the human brain’s neuron
Reduces overfitting and improves accuracy. connections.
Useful for complex tasks like image and
speech recognition.
Regression
Definition: Regression is a supervised learning technique used to predict continuous numerical values rather than categories.
Nature of Output: Output is a real-valued number (e.g., price, temperature, sales).
Purpose:
Identify relationships between input features and output variable.
Make future predictions based on historical data.
Process:
[Link] labeled dataset with numeric outcomes.
[Link] a regression model to fit a curve/line.
[Link] continuous values for unseen inputs.
Examples:
Predicting house price based on location, size, and age.
Forecasting stock market trends.
Estimating student marks based on study hours.
Difference from Classification:
→
Classification categorical outputs (Yes/No).
→
Regression continuous outputs (numbers).
Common Regression Algorithms
1. Linear Regression
Models relationship between input (X) and output (Y) as a straight line.
2. Multiple Regression
Uses multiple independent variables to predict a dependent variable.
3. Polynomial Regression
Fits curved data by adding higher-order polynomial terms.
4. Logistic Regression
Used for binary classification, outputs probability (0–1).
5. Ridge & Lasso Regression
Add penalty terms to reduce overfitting and handle multicollinearity.
Linear Regression
Equation: Y=mX+cY = mX + cY=mX+c (simple linear form).
Goal: Minimize difference between predicted and actual values.
Method Used: Least Squares Method to reduce errors.
Applications:
Predicting salary based on experience.
Estimating sales from advertising spend.
Advantages: Simple, interpretable, works well for linear data.
Multiple Regression
Definition: Extends linear regression by using two or more independent variables.
Equation: Y=b0+b1X1+b2X2+…+bnXnY = b0 + b1X1 + b2X2 + … +
bnXnY=b0+b1X1+b2X2+…+bnXn.
Purpose: Capture influence of multiple factors on a dependent variable.
Example: Predicting house price using area, location, number of rooms, and age.
Challenges: Risk of multicollinearity (when predictors are highly correlated).
Regression Assumptions
Linearity → Relationship between predictors and output must be
linear.
Independence → Observations should not influence each other.
Normality→ Residuals (errors) should follow a normal distribution.
Homoscedasticity → Constant variance of errors across data points.
No Multicollinearity → Predictors should not be highly correlated.
Polynomial Regression
Definition: Regression technique where the relationship is modeled
as an nth degree polynomial.
Equation: Y=b0+b1X+b2X2+b3X3+…+bnXnY = b0 + b1X + b2X^2 +
b3X^3 + … + bnX^nY=b0+b1X+b2X2+b3X3+…+bnXn.
Use Case: When data shows a non-linear pattern.
Example: Growth rate of a plant over time (curved trend).
Limitation: High-degree polynomials may lead to overfitting.
Logistic Regression
Purpose: Used for classification problems, not regression.
Output: Probability between 0 and 1.
Sigmoid Function: Converts linear output into probability.
Decision Boundary:
→
Probability > 0.5 Class 1
→
Probability ≤ 0.5 Class 0
Applications:
Medical diagnosis (disease/no disease).
Credit risk prediction (default/no default).
Maximum Likelihood Estimation (MLE)
Definition: A statistical method used to estimate parameters of a model.
Idea: Choose parameters that make the observed data most likely.
Steps:
Define likelihood function based on model.
Calculate probability of data given parameters.
Adjust parameters to maximize likelihood.
Applications in ML:
Used in Logistic Regression for parameter estimation.
Widely applied in probabilistic models (Naïve Bayes, Hidden Markov Models).
