MACHINE LEARNING

UNIT -1
UNIT I: Introduction to Machine Learning and Linear Models
Definition and Scope of Machine Learning, Applications and Types of Learning:
Supervised, Unsupervised, Reinforcement, Linear Regression: Least Squares, Cost
Function, Gradient Descent, Polynomial Regression and Overfitting, Evaluation Metrics:
RMSE, MAE, R² Score, Bias-Variance Trade off.

Machine Learning :
 It is a Subset Of Arttificial Intelligence
Definition : Machine learning enables a machine to automatically learn from
data, improve performance from experiences, and predict things without being
explicitly programmed.
 Machine learning constructs or uses the algorithms that learn from historical data (Training Data). The
more we will provide the information, the higher will be the performance.

Scope Of Machine Learning:

1. Broad Industry Adaption:
 Healthcare: Predictive diagnostics, medical imaging, drug discovery
 Finance: Real-time fraud detection, algorithmic trading, risk analytics
 Retail/E-commerce: Recommendation engines, customer segmentation, dynamic pricing
 Automotive: Self-driving cars, traffic prediction, autonomous maintenance
 Manufacturing: Smart factories, predictive maintenance, process optimization
 IoT & Smart Devices: Predictive models on edge devices

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

2. Technical Advancements and Trends :

 Deep Learning & Generative AI: Powerful tools revolutionizing NLP and CV
 Edge ML & IoT integration: Enabling real-time intelligent processing
 Quantum ML: Early-stage field promising major leaps in computation

APPLICATIONS OF MACHINE LEARNING

1. Image Recognition:
It is used to identify objects, persons, places, digital images, etc. example, Facebook provides us
a feature of auto friend tagging suggestion. Whenever we upload a photo with our Facebook
friends, then we automatically get a tagging suggestion with name, and the technology behind
this is machine learning's face detection and recognition algorithm.
2. Speech Recognition:
Speech recognition is a process of converting voice instructions into text, and it is also known as
"Speech to text", or "Computer speech recognition." While using Google, we get an option of
"Search by voice," it comes under speech recognition, and it's a popular application of machine
learning.
3. Traffic Prediction :
If we want to visit a new place, we take help of Google Maps, It predicts the traffic conditions
such as whether traffic is cleared, slow-moving, or heavily congested with the help of two ways:

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

Real Time location of the vehicle form Google Map app and sensors, Average time taken on Past
days at the same time.
4. Product Recommendations :
Whenever we search for some product on Amazon, then we started getting an advertisement for
the same product while internet surfing on the same browser and this is because of machine
learning. Google understands the user interest using various machine learning algorithms and
suggests the product as per customer interest.
5. Self-driving Cars:
Tesla, the most popular car manufacturing company is working on self-driving car. It is using
unsupervised learning method to train the car models to detect people and objects while driving.
6. Virtual Personal Assistant :
Now a days various virtual personal assistants such as Google assistant, Alexa, Cortana, Siri. As
the name suggests, they help us in finding the information using our voice instruction.
7. Online Fraud Detection :
Machine learning is making our online transaction safe and secure by detecting fraud transaction.
For each genuine transaction, there is a specific pattern which gets change for the fraud
transaction hence, it detects it and makes our online transactions more secure.
8. Stock Market Trading:
In the stock market, there is always a risk of up and downs in shares, so for this machine
learning's long short term memory neural network is used for the prediction of stock market
trends.
9. Medical Diagnosis:
Medical technology is growing very fast and able to build 3D models that can predict the exact
position of lesions in the brain.It helps in finding brain tumors and other brain-related diseases
easily.
10. Automatic Language Translation :
Nowadays, if we visit a new place and we are not aware of the language then it is not a problem
at all, as for this also machine learning helps us by converting the text into our known languages.

TYPES OF LEARNING:

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

1. Supervised Learning

The model is trained on labeled data (data with input-output pairs). It learns a mapping from
inputs (features) to outputs (labels) to predict outputs for the new data.

Example:
Email Spam Detection:
Input: Email text
Output: "Spam" or "Not Spam"
The algorithm is trained on many emails that are already labeled as spam or not spam.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

Common Algorithms:
Linear Regression, Logistic Regression, Decision Trees, Random Forest, Support Vector
Machines (SVM).
 Supervised learning is broadly divided into two main types based on the type of output:
1. Classification – When the output is a category or label (discrete values).
2. Regression – When the output is a continuous value (numeric values).

1. Regression:
Regression is a supervised learning task where the output variable is continuous. The goal is
to predict a real-valued output based on input features.
Example:
 Predicting the price of a house based on its area, location, and number of rooms.
 Forecasting the temperature for the next day.
2. Classification :
Classification is a supervised learning task where the output variable is categorical. The
goal is to classify input data into one of the predefined classes or categories.
Example:
 Email classification: Predicting whether an email is “Spam” or “Not Spam”.
 Disease detection: Classifying whether a patient has “Diabetes” or “No Diabetes”.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

2. Unsupervised learning
The model is trained on unlabeled data (no predefined output). It tries to find hidden patterns
or structures and group or reduce the data into meaningful patterns

Example:

Customer Segmentation:
Input: Customer purchasing behavior (no labels)
Output: Groups of customers with similar buying habits.
Common Algorithms:
K-Means Clustering, Hierarchical Clustering, PCA (Principal Component Analysis).
 Unsupervised Learning is broadly divided into 2 types :
1. Clustering
2. Association

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

1. Clustering:
Clustering is the process of grouping similar data points together based on similarities or
distances.
Example:
 Grouping customers based on purchasing habits.
 Grouping news articles by topics without predefined labels.

2. Association:
Association is the process of finding rules and relationships between variables in large datasets
Example:
 Market basket analysis: “If a customer buys bread, they are likely to buy butter.”
 Online recommendations (e.g., “People who bought this product also bought…”).

3. Reinforcement Learning:
The model (called an agent) learns by interacting with an environment and receiving feedback in
the form of rewards or penalties. it maximize cumulative reward over time by learning the best
actions to take.

Example:

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

Self-driving Car:
The car (agent) gets rewards for staying on track and penalties for hitting obstacles. Over
time, it learns to drive safely.

Applications :

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

Characteristics Supervised Learning Unsupervised Reinforcement

Learning Learning
Definition Supervised learning Unsupervised Reinforcement
is when the learning is when the learning is the
algorithm learns on algorithm learns on training of machine
a labeled dataset and unlabeled data, learning models to
analyses the training inferring more about make a sequence of
data. These labeled hidden structures to decisions. It
data sets have inputs produce accurate concerns how
and expected and reliable outputs. intelligent agents
outputs. are to take action in
an environment to
maximize
cumulative reward.
Types Of Data Labeled Unlabeled No Predefined Data
Goal To Predict correct To find hidden patterns To find an effective
Labels for newly through similarities model that maximize
inputted data and differences the benefits of the
between data points reward signal through
the trail- error method
Output Mapping Classes Action

LINEAR REGRESSION :

 Linear Regression is a supervised machine learning algorithm used for predicting a

continuous target variable based on one or more independent (input) variables.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

 It models the relationship between the dependent variable Y and the independent variable(s)
X by fitting a straight line (linear equation) to the data.

Example :
A data of 14 students is given below

Internal
Exam 16 15 23 18 23 24 22 22 19 19 16 24 11 24
External
Exam 49 49 63 58 60 58 61 60 63 60 52 62 30 59

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

 Residual ( ) : It is the distance between the predicted point and the actual point

LEAST SQUARES
 Least squares linear regression is a method used to find the best-fitting straight line
through a set of data points by minimizing the sum of the squares of the differences
between the observed values and the values predicted by the line.
 The most common implementation of least squares linear regression is Ordinary Least
Squares(OLS)
Algorithm :
1. Calculate the mean of x and y
2. Calculate the errors of x and y
3. Get the product of Errors
4. Get the summation of product of errors in step3
5. Square the difference of X
6. Get the sum of the squared difference
7. Divide the output of Step 4 by output step6 to calculate b

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

8. Calculate the a value by using b

a=Y-bX

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

COST FUNCTION
The cost function evaluates how well the regression line fits the data by calculating the difference
between the actual and predicted values.
 Goal: The smaller the cost, the better the model.
 In linear regression, the cost function is used to evaluate how far the predicted line is from
the actual data points.
Most Common Cost Function: Mean Squared Error (MSE)

Ex:
House Price Prediction
Actual Prices: [200, 250, 300]
Predicted Prices: [210, 240, 310]

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

GRADIENT DESCENT
Gradient Descent is an optimization algorithm used to minimize the cost function by adjusting the
parameters of the model (slope m and intercept c) in small steps until the best-fit line is found.
 The cost function J(m,c) tells us how far the predicted values are from the actual values.
 Gradient Descent iteratively changes m and c to reduce J(m,c)until it reaches the
minimum value (global minimum).
Steps:
1. Initialize parameters:
Set m and c to 0 or small random values.
2. Calculate Predictions:
Use the current values of m and c using to predict

3. Compute the cost:

Find the cost function J(m,c) using mean squared Error(MSE).
4. Calculate the gradients:
Derivates of Cost function with respect to m and c:

5. Update Parameters:
Update m and c using :

6. Repeat:
Keep repeating steps 2–5 until the cost function converges (i.e., no significant
improvement).

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

 Learning Rate (α):

Controls how big the steps are toward the minimum.
If α is too large → overshoots the minimum.
If α is too small → slow convergence.
 Convergence:
The point at which the cost function reaches its minimum value.
Example:
Imagine you are on a hill (cost function surface) and want to reach the lowest point (global
minimum).
 At each step, you look at the slope (gradient) and move downwards in the direction of steepest
descent.
 Step size = learning rate.

POLYNOMIAL REGRESSION:
 Polynomial Regression is a regression algorithm that models the relationship between a
dependent(y) and independent variable(x) as nth degree polynomial.
 The Polynomial Regression equation is given below:
y= b0+b1x1+ b2x12+ b3x13+...... bnx1n
 It is a linear model with some modification in order to increase the accuracy.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

 The dataset used in Polynomial regression for training is of non-linear nature.

 It makes use of a linear regression model to fit the complicated and non-linear functions
and datasets.
 Hence, "In Polynomial regression, the original features are converted into Polynomial
features of required degree (2,3,..,n) and then modeled using a linear model."

OVERFITTING:
 Overfitting happens when the model tries too hard to learn the training data, including
every small detail or noise.
 It’s like memorizing answers instead of understanding the concept.
 The model performs great on training data but fails to give good predictions on new
data.
In Linear Regression:
 A simple line might not capture the pattern (underfitting).
 A very complex curve that passes through every point is overfitting.
 The best-fit line captures the main pattern and generalizes well.
Avoid Overfitting:
 Use simpler models (not too many polynomial terms).
 Use regularization to reduce unnecessary complexity.
 Use more data or test the model on unseen data (validation).

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

EVALUATION METRICS:
Evaluation metrics are quantitative measures used to assess how well a machine learning model
performs on given data. They help us understand the accuracy, error, and overall quality of the
predictions made by the model.
Evaluation Metrics in Linear Regression:
In regression problems, common evaluation metrics include:
1. RMSE (Root Mean Squared Error): Measures the square root of average squared
errors.
2. MAE (Mean Absolute Error): Measures the average of absolute differences between
actual and predicted values.
3. R² Score (Coefficient of Determination): Explains how much of the variance in the
target variable is captured by the model.
1. RMSE:
Formula :

Key point:
Lower RMSE means better predictions. It penalizes large errors more because of the square.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

2. MAE:

3. R2 Score:

Bias Variance Trade Off

In supervised learning, the class value assigned by the learning model built based on the
training data may differ from the actual class value. This error in learning can be of two types –
errors due to ‘bias’ and error due to ‘variance’.
 Bias: Error due to overly simple assumptions (underfitting).
Errors due to bias arise from simplifying assumptions made by the model to make the target
function less complex or easier to learn
 Variance: Error due to too much complexity (overfitting).
Errors due to variance occur from difference in training data sets used to train the model.
Different training data sets (randomly sampled from the input data set) are used to train the
model. Ideally the difference in the data sets should not be significant and the model trained
using different training data sets should not be too different.

G.VIDYA SAGAR, Asst.Prof

 MACHINE LEARNING

 Trade-off: A good model balances bias and variance to achieve high accuracy on both
training and test data
Analogy:
 High bias = always making the same mistake.
 High variance = predictions change a lot depending on the dataset.

Fig: Bias variance trade off

G.VIDYA SAGAR, Asst.Prof