Like a teacher teaches a student ie by quoting examples
Discrete values means spam or not spam etc
Here the output is categorical.
Ie whether the student is going to pass or fail the exam.
So here only two categories- F or P – so this is Binary classification
In the output if we have more than two categories then it is multiclass
classification
Eg:
Ie. Here getting continuous value (Percentage of marks, housing price etc)in the
output – so this is regression.
classifier is an algorithm that automatically assigns data points
to a range of categories or classes
Data >=0.5 Malignant
Data<0.5-Benign
hθ(x) typically represents the hypothesis function or model's predicted
output for a given input x, where θ denotes the parameters of the model.
Taking 0.5 as threshold and the points left to the vertical line indicated No cancer
and right of the line indicates cancer
So this can be solved using Linear Regression
But
The points left to the second vertical lines indicates no cancer and this is wrong.
Binary Classification
1. Logistic Regression(Classification Problem( not regression by its name)
Logistic Regression is a supervised learning algorithm used for classification problems,
especially binary classification (yes/no, true/false, 0/1).
Unlike linear regression, which predicts continuous values, logistic regression predicts the
probability that a given input point belongs to a certain class. It uses the logistic (sigmoid)
function to map predicted values to probabilities between 0 and 1.
When we plot Z in X axis and sig(Z)(after calculating) in Y axis we will get
a graph like this. When we observe the graph we can see that the
equations values lies between 0 and 1
Also the function cuts the Y axis at 0.5 value for z=0
wi is the weight(importance) of the feature xi.
so z formula represents a weighted sum of the input features plus a bias term
w0(intercept).
Working of Logistic Regression
1. Compute the linear combination of input features.
2. Apply the sigmoid function to get a probability.
3. Choose a threshold (e.g., 0.5) to classify into class 0 or 1.
Eg: Predict whether a student will pass an exam (1) or not (0) based on study
hours.
The probability that Y equals 1, given X
In logistic regression, we are not predicting exact values, but rather probabilities that a data
point belongs to a particular class (usually 1 or 0).
So we need a special cost function that works well with probabilities.
A cost function is a mathematical formula used to measure how wrong your
model's predictions are compared to the actual values.
The goal of training a machine learning model is to minimize the cost function
— meaning, we want our predictions to be as close as possible to the actual
labels.
So we need a cost function that:
Gives a low cost when prediction is good
Gives a high cost when prediction is bad
It calculates the average prediction error (in terms of log loss) across all
training examples, based on how far the predicted probabilities y^are from the
actual outcomes y
Eg:
Eg:
Calculate a0 and a1 using Gradient Descent
Gradient Descent is an optimization algorithm used to find the best values of parameters
(like a0,a1) in a machine learning model by minimizing the cost function (how wrong the
model is).
Step 1: Define the cost function
Log loss (Binary Cross-Entropy):
Learning Rate - The learning rate is a small number that controls how fast or slow a
machine learning model learns during training.
ITERATION 2
Gradient - In machine learning, it's the direction and amount by which we should
adjust the model's parameters (like weights) to reduce the error (cost).
