Bayesian linear regression

Bayesian linear regression is a statistical technique that integrates lincar regression with Bayesian
inference.

"It accounts for uncertainty in model parameters by considering them as random variables and
assigning probability distributions to them.

Bayesian linear regression is particularly valuable in machine learning (ML) for several reas

especially when dealing with uncertainty, incorporating prior knowledge, and improving model
interpretability.
" Bayesian lincar regression is a statistical technique that integrates linear regression with
Bayesian
inference.

" It accounts for uncertainty in model parameters by considering them as random variables and
assigning probability distributions to them.

"Bayesian lincar regression is particularly valuable in machine leaming (ML) for several reasons,
especially when dealing with uncertainty. incorporating prior knowledge, and improving model
interpretability.
[Link] Linear Regression Bayesian Linear Regression

Provides point estimates for the model parameters (like Bayesian methods provide a full probability distribution for
slope and intercept) and predictions and does not parameters and predictions, enabling uncertainty
inherently express the uncertainty or confidence in these quantifcation and confidence estimation for predictions.
estimates.

Does not easily incorporate prior beliefs or domain Bayesian approaches enable the use of prior knowledge or
2 knowledge into the model. historical data to inform model parameters, leading to more
accurate predictions.

overfiting can occur, especially with small datasets or Bayesian linear regression uses prior distributions to
3 complex models. Regularization techniques (like Ridge) regularize the model, reducing overfitting by favoring
are often used to address this. parameters close to zero.
 Bayesian models provide more interpretable results because they produce distributions over
parameters.

" Atraditional linear regression model might give you a single predicted price, but it won't tell how
confident the model is in that prediction.

" Bayesian linear regression, on the other hand, would provide a distribution of possible prices,
reflecting the uncertainty in he market. This probabilistic predietion is far more useful in making
informed decisions, such as risk management
Linear Regression

"To predict an outcome y(e.g., income) based on an input x(e.g.. years of experience).
" The relationship is modeled as: y=B+B;xte

where B, is the intercept. ß, is the slope, and c is the error term.

Bayesian Approach

" Bayesian approach finds a distribution of possible parameters.

" Start with a prior belief about the parameters and update this belief using the data to get a posterior
distribution.
[Link] Specification

Assume a linear relationship between the input x and the output y: y-Bo+BiXte

where:

" Bo is the intercept,

" B, is the slope,

" cis the error term, typically assumed to be normally distributed: e~N(0,o)
2. Prior Distributions

Assign prior distributions to the model parameters Bo and B. These represent your initial beliefs about

the parameters before secing the data:

Bo-N(Hooo)
P~N(Ho)

where Ho-H are the means and o, o are the variances of the priors.
2. Prior Distributions

Assign prior distributions to the model parameters Bo and B. These represent your initial beliefs about
the parameters before seeing the data:

Po-N(Hoao)
B-N(4.G;)

where Ho., are the means and o'. Gj' are the variances of the priors.
3. Likelihood

" The likelihood function represents the probability of observing the data given the paramcters.
Assuming normally distributed errors:

yi~N(B+B,x,o)
This is the likelihood of observing y,given x, and the parameters Band Bi
4. Posterior Distribution

" Using Bayes' theorem, combine the prior and the likelihood to obtain the posterior distribution of the

parameters:

P(Ê,-B,ldata)ocP(datalP,-ß)*P(,)*PG)
" Prior P(B) and P(P,)

" Likelibood P(datalfo. B)

" Posterior P(ß. B,ldata)

" Using Bayes' theorem, combine the prior and the likelihood toobtain the posterior distribution of the
parameters:

P(P.B,ldatajocP(data|.,)*P(P,)P(P)
" Prior PfB) and P(ß,)

" Likelibood P(datalß,. B,)

" Posterior P(ß B,ldata)

This formula provides the updated beliefs about the parameters after considering the observed data.
5. Prediction

" Topredict the output y' for a new input x*, use the posterior distribution of the parameters:

y'~NB +`,z',oiedktive)
*accountsfor both the uncertainty in the
parameters and the noise in the data.
posterior means

This approach allows for the incorporation of prior knowledge and the estimation of uncertainty in the
model parameters and predictions.