ENGINEERING DATA ANALYSIS

Joint Probability Distribution

Joeniel Abasolo, E-Jay Albores, Hasmer Khalil Madijanon, Harvey James Merto
Bachelor of Science in Civil Engineering
University of Science and Technology of Southern Philippines, October 2024

Summary

This paper explores the concept of joint probability distributions, which

quantify the likelihood of multiple events occurring simultaneously. By understanding
the relationships between random variables, we can gain insights into complex
phenomena and make informed predictions. The paper delves into the key concepts
of joint probability, including the notation P(A ∩ B) for the intersection of events A
and B. It also discusses the distinction between joint probability and conditional
probability, emphasizing the independence of events in joint probability. Furthermore,
the paper examines linear functions of random variables, which involve combining
multiple random variables using a linear combination. It provides formulas for
calculating the expected values, variances, and standard deviations of linear
functions of random variables. These calculations are crucial for understanding the
behavior of transformed random variables and their relationships to the original
variables. The concepts presented in this paper have practical applications in various
fields, including finance, engineering, and social sciences. For example, in finance,
joint probability distributions can be used to model the risks associated with portfolios
of assets, while linear functions can be applied to analyze the impact of changes in
interest rates or market conditions on investment returns. In engineering, joint
probability distributions can be used to assess the reliability of systems with multiple
components, while linear functions can be applied to model the relationships
between different variables in engineering models. In the social sciences, joint
probability distributions can be used to study the relationships between social
phenomena, while linear functions can be applied to analyze the impact of policy
changes on social outcomes. By understanding joint probability distributions and
linear functions of random variables, researchers and practitioners can gain valuable
insights into the interconnectedness of multiple phenomena and make informed
decisions based on probabilistic models.

Keywords: joint probability distribution, random variable, discrete, continuous, linear function

JOINT PROBABILITY DISTRIBUTION

This topic delves into the concept of joint probability distribution, particularly

focusing on the implications of linear functions of random variables. Joint probability

distributions allow us to examine the relationships between multiple random

variables, providing a comprehensive framework for analyzing how the occurrence

of one variable influences the other. This is essential for various applications, as it
enables us to understand the joint behavior of random phenomena, quantify their

interactions, and make informed predictions.

Probability refers to the likelihood that a specific event will occur, expressed

as a number between 0 and 1. A probability of 0 indicates that the event will not

happen, while a probability of 1 means the event is certain. In many real-world

scenarios, we are often concerned with more than just a single event or outcome.

When two or more random variables are involved, the concept of joint probability

comes into play.

I. DEFINITION OF TERMS

● JOINT PROBABILITY DISTRIBUTION

Joint probability is a statistical concept used to determine the chance

that two events will happen simultaneously. The term joint probability can be

referred to as a statistical measure that calculates the likelihood of two events

occurring together and at the same point in time according to Will Kenton

(2024). In simpler terms, joint probability looks at the relationship between two

events and calculates the probability that both will happen together.

For example, if you want to know the likelihood of it raining and having

strong winds on a given day, joint probability helps to quantify how often these

two events coincide. This measure is important when dealing with multiple

variables or outcomes that might influence each other or occur concurrently,

as it allows for a more comprehensive analysis of combined events.

Since joint probability is the probability of two events happening

together, we can let A and B be the two events. In probability terminology, it

can be written as P (A and B). Furthermore, Geraghty (2022) similarly

 describes that joint probability is also called as the probability of the

intersection of two (or more) events and it can be denoted as P (A ∩ B). We

can visualize this by a Venn diagram shown in Figure 1.

Figure 1. Joint Probability Distribution

Adapted Diagram: A.G. Geraghty (5.5: Joint Probability and Additive Rule (2022))

The Venn diagram shows the overlapping area of two circles which

represents the joint probability. This area is referred to as the intersection of

two events. Therefore,

P (A ∩ B) = P (A) × P (B)
where:

● P(A ⋂ B) is the notation for the joint probability of event “A” and “B”.

● P(A) is the probability of event “A” occurring.

● P(B) is the probability of event “B” occurring.

This shows a probability distribution for two (or more) random

variables. Instead of events being labeled A and B, the norm is to use X and

Y. The formal definition is:

f (x, y) = P (X = x, Y = y)

However, not every function defined for the values of a random variable

can serve as a probability distribution. Since the values of probability

distributions are probabilities and one value of a random variable must always

occur, it follows that if f(x) is a probability distribution, then f(x) ≥ 0 for all x,
 and ∑ 𝑓(𝑥) = 1. The whole point of the joint distribution is to look for a
𝑎𝑙𝑙 𝑥

relationship between two variables. The joint probability distribution can be

represented differently depending on the type of variable. For discrete

variables, it is shown as a joint probability mass function. For continuous

variables, it can be expressed through a joint cumulative distribution function

or a joint probability density function. Joint probability depends on the two

events acting independently from one another (Kenton, 2024). To determine

whether they are truly independent, it's important to establish whether one's

outcome affects the other. If they do, they are dependent, which means they

lead to conditional probability. If they don't, you end up with joint probability.

● LINEAR FUNCTIONS

In mathematics, linear functions play a fundamental role in various

fields, such as calculus, geometry, probability theory, and linear algebra. A

linear function is one of the simplest types of functions, and its properties are

essential for understanding more complex mathematical concepts.

The term linear function in calculus, geometry, and plotting refers to a

function whose graph forms a straight line. In calculus, geometry, and plotting

contexts, the term "linear function" means a function whose graph is a straight

line, i.e., a polynomial function of degree 0 or 1 (Weisstein, 2024). This

typically means that the function is a polynomial of degree 0 or 1. In one

variable, the general form of a linear function is given as:

f(x) = 𝑎𝑥 + b,
 where a and b are constants, and x is the independent variable. This function

shows a constant rate of change and represents a direct proportionality

between x and f(x). For multiple variables, a linear function takes the form:

f(𝑥1,....., 𝑥𝑛) = b + 𝑎1𝑥1 + … + 𝑎𝑛𝑥𝑛,

where 𝑎1, …, 𝑎𝑛 are the coefficients corresponding to each variable 𝑥1, …, 𝑥𝑛

and b is the constant term. In the context of linear algebra and functional

analysis, a linear function (or linear map) satisfies two key properties of

linearity: additivity and homogeneity. This means that for any two elements x

and y, the linear function f satisfies the condition:

f(x + y) = f(x) + f(y)

Additionally, it follows that f(α x) = αf(x), where α is a scalar. These

properties make linear functions crucial in various mathematical applications,

including solving systems of linear equations, performing transformations in

vector spaces, and simplifying complex relationships in higher-dimensional

spaces. A joint probability distribution describes the probability of different

outcomes for two or more random variables. A linear function applied to

random variables in a joint probability distribution involves creating a new

random variable by combining the original random variables through a linear

combination.

● RANDOM VARIABLES

When analyzing random experiments, we typically focus on specific

numerical outcomes that provide valuable insights. In a book of Pish-ro Nik

(2014), a random variable is a real-valued variable whose value is determined

by an underlying random experiment. Thus, random variables allow us to

translate unpredictable phenomena into quantifiable data. These variables

can take on different values depending on the particular realization of the

experiment.

A random variable is a function that assigns numerical values to the

possible outcomes of a random phenomenon. In mathematical terms, a

random variable X is defined as a mapping from the sample space S which

consists of all possible outcomes of an experiment, to the set of real numbers

R. This can be expressed as:

X: S → R

where each element in the sample space is associated with a real number.

Random variables are essential in probability theory as they allow us to

quantify and analyze random processes by transforming qualitative outcomes

into measurable data.

There are two main types of random variables: discrete and

continuous. A discrete random variable (RV) takes on specific, distinct values,

often representing counts or individual events. For example, the number of

heads in a series of coin tosses is a discrete RV. On the other hand, a

continuous random variable can take on an infinite number of values within a

continuous range, such as the time taken for an event to occur or the exact

height of individuals in a population. These two types of random variables are

fundamental in defining probability distributions and modeling various

real-world processes.

Essentially, a random variable is a real-valued function that assigns a

numerical outcome to each possible result of a random process. This makes

random variables crucial in probability theory and statistics, as they help us

 model uncertainty and make informed predictions based on data.

II. GENERAL FUNCTION OF RANDOM VARIABLES

● DISCRETE RANDOM VARIABLE

A random variable is one whose value is determined by chance. It can be

either discrete or continuous, and understanding the characteristics of each type is

crucial for various statistical applications. A random variable is discrete if its range is

a countable set. Moreover, Frost (2022) stated that a discrete random variable has

distinct values that are countable and finite or countably infinite. This data type often

occurs when you are counting the number of event occurrences.

If you have a discrete random variable X with a known probability distribution

f(x), and you transform it using a one-to-one function u(x), you can find the

probability distribution of the transformed variable Y by:

g(y) = f[w(y)]

where:

● g(y): The probability density function of the transformed random variable Y.

● w(y): This is the inverse function of u(x).

● f[w(y)]: The probability density function of X evaluated at the inverse function

w(y).

Sample Problem:

Let X be a random variable that follows a geometric distribution. Find the probability
2
distribution of Y = 𝑋 .
 2
𝑥 +1
𝑓(𝑥) = 14
, x = 1, 2, 3, ….

Solution:

The values of X are all positive and distinct, squaring them (to get Y) creates a

one-to-one correspondence. For every different value of X, we get a different value

of Y.

2 2 2 2 2 2
𝑌 = 𝑋 = 1 = 1; 𝑌 = 𝑋 = 2 = 4; 𝑌 = 𝑋 =3 = 9

2
Since Y = 𝑋 , we have 𝑋 = 𝑌 as our w(y). Substitute into the PMF of X:

2
𝑔( 𝑦) = 𝑓( 𝑦) = (( 𝑦) + 1)/14 ⇒ (𝑦 + 1)/14

Hence,
𝑦+1
⎰𝑓( 𝑦) = 14
, y = 1, 4, 9, ….

⎱0, elsewhere.

● CONTINUOUS RANDOM VARIABLE

A continuous random variable possesses an uncountably infinite number of

values that create a continuous spectrum. This means it can assume any value

within a specified interval, allowing for a wide range of possible outcomes (Frost,

2022). For example, the height of individuals or the time taken to complete a task

can be modeled as continuous random variables, as they can take on any value

within a certain range.

If we want to find the probability density (PDF) distribution of Y = u(X), the

function u(x) must first satisfy the following conditions: it must be differentiable and is

strictly increasing. If the function satisfies both conditions, the method of

transformations can then be used (Pish-ro Nik, 2014). Hence, let X be a continuous

random variable with PDF f(x). Suppose there exists a one-to-one function u(x) that

maps values of X to values of Y. If the inverse function, w(y), exists and is

differentiable, then the PDF of Y is given by:

g(y) = f[w(y)] × |J|

where:

● g(y): The probability density function of the transformed random variable Y.

● f[w(y)]: The probability density function of X evaluated at the inverse function

w(y).

● |J|: The absolute value of the Jacobian of the transformation, which is the

derivative of the inverse function: J = w'(y). C

Sample Problem:

Let X be a continuous random variable with probability distribution. Find the

1
probability distribution of the random variable 𝑌 = 𝑥 .

⎰6𝑥5 3<x<5
⎱0, elsewhere.

Solution:

1 1
Since 𝑌 = 𝑥
, we have 𝑋 = 𝑌
as our w(y). Solve for J = w’(y), and we get

1
𝑤'(𝑦) =− 2 .
𝑦

Hence,

|1|
⎰ 𝑦6 5 × | 2|=
|𝑦 |
6
7
𝑦
, 1
5
<y<
1
3

⎱0, elsewhere.
III. LINEAR FUNCTIONS OF RANDOM VARIABLES

In probability theory and statistics, a random variable is a variable whose

values are determined by the outcomes of a random phenomenon. These random

variables can be combined and manipulated through various mathematical

operations to understand relationships and dependencies between them. A common

operation involving random variables is forming linear functions. Linear functions in

joint probability distributions refer to the transformation of random variables in a joint

distribution by applying a linear combination. If 𝑋1, 𝑋2, …, 𝑋𝑛 are random variables

with a joint probability distribution, a linear function of these variables can be

expressed as: [Y = 𝑎1𝑋1 + 𝑎2𝑋2 + … 𝑎𝑛𝑋𝑛 + b] where 𝑎1𝑋1 + 𝑎2𝑋2 + … 𝑎𝑛𝑋𝑛 are

constants (coefficients) and b is a constant. The new random variable Y is a linear

combination of the original random variables (Bain & Engelhardt, 1987). A linear

function of random variables can be expressed in the form:

Y = aX + b,

where:

● X is a random variable

● a and b are constants

● Y is a new random variable derived from X

Linear functions are particularly important in many statistical applications because

they simplify the process of analyzing and predicting outcomes in situations involving

uncertainty. This is due to the straightforward properties of linearity, which makes

them easier to handle when calculating expectations (means), variances, and

standard deviation.

● Expected Value or Mean of Linear Functions of Random Variables

 In finding the expected value of linear functions of random variables, there are

three conditions. First, the expected value of a constant, a, is simply a itself and it

can be expressed as E(a) = a. Second, If each value in a probability distribution is

multiplied by a constant, a, the expected value of the distribution will be multiplied by

the same constant, a, which can be expressed as E(aX) = a × E(X). Lastly, if a

constant, b, is added or subtracted from each value in the distribution, the expected

value will be increased or decreased by b, respectively and can be expressed as

E(aX + b) is equal to a × E(X) + b (BestMaths, 2014). These properties hold true for

any constants a and b.

For example, the expected value, E(X), of a probability distribution of the

random variable X is 10, find E(3X + 4).

E(3X + 4) = E(3X) + E(4)

= 3[E(X)] + E(4)

= 3(10) + 4

= 30 + 4

= 34

Therefore, the expected value of the linear function, 3X +4, of the random variable X

is 34.

● Variance of Linear Functions of Random Variables

In finding the variance of linear functions of random variables. It presents

three conditions. First, σ²(a) = 0, states that the variance of a constant, a, is always

zero. This is because a constant value has no variation. Second, σ²(aX) = a² × σ²(X),

shows that if each value in a probability distribution is multiplied by a constant, a, the

variance of the distribution will be multiplied by the square of a. This means that if a
random variable's values are doubled or squared, its variance will be quadrupled.

Last condition, σ²(aX + b) = a² × σ²(X), the third condition states that adding or

subtracting a constant value, b, to each value in a probability distribution does not

affect the variance of the distribution (BestMaths, 2014). In other words, shifting the

values of a random variable by a constant will not change its variance.

For example, the variance of a probability distribution of the random variable

X is 4. Find σ²(3X + 4).

σ²(3X + 4) = σ²(3X) + Var(4)

= (3²) (σ²(X) + σ²(4)

= (3²) (4) + 0

= (9) (4)

= 36

Therefore, the variance of the linear function, 3X + 4, of the random variable X is 18.

● Standard Deviation of Linear Functions

In finding the standard deviation of linear functions of random variables, it

presents three conditions. First, σ(a) = 0, states that the standard deviation of a

constant, a, is always zero. Second, σ(aX) = a × σ(X), If each value in a probability

distribution is multiplied by a constant, a, the standard deviation of the distribution

will also be multiplied by “a.” Last condition, σ(aX + b) = a × σ(X), if a constant, b, is

added or subtracted from each value in the distribution, the standard deviation

remains unchanged.

For example, the standard deviation of a probability distribution of the random

variable X is 2. Find σ(3X + 4).

 σ(3X + 4) = σ(3X) + σ(4)

= (3) σ(X) + σ(4)

= (3) (2) + 0

=6

Therefore, the standard deviation of the linear function, 3X + 4, of the random

variable X is 6.

Sample Problem 1

Given the following function of the random variable X, find E(X), σ²(X), SD(X),

E(3X+4), σ²(3X+4), and SD(3X + 4).

x 5 10 15

P(X=x) 0.2 0.5 0.3

● Find Expected Value of the Random Variable X:

To get the expected value of the random variable X we have,

E(X) = ∑Xi • P(Xi) = (5) (0.2) + (10) (0.5) + (15) (0.3) = 10.5

● Find Variance of the Random Variable X:

To get the variance of the random variable X we have,

σ²(X) = ∑ E(Xi2) − [E(Xi)]2 = (52) (0.2) + (102) (0.5) + (152) (3) − 10.52 = 12.25

● Standard Deviation of the Random Variable X:

To get the standard deviation of the random variable X we have,

σ(X) = √σ²(X) = √12.25 = 3.5

Next, we will add rows to the table.

 x 5 10 15

3x+4 19 34 49

P(X=x) 0.2 0.5 0.3

● Find Expected Value of the Linear Function, 3X + 4:

To get the expected value of the linear function, 3X + 4 we have,

E(3X + 4) = [((3× 5) + 4) (0.2)] + [((3× 10) + 4) (0.5)] + [((3× 15) + 4) (0.3)]

= [((15) + 4) (0.2)] + [((30) + 4) (0.5)] + [((45) + 4) (0.3)]

= [(19) (0.2)] + [(34) (0.5)] + [(49) (0.3)]

= (3.8) + (17) + (14.7)

= 35.5

Linear Function Solution:

Since the expected value of the random variable X is 10.5, then,

E(3X + 4) = E(3X) + E(4) = (3) E(X) + E(4)

= (3) (10.5) + 4

= 31.5 + 4 = 35.5

● Find Variance of the Linear Function, 3X + 4:

To get the variance of the linear function, 3X + 4 we have,

σ²(3X + 4) = (192) (0.2) + (342) (0.5) + (492) (0.3) – 35.52

= (72.2) + (578) + (720.3) - 1260.25

= 1370.5 - 1260.25

= 110.25

Linear Function Solution:

Since the variance of the random variable X is 12.25, then,

σ²(3X + 4) = σ²(3X) + σ²(4)

= (3) σ²(X) + σ²(4)
= (3²) (12.25) + 0
 = (9) (12.25)
= 110.25
● Find Standard Deviation of the Linear Function, 3X + 4:
To get the standard deviation of the linear function, 3X + 4, we have:

σ(3X + 4) = √σ²(3X + 4)

= √110.25

= 10.5

Linear Function Solution:

Since the standard deviation of the random variable X is 3.5,

σ(3X + 4) = σ(3X) + σ(4)

= 3 • σ(X) + σ(4)

= (3 • 3.5) + 0

= 10.5

Therefore, the expected value, variance, and standard deviation of the random

variable X is 10.5, 12.25, and 3.5. Meanwhile, the expected value, variance, and

standard deviation of the linear function, 3X + 4, of the random variable X is 35.5,

110.25, and 10.5 respectively.

REFERENCES

Books

Bain, L. J., & Engelhardt, M. (1987). Introduction to Probability and Mathematical Statistics.
[Link]
.html?id=MkFRIAAACAAJ&redir_esc=y

Pish-ro Nik, H. (2014). Introduction To Probability, Statistics, and Random Processes. Basic
Concept, Chapters 3.1.1 Random Variables. Retrieved from
[Link]

Electronic Sources

BestMaths (2014). Mean and Variance of a Linear Function of a Random Variable. Year 12
Topics, Probability, Functions of Random Variables. Retrieved from
[Link]
-random-variables/?fbclid=IwY2xjawF0F69leHRuA2FlbQIxMQABHcKVspRw6GzPUk
P_ubQWCQVe5HV42al-0BfUgwDupOlEbOjWJ1f7vwSVbQ_aem_e5nw_G8FIZUQj_
4weaef-w

Frost, J. (2022). Random Variable: Discrete & Continuous. Statistics By Jim. What is A
Random Variable?. Retrieved October 8, 2024, from
[Link]

Geraghty, M. A. (2022). Joint Probability - Definition, Formula, and Examples. Corporate

Finance Institute. Resources — Data Science, Joint Probability. Retrieved October 8,
2024, from
[Link]

Kenton, W. (2024). Joint Probability: Definition, Formula, and Example. Investopedia.

Retrieved October 8, 2024, from
[Link]

Pish-ro Nik, H. (2014). Introduction To Probability, Statistics, and Random Processes. Basic
Concept, Chapters 3.1.1 Random Variables.
[Link]

Weisstein, E. W. (2024). Linear Function. MathWorld - A Wolfram Web Resource. Calculus

and Analysis, Functions. Retrieved from
[Link]