ECT305 Analog and Digital Communication

Module 4 – Vector model of AWGN channel, MAP and

Maximum likelihood Rule

Lakshmi V.S.
Assistant Professor
Electronics & Communication Department
Sree Chitra Thirunal College of Engineering, Trivandrum
 Signal Space Analysis

Statistical model for a generic digital communication system

• Message source: Generates one of the M messages with apriori probability 𝑝𝑖 = 𝑃 𝑚𝑖 for 𝑖 = 1,2, . . . , 𝑀.
• Transmitter: The transmitter converts the message 𝑚𝑖 into signal waveform 𝑠𝑖 (𝑡). Assume that 𝑠𝑖 (𝑡) must
be a real-valued energy signal (i.e., a signal with finite energy) with duration 𝑇𝑏 .
𝑇𝑏
𝐸𝑖 = න 𝑠𝑖2 (𝑡)𝑑𝑡 < ∞.
0
• Channel: The channel is linear and with a bandwidth wide enough to pass
𝑠𝑖 (𝑡) with no distortion. The channel noise, w t is a sample function of zero-
mean white Gaussian noise process. Such a channel is called additive white
Gaussian noise (AWGN) channel.
• 𝑥 𝑡 = 𝑠𝑖 𝑡 + w(t) AWGN model of a channel
2
 Signal Space Analysis

Statistical model for a generic digital communication system

• Receiver: The receiver has the task of observing the received signal x(t) for a duration of Tb seconds and
then making a best estimate of the transmitted signal 𝑠𝑖 𝑡 or 𝑚𝑖 .
• Design the receiver so as to minimize the average probability of symbol error 𝑃𝑒 , which is defined as,
𝑀

𝑃𝑒 = ෍ 𝑝𝑖 𝑃 𝑚
ෝ ≠ 𝑚𝑖 𝑚𝑖
𝑖=1
✓ 𝑚𝑖 - transmitted symbol
ෝ - estimate produced by the receiver
✓ 𝑚
ෝ ≠ 𝑚𝑖 𝑚𝑖 - conditional error probability given that ith symbol was sent.
✓ 𝑃 𝑚
• The receiver that minimizes 𝑃𝑒 is said to be optimum in the minimum probability of error sense.
• To design an optimum receiver, we can use geometric representation of known set of signals 𝑠𝑖 𝑡 . 3
 Conversion of the Continuous AWGN channel into a Vector Channel
(Vector Model of AWGN)
• To design an optimum receiver (minimizing prob. of error), we
can use signal space concept and convert signals into vectors.
• The received signal, 𝑥 𝑡 from AWGN channel can be
converted into vector format, 𝑥𝑗 by passing 𝑥 𝑡 through a
bank of N product integrators or correlators

0≤𝑡<𝑇
𝑥 𝑡 = 𝑠𝑖 𝑡 + w(t), ቊ
𝑖 = 1,2, . . . , 𝑀
where w(t) is a sample function of
white Gaussian noise process, W(t) (b) Analyzer for reconstructing the signal vector {𝑠𝑖𝑗 }.
with zero mean and PSD or variance, 𝑇
N0/2. 0≤𝑡<𝑇
𝑠𝑖𝑗 = න 𝑠𝑖 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡 , ቊ
AWGN model of a channel 𝑖 = 1,2, .4. . , 𝑀
0
Conversion of the Continuous AWGN channel into a Vector Channel
(Vector Model of AWGN)
• If the received signal, 𝑥 𝑡 from AWGN channel, is applied to
𝑥1
the bank of N product integrators or correlators, the output
will be the observation vector, x which consists of N
components, {𝑥1 , 𝑥2 ,…, 𝑥𝑁 }.
• The output of correlator, j is the sample value, 𝑥𝑗 of random
𝑥 𝑡 𝑥2
variable Xj
𝑇 𝑇

𝑥𝑗 = න 𝑥 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡 = න 𝑠𝑖 𝑡 + w(t) . 𝜙𝑗 𝑡 𝑑𝑡
0 0

𝑥𝑗 = 𝑠𝑖𝑗 + 𝑤𝑗 , 𝑗 = 1,2, . . . , 𝑁 𝑥𝑁

𝑇 𝑇

where 𝑠𝑖𝑗 = න 𝑠𝑖 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡 , 𝑤𝑗 = න 𝑤 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡
Vector representation of Received Signal
0 0
5
Vector Model of AWGN – Statistical Characterization of the Correlator
Outputs
𝑥1 𝑠𝑖1 𝑤1
𝑥𝑗 = 𝑠𝑖𝑗 + 𝑤𝑗 , 𝑗 = 1,2, . . . , 𝑁 ⋮ = ⋮ + ⋮
𝑥𝑁 𝑠𝑖𝑁 𝑤𝑁
In vector form, x = 𝒔𝒊 + 𝒘
𝑇 𝑇

where 𝑠𝑖𝑗 = න 𝑠𝑖 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡 , 𝑤𝑗 = න 𝑤 𝑡 . 𝜙𝑗 𝑡 𝑑𝑡
0 0

deterministic Random variable

• Since 𝑠𝑖𝑗 is deterministic, the distribution of 𝑥𝑗 or x

depends on 𝑤𝑗 or 𝒘.
• Since 𝒘 is white Gaussian distributed noise with
zero mean value and PSD N0/2, each correlator
output 𝑥𝑗 will be a Gaussian random variable with
mean, 𝑠𝑖𝑗 and variance, N0/2.
6
Vector Model of AWGN – Statistical Characterization of the Correlator
Outputs
• Each correlator output 𝑥𝑗 is a Gaussian random variable with mean 𝜇𝑋𝑗 = 𝑠𝑖𝑗 and variance 𝜎𝑋2𝑗 = N0/2
for all j.
• This implies that all 𝑋𝑗 ‘s are mutually uncorrelated and have a variance equal to the PSD of noise.
Proof 𝑇 𝑇
𝐸[𝑤𝑗 ] = 𝐸 න 𝑤(𝑡)𝜙𝑗 (𝑡)𝑑𝑡 = න 𝐸[𝑤(𝑡)]𝜙𝑗 (𝑡)𝑑𝑡 = 0
0 0

𝑇 𝑇
𝐸[𝑤𝑖 𝑤𝑗 ] = 𝐸 න 𝑤(𝑡)𝜙𝑖 (𝑡)𝑑𝑡 න 𝑤(𝑢)𝜙𝑗 (𝑢)𝑑𝑢
0 0
𝑇 𝑇
= න න 𝐸[𝑤(𝑡)𝑤(𝑢)]𝜙𝑖 (𝑡)𝜙𝑗 (𝑢)𝑑𝑡𝑑𝑢
0 0
𝑇 𝑇
𝑁0
=න න 𝛿(𝑡 − 𝑢)𝜙𝑖 (𝑡)𝜙𝑗 (𝑢)𝑑𝑡𝑑𝑢 𝑁0
2 ,𝑖 =𝑗
0 0 Cov[𝑋𝑖 𝑋𝑗 ] = ቐ2
𝑁0 𝑇 𝑁0 0, 𝑖 ≠ 𝑗
= න 𝜙𝑖 (𝑡)𝜙𝑗 (𝑡)𝑑𝑡 = 𝛿
2 0 2 𝑖𝑗

So [w1, w2, …, wN] are zero-mean i.i.d. Gaussian distributed with variance N0/2. 7
Vector Model of AWGN – Statistical Characterization of the Correlator
Outputs

Statistical model for a generic digital communication system

• To design a receiver with minimum probability of error, we need to represent the conditional probability
density function (pdf) of received vector, 𝐱, given signal, 𝑠𝑖 𝑡 or symbol, 𝑚𝑖 was transmitted.
• Since the elements of 𝐱 are statistically independent, the conditional pdf of 𝐱 can be expressed as product
𝑥1
of individual pdfs. (𝜇𝑋𝑗 = 𝑠𝑖𝑗 and variance 𝜎𝑋2𝑗 = N0/2)
𝐱= ⋮
𝑁 𝑥𝑁
1 1 2
𝑓𝐱 𝐱/𝑚𝑖 = ෑ 𝑓𝑥𝑗 𝑥𝑗 /𝑚𝑖, 𝑖 = 1,2, . . . , 𝑀 where 𝑓𝑥𝑗 𝑥𝑗 /𝑚𝑖 = 𝑒𝑥𝑝 −
𝑁0
𝑥𝑗 − 𝑠𝑖𝑗
𝜋𝑁0
𝑗=1
−(𝑥−𝜇)2
𝑁 1
1 2 Gaussian PDF, 𝑓 𝑋 = 𝑒 2𝜎2
𝑓𝐱 𝐱/𝑚𝑖 = 𝜋𝑁0 −𝑁/2 𝑒𝑥𝑝 −෍ 𝑥𝑗 − 𝑠𝑖𝑗 , 𝑖 = 1,2, . . . , 𝑀 2𝜋𝜎2
𝑁0
𝑗=1 8
 Coherent Detection of Signals in Noise
• Suppose that, in each time slot of duration T seconds, one of the M
possible signals 𝑠1 𝑡 , 𝑠2 𝑡 ,…, 𝑠𝑀 𝑡 is transmitted.
• For geometric signal representation, the signal 𝑠𝑖 𝑡 , i = 1, 2, …, M, is
applied to a bank of correlators with a common input and supplied
with an appropriate set of N orthonormal basis functions.
• Thus each, 𝑠𝑖 𝑡 can be represented as a point in a N - dimensional
Euclidean space (N ≤ M).
location of the received signal point
• These points are referred to as the transmitted signal points, or
message points and the set of message points/ transmitted signals is
called a signal constellation.
• Similarly, the received signal 𝑥(𝑡) when applied to the bank of N
correlators, the correlator outputs define the observation vector x.
• x differs from the signal vector 𝒔𝒊 by the noise vector w, whose
orientation is completely random. (x = 𝒔𝒊 + 𝒘)
• x can also be represented as a point in the same Euclidean space and
Illustrating the effect of noise perturbation
this point is the received signal point.
on the location of the received signal9 point
 Coherent Detection of Signals in Noise – MAP Rule
Signal Detection Problem:
• Given the observation vector x, perform a mapping from x to an estimate 𝑚 ෝ of the transmitted
symbol, 𝑚𝑖 , in a way that would minimize the probability of error in the decision-making process.

Probability of error
• Suppose that, given the observation vector x, we make the decision 𝑚
ෝ = 𝑚𝑖 . The probability of
error in this decision, denoted by 𝑃𝑒 𝑚𝑖 𝐱 , is simply
𝑃𝑒 𝑚𝑖 𝐱 = 𝑃 𝑚𝑖 not sent 𝐱
= 1 − 𝑃 𝑚𝑖 sent 𝐱
Optimum decision rule
• The decision making criterion is to minimize the probability of error in mapping a given x into a
decision.
• The maximum a posteriori probability (MAP) rule is,
Set 𝑚
ෝ = 𝑚𝑖 if
𝑃 𝑚𝑖 sent 𝐱 ≥ 𝑃 𝑚𝑘 sent 𝐱 , for all 𝑘 ≠ 𝑖
where k = 1, 2, …, M 10
 Coherent Detection of Signals in Noise – MAP Rule
Optimum decision rule
• The maximum a posteriori probability (MAP) rule is,

• set 𝑚
ෝ = 𝑚𝑖 if 𝑃 𝑚𝑖 sent 𝐱 ≥ 𝑃 𝑚𝑘 sent 𝐱 , for all 𝑘 ≠ 𝑖

• Using Bayes’ rule, the MAP rule can be written as, 𝑃(𝑚𝑘 sent, 𝐱)
∵𝑃 𝑚𝑘 sent 𝐱 =
𝑓𝐱 𝐱
𝑝𝑘 𝑓𝐱 𝐱/𝑚𝑘
• Set 𝑚
ෝ = 𝑚𝑖 if is maximum for all 𝑘 = 𝑖 𝑃 𝑚𝑘 𝑓𝐱 𝐱/𝑚𝑘
𝑓𝐱 𝐱 =
𝑓𝐱 𝐱
• where
✓ 𝑝𝑘 is the a priori probability of transmitting symbol, 𝑚𝑘 (𝑝𝑘 = 𝑃 𝑚𝑘 )
✓ 𝑓𝐱 𝐱/𝑚𝑘 is the conditional pdf of the observation vector, 𝐱 given symbol 𝑚𝑘 was transmitted. Also
referred to as the Likelihood function, 𝐿 𝑚𝑘 = 𝑓𝐱 𝐱/𝑚𝑘
✓ 𝑓𝐱 𝐱 is the unconditional pdf of the observation vector, 𝐱
• Note : Likelihood function is used to estimate the message symbol, 𝑚𝑘 which generates the
observation vector, 𝐱 (or how likely a particular 𝑚𝑘 generates 𝐱 ).
11
 Coherent Detection of Signals in Noise – ML Rule
𝑝𝑘 𝑓𝐱 𝐱/𝑚𝑘
• MAP Rule: Set 𝑚
ෝ = 𝑚𝑖 if is maximum for all 𝑘 = 𝑖
𝑓𝐱 𝐱
• a priori probability 𝑝𝑘 = 𝑝𝑖 , when all source symbols are transmitted with equal probability.
• Denominator term 𝑓𝐱 𝐱 is independent of the transmitted symbol.
• The conditional pdf or Likelihood function, 𝐿 𝑚𝑘 = 𝑓𝐱 𝐱/𝑚𝑘 bears a one-to-one relationship to the
log-likelihood function, 𝑙 𝑚𝑘 = ln 𝐿 𝑚𝑘 .
• So, the decision rule can be restated as,
• ML Rule: Set 𝑚ෝ = 𝑚𝑖 if 𝑙 𝑚𝑘 is maximum for 𝑘 = 𝑖
• This decision rule is known as Maximum-Likelihood (ML) rule and the device based on this rule is
called Maximum likelihood decoder.
• According to ML decision rule, a maximum likelihood decoder computes the log-likelihood functions
as metrics for all the M possible message symbols, compares them, and then decides in favor of the
maximum.
• Thus, the maximum likelihood decoder is a simplified version of the maximum a posteriori (MAP)
decoder, in that the M message symbols are assumed to be equally likely.
• For equally probable symbols, the MAP rule becomes equivalent to the Maximum likelihood (ML) rule
12
THANK YOU…