ECT305 Analog and Digital Communication

MODULE-4 , PART-1

Dr. Susan Dominic

As s i stant Profess or
De p a rtm ent O f E l e ct ronic s a n d C o m m u n i cat ion E n g i n eeri ng
R a ja giri S c h o ol Of E n gi neering A n d Te c hno logy

RAJAGIRI SCHOOL OF ENGINEERING AND TECHNOLOGY, KOCHI

In this module we will learn about some basic issues pertaining to the transmission of
signals over an AWGN channel

Consider a most basic form of a digital communication system as above

• A message source emits one symbol every T seconds with the symbols belonging to
an alphabet of M symbols denoted by 𝑚1, 𝑚2 , … . . , 𝑚𝑀
• In general each symbol is encoded into a distinct signal 𝑠𝑖 (𝑡) which is suitable for
transmission over the channel
• The signal 𝑠𝑖 (𝑡) occupies the full duration T allotted for symbol 𝑚𝑖
• 𝑠𝑖 (𝑡) is a real-value energy signal (i.e., with finite energy), as shown by

• The channel is represented by an AWGN model with the received signal as

• The receiver observes the received signal for a duration of T seconds and then makes an
estimate of the transmitted signal 𝑠𝑖 (𝑡) or equivalently the 𝑖𝑡ℎ symbol 𝑚𝑖
• Due to the presence of noise the receiver will occasionally make erroneous decisions
• The requirement is to design the receiver so as to minimize the average probability of
symbol error
which is defined as Conditional probability of making
𝑀 an error( conditional error
𝑃𝑒 = ෍ 𝑝𝑖 𝑃(𝑚
ෝ ≠ 𝑚𝑖 |𝑚𝑖 ) probability) given that 𝑖𝑡ℎ symbol
𝑖=1 was sent

Prior probability
of transmitting 𝑚𝑖
• The aim is to make the system as reliable as possible by minimizing the average
probability of symbol error between the transmitted symbol by the source and the
receiver output
• For this, we have to tackle two basic issues
1. How to optimize the design of the receiver to minimize the average
probability of error
 2. How to chose the set of signals 𝑠1 (𝑡), 𝑠2 (𝑡)…., 𝑠𝑀 (𝑡) for representing the
symbols 𝑚1, 𝑚2 , … . . , 𝑚𝑀 resp. since this affects the average probability of
symbol error
Both require an understanding of the geometric representation of signals
Geometric Representation of Signals

• The essence of geometric representation of signals is to represent any set of M

energy signals {𝑠𝑖 (𝑡) } as linear combinations of N orthonormal basis functions,
where N ≤ M,
• i.e given a set of real-valued energy signals 𝑠1 (𝑡), 𝑠2 (𝑡) ,…, 𝑠𝑀 (𝑡), each of duration T
seconds ,

1
where the coefficients 𝑠𝑖𝑗 are defined as

• The real-valued functions form an orthonormal set , i.e.

where is Kronecker delta

First condition of 3 ensures that each basis function is normalized to have unit energy
Second condition ensures that the basis functions are orthogonal wrt to each other within
the interval 0 ≤ t ≤ T
• For a given i, the coefficients can be viewed as an N-dimensional vector
denoted by
• The vector bears a one-to-one relationship with the transmitted signal 𝑠𝑖 (𝑡), i.e.

1. Given the N elements of the vector (i.e., 𝑠𝑖1 , 𝑠𝑖2 , … , 𝑠𝑖𝑁 ) operating as input, we can
generate the signal 𝑠𝑖 (𝑡) from 1

We can use a bank of N multipliers with each multiplier having its own basis function
followed by a summer can be viewed as a synthesizer
2. Conversely, given the signals operating as inputs, the
coefficients can be calculated using 2

The scheme consists of a bank of N product integrators or correlators with a common

input and each one of them supplied with its own basis function can be viewed
as an analyzer
We may state that each signal 𝑠𝑖 (𝑡) can be completely determined by the vector
of its coefficients

The vector is called a signal vector

We may visualize the set of signal vectors as corresponding set
of M points in an N-dimensional Euclidean space, with N mutually perpendicular axes
labeled . This N-dimensional space is called as the signal space
Example: Consider the case of a two dimensional signal space ( N=2) with three signals (M=3)
• In the N-dimensional space, we may define lengths of vectors and angles between
vectors
• The length ( called absolute value or norm) of a signal vector is represented by the
symbol
• The squared length of any signal vector is defined as the inner product or dot
product of with itself , i.e.

𝑠𝑖𝑗 is the jth element

of and ‘T’ stands
for transpose
• In the N-dimensional space, we may define lengths of vectors and angles between
vectors
• The length ( called absolute value or norm) of a signal vector is represented by the
symbol
• The squared length of any signal vector is defined as the inner product or dot
product of with itself , i.e.

𝑠𝑖𝑗 is the jth element

of and ‘T’ stands
for transpose
Relationship between the energy of a signal and its representation as a vector

Energy of a signal 𝑠𝑖 (𝑡) of duration T seconds,

Substituting

We have
Interchanging the order of summation and integration and then rearranging the terms,
we have

Since { } form an orthonormal set,

Hence, the energy of a signal 𝑠𝑖 (𝑡) is equal to the squared length of the signal vector
representing it
Similarly, for a pair of signals 𝑠𝑖 (𝑡) and 𝑠𝑘 (𝑡) represented by vectors 𝐬𝑖 and 𝐬𝑘
respectively, we may show that

Shows that inner product of energy signals 𝑠𝑖 (𝑡) and 𝑠𝑘 (𝑡) over an interval [0, T ] using their
time-domain representations is equal to the inner product of their respective vector
representations 𝐬𝑖 and 𝐬𝑘
 Another useful relation is

represents the Euclidean distance between the points represented by the

signal vectors 𝐬𝑖 and 𝐬𝑘
The angle subtended between two signal vectors 𝐬𝑖 and 𝐬𝑘 is given by

Hence, two vectors 𝐬𝑖 and 𝐬𝑘 are orthogonal if their inner product is equal to zero ,
i.e. equal to zero in which case, = 90 degrees
Therefore, geometric representation of signals can provide a compact characterization of
signals and can simplify analysis of their performance as modulation signals
How to choose the orthonormal basis functions ?

For a given set of M energy signals, , the orthonormal basis

functions can be extracted from the signals by following Gram-Schmidt orthogonalization
procedure
1
 2

1 2

3
2

3
Geometric Representation of Signals
Any signal 𝑠𝑖 𝑡 of finite energy can be represented as a linear combination of N orthonormal
basis functions

Synthesizer Analyzer

44
Conversion of continuous AWGN channel into a vector channel
• The received signal in accordance with idealized AWGN channel is given by
0≤t≤T
x t = si t + w t ቊ
i = 1, 2, . . M
where 𝑤(𝑡) is a sample function of white gaussian noise process 𝑊(𝑡) of zero mean and
power spectral density 𝑁0 /2

• On giving 𝑥(𝑡) as input to a bank of 𝑁 product integrators or correlators, the output of

𝑗th correlator is given by
T

xj = න x t ϕj t dt In terms of RVs
0 𝑿𝒋 = 𝒔𝒊𝒋 + 𝑾𝒋
= 𝒔𝒊𝒋 + 𝒘𝒋 for 𝑗 = 1, 2, . . 𝑁
where 𝑠𝑖𝑗 is a deterministic quantity and 𝑤𝑗 is the sample value of a random variable 𝑊𝑗 due
to channel noise w(t)
45
• Consider next a random process 𝑋 ′ (𝑡) whose sample function 𝑥 ′ (𝑡) is related to received signal
𝑥(𝑡) as

𝑥 ′ 𝑡 = 𝑠𝑖 𝑡 + 𝑤 𝑡 − ෍(𝑠𝑖𝑗 +𝑤𝑗 )𝜙𝑗 𝑡

𝑗=1
𝑁 𝑁

= 𝑠𝑖 𝑡 + 𝑤 𝑡 − ෍ 𝑠𝑖𝑗 𝜙𝑗 𝑡 − ෍ 𝑤𝑗 𝜙𝑗 𝑡
𝑗=1 𝑗=1
𝑁

= 𝑤 𝑡 − ෍ 𝑤𝑗 𝜙𝑗 𝑡 = 𝑤′(𝑡)
𝑗=1
• The sample function 𝑥 ′ (𝑡) depends
solely on channel noise w(t)
• Thus the received signal is expressed as On comparing with expression of transmitted
signal, 𝑁

𝑠𝑖 (𝑡) = ෍ 𝑠𝑖𝑗 𝜙𝑗 (𝑡) Deterministic

𝑗=1

Stochastic / Random
46
Statistical Characterization of Correlator Output

• Let 𝑋𝑗 denote random variable (sample value is represented by correlator output 𝑥𝑗 ,

𝑗 = 1, 2 … 𝑁) in response to received signal 𝑥 𝑡
• To find mean and variance of observation vector, X 𝑥1

• The mean of 𝑋𝑗 is given by

𝝁𝑿𝒋 = E Xj x(t)
𝑥2

= E s𝑖j + 𝑊𝑗
= s𝑖j + E 𝑊𝑗
= s𝑖j [𝑾𝒋 has zero mean (AWGN)]
𝑥𝑁

𝝁𝑿𝒋 = 𝒔𝒊𝒋

47
• The variance of 𝑋𝑗 is given by
𝑤1
𝟐
𝑬 𝑿𝒋 − 𝝁𝑿𝒋

w(t)
𝑤2

• The random variable 𝑊𝑗 is defined as

Therefore 𝑤𝑁

48
• Since noise process W(𝑡) is stationary, 𝑹𝑾 𝒕, 𝒖 = 𝑹𝑾 (𝒕 − 𝒖)

Autocorrelation of noise process =

49
Covariance of observation vector:

• Since 𝜙𝑗 (𝑡) form an orthogonal set, 𝑋𝑗 are mutually uncorrelated

50
 Hence 𝑋𝑗 and 𝑋𝑘 are uncorrelated

• The vector 𝑋 is defined as vector of 𝑁 random variables. Each element is an independent

Gaussian R.V. with mean 𝑠𝑖𝑗 and variance 𝑁0 /2

51
 𝑁𝑜
Since each 𝑋𝑗 is a Gaussian RV with mean 𝑠𝑖𝑗 and variance , we can write the conditional PDF
2

as

𝟏 𝒙 − 𝝁𝑿 𝟐
𝒇𝑿 𝒙 = 𝐞𝐱𝐩
𝟐𝝅𝝈𝟐 𝟐𝝈𝟐

Since the elements of X are independent , the conditional probability density function of
vector X, given that signal 𝑠𝑖 (𝑡) or correspondingly the symbol 𝑚𝑖 was transmitted, can be
expressed as the product of the conditional PDFs of its individual elements as

52
Hence we have

Here vector x and 𝑥𝑗 are sample values of random vector X and random variable 𝑋𝑗 ,
resp.
• Vector x is called observation vector
• 𝑥𝑖 is called observable element
• A channel satisfying the above equation is called a memoryless channel

53
Now we have fully characterised the first part of the above equation. Now we have to
account for 𝑤 ′ (𝑡).
We can prove that 𝑤 ′ 𝑡 is orthogonal to σ𝑁
𝑗=1 𝑥𝑗 𝜙𝑗 (𝑡)

Hence we can conclude that

Insofar as signal detection in additive white Gaussian noise is concerned, only the
projections of the noise onto the basis functions of the signal set affects the
sufficient statistics of the detection problem; the remainder of the noise is
irrelevant
54
Putting the theorem into a mathematical context we can say that the AWGN channel
below

is equivalent to

55
 Maximum Likelihood Decoding
• At the receiver, given the observation vector 𝐱, we need to estimate the message symbol
responsible for generating 𝐱
• The likelihood function is defined as
𝐿 𝑚𝑖 = 𝑓𝐗 (𝐱/𝑚𝑖 ) where 𝑖 = 1, 2, . . 𝑀
where 𝑓𝐗 (𝐱/𝑚𝑖 ) is the conditional probability density function (pdf) of the vector X given that
symbol 𝑚𝑖 was transmitted and is given by

• Each element 𝑋𝑗 is a gaussian R.V with mean 𝑠𝑖𝑗 and variance 𝑁0 /2

56
• In practice it is more convenient to use the log likelihood function for the AWGN channel is
defined as

𝑙 𝑚𝑖 = l𝑛(𝐿 𝑚𝑖 ) = l𝑛[𝑓𝑋 (x/𝑚𝑖 )]

𝑁 2
− 1
= 𝑙𝑛 𝜋𝑁0 2 𝑒𝑥𝑝 − σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑖𝑗
𝑁0

𝑁 1 2
=− 𝑙𝑜𝑔 𝜋𝑁0 − σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑖𝑗 ,
2 𝑁0

1 2
=− σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑖𝑗 , 𝑖 = 1, 2, . . , 𝑀
𝑁0

𝑁
1 2
𝑙 𝑚𝑖 = − ෍ 𝑥𝑗 − 𝑠𝑖𝑗 , 𝑖 = 1, 2, . . , 𝑀
𝑁0
𝑗=1
57
Maximum Likelihood Detection
• Suppose one of M possible signals 𝑠1 𝑡 , 𝑠2 𝑡 … 𝑠𝑀 𝑡
transmitted over a duration of T seconds with equal
1
probability
𝑀

• The signal 𝑠𝑖 𝑡 (or correspondingly the signal vector 𝐬𝑖 ) can

be represented by a point in Euclidean space of dimension
N ≤ 𝑀.
• This point is known as transmitted signal / message point.
• The set of message points corresponding to the set of
transmitted signals {𝑠𝑖 (𝑡)} is called signal constellation
• The correlator outputs determined by received signal x(t)
define observation vector 𝐱
• The vector 𝐱 differs from signal vector 𝐬𝑖 by noise vector 𝐰

58
 • The signal detection problem can be stated as follows:

Given the observation vector x, the mapping from x to an estimate m

ෝ of transmitted symbol 𝑚𝑖
must be done to minimize the probability of error in decision making process
• Suppose given an observation vector x, a decision is made, 𝑚
ෝ = 𝑚𝑖 . The probability of error in this
decision is defined as

• The optimum decision rule to minimize the probability of error is stated as

where k=1,2..M. This decision rule is known as Maximum A Posteriori Probability Rule (MAP)
59
The decoder which implements the MAP decision rule is called as a MAP decoder.

• Using Bayes’ probability rule,

𝑃 𝐴 𝑃(𝐵Τ𝐴)
𝑃(𝐴Τ𝐵) =
𝑃(𝐵)
• The MAP decision rule can be now rewritten as

where 𝑝𝑘 is the prior probability of message symbol 𝑚𝑘 .

60
• The denominator is independent of the transmitted symbol
• Since all message symbols are transmitted with equal probabilities, the prior
probabilities are equal
• The conditional probability density function bears a one-to-one relationship
with the log likelihood function
The decision rule can be hence restated as

Set m
ෝ = mi if l mk is maximum for k = i

This decision rule is called as maximum likelihood (ML) rule and the decoder that
implements this is called as maximum likelihood decoder (MAP rule becomes ML rule
when messgaes are assumed to be equiprobable)
61
• maximum likelihood decoder computes the log-likelihood functions for all M
message symbols, compares them and decides in favour of the maximum
• Thus the maximum likelihood decoder is simplified version of the MAP decoder where
the messages are assumed to be equally likely
1 2
• For AWGN channel, since 𝑙 𝑚𝑖 = − σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑖𝑗 , 𝑖 = 1, 2, . . , 𝑀, maximum
𝑁0

2
value for log-likelihood function would mean, minimum value for σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑖𝑗 ,
hence the decision rule can be restated as
2
Set m
ෝ = mi if σN
j=1 xj − skj is minimum for k = i

62
• Hence maximum likelihood decision rule is nothing but choosing the the message
point in an N-dimensional signal space that is closest to the received signal point
2
(observation vector) since σN
j=1 x j − skj determines the distance between the
observation vector x and the signal vector 𝒔𝒌

• Hence decision rule may be restated as

Set m
ෝ = mi if ||x-𝒔𝑘 || is minimum for k=i

63
Graphical Interpretation of Maximum Likelihood Rule

• Since ML rule is simply choosing the message point closest to the received signal
point, all points that are closest to a message point can be thought of as a region
around the message point called as the decision region
• Since there are M number of message points, we can picture the N-dimensional
space to be divided into M such decision regions surrounding the M message
points
• Hence by simply observing which decision region a received signal point falls in,
the estimate of the message point can be easily obtained

64
65
Hence we can restate the decision rule as
𝐎𝐛𝐬𝐞𝐫𝐯𝐚𝐭𝐢𝐨𝐧 𝐯𝐞𝐜𝐭𝐨𝐫 𝐱 𝐥𝐢𝐞𝐬 𝐢𝐧 𝐫𝐞𝐠𝐢𝐨𝐧 𝐙𝐢 𝐢𝐟
𝐭𝐡𝐞 𝐄𝐮𝐜𝐥𝐢𝐝𝐞𝐚𝐧 𝐝𝐢𝐬𝐭𝐚𝐧𝐜𝐞 𝐱 − 𝐬𝐤 𝐢𝐬 𝐦𝐢𝐧𝐢𝐦𝐮𝐦 𝐟𝐨𝐫 𝐤 = 𝐢

• In practice, the decision rule can be further simplified by expanding the

summation as
N N N N
2 2
෍ xj − skj = ෍ xj2 − ෍ 2xj skj + ෍ skj
j=1 j=1 j=1 j=1
• The first summation term is independent of k and can be ignored
• The second term is the inner product of observation vector and signal
vector
• The third term is the energy of the transmitted signal
2 1
σ𝑁
𝑗=1 𝑥𝑗 − 𝑠𝑘𝑗 ≅ −(σ𝑁
𝑗=1 𝑥𝑗 𝑠𝑘𝑗 − 𝐸𝑘 )
2

66
 • The maximum likelihood decision rule for AWGN channel which was stated as

Observation vector 𝐱 lies in region Zi if

N
2
෍ xj − skj is minimum for k = i
j=1

can be restated as:

Observation vector 𝐱 lies in region Zi if
1
σN
j=1 x j skj − Ek is maximum for k = 𝑖
2

Next we look at the optimum receivers for an AWGN channel

67
Correlation Receiver

For an AWGN channel, for the case when the transmitted signals 𝑠1 (𝑡), 𝑠2 (𝑡), … , 𝑠𝑀 (𝑡) are
equally likely, the optimum receiver consists of two subsystems namely:

The detector/ Demodulator: It consists of a bank of product integrators/correlators each

supplied with orthonormal basis functions operating on the received signal x(t) to produce the
observation vector x

The signal transmission decoder: that operates on the observation vector x to produce an
estimate of transmitted symbol to minimize probability of error – implemented in the form of a
maximum likelihood decoder

68
Detector

69
 Signal
Transmission
Decoder

Maximum Likelihood Decoding Principle:

𝑶𝒃𝒔𝒆𝒓𝒗𝒂𝒕𝒊𝒐𝒏 𝒗𝒆𝒄𝒕𝒐𝒓 𝐱 𝒍𝒊𝒆𝒔 𝒊𝒏 𝒓𝒆𝒈𝒊𝒐𝒏 𝒁𝒊 𝐢𝐟
𝟏
σ𝑵
𝒋=𝟏 𝒙𝒋 𝒔𝒌𝒋 − 𝟐 𝑬𝒌 𝐢𝐬 𝐦𝐚𝐱𝐢𝐦𝐮𝐦 𝐟𝐨𝐫 𝐤 = 𝐢

Inner product of x
and 𝒔𝑘

Detector + Signal transmission

decoder = Correlation receiver

70
71
Matched Filter Receiver

• The detector discussed has a set of correlators

• Alternatively, we may use an equivalent structure in place of correlators (each
correlator can be replaced by an LTI filter)
• Consider an LTI filter with IR ℎ𝑗 (𝑡) (j th arm of the bank of filters)
• With the received signal 𝑥(𝑡) as input, the output is defined by convolution integral

• Evaluating this integral over the duration of the transmitted signal 0 ≤ 𝑡 ≤ 𝑇 seconds
and observing the output at t=T, we have

72
73
• Consider next, a detector based on a bank of correlators
• The output of the j th correlator is

• For to equal , we choose

• Or equivalently , the IR of the desired filter can be expressed as

74
• An LTI filter with IR defined in such a way is called as a matched filter
• Hence we can have an equivalent receiver with matched filters in place of correlators
followed by a signal transmission decoder which implements the ML detection rule

75
76
Probability of Error for a correlation receiver or an equivalent matched-
filter receiver in the presence of AWGN (probability of error of maximum
likelihood detection)

• We need to evaluate the performance of the receivers in the presence of AWGN

𝑀
• Assume that the observation space Z is partitioned into a set of M regions, 𝑍𝑖 𝑖=1

, in accordance with the maximum likelihood decision rule

• Suppose also that symbol 𝑚𝑖 (or, equivalently, signal vector 𝒔𝑖 ) is transmitted and
an observation vector x is received
• Then, an error occurs whenever the received signal point represented by x does
not fall inside region 𝑍𝑖 associated with the message point 𝒔𝑖

77
• Averaging over all possible transmitted symbols assumed to be equiprobable, we
see that the average probability of symbol error is

An N-dimensional integration

78
Capacity of an AWGN channel
• The channel capacity, C, is defined to be the maximum rate at which information
can be transmitted through a channel.
• For an AWGN (zero mean) channel of bandwidth B Hz, and PSD of , the
channel capacity is given by
P- received signal power (at the
channel o/p)
𝜎 2 - variance of noise
P/ 𝜎 2 and P/𝑁0 𝐵 are the
expressions for SNR at the
channel output
79
• The channel capacity is an extremely important quantity, since it is possible to
transmit information through a channel at any rate less than the channel
capacity with smallest possible probability of error
• Completely reliable transmission is not possible if the information is sent at a
rate greater than the channel capacity

80
81
82