Data transmittion
-
~
panael
-
Encoder hi
-
X
PyYx2) ·
1 /
.
Discrete Memoriless Channel (DMC :
P(Yp(x" y(y) =
P(((x1) Vk = wh
Without Feedback :
P(x , /x*ypy) =
P(xx)x yVk
*
= wh
DMC without Feedback :
p(yn/x4 = P13 : 1411
Channel Coding :
Edelt-
f i
=
-
Defi )E"
channel PlYI)
,
:
,
↓ J
input channel output
(n M) code M symbols length n
:
,
message encodes
into codewords with
Encode f [1
"
: :
,
2. 3--
,
M3 + *
Decode : g :
E -
> &1 ,
43 .
My
....
Prob , of error :
P(g(y) + :(w =
i)
Transmittion rate :
A
M
R = bits per channel
usage
Achievable rate :
There exist a
coding sequence (n .
T2VRT) code
That Be to for n+ S
,
we
say
R is a achievable rate
Channel Capacity :
Supremum of achievable rate
I
Joint typical set Pr[Tn-E
↑" (6) =
(( ,
3)) /logy -
H(X 106 ,
↓o
Joint AEPi
-
n(H(x ,)) + 8) n(H(X Y) 8) -
P(( YET* 81)
-
(1) 2 > 2
(2) (1 3) 24(H(x 4) 8)))T * n(H(x Y) + 8)
(8)112
- - ,
·
,
(3) Define ( **, Y) , Pyrn)4 3)
, = Pxyx) Pyly
nEWiYI +
381pp((Y, 2) fin 1611
-
(l-E) 2 *" :
EXiY) 30
+
< 2
proof of 131
P
O =
8
2 -2(H(y)
+
8)
2n(H(4
+
=
xi
.
"(H(x1 + H(31 + 28
/T(8))2
:
(H(xix)
(1-8)2 8)
~
n(H(x) +
H(()
-
-
2
+ 5)
z
.
: (1-3) n([(Xiy) +
38)
2
-
P
& =
-
n(H(x) -
6) 2(H(Y)
-
-
8/
(
-xi
.
n((f(x) + H(y) 28)
(in(s)
-
-
=
2
~ (((xix) + 8) -
n(H(x) +
H)() -
287
-2 ·
(h(1(Xiy) -
38)
Conditional AEP :
+ Y) =
99"/(x" , 2) Th "Cas
*
,
xv-) In 181
*
***
By the definition
*"
18) in "I
Th 181_ Th "(S)
of
(f
,
Tn" !6) in" Is
when X"is transmitted , most of the time Y" falls
In" (8)
:
in
-(((i(xy
28) +Y (81)26 n(H(y(x4) +
-
(1 3)2
-
28)
Shannon Capacity :
C =
max I(xi,
s
⑪ Shannon Channel Coding Theorem :
0
for DMc /1 PLYI)
a
, ,
#)
RCE R is acheivable
①
Random coding ; for a (n ,
24B) code
,
there
are n . 2"R possible codebook,
denote X (W) be the ith bits of with symbol
,
Then X (1 Xn)
I
code book
I
is
...
, the
X(z)
I
-
- -
Xn(4
C =
! !
**
X
,
(2m) - -
-
Xn(2 )
The prob of this codebook being chosen is
znRn
P(c) = P(X : M )
* symmetricity of random
prob of error :
coding
Pr(E) = [P(c)Pe"(C)
C
[PK)w
a
=
[P(c(X (c) ,
= Pr(E (w =
1)
error prob of all codeword = error prob of the first codeword
Ei =
E((i) Y ,
ET *1813 error 0:
·
P(E)w 1) =
i
= plEi)(w ht
=
)
> PLE"/w =
1) + PE/w = error & :
·
n(t(xiY) 38)
(
-
-
= 3 + -
1) -
2
y
AEP
nR
-
n([(xiY) -
38)
-E + 2
23n8 [h([(xi R)
) -
=E +
.
2E
TY8) :
9(04341 /lospin H(x11 -
/log pin-H1Y 1138
/log piga - H(Xis)) 83 <
①
by hLLN ②
-[20 ,
EN
, s %
.
, n > N
I log pyn -H(Xis))
,
S
-n(H(X y
n(H(x,y)
-
8)
f((p(xyyn))2
- + ,
Pr[(ET"181331-8 2
I'l
8 n(H(X ,) -
8)
1-31Playas
-
< /T* 161) . 2
,
/i 11 2- H(Xi
+
(2) 2
:
13 P1x4 ya)
[ ,
> .
(x4y4)ETY18 ,
8)
2n(((X Y)
+
=
It** (8) -Hi-8
,
<
< (1-3)
Thlf Trc8
x
(3)(8
=
3)2(H(Y(x) / + *
28,
2"(H(Y(x
-
(1 +
28
Y (5))
-
B(x) Px(X) Py(bl = Py(y) Par(x3 = (x14 y 19)
Let *, Y .
=
,
-
n([(x;y + 38/
= (1 9)2
-
2-v(H(xi 8) z n(H(x) + 8) -(H(y) + 8)
- (1-3)
-
-
-
.
z
*
(H(x) + 8) -(H)y) + 8)
-(Tr )(z
- -
.
z
[Par(s) =
giP(x)p(y)
T
*
y7
- (H(x) 8) -(H)y) 8)
(y()2
-
-
-
? .
z
- 24(H(i) + 8)
.
z
-(H(x) 8) -
.
z
-
-(H)y) 8)
-
n([(X : Y) + 36)
= 2
: Ec *, PLE
= 12
=> less then half of the codewords has error of probability higher than 42
chose the best half of the codewords to form a (n LP-1)
,
code
with rate R-m
n+ 0 rate + R Pe +
, ,
: R is a cheirable
#
② (YW X - W
Y: +
① Fano's inequality :
Pelog)2 1
H(w(yY) = hy(Pe) +
[hn(Pe) + PenR
② DPI
(x" : M H(w/YY
& [[(wiYH(W) -
= nR -
(hy(Pe) + PenR)
③
](x * H(yY) H(-y(xY
*
Y ) = -
> H(Y) - H(Y 1 Y: XY
- :
= HY- H :(i)
DiC
= (Xi Yi) C in
from 8 8
.
,
nR-hy(Pel-Pour RsFrenhn(e)
Inc .
+
4e + 0
n+ 0 , = R(C
channel
Q : matrix
[Q]ij = P(y i(x
=
=
j)
Symmetric channels
columns are a
permutation of each other
vow -
channel
Weakly-symmetric
:
onlyrow 11
Quasi-symmetric Channel :
exist a partition of rOWS that makes the channels weakly
-
symmetric
channel capacity of weakly symmetric channel :
C max
I(X ! ") H(Y(X)
I
=
I
P(X)
max (H(y))
=
ExH(Y(X :
x
1)
P(X)
-
H(Y(X =
x) H(Y(X =
x)
=
H(Y(X x) =
log(I)-P(Y(x) log independent of P(X)
is
=
55
I Ei)
exi
5 +
C =
log3
-
(log2 + log 3 -10g6) +
=
Blogb-Glogh flog -
channel capacity of quasi-symmetric channel :
X
[i =
log (In)- P(x) log **
[(xiY) = 1-log
En ,
= P(YEIn)E Flog In
=I (i
if IikX : ' has the maximum with the same P(X)
= Ci
BEC : A quasi-symmetric channel
a =
[
X Y
,
a =
( is]
<
,
ac :
( *] it
O Laximized at P(x =
1) = P(x= 0) =
i
①
G is constant at all PIX)
· /talC , &C
C = +
= 1 -
4
+
a+ P(x =
1) =
p(X o =
=
