Introduction to Image

Processing
Yao Wang
Dept. Electrical & Computer Engineering
Brooklyn NY 11201
Brooklyn,

Adapted from slides by Zhu Liu

 Lecture Outline
• Applications of image processing
• Demonstration of basic image processing
tools
• Image formation and perception
• Image representation
• Matrix/Matlab primer

Yao Wang, NYU-poly EL5123: Introduction 2

 Application Areas of Image Processing
• Purpose of image processing
– Improvement of pictorial information for human interpretation
– Compression
C i off iimage ddata
t ffor storage
t and
d ttransmission
i i
– Preprocessing to enable object detection, classification, and
tracking
• Typical
T i l application
li ti areas
– Television Signal Processing
– Satellite Image Processing
– Medical Image Processing
– Robotics
– Visual Communications
– Law Enforcement
– Etc.

Yao Wang, NYU-poly EL5123: Introduction 3

 Television Signal Processing
• Image brightness, contrast, color hue
adjustment
• Video compression for efficient delivery
and storage
• Conversion among different video formats
– QVGA<->VGA<->XVGA
– SDTV<->HDTV
– NTSC<->PAL

Yao Wang, NYU-poly EL5123: Introduction 4

 Medical Image Processing
• Images are acquired to get information about Anatomy and
Physiology of a patient
• g from captured
How to reconstruct the image p data
• How to process/analyze the image to help diagnosis/treatment?

– Ultra Sound (US)

– Magnetic resonance Imaging

– Positron Emission Tomography (PET)

– Computer Tomography (CT)

– XRays

Yao Wang, NYU-poly EL5123: Introduction 5

 Visual Communication
• Videophone
• Tele-conferencing
• Tele-shopping
• How to compress the
video to reduce
bandwidth/storage
requirements
• How to conceal
artifacts due to
transmission losses?

Yao Wang, NYU-poly EL5123: Introduction 6

 Law Enforcement
• Biometric identification / verification
– Fingerprint
g p

– Face
F

– Iris

AR Face Database

– How to extract features that can be used to

g different images?
differentiate among g

Yao Wang, NYU-poly EL5123: Introduction 7

 Robot Control
• Automatic inspection
• Unmanned operations
– Autonomous Vehicle
driving
Mars Rover
• How to detect and track
th ttarget?
the t?

Yao Wang, NYU-poly EL5123: Introduction 8

 Satellite Image Processing
• Remote sensing
• Climate
• Geology
• Land resource
• Flood monitor
• How to enhance the
image to facilitate
i t
interpretation?
t ti ?
• How to analyze the
image
g to detect certain
phenomena?

New York (from Landast-5 TM)

Yao Wang, NYU-poly EL5123: Introduction 9

 Components in Digital Image Processing

Output are images

Wavelets and
Color image Morphological

Outputt are image attribute

Multiresolution Compression
processing processing
processing

Image
Segmentation
restoration

Image Representation
Knowledge base
enhancement & description

es
Image Object
acquisition recognition
Input
Image We will deal with mainly the light green boxes.
Yellow boxes belong to “computer vision” and “pattern recognition”

Yao Wang, NYU-poly EL5123: Introduction 10

 Basic Image Processing Operations
• Simple point processing
• Special effects
• Noise reduction
• Image enhancement
• Image restoration
• Face detection
• Image segmentation

Yao Wang, NYU-poly EL5123: Introduction 11

 Simple point processing
age

Horizontal flip
Original ima

H
O

Negative
al flip
Vertica

Digital N

Yao Wang, NYU-poly EL5123: Introduction 12

 Contrast Enhancement

Original image with low contrast Enhanced image

Yao Wang, NYU-poly EL5123: Introduction 13

 Noise Reduction

Degraded Image Noise reduced Image

(salt and pepper noise) (after median filtering)

Yao Wang, NYU-poly EL5123: Introduction 14

 Image Sharpening

Observed Image Enhanced Image

( h
(sharpening)
i )

Yao Wang, NYU-poly EL5123: Introduction 15

 Image Restoration

Degraded Image Restored Image

Yao Wang, NYU-poly EL5123: Introduction 16

 Image debluring

Yao Wang, NYU-poly EL5123: Introduction 17

 Special Effects
age
Original Ima

Rotation
O ave

Swirrl
Wa

Yao Wang, NYU-poly EL5123: Introduction 18

 Face Detection
• Face detection in an image or video
• Face tracking in a video
ideo

Yao Wang, NYU-poly EL5123: Introduction 19

 Image Segmentation
• Segmentation of different object in the
scene

Yao Wang, NYU-poly EL5123: Introduction 20

 Image Formation and Representation
• Overview
• Light physics
ph sics and color perception
• Grayscale image capture
• Color image capture
• Digital image representation

Yao Wang, NYU-poly EL5123: Introduction 21

 Image Formation
• Light source (λ: wavelength of the source)
– E(x, y, z, λ): incident light on a point (x, y, z world coordinates of
the point)
• Each point of the scene has a reflectivity function.
– r(x, y, z, λ): reflectivity function
• Light reflects from a point and the reflected light is
captured by an imaging device.
– c(x,
( y y, z, λ)) = E(x,
( y y, z, λ)) * r(x,
( y y, z, λ):
) reflected light.
g

E(x, y, z, λ)
c(x yy, zz, λ) = E(x,
c(x, E(x y,
y z,
z λ)
λ)*r(x
r(x, yy, zz, λ)

Camera( c(x, y, z, λ) )

Yao Wang, NYU-poly EL5123: Introduction 22

 Light is part of the EM wave

Yao Wang, NYU-poly EL5123: Introduction 23

 Three Attributes of Color
• Luminance (brightness)
• Chrominance
– Hue (color tone) and Saturation (color purity)
• Represented by a “color cone” or “color solid”

Yao Wang, NYU-poly EL5123: Introduction 24

 Tri-chromatic Color Mixing
• Tri-chromatic color mixing theory
– Any color can be obtained by mixing three
primary colors with a right proportion

Yao Wang, NYU-poly EL5123: Introduction 25

 Gray and Color Image Capture
• Gray images are captured by a single sensor,
sensitive to the entire visible spectrum, similar to
the rods
• Color images
g are captured
p by
y havingg three
sensors, each sensitive to a different primary
color, similar to the cones

Yao Wang, NYU-poly EL5123: Introduction 26

 Gray and Color Image Display/Printing
• Gray images are displayed by a single light
sensitive diode, with intensity proportional to
gray level.
• LCD monitors display
p y color images
g are
displayed by having three phosphors at each
pixel, each generating a different primary color
(red, green, blue)
• Color images are printed by having three color
i k ((cyan, magenta,
inks t yellow)
ll )

Yao Wang, NYU-poly EL5123: Introduction 27

 Eye Anatomy

From [Link]

Yao Wang, NYU-poly EL5123: Introduction 28

 Eye vs. Camera

Camera components Eye components

Lens Lens, cornea
Shutter Iris, pupil
Fil
Film R ti
Retina
Cable to transfer images Optic nerve send the info to
the brain
Yao Wang, NYU-poly EL5123: Introduction 29
 Receptors in the Retina
• Rods
– night vision
– Low acuity
– Achromatic
• Cones
– Day vision
– High acuity
– Chromatic
– Three sets, with different sensitivity functions
• 700nm (R), 546nm (G), 435nm (B)

Yao Wang, NYU-poly EL5123: Introduction 30

 Color Representation
• Specify three primary colors directly
– Red, Green, Blue (RGB)
– Cyan, Magenta, Yellow (CMY)
• Specify the luminance and chrominance
– HSB or HSI (Hue
(Hue, saturation
saturation, and brightness or
intensity)
– YIQ (used in NTSC color TV)
– YCbCr (used in digital color TV)
– Can be determined from RGB or CMY
• Amplitude specification:
– 8 bits per color component, or 24 bits per pixel
– Total of 16 million colors
– A 1kx1k true RGB color requires 3 MB memory

Yao Wang, NYU-poly EL5123: Introduction 31

 Analog to Digital Image Conversion
• Sampling: Dividing a continuous region into small
squares (pixels), taking average value of each square
• Quantization: Map each value into one in a set of
discrete values

Yao Wang, NYU-poly EL5123: Introduction 32

 Digital Image Capture by CCD Array
• Continuous Scene -> Digital image
– Each CCD sensor averages the light intensity in a
small region and output a discretized value

Yao Wang, NYU-poly EL5123: Introduction 33

 Grayscale Image Specification
• Each pixel value represents the brightness of the pixel. With 8-bit
image, the pixel value of each pixel is 0 ~ 255
• M t i representation:
Matrix t ti An
A image
i off MxN
M N pixels
i l is
i represented
t dbby an
MxN array, each element being an unsigned integer of 8 bits

160 162  166 154

162 158  122 69 
 
M       
 
 60 55  79 94 
 58 55  99 109

Yao Wang, NYU-poly EL5123: Introduction 34

 Color Image Specification
• Three components
– M = {R
{R, G
G, B}
 73  87 66  98 31  61
R      , G      , B      
27  17  36  13 36  14

G B
R

Red nose is brightest! Blue Cheek is brightest

Yao Wang, NYU-poly EL5123: Introduction 35

 A Brief Matrix Tutorial
• A matrix is an NxM array of numbers
– If M
M=N,N, A is a square matrix  a11 a12  a1M 
a  a2 M 
– If N = 1, A is a row vector  a22
A 21

– If M = 1, A is a column vector      
 
– If N=M=1, A is a scalar a N 1 a N 2  a NM 
– aij is the element of A at row i and column j.
• Special Matrix 1
0
0  0
1  0
– Identity (unit) matrix I A
   
 
0 0  1 a11 0  0 
0  0 
– Diagonal matrix A
a22
    
 
0 0  a NN 

Yao Wang, NYU-poly EL5123: Introduction 36

 Matrix Operations
• Addition / Subtraction of matrices.
– Matrices A and B are of the same order (NxM)
[C ]  [ A]  [ B], cij  aij  bij ;
[ D]  [ A]  [ B], d ij  aij  bij .
• Multiplication of two matrices
– A is an NxM matrix, B is an MxL matrix, note that
the column number of A should be the same as the
row number of B. M

– C=A
C=A*B B, an NxL matrix
cij  aik bkj
k 1


* =

Yao Wang, NYU-poly EL5123: Introduction 37

 Matrix Operations
• Multiplication of a matrix by a scalar [C ]  [ A], cij  aij

• Transpose of a matrix [C ]  [ A]T , cij  a ji

– Interchange of rows and columns
• If A = AT, A is symmetric
• Matrix inverse [C ]  [ A]1 , if AC  I
– A must be a square
q matrix
– Some matrices are invertible (singular) N
• Trace of a square matrix trace( A)   aii
i 1
• Determinant
D t i t off a square matrix
ti
a11 a12
A,  a11 * a22  a12 * a21
a21 a22

Yao Wang, NYU-poly EL5123: Introduction 38

 A Brief Matlab Tutorial
• An interactive program from The MathWorks for
high-performance
high performance numeric computation and
visualization.
• Refer to Matlab Primer for g
general use
• Type “help plot” to see help information of
function plot.
• Type “help image” to see functions in image
processing toolkit

Yao Wang, NYU-poly EL5123: Introduction 39

 Matrix, Vector, and Scalar in Matlab
>> a=[1 2 3; 4 5 6; 7 8 9]
• A matrix is a
a=
2 dimension arra
array
1 2 3
>> a = [1 2 3]
4 5 6
7 8 9
a=

1 2 3
• A vector is a 1XN or Nx1 matrix
>> b = [1;2;3]

b=
• A scalar is a 1 x 1 matrix >> a=1
1
a= 2
3
1

Yao Wang, NYU-poly EL5123: Introduction 40

 Colon Operator in Matlab
• If two integer numbers are separated >> a=[1:5]
a=
by a colon, Matlab will generate all of 1 2 3 4 5
the integers between these two
integers. >> b=[0:2.0:9.6]
b=
• If three numbers
numbers, integer or non- 0 2 4 6 8
integer, are separated by two colons,
the middle number is interpreted to be >> c=[1 2 3; 4 5 6; 7 8 9]
a ”step"
” t " andd the
th fifirstt and
d thi
third
d are c=
1 2 3
interpreted to be "limits” 4 5 6
• The colon operator is useful in 7 8 9
>> d=c(1:2,2:3)
extracting smaller matrices from larger d=
matrices. 2 3
5 6

Yao Wang, NYU-poly EL5123: Introduction 41

 Plotting figures in Matlab
Line plot: Stem plot:
x=0:0.05:5;,y=sin(x.^2);,plot(x,y); x = 0:0.1:4;, y = sin(x.^2).*exp(-x);stem(x,y)

Surface plot:
zz=peaks(25);
peaks(25);, surf(z);
surf(z);,
colormap(jet);

Yao Wang, NYU-poly EL5123: Introduction 42

 Image Toolbox in Matlab

>> x = imread('[Link]');
>> whos
Name Size Bytes Class

x 256x256x3 196608 uint8 array

Grand total is 196608 elements using 196608 bytes

>> imshow(x);
( )
>> g = rgb2gray(x);
>> imshow(g);
>> g(1:2, 1:2)

ans =

64 46
77 75

Yao Wang, NYU-poly EL5123: Introduction 43

 Recommended Readings
• Gonzalez & Woods, “Digital Image Processing”,
Chapter 1, Chapter 2 (Section 2.1 – 2.3)
• MATLAB tutorial
– [Link]
p pp _ g
[Link].
• Matrix tutorial:
– [Link]
[Link]
ece utk edu/~gonzalez/DIP WEB V3/downloads V3/dip3e
downloads/dip3e_classroom_presentations/Review_of_matrices_and%
[Link]
– [Link]
eview_material/[Link]
i t i l/ i i
– Introduction to Matrix Algebra.
[Link]
– Free chapter texts are available at
[Link]

Yao Wang, NYU-poly EL5123: Introduction 44

 Homework (Due Next Lecture)
1. If you do not have a copy of MATLAB, get one installed by Poly IS helpdesk or use
the PC Lab at Poly.
2. Get a picture of yourself taken in full colors. You could take the picture with a digital
camera, and save it in the JPEG format. Alternatively, you could take the picture
using the WebCam available at the Multimedia Lab (LC008).
3. Write a matlab program that can i) read your color image into a matrix (you can use
imread() function in MATLAB); ii) covert it to a 8-bit grayscale image (you can use
rgb2gray() in MATLAB), iii) generate a digital negative version of your image, and iv)
display and print both the color image
image, the original gray scale and the negative
image.
4. If you have not used Matlab before, go through Matlab getting started guide at
[Link] Go through all
exercises step by step. Make sure that you know basic operations and how to write
m files (scripts or functions)
m-files functions).
5. If you have forgotten your matrix operations, go through first four chapters of
Introduction to Matrix Algebra. Free chapter texts are available at
[Link]

Note: For this (and future) computer assignment, you should submit a lab report that
includes a copy of your matlab programs, the pictures you created, as well as a
description of your findings if applicable. All printed figures should have a caption that
explains what they are. You should submit a printed copy of your report together with
your other written homework
homework.

Yao Wang, NYU-poly EL5123: Introduction 45