What is an image?

An image is a data set recording (visual)

information at spatial locations (pixels).
Lecture 0:
What is Image Processing?

Math 490 95
Prof. Todd Wittman Pixel
“Picture element”
The Citadel

Common Test Images

Optical images Lena Barbara Cameraman

An image that corresponds to human vision is called an

optical image.

Optical cameras are engineered to record visual data as
a human would perceive it.

Most of our semester will be spent working with optical
images.

Mandrill Peppers Shepp-Logan Phantom

Uncommon Test Image The EM spectrum

But human vision takes up a very small part of the
electromagnetic spectrum.

Schrodinger

Engineers can build cameras to respond to wavelengths
beyond the human visual system.

1
Beyond human vision Geoscience & remote sensing
Infrared Radar
(Just to the right of visible) (Radio waves – very long wavelength)

An imaging modality is a system (camera) designed to
record a specific EM wavelength or set of wavelengths.

Infrared images

Sonar Lidar
(Acoustic waves travel underwater) (Range information from lasers. Pew pew!)

Medical imaging Image scale

Magnetic Resonance Imaging (MRI)

Positron Emission Tomography (PET)
From very large things in telescopy

The Crab Nebula

X-ray
Hubble Telescope

Computed Tomography (CT)

... to very small things in microscopy.

Oxidation
Atomic Force Microscope (AFM)
Ultrasound

1D Images 3D Images

Normally, images are 2D.
We could also add a z-axis to our images and

But if we take just one row of an image, we get a 1D construct a 3D image or volume.
signal.

Or if we imagine the 3rd axis is time, we can

Many ideas in image processing also work in signal
processing, e.g. enhancing audio signals. create a sequence of images to form a video.
x

125

y
Voxel
“Volume element”

2
 The Imaging Science Pipeline What is image processing?

Image processing is the study of how to:
1. enhance a damaged or inadequate
image.
2. extract meaningful features or statistics
from images.

1. Enhance an image 2. Extract image features

Change a “bad” image f into a “good” image u
Segmentation partitions an image
into meaningful pieces (segments).
Input: Output:
Image Processing
Bad image f Good image u

Detection looks for specific objects or
shapes in an image.

Examples of “bad” images.
Recognition identifies the objects in
Problem: Noisy Blurry Missing data Low resolution File too large an image.

Registration seeks a way to align

29.3 MB multiple images.

Task: Denoising Deblurring Inpainting Zooming / Compression
We might also be interested in a % of image that is in
focus = 63.1%
Super-resolution meaningful statistic about the image.

Levels of image processing Image Processing Tasks

An image processing task that relies primarily on
geometric properties is called a low-level task.

A task that requires (human) knowledge of the subjects
in the image is high-level task.

High-level tasks take us into the realm of artificial

intelligence.

Depending on how you incorporate knowledge, any low-
level task can be made into a high-level task.

3
 Who’s That Pokemon?!?
0 0 0 0 0 0 0 0 102 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 102 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 102 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 102 0 0 0 0 0 0 0
0 0 0 0 0 0 0 102 136 136 136 136 245 245 245 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 136 102 102 102 46 0 0 0 0 0
0 0 0 0 0 0 0 102 136 136 245 245 245 245 245 136 136 136 136 136 136 136 136 245 245 245 238 136 136 136 136 102 102 102 102 46 0 0 0 0 0
0 0 0 0 0 0 0 102 136 245 245 245 245 245 245 245 136 136 136 136 136 136 245 245 245 245 245 238 238 102 102 102 102 102 102 46 0 0 0 0 0
0 0 0 0 0 0 102 136 136 245 245 245 245 245 245 245 136 136 136 136 136 245 245 245 245 245 245 238 238 238 102 102 102 102 102 46 0 0 0 0 0
0 0 0 0 0 0 102 136 245 245 245 245 245 245 245 245 245 136 136 245 245 245 245 245 245 245 245 245 238 238 238 102 102 102 102 0 46 0 0 0 0
0 0 0 0 0 0 102 136 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 102 102 102 102 0 46 46 0 0 0
0 0 0 0 0 0 102 136 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 102 102 102 0 102 102 46 0 0
0 0 0 0 0 0 72 136 245 115 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 102 102 102 0 102 102 102 46 0
0 0 0 0 0 0 72 0 245 245 0 115 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 102 102 46 102 102 102 102 102 46
0 0 0 0 0 72 72 0 245 245 245 161 0 0 115 245 245 245 245 245 245 245 245 245 245 245 245 115 115 238 238 238 102 102 46 102 102 102 102 102 102
0 0 0 0 72 102 102 102 0 245 245 245 245 245 245 245 245 245 245 245 245 245 115 0 0 0 115 238 238 238 238 238 102 46 102 102 102 102 102 102 102
0 0 0 72 102 102 102 102 0 245 245 245 245 115 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 102 102 46 102 102 102 102 102 102 102
0 0 72 136 136 102 102 102 102 0 245 245 0 255 115 245 245 245 245 245 245 245 115 245 245 245 238 238 238 238 238 102 46 102 102 102 102 102 102 102 102
0 102 136 136 136 136 102 102 102 102 0 245 238 0 0 115 245 245 245 245 245 115 255 115 238 238 238 238 238 238 102 102 102 102 102 102 102 102 102 102 102
72 102 136 136 136 136 136 102 102 102 46 0 0 238 238 238 115 0 0 0 0 0 0 238 238 238 238 238 0 46 102 102 102 102 102 102 102 102 102 102 102
72 136 136 136 136 136 136 102 102 102 161 245 245 115 115 161 238 238 238 238 238 238 238 238 238 115 0 0 161 161 102 102 102 102 102 102 102 102 102 136 136
136 136 136 136 136 136 136 136 102 245 245 245 245 245 209 209 161 161 161 161 115 115 115 115 115 209 209 209 209 209 209 161 102 102 102 102 102 136 136 136 136
136 136 136 136 136 136 136 136 102 245 245 245 245 245 245 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 161 46 102 136 136 136 136 136 136
136 136 136 136 136 136 136 136 102 245 245 245 245 245 245 245 245 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 46 136 136 136 136 136 136 136
136 136 136 136 136 136 136 102 245 245 245 245 245 245 245 245 245 245 245 209 209 209 209 209 209 209 209 209 209 209 209 209 209 46 136 136 136 136 136 136 136
136 136 136 136 136 136 136 102 245 245 245 245 245 245 245 245 245 245 245 245 245 209 209 209 209 209 209 209 209 209 209 209 46 136 136 136 136 136 136 136 136
136 136 136 136 136 136 136 72 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 209 209 209 209 209 209 209 209 46 136 136 136 136 136 136 136 136
136 136 136 136 136 136 136 72 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 238 238 238 46 136 136 136 136 136 136 136 136 136
136 136 136 136 136 136 102 72 238 245 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 238 238 238 46 136 136 136 136 136 136 136 136 136
136 136 136 136 136 136 205 46 238 245 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 238 238 238 46 136 136 136 136 136 136 136 136 136 136
102 136 136 136 102 102 255 0 238 245 245 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 238 238 238 238 46 136 136 136 136 136 136 136 136 136 136
102 102 102 102 205 46 0 161 238 238 245 245 245 245 245 245 245 245 245 245 245 238 238 238 238 238 238 238 238 46 46 136 136 136 136 136 136 136 136 136 136
102 72 205 46 255 0 209 209 238 238 238 238 245 245 245 245 245 245 245 238 238 238 238 238 238 238 238 238 46 205 205 136 136 136 136 136 136 136 136 136 136
205 0 255 0 0 161 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 46 255 136 136 136 136 136 136 136 136 136 136 46
0 161 0 161 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 46 46 136 136 136 136 136 136 136 136 136 46 161
209 209 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 0 255 205 136 136 136 136 136 136 136 136 46 209 209
209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 0 0 0 102 136 136 136 136 136 102 46 209 209 209
209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 91 255 205 102 136 136 102 102 46 209 209 209 209
209 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 91 0 205 0 205 0 46 209 209 209 209 209
91 209 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 91 91 46 209 209 209 209 209 209 209
209 91 209 209 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 209 209 209 209 209 0 209 209 209 161
209 209 91 209 209 209 209 209 209 209 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 238 209 209 209 209 209 209 0 255 0 209 161 102
209 209 91 209 209 209 209 209 209 209 209 209 209 209 238 238 238 238 238 238 238 238 238 209 209 209 209 209 209 209 209 209 209 209 209 0 255 255 91 0 46
209 209 209 91 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 209 0 255 255 161 238 46

Snorlax!!!

Close but not quiteI Projects I Have Worked On

There are many fields that are related to or
I thought it might get you
use image processing, but are not quite interested in the course
the same. and image processing if I
 Imaging told you about some
 Computer graphics research projects I have
 Graphic design & image editing worked on in the last few
 Robotics & computer vision years.
 Machine learning / Artificial intelligence

Project: Pan-sharpening Project: Barcode Super-resolution

When building a camera, there is a trade-off between
spectral and spatial accuracy.
Taking a picture of a barcode with a camera does not result in
 More accurate color  Bigger pixels resolution as high as scanning it with a laser.

Super-resolution: Combine the information in the barcode image to

Pan-sharpening: Enhancing a satellite image using a produce a high-resolution signal.
high-resolution panchromatic image.

4
Project: Placental Analysis Project: Density Estimation

Data sometimes comes as a set of discrete events (e.g. crime

Analyzing the blood vessel network of the placenta (afterbirth) can reports, weather stations, census data).
help reveal health problems in the baby, e.g. autism, diabetes.
Density Estimation: Create an image reflecting the density of the

Blood Vessel Detection: Trace the blood vessel network in a events.
placenta image using a neural network.