Image Analysis (FMAN20, MATC20, FMA171F)

Lecture 1, 2025
Magnus Oskarsson
 Overview

1. Course information
2. Image Analysis -- examples
3. Image Models (continuous vs discrete)
4. Sampling
5. Discrete geometry
 Image Analysis - Motivation

A bowl of soup that

looks like a monster
knitted out of wool

DALL-E 2 Text to
image generation
 Image Analysis - Motivation

A bowl of soup that

looks like a monster
knitted out of wool

DALL-E 2 Text to
image generation
 Image Analysis - Motivation

DALL-E 2 Text to
image generation
Image Analysis - Motivation
Image Analysis - Motivation

https://www.youtube.com/watch?v=N--qC3T2wc4

Towards An End-to-End Framework for Flow-Guided Video Inpainting

Zhen Li1 * Cheng-Ze Lu1 * Jianhua Qin2 Chun-Le Guo1† Ming-Ming Cheng1
1
TMCC, CS, Nankai University 2 Hisilicon Technologies Co. Ltd.
[email protected], [email protected], [email protected]
{guochunle, cmm}@nankai.edu.cn

Abstract
Course Evaluation

”The amount of work in this course is way too much.”

”Even though the assignments are almost all coding, there's almost
no useful coding examples of the techniques. ”

”The amount of methods made it difficult to learn them in depth.”

”The people grading the assignments have very different ways of

grading.”

”More feedback would be appreciated.”

”Review faster.”
Course information
• All information will be on the Canvas page
• Lectures: two or three per week
• Assignments: four mandatory (more information next lecture)
• Exercises: if you need help with assignments, two per week
• Passed assignments gives passing grade
• For higher grade there is an optional take-home/oral exam
• Possibility to take a project course next study period
Canvas page
Canvas page
 Additional Text
http://szeliski.org/Book/
CVML
• Division of Computer Vision and Machine Learning
• 3 prof, 5 lecturers, 3 postdocs, 15-20 phd students
• Mathematics and mathematical statistics
• Centre for Mathematical Sciences
• CVML (appr. 25 employees)
• Analysis, Algebra and Dynamical Systems
(appr. 20 employees)
• Applied Mathematics (appr. 20 employees)
• Mathematical statistics (appr. 30 employees)
• Mathematics NF (appr. 20 employees)
Research
• Geometry (3D shape, camera calibration, camera
motion , structure and motion, robotics)
• Medical Image Analysis (Shape variation,
segmentation, tomography, decision support)
• Cognitive Vision (recognition, detection, scene
interpretation, attention, segmentation, handwriting
recognition)
The goal of Image analysis
• To bridge the gap between pixels and “meaning”

•What a computer sees

•Source: S. Narasimhan
The goal of Image analysis
• To bridge the gap between pixels and “meaning”

•What a computer sees

•Source: S. Narasimhan
The goal of Image analysis
• Images are functions. Each pixel measures brightness

•What we see

•What a computer sees

•Source: S. Narasimhan
Why images?
• As image sources multiply, so do applications
• Relieve humans of boring, easy tasks
• Enhance human abilities: human-computer interaction, visualization
• Perception for robotics / autonomous agents
• Organize and give access to visual content
What kind of information can we
extract from an image?
• Metric 3D information
• Semantic information
• Think about tasks that you solve with your own eyes!
 Vision as measurement device
•Reconstruction from
•Real-time stereo •Structure from motion Internet photo collections

3D-
Reconstruction

•NASA Mars Rover Mapping

and
Localization

Autonomous
•Pollefeys et al.
Vehicles
•Goesele et al.
•Vision as a source of semantic information

•slide credit: Fei-Fei, Fergus & Torralba

•Object categorization

•sky
•building

•flag

•face
•banner
•wall
•street lamp
•bus •bus

•cars •slide credit: Fei-Fei, Fergus & Torralba

•Scene and context categorization
• outdoor
• city
• traffic
•…

•slide credit: Fei-Fei, Fergus & Torralba

•Qualitative spatial information

•slanted

•non-rigid
moving object

•vertical

•rigid moving •rigid moving

object object
•horizontal •slide credit: Fei-Fei, Fergus & Torralba
Current state-of-the-art example

Mask DINO: Towards A Unified Transformer-based Framework for Object

Detection and Segmentation

Feng Li1,3 *†, Hao Zhang1,3⇤† , Huaizhe Xu1,3 , Shilong Liu2,3 ,

Lei Zhang3‡, Lionel M. Ni1,4 , Heung-Yeung Shum1,3
1
The Hong Kong University of Science and Technology.
2
Dept. of CST., BNRist Center, Institute for AI, Tsinghua University.
3
International Digital Economy Academy (IDEA).
4
The Hong Kong University of Science and Technology (Guangzhou).
{fliay,hzhangcx,hxubr}@connect.ust.hk {liusl20}@mails.tsinghua.edu.cn {leizhang}@idea.edu.cn {ni,hshum}@ust.hk
Why is working with images
challenging/difficult?
•Challenges: viewpoint variation
•Challenges: illumination

•image credit: J. Koenderink

•Challenges: scale
•Challenges: deformation

Théodore Géricault, Derby at Epsom, 1821

•Challenges: occlusion

•Magritte •slide credit: Fei-Fei, Fergus & Torralba

•Challenges: background clutter
•Challenges: Motion
Challenges: object intra-class variation
Challenges: local ambiguity

•slide credit: Fei-Fei, Fergus & Torralba

Challenges: context
Challenges: context
Challenges: context
Challenges: context/viewpoint
Challenges: context/viewpoint
In this course
• Tools:
• Basics of image modelling
• Linear Algebra, Linear System Theory
• Filters
• Mathematical Statistics
• Machine Learning
• Segmentation
• System development
• Based on the tools
• Ground truth, evaluation, benchmarking
After the course
• You should be able to develop and test your own image
analysis system
• You should have tools for understanding and working with big
data
• You should have improved your skills in programming and
modelling.
Continuous model

Continuous Model
An image can be seen as a function

f : ⌦ 7! R+ ,
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-l

2
where ⌦ = { (x, y ) | a model
Continuous x  b, c  y  d} ✓ R and
(ctd.)
R+ = {x 2 R | x 0}. f (x, y ) = intensity at point (x, y ) =
Change to gray-scale [0, L] where 0=’black’ and L=’white’.
gray-level
(f does not have to be continuous)
0  Lmin  f  Lmax  1
[Lmin , Lmax ] = gray-scale

Kalle Åström Image Analysis - Lecture 1

 Continuous model

Continuous Model
An image can be seen as a function

f : ⌦ 7! R+ ,
2
where ⌦ = { (x, y ) | a  x  b, c  y  d} ✓ R
R+ = {x 2 R | x 0}. f (x, y ) = intensity at point
gray-level
(f does not have to be continuous)
0  Lmin  f  Lmax  1
[Lmin , Lmax ] = gray-scale

Kalle Åström Image Analysis - Lecture

Discrete Image Model
• Discretize x,y -> sampling M rows, N columns
• Discretize f -> quantization
• (often in 2m levelsGeneral
) Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-le

• Color depth Continuous model (ctd.)

• ”8 bit grayscale”, 2 8 = 256 levels, 0-255
Change to gray-scale [0, L] where 0=’black’ and L=’white’.

• Decreasing M and N
• Chess patterns
• Decreasing m
• False contours
Sampling, decreasing M and N
Sampling, decreasing M and N
Sampling, decreasing M and N
Sampling, decreasing M and N
Sampling, decreasing M and N
Quantization, decreasing m
Quantization, decreasing m
Quantization, decreasing m
Quantization, decreasing m
 x= 1 y= 1

nterpolation and Discretization

normalized sinc function, sinc, is defined as
he fourier transform is zero outside ( the interval [ ⇡, ⇡] ⇥ [ ⇡, ⇡].
1.3 Interpolation and Discretization
sin ⇡x
of discrete functions f : sinc(x) Z !=R that
2 ⇡x , arexalso
6= 0 square integrable, i.e. suc
Interpolation X1 X1
1, x=0

discretization
ofThe or sampling 2 is
idea of discretization or sampling is to find a way to m
|f
duce the ideal interpolation operator I : `2 ! B, such that
(i, j)| to find a way to ma
er the
points,
integer
• i.e.
Discrete f
points, :
image Z
i=
i.e.
f
F (x, y) = I(f )(x, y) =
21!
1 j= R.
X
f 1
: Z
1
X 2 !The R. idea
The of
idea interpolation
of interpolatio
sinc(x i) sinc(y j)f (i, j).
and produce a function F : R !2 R.
as input • and produce
Continuous image aF function
i= 1 j= 1 F2 : R 2 ! R.
troduce
Let
we have the
B
that discretization
denote
Going
ideal the
from
interplation F set
to operator
f
restores of
(sampling)
the D
sampled :
continuousB !
function, `
i.e. , such
functionsthat
F : R 2 ! R
enote the set of continuous functions F : R ! R
• 2
integral f (i, j) = D(F )(i,
I(D(F j)F.= F (i, j).
)) =
Z Z
1 1
function, • Going from f to F (interpolation)
sinc, is
interpolation methods candefined Z
as in a similar way. Introduce
often be written 1 Z 1 |F (x
(
X1
sin 1
⇡xX
Fh (x, y) = Ih (f )(x, y) = ⇡x , h(xx 6=
x= 1
i, y 0 j)f (i, j).
y= 1|F (x,
exists and such that the fourier
sinc(x) =i= 1 j= transform
1, 1 x=
x=0 1is zero
y= outside
1 the
d Let ` 2 denote
More on the set
technicalof discrete
details on howfunctions
to do Z
interpolation
such that the fourier transform is zero outside the i
•
3
f : 2 !
next R that a
terpolation operator
week… I : `2 ! B, such that
1 1
enote the set of discrete1 functions
1
X X
f : Z ! R that|far(i
2
X X
F (x, y) = I(f )(x, y) = sinc(x i) sinc(y
1 j)f
1 (i, j).
i= 1 j= 1 X X
i= 1 j= 1
 Gray level transformations
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation

Pixelwise
Gray-level operations
transformation
A simple method for image enhancement
Definition
Let f (x, y ) be the intensity function of an image. A gray-level
transformation, T , is a function (of one variable)

g(x, y ) = T (f (x, y ))
s = T (r ) ,

that changes from gray-level f to gray-level g. T usually fulfils

I T (r ) increasing in Lmin  r  Lmax ,
I 0  T (r )  L.

In many examples we assume that Lmin = 0 och Lmax = L = 1.

The requirements on T being increasing can be relaxed, e.g.
with inversion.
Kalle Åström Image Analysis - Lecture 1
 Gray level transformations
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gr

Thresholding
Example: Thresholding
Let (
0 r m
T (r ) =
1 r > m,
for some 0 < m < 1.
Gray level transformations
Example: Histogram equalization
Histograms
Histograms
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray

Continuous images
• Let s = T(r) be a gray level transformation
• Let pr be the histogram before the transformation
• Let ps be the histogram after the transformation
• Assume that T is a monotonically increasing function.
I Let s = T (r ) be a gray-scale transformation (r = T 1 (s))
• The pixels that were darker than level r before are darker than
I Let pr (r ) be the frequency function for the original image.
s after.
I Let ps (s) be the frequency function for the resulting image.

It follows that Z s Z r
ps (t)dt = pr (t)dt.
0 0
et s = T (r ) be a gray-scale transformation (r = T 1 (s))

Histogram equalization
et pr (r ) be the frequency function for the original image.
et p (s) be the frequency function for the resulting image.
s General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation

ws that Histogram
Z s equalization
Z r
ps (t)dt = pr (t)dt.
0 0
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation

ogram equalization (ctd.)

Take T so that ps (s) = 1 (constant).
Z r Z s Z r
pr (t)dt = 1dt = s ) s = T (r ) = pr (t)dt
his transformation is called
0 histogram
0 equalization. 0

or
Kalle Åström ds - Lecture 1
Image Analysis
= pr (r )
dr

Kalle Åström Image Analysis - Lecture 1

Histogram equalization
Histogram equalization
1000 pr

800

600

400

200

0
r
0 50 100 150 200 250
Histogram equalization
1000 pr

800

600

400

200

0
r
0 50 100 150 200 250

250

200 s = T(r)

150
s

100

50

0
0 50 100 150 200 250
r
Histogram equalization
1000 pr

800

600

400

200

0
r
0 50 100 150 200 250

250

200 s = T(r)

150
s

100

50

0
0 50 100 150 200 250
r
Histogram equalization
1000 pr

800

600

400

200

0
r
0 50 100 150 200 250

1000 ps

800

600

400

200

0 50 100 150 200 250