Sensor fusion
What is it?
Web Economy Bullshit Generator
[Link]
•synthesize holistic deliverables
•architect robust interfaces
•facilitate web-enabled interfaces
Combining data from different
sensor, in order to get a optimal
result.
1
Sensors
• Radar
• Sonar (under water) and Ultrasonic (above water)
• Infra-red / thermal imaging camera
• Cameras
• Tilt sensors
• Gyro
• GNSS(Global NavigationSatelliteSystem)
• Laserscanners
• Odometri encoders
• Kinect(IR)
Bjoern Griesbach griesbac@[Link]
2
Sensors
Can we use the result or data from one sensor alone?
That depends on the sensor.
☺
NO!
And the application it is used in.
3
Sensor fusion, why?
• Each sensor has its strengths and weaknesses
• One sensor is never sufficient for reliable
tracking/measuring
• Optimal tracking/measuring = use multiple
sensors
Precise Estimation
Data Fusion Component
Noisy Noisy
Data Noisy
Data
Data
Sensor Sensor Sensor
Bjoern Griesbach griesbac@[Link]
4
Sensor fusion/confusion
Said the eye one day, "I see beyond these valleys a mountain veiled with
blue mist. Is it not beautiful?“
The ear listened, and after listening intently awhile, said, "But where is
any
mountain? I do not hear "
Then the hand spoke and said, "I am trying in vain to feel it or touch it,
and I
can find no mountain.“
And the nose said, "There is no mountain, I cannot smell it.“
Then the eye turned the other way and they all began to talk together
way, about the eye's strange delusion. And they said, "Something
must be
the matter with the eye.“
[Link]: „The Madman“ (1946)
5
Mathmatical sensorfusion methods
Simple:
• Use normal math to correct a signal
• Use simple algorithms to select the most
accurate signal.
Complex:
• Kalman filter
• Dempster–Shafer
• Bayesian network
6
Simple Sensor fusion
Line drawed without sensor
fusion
7
Simple sensor fusion
8
Simple Sensor fusion
Line drawed with sensor
fusion
9
Simple Sensor fusion
Gyrodometri
10
Complex sensor fusion
Wanted: Fusion of odemetri and
GNSS data to detemine the most
accurate possition.
Estimate pose of position
How to realize?
Kalmanfilter
Bjoern Griesbach griesbac@[Link]
11
Complex Sensor fusion -
Kalmanfilter
Optimal data processing algorithm
• Filter out noise of measurement data
• Can fuse different measurements to find the optimal
result
• Optimal: incorporates all information (i.e. measurement
data) that can be provided to it
• Does not need to keep all previous measurement data in
storage!
Bjoern Griesbach griesbac@[Link]
12
Complex Sensor fusion -
Kalmanfilter
• Pose of an object is represented as a vector
=(x,y,θ);
• By transforming poses of two different sensors
into the same coordinate system, one will get
two (noisy) measurements and for the same
object
• Each measurement is weighted differently
depends on the variance of the measurement
• Each measurement has its own variance σi2
represented by a matrix Pi
Bjoern Griesbach griesbac@[Link]
13
Complex Sensor fusion -
Kalmanfilter
• Given: pose measurements: z1 z2 with
covariance matrices P1 P2 from two different
sensors
• Wanted: optimal weights to get optimal estimate
• Solution: Optimal estimate x with minimal
combined covariance matrix P:
r P2 r P1 r
x= z1 + z2 ;
P1 + P2 P1 + P2
P2
P= P1 ;
P1 + P2
Bjoern Griesbach griesbac@[Link]
14
Basic Kalman Filter
r r r r
• Process Model: x (t k ) = A ⋅ x (t k −1 ) + Bu (t k ) + w(t k −1 ) w(t ) → N (0, Q (t ))
r r r
• Measurement Model: z (t k ) = H ⋅ x (t k ) + v (t k ) v(t ) → N (0, R(t ))
• Algorithm: (vector arrows omitted!)
xˆ − (t k ) = Ax(t k −1 ) + Bu (t k ) [
xˆ(tk ) = xˆ− (tk ) + K ⋅ z(tk ) − H ⋅ xˆ− (tk ) ]
P − (t k ) = AP(t k −1 ) AT + Q (t k ) P(tk ) = [I − KH]P− (tk )
1. Predict 2. Correct
P − (t k ) H T
Kalman Gain K= −
HP (t k ) H T + R (t k )
Bjoern Griesbach griesbac@[Link]
15
Combine simple and complex
sensor fusion
16
Sensor Fusion
What can it be used for?
Your imagination is the limit!
17
