Lab Assignment 1: Homography estimation and interpretation
The objective of this lab assignment is to present the student with various techniques related to the
application of concepts of projective geometry to the field of computer vision, in particular to the
estimation of the geometric structure of the scene.
For the understanding of the behavior of the functions that are proposed, it is essential to navigate
through the MatLab help: this script does not include precise explanations on how to use MatLab
functions.
The answers to the proposed exercises should be included in a PDF report named
‘LA1_name_SURNAME.pdf’. This file, along with the code for every exercise (in separate .m files)
should be combined in a ‘LA1_name_SURNAME.zip’ file and uploaded to the moodle platform.
1.1 Homography computation
A homography between two planes or images x, y and x, y is given by the expression
Xθ = H× Xψ , where H is a 3x3 matrix of coefficients:
h11 h12 h13
H = h21 h22 h23
h31 h32 h33
One of the objectives of this lab assignment is to illustrate with MatLab several methods to obtain a
homography starting from different types of information.
1.1.1 Exercise 1: homography defined by four pairs of transformed points
The objective is to obtain the homography defined by the transformation of four points, that is, by four
pairs of transformed coordinates, and then apply it on a specific image, the image 'hopper.jpg', visualizing
both the original image and the result of the application of the transformation
Take as transformed points those defined by each of the column vectors of the following matrices:
xy_origin = [41 484 586 72; 124 30 299 414];
xy_target = [32 632 632 32; 25 25 468 468];
To apply a homography in MatLab, we propose to use the imtransform function. This function
applies to the image defined by its first parameter, the transformation defined by its second parameter 1.
This transformation is defined by a data structure that can be generated using various methods. To define
the transformation use two alternative methods:
1. Use the maketform function of MatLab directly, in the version in which it supports taking as
parameters the four transformed points. This MatLab function directly computes the matrix
H from the four points given.
2. Use the same function, but now in the version whose second parameter is now the matrix that
1
To force the transformed image to be the same size as the input image, specify the output size in the function:
tr_ima = imtransform(ima,tform,'XData',[1 size(ima,2)], 'YData',[1 size(ima,1)]);
defines the homography, transposed. In this case, it is necessary to first obtain the H matrix,
using the homography_solve_vmmc function, whose code is provided.
Note that the result obtained is visually the same in both cases. Check that the homographies that are
applied using these methods (accessible through the tdata.T component of the data structure that defines
the transformation) are not equal but proportional. Finally check, calculating the energy of the difference
between both results (use the provided get_error_energy_vmmc function for this), whether both
transformations are identical.
Include in your report the original image, the two transformed images, the two homography matrices
applied, and the energy of the transformed images’ difference.
Later work: identify the code of the homography_solve_vmmc function for the case in which four pairs
of transformed points are passed, and describe how the following stages are implemented: generation of
the system of equations (note that what is done is to solve the homogeneous system Xθ − H× Xψ = 0 ),
resolution based on singular value decomposition, and obtaining the final solution H that defines the
homography
1.1.2 Exercise 2: problems in the definition of a homography from four pairs of
transformed points.
Although in the previous exercise the coordinates of the four pairs of transformed points have been
provided, in a real situation these coordinates usually come either from a manual selection or from the
result of an algorithm which automatically searches for characteristic points. In both situations, more in
the second than in the first, the imprecision of the process in terms of the location of the points suggests
the use of more than four pairs of points in order to estimate the homography that fits all couples with
minimum error.
This exercise first proposes to repeat the previous exercise, but now by using a function
get_user_points_vmmc that returns the coordinates of the points marked by a user. To do this, follow the
following steps:
• Load the image 'hopper.jpg', which will act as the original or initial image, and mark four
points on it (initially mark the four corners of the painting).
• Generate an empty image (i.e., a black image) of the same size and type as the starting
image. This image is only going to be used to mark points on it and thus obtain the region in
which you want to transform the picture of the original image. Mark, therefore, four points
on this second image.
• Finally, apply any of the two methods seen in the previous exercise to obtain the transformed
image.
In order to evaluate quantitatively the imprecision committed with manual marking, the following
experiment is proposed:
Load the image 'perspective_pattern.bmp', and apply the homography defined by the following pairs
of transformed points:
xy_origin = [18 475 599 41; 59 11 420 446];
xy_target = [4 604 604 4; 4 4 466 466];
The objective of this homography is to eliminate the perspective of the original image, for which the
four corners of the white pattern are taken as starting points (that is, without taking into account the thin
gray frame that surrounds it), and as points of destination the four corners aligned two to two.
To verify that the homography performs this operation with precision, load the objective image,
'reference_pattern.bmp', and calculate the error between the obtained image and this objective image (use
the function get_error_energy_vmmc for this). The error must be null.
Repeat the above process now, but instead of using the given xy_origin point vector, try manually
selecting the four corners using the get_user_points_vmmc function. Observe, through the error obtained,
the difficulty of selecting these four points, even zooming in the image to be marked.
Include in your report the images, selected points and energies that you consider relevant to
demonstrate that manual selection of the points leads to imprecise estimation of the homography.
1.1.3 Exercise 3: homography that best fits more than four pairs of transformed points
As mentioned in the previous exercise, one way to reduce the error in the calculation of a homography
is to use more than four points to compute it. In order to verify the effectiveness of this technique, we
propose to perform a couple of experiments.
The first is to repeat the last part of the previous exercise but using a greater number of points. To do
this, load the starting image, 'perspective_pattern.bmp', and the target image, 'reference_pattern.bmp'.
Then use the function get_user_points_vmmc to mark the same number of characteristic points in each
image, obtain the homography that converts the starting image into the objective image using the
homography_solve_vmmc function, apply the homography, and obtain the error between the obtained
image and the objective image.
Perform the experiment first with 4 points (it is recommended to use the four corners of a quadrilateral
as wide as possible), then with 8 (adding to the previous 4 those of the next larger quadrilateral), and then
with 12. Try the three experiments with the same level of concentration (no matter what it is) in the
marking of points and write down the resulting error in each case.
Nº points: 4 points 8 points 12 points
Error:
The second consists of replacing the manual marking of homologous points with a technique that
automates this process. To evaluate the operation of this technique, we propose to take as a reference the
result of a manual marking. To obtain this reference result, load the starting image 'hopper.jpg' and the
target image 'hopper_reference.bmp'. Then, use the function get_user_points_vmmc to mark the same
number of characteristic points in each image (the more you mark, the better, as verified in the previous
experiment), get the homography that converts the starting image into the objective image using the
homography_solve_vmmc function, apply it, and obtain and write down the error between the image
obtained and the objective image.
Nº points:
Error:
Then use the homography_auto_vmmc function, which takes two images as a parameter and returns the
homography that best transforms the one into the other. Apply it over the images 'hopper.jpg' (ima1) and
'hopper_reference.bmp' (ima2). Once this homography is obtained, apply it over the starting image and
compute the error between the obtained image and the objective image, noting the error obtained, the
number of points that the function has used to evaluate the homography and the number of iterations that
has performed the search algorithm to obtain the best homography (these two numbers will appear on the
command line). As this process is not deterministic, repeat the experiment several times and write down
your results.
Nº realization 1 2 3 4 5
Error:
Nº used points:
Nº iterations:
Further work: identify the code of the homography_auto_vmmc function, qualitatively describing the
following stages: 1) obtaining characteristic points and correspondences between them and 2) obtaining
the homography that best fits the characteristic points located.
1.2 Homography interpretation
The objective of this section is to understand several of the concepts associated with projective
transformations, particularly those related to the ideal points or points at infinity.
1.2.1 Exercise 4: vanishing points and the line at infinity
This exercise illustrates, from the homography between two images, how the line at infinity defined
by the homography itself coincides with that which can be obtained graphically by identifying the
vanishing points.
First, we propose to obtain the line at infinity graphically. To do this, load and display the image
'hopper_vanisging.bmp'. Now using a drawing application, draw four straight lines extending the sides of
the trapezoid that defines the picture until these lines intersect at their vanishing points. These points,
where parallel lines cut in the projective plane, are points on the infinity line. Join these two points with a
line to get the line at infinity:
Next, we propose to obtain that same line from a homography. Following the process indicated in
previous exercises, obtain the homography that transforms the picture of the image
'hopper_vanisging.bmp' in the image 'hopper_reference.bmp'. To do this, manually mark the four corners
of the box in 'hopper_vanisging.bmp' and use as objective coordinates the four corners of an image of
equal size to the initial one.
Once the homography is obtained, deduce the expression, in homogeneous coordinates, of the line
containing the points of the original image that are transformed into infinite points of the target image.
Then, obtain the equation of this line in affine coordinates and represent it over the original image. Check
that the line obtained matches the line you plotted graphically.
Transfer to the report the result obtained graphically and the one obtained analytically, detailing in this
last case the process of obtaining the equation of the line.
1.3 Generation of panoramic images
The generation of panoramas (or image stitching) is the process by which multiple images, captured
from a fixed pan-tilt camera, are combined to produce a panoramic or high-resolution image. Among the
existing techniques, this section will explore the techniques based on the automatic calculation of
homographies.
1.3.1 Exercise 5: combining two images in a panorama
This technique to generate panoramas is based on calculating homographies between pairs of images
that share areas with the same field of vision and subsequently combining one of these images, let us say
the initial image, with the transformation of the other image via an homography H , according to the
following figure:
For the computation of the homography H we will use the homography_auto_vmmc_2 function,
modified to allow the selection of the amount of points to use in the calculation of the homography
(through a threshold, th) and to visualize matched and unmatched points. For the combination of the
initial image with the transformed image, we provide the stitch_vmmc function.
Next, operate with the image pairs ‘pan_demo_1.png’ - ‘pan_demo_2.png’ and ‘mountain1.jpg’ -
‘mountain2.jpg’ to create panoramas corresponding to both image pairs varying the number of points
selected to obtain the homography. Fill in the following table:
pan_demo_1 - pan_demo_2 mountain1 -mountain2
Nº Points Quality2 Nº Points Quality
th=0.01
th=0.05
th=0.1
th=0.5
th=1
2
To indicate the quality of the generated panorama, consider three subjective categories (low, medium
and high) according to the final result you expect to visualize
1.3.2 Exercise 6: combining several images in a panorama
The process presented in the previous section can be applied to combine more than two images in a
panorama: a first panorama of two images is generated; then, a new panorama is generated combining the
first panorama with a new image, and so iteratively...
Follow this approach to generate the resulting panorama with 3 and 4 images of the type
‘mountainX.jpg’ (the images are arranged taking into account that they share the field of vision). Vary the
number of selected points to achieve the best possible result. Comment if the results obtained correspond
to your expectations and describe what would you do to enhance them.
Further work: Capture a sequence of 5 overlapping images with your mobile, taking special care of
not translating the camera, but just rotating it. Try to capture a far plane and make sure there is
considerable overlap between consecutive images (at least 30%). Obtain the panoramic image so that the
central image is not transformed.
