I will begin with a general overview of the two types of systems and compare them at a general le... more I will begin with a general overview of the two types of systems and compare them at a general level. The second section will describe the eye and camera as the input devices. Later sections will discuss segmentation of the scene, visual imagery and encoding of the scene, perception, depth and space perception, and perception of movement.
Indoor localization using fine time measurement (FTM) round-trip time (RTT) with respect to coope... more Indoor localization using fine time measurement (FTM) round-trip time (RTT) with respect to cooperating Wi-Fi access points (APs) has been shown to work well and provide 1–2 m accuracy in both 2D and 3D applications. This approach depends on APs implementing the IEEE 802.11-2016 (also known as IEEE 802.11mc) Wi-Fi standard (“two-sided” RTT). Unfortunately, the penetration of this Wi-Fi protocol has been slower than anticipated, perhaps because APs tend not to be upgraded as often as other kinds of electronics, in particular in large institutions—where they would be most useful. Recently, Google released Android 12, which also supports an alternative “one-sided” RTT method that will work with legacy APs as well. This method cannot subtract out the “turn-around” time of the signal, and so, produces distance estimates that have much larger offsets than those seen with two-sided RTT—and the results are somewhat less accurate. At the same time, this method makes possible distance measure...
This report describes research done at the Artificial Intelligence Laboratory of the Massachusett... more This report describes research done at the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Department of Defense under Office of Naval Research contract N00014-75-C-0643.MIT Artificial Intelligence Laboratory Department of Defense Advanced Research Projects Agenc
Convolutions have long been regarded as fundamental to applied mathematics, physics and engineeri... more Convolutions have long been regarded as fundamental to applied mathematics, physics and engineering. Their mathematical elegance allows for common tasks such as numerical differentiation to be computed efficiently on large data sets. Efficient computation of convolutions is critical to artificial intelligence in real-time applications, like machine vision, where convolutions must be continuously and efficiently computed on tens to hundreds of kilobytes per second. In this paper, we explore how convolutions are used in fundamental machine vision applications. We present an accelerated n-dimensional convolution package in the high performance computing language, Julia, and demonstrate its efficacy in solving the time to contact problem for machine vision. Results are measured against synthetically generated videos and quantitatively assessed according to their mean squared error from the ground truth. We achieve over an order of magnitude decrease in compute time and allocated memory ...
This is a primer on extended Caussian images. Extended Gauss/an images are useful for representin... more This is a primer on extended Caussian images. Extended Gauss/an images are useful for representing the shapes of surfaces. They can be computed easily from: 1. needle maps obtained using photometric stereo; or 2. depth maps generated by ranging devices or binocular stereo. Importantly, they can also be determined simply from geometric models of the objects. Extended Caussian images can be of use in at least two of the tasks facing a machine vision system: 1. recognition, and 2. determining the attitude in space of an object. Here, the extended Caussian image is defined and some of its properties discussed. An elaboration for nonconvex objects is presented and several examples are shown.
It appears that the development of machine vision may benefit from a detailed understanding of th... more It appears that the development of machine vision may benefit from a detailed understanding of the imaging process. The reflectance map, showing scene radiance as a function of surface gradient, has proved to be helpful in this endeavor. The reflectance map depends both on the nature of the surface layers of the objects being imaged and the distribution of light sources. Recently, a unified approach to the specification of surface reflectance in terms of both incident and reflected beam geometry has been proposed The reflecting properties of a surface are specified in terms of the bidirectional reflectance-distribution function (BRDF). Here we derive the reflectance map in terms of the BRDF and the distribution of source radiance. A number of special cases of practical importance are developed in detail. The significance of this approach to the understanding of image formation is briefly indicated.
It is well known that a function can be decomposed uniquely into the sum of an odd and an even fu... more It is well known that a function can be decomposed uniquely into the sum of an odd and an even function. This notion can be extended to the unique decomposition into the sum of four functions — two of which are even and two odd. These four functions are eigenvectors of the Fourier Transform with four different eigenvalues. That is, the Fourier transform of each of the four components is simply that component multiplied by the corresponding eigenvalue. Some eigenvectors of the discrete Fourier transform of particular interest find application in coding, communication and imaging. Some of the underlying mathematics goes back to the times of Carl Friedrich Gauss.
Acta Crystallographica Section A Foundations and Advances
How should we represent rotations? Orthonormal (3 Â 3) matrices? Euler angles (pitch, yaw and rol... more How should we represent rotations? Orthonormal (3 Â 3) matrices? Euler angles (pitch, yaw and roll)? Axis and angle? Gibbs vector? Pauli (2 Â 2, complex) spin matrices? Cayley-Klein (complex) parameters? Euler-Rodrigues parameters? Hamilton's quaternions? Why are there so many choices (Goldstein, 1980; Varshalovich et al., 1988; Shuster, 1993)? We don't see this variety when it comes to translation: we just use vectors. The big difference is that translations live in a Euclidean vector space. Rotations do not, and so all the nice linear algebra machinery that we use for manipulating translations seems not to apply. Why does it matter how we represent rotation? Solving problems often depends on finding the 'right' representation. Sometimes a good notation can make it possible to solve a problem in closed form. Without a notion of the 'space of rotations' many problems are hard to formulate and solve. Spoiler alert: unit quaternions provide 'the' way to represent rotations. Why? Unit quaternions allow a clear visualization (see Hanson, 2006) of the space of rotations as the unit sphere S 3 in four dimensions (with antipodal points identified). Unit quaternions make it possible to differentiate an expression 'with respect to rotation'. Unit quaternions make it possible to find extrema of expressions by setting that derivative equal to zero! Unit quaternions make it easy to compose rotations (unlike, e.g., axis-and-angle notation). Unit quaternions do not suffer from singularities (as do, e.g., Euler angles when two axes line up-see gimbal lock). Unit quaternions, while redundant (four parameters for three degrees of freedom), have only one constraint on their components (unlike orthonormal matrices, which have six non-linear constraints on the rows or columns-plus one on the determinant). Unit quaternions make it possible to compute averages, to interpolate, to sample in the space of rotations and to represent densities over orientations. And, as demonstrated in Andrew Hanson's (2020) article in the previous issue of Acta Crystallographica Section A, they also make possible insightful illustrations relating to orientations. Which came first, the vector or the quaternion? Mathematicians extended algebra from real number to complex numbers. We can add, subtract, multiply and divide these. The question that obsessed Hamilton was whether there would be further generalizations to algebras with three components. His big breakthrough came when he realized it was not possible with three components-but it was possible with four (Hamilton, 1844)! That is how the quaternion was born-on 16 October 1843. Hamilton showed that for some applications in physics it was convenient to split the quaternion into a 'scalar' and a 'vector' part. For a few decades, quaternions were used in physics to represent spatial quantities, often with the scalar part set to zero. Maxwell used both notations in some of his work (Maxwell, 1873). Some got tired of using four components to represent positions in space which clearly only need three, and so lopped off the scalar component and switched to using just vectors. Gibbs and Heaviside were the main propagandizers of this approach (Crowe, 1967). The price they paid was that there is no algebra for vectors (e.g., no inverses with respect to multiplication). Some critics called vector analysis a 'hermaphrodite monster' because it 'compounded the notations of Hamilton and Grassmann' (Tait, 1867; Prichard, 1998). After an acrimonious war of words, the proponents of vectors finally won, and the rest is history.
IEEE Transactions on Intelligent Transportation Systems, Oct 1, 2018
In the United States, every year, several people whose job takes them to the sides of roads, are ... more In the United States, every year, several people whose job takes them to the sides of roads, are injured or killed by roadside collisions. This could be avoided if a warning signal could be sent to them. In this paper, we describe a machinevision based alerting system which detects and tracks headlamps of cars in night traffic. The system automatically computes a "normal traffic" region in the image. Unusual trajectories of cars are detected when the images of their headlamps move out of that region. The system promptly sends a warning signal once a risk has been identified. The system runs on the Android smart phones, which are mounted on cars or on roadside fixtures.
In this paper, we study the chain stability of vehicles under bilateral control (BCM), and prove ... more In this paper, we study the chain stability of vehicles under bilateral control (BCM), and prove that vehicles under bilateral control are chain stable, i.e., all input perturbations to the chain decay exponentially (with the length of the chain). Chain stability analysis tells us how vehicles under bilateral control will act in traffic when mixed with cars driven by human drivers. It shows that self-driving cars using bilateral control can reduce traffic flow instabilities in mixed traffic. Indeed, chains of BCM vehicles become perturbation-consuming dampers when inserted in traffic, since they split chains of human-driven vehicles and prevent perturbations from being transmitted from one chain of car-following cars to the next. Thus, today's traffic can be improved greatly by the insertion of BCM vehicles. The simulation results validate the theoretical analysis.
In this paper, we study a mixed traffic flow in which some cars are under car-following control a... more In this paper, we study a mixed traffic flow in which some cars are under car-following control and others are under bilateral control. We also provide the necessary (modular string) stability condition for this type of mixed traffic, which can be viewed as an extension of the condition for pure car-following control based traffic. This necessary stability condition provides some indication of how the introduction of self-driving cars (under bilateral control) will affect today's traffic. Index Terms-bilateral control, car-following model, mixed traffic, stability analysis, adaptive cruise control (ACC).
I will begin with a general overview of the two types of systems and compare them at a general le... more I will begin with a general overview of the two types of systems and compare them at a general level. The second section will describe the eye and camera as the input devices. Later sections will discuss segmentation of the scene, visual imagery and encoding of the scene, perception, depth and space perception, and perception of movement.
Indoor localization using fine time measurement (FTM) round-trip time (RTT) with respect to coope... more Indoor localization using fine time measurement (FTM) round-trip time (RTT) with respect to cooperating Wi-Fi access points (APs) has been shown to work well and provide 1–2 m accuracy in both 2D and 3D applications. This approach depends on APs implementing the IEEE 802.11-2016 (also known as IEEE 802.11mc) Wi-Fi standard (“two-sided” RTT). Unfortunately, the penetration of this Wi-Fi protocol has been slower than anticipated, perhaps because APs tend not to be upgraded as often as other kinds of electronics, in particular in large institutions—where they would be most useful. Recently, Google released Android 12, which also supports an alternative “one-sided” RTT method that will work with legacy APs as well. This method cannot subtract out the “turn-around” time of the signal, and so, produces distance estimates that have much larger offsets than those seen with two-sided RTT—and the results are somewhat less accurate. At the same time, this method makes possible distance measure...
This report describes research done at the Artificial Intelligence Laboratory of the Massachusett... more This report describes research done at the Artificial Intelligence Laboratory of the Massachusetts Institute of Technology. Support for the laboratory's artificial intelligence research is provided in part by the Advanced Research Projects Agency of the Department of Defense under Office of Naval Research contract N00014-75-C-0643.MIT Artificial Intelligence Laboratory Department of Defense Advanced Research Projects Agenc
Convolutions have long been regarded as fundamental to applied mathematics, physics and engineeri... more Convolutions have long been regarded as fundamental to applied mathematics, physics and engineering. Their mathematical elegance allows for common tasks such as numerical differentiation to be computed efficiently on large data sets. Efficient computation of convolutions is critical to artificial intelligence in real-time applications, like machine vision, where convolutions must be continuously and efficiently computed on tens to hundreds of kilobytes per second. In this paper, we explore how convolutions are used in fundamental machine vision applications. We present an accelerated n-dimensional convolution package in the high performance computing language, Julia, and demonstrate its efficacy in solving the time to contact problem for machine vision. Results are measured against synthetically generated videos and quantitatively assessed according to their mean squared error from the ground truth. We achieve over an order of magnitude decrease in compute time and allocated memory ...
This is a primer on extended Caussian images. Extended Gauss/an images are useful for representin... more This is a primer on extended Caussian images. Extended Gauss/an images are useful for representing the shapes of surfaces. They can be computed easily from: 1. needle maps obtained using photometric stereo; or 2. depth maps generated by ranging devices or binocular stereo. Importantly, they can also be determined simply from geometric models of the objects. Extended Caussian images can be of use in at least two of the tasks facing a machine vision system: 1. recognition, and 2. determining the attitude in space of an object. Here, the extended Caussian image is defined and some of its properties discussed. An elaboration for nonconvex objects is presented and several examples are shown.
It appears that the development of machine vision may benefit from a detailed understanding of th... more It appears that the development of machine vision may benefit from a detailed understanding of the imaging process. The reflectance map, showing scene radiance as a function of surface gradient, has proved to be helpful in this endeavor. The reflectance map depends both on the nature of the surface layers of the objects being imaged and the distribution of light sources. Recently, a unified approach to the specification of surface reflectance in terms of both incident and reflected beam geometry has been proposed The reflecting properties of a surface are specified in terms of the bidirectional reflectance-distribution function (BRDF). Here we derive the reflectance map in terms of the BRDF and the distribution of source radiance. A number of special cases of practical importance are developed in detail. The significance of this approach to the understanding of image formation is briefly indicated.
It is well known that a function can be decomposed uniquely into the sum of an odd and an even fu... more It is well known that a function can be decomposed uniquely into the sum of an odd and an even function. This notion can be extended to the unique decomposition into the sum of four functions — two of which are even and two odd. These four functions are eigenvectors of the Fourier Transform with four different eigenvalues. That is, the Fourier transform of each of the four components is simply that component multiplied by the corresponding eigenvalue. Some eigenvectors of the discrete Fourier transform of particular interest find application in coding, communication and imaging. Some of the underlying mathematics goes back to the times of Carl Friedrich Gauss.
Acta Crystallographica Section A Foundations and Advances
How should we represent rotations? Orthonormal (3 Â 3) matrices? Euler angles (pitch, yaw and rol... more How should we represent rotations? Orthonormal (3 Â 3) matrices? Euler angles (pitch, yaw and roll)? Axis and angle? Gibbs vector? Pauli (2 Â 2, complex) spin matrices? Cayley-Klein (complex) parameters? Euler-Rodrigues parameters? Hamilton's quaternions? Why are there so many choices (Goldstein, 1980; Varshalovich et al., 1988; Shuster, 1993)? We don't see this variety when it comes to translation: we just use vectors. The big difference is that translations live in a Euclidean vector space. Rotations do not, and so all the nice linear algebra machinery that we use for manipulating translations seems not to apply. Why does it matter how we represent rotation? Solving problems often depends on finding the 'right' representation. Sometimes a good notation can make it possible to solve a problem in closed form. Without a notion of the 'space of rotations' many problems are hard to formulate and solve. Spoiler alert: unit quaternions provide 'the' way to represent rotations. Why? Unit quaternions allow a clear visualization (see Hanson, 2006) of the space of rotations as the unit sphere S 3 in four dimensions (with antipodal points identified). Unit quaternions make it possible to differentiate an expression 'with respect to rotation'. Unit quaternions make it possible to find extrema of expressions by setting that derivative equal to zero! Unit quaternions make it easy to compose rotations (unlike, e.g., axis-and-angle notation). Unit quaternions do not suffer from singularities (as do, e.g., Euler angles when two axes line up-see gimbal lock). Unit quaternions, while redundant (four parameters for three degrees of freedom), have only one constraint on their components (unlike orthonormal matrices, which have six non-linear constraints on the rows or columns-plus one on the determinant). Unit quaternions make it possible to compute averages, to interpolate, to sample in the space of rotations and to represent densities over orientations. And, as demonstrated in Andrew Hanson's (2020) article in the previous issue of Acta Crystallographica Section A, they also make possible insightful illustrations relating to orientations. Which came first, the vector or the quaternion? Mathematicians extended algebra from real number to complex numbers. We can add, subtract, multiply and divide these. The question that obsessed Hamilton was whether there would be further generalizations to algebras with three components. His big breakthrough came when he realized it was not possible with three components-but it was possible with four (Hamilton, 1844)! That is how the quaternion was born-on 16 October 1843. Hamilton showed that for some applications in physics it was convenient to split the quaternion into a 'scalar' and a 'vector' part. For a few decades, quaternions were used in physics to represent spatial quantities, often with the scalar part set to zero. Maxwell used both notations in some of his work (Maxwell, 1873). Some got tired of using four components to represent positions in space which clearly only need three, and so lopped off the scalar component and switched to using just vectors. Gibbs and Heaviside were the main propagandizers of this approach (Crowe, 1967). The price they paid was that there is no algebra for vectors (e.g., no inverses with respect to multiplication). Some critics called vector analysis a 'hermaphrodite monster' because it 'compounded the notations of Hamilton and Grassmann' (Tait, 1867; Prichard, 1998). After an acrimonious war of words, the proponents of vectors finally won, and the rest is history.
IEEE Transactions on Intelligent Transportation Systems, Oct 1, 2018
In the United States, every year, several people whose job takes them to the sides of roads, are ... more In the United States, every year, several people whose job takes them to the sides of roads, are injured or killed by roadside collisions. This could be avoided if a warning signal could be sent to them. In this paper, we describe a machinevision based alerting system which detects and tracks headlamps of cars in night traffic. The system automatically computes a "normal traffic" region in the image. Unusual trajectories of cars are detected when the images of their headlamps move out of that region. The system promptly sends a warning signal once a risk has been identified. The system runs on the Android smart phones, which are mounted on cars or on roadside fixtures.
In this paper, we study the chain stability of vehicles under bilateral control (BCM), and prove ... more In this paper, we study the chain stability of vehicles under bilateral control (BCM), and prove that vehicles under bilateral control are chain stable, i.e., all input perturbations to the chain decay exponentially (with the length of the chain). Chain stability analysis tells us how vehicles under bilateral control will act in traffic when mixed with cars driven by human drivers. It shows that self-driving cars using bilateral control can reduce traffic flow instabilities in mixed traffic. Indeed, chains of BCM vehicles become perturbation-consuming dampers when inserted in traffic, since they split chains of human-driven vehicles and prevent perturbations from being transmitted from one chain of car-following cars to the next. Thus, today's traffic can be improved greatly by the insertion of BCM vehicles. The simulation results validate the theoretical analysis.
In this paper, we study a mixed traffic flow in which some cars are under car-following control a... more In this paper, we study a mixed traffic flow in which some cars are under car-following control and others are under bilateral control. We also provide the necessary (modular string) stability condition for this type of mixed traffic, which can be viewed as an extension of the condition for pure car-following control based traffic. This necessary stability condition provides some indication of how the introduction of self-driving cars (under bilateral control) will affect today's traffic. Index Terms-bilateral control, car-following model, mixed traffic, stability analysis, adaptive cruise control (ACC).
Uploads
Papers by Bkp Horn