Connected Vehicle Insights: Trends in Computer Vision

Overview

Drivers must keep their eyes on the road, but can always use some assistance in maintaining their awareness and directing their attention to potential emerging hazards. In the last decade, the auto industry and the auto aftermarket have experimented with devices that provide drivers with a second pair of “electronic eyes,” enabled by simple vision-based data acquisition and processing technology. In 2000, Iteris introduced one of the first commercially available large-scale computer vision applications, lane departure warning, in Mercedes Actros trucks.1 Since then, a number of computer-based vision products have been made available in vehicles and, just recently, in aftermarket automotive devices. By contrast, road operators have for a long time used computer vision to monitor and analyze the performance of their highway networks. Computer vision is the process of using an image sensor to capture images, then using a computer processor to analyze these images to extract information of interest. A simple computer vision system can indicate the physical presence of objects within view by identifying simple visual attributes such as shape, size, or color of an object. More sophisticated computer vision systems may establish not only the presence of an object, but can increasingly identify (or classify) the object based upon the requirements of an application. In intelligent transportation systems (ITS), computer vision technology is broadly applied to either 1) detect objects and events that may represent safety risks to drivers, or 2) detect hindrances to mobility or otherwise improve the efficiency of road networks. Computer vision’s advantages over many other detection sensors or location technologies are generally twofold. First, computer vision systems are relatively inexpensive and can be easily installed on a vehicle or road infrastructure element, and they can detect and identify objects without the need for complementary companion equipment such as transponders. Second, computer vision systems can capture a tremendous wealth of visual information over wide areas, often beyond the longitudinal and peripheral range of other sensors such as active radar. Through continual innovations in computer vision processing algorithms, this wealth of visual data can be exploited to identify more subtle changes and distinctions between objects, enabling a wide array of ever more sophisticated applications. The ability to upgrade computer vision processing algorithms allows image sensors, with some limitations, to be improved or even repurposed to support new applications. In vehicles, this means computer vision-based vehicle safety systems may be updated to detect and identify not just other vehicles, but also road signs and traffic signals. For roadside infrastructure, closed circuit TV cameras at an intersection may be repurposed to support not just signal timing, for example, but also traffic incident detection. Although computer vision can acquire tremendous amounts of visual data, turning it into useful information is in many cases very challenging. Current computer vision technology suffers from the problem of robustness, or the inability to detect objects under a wide variety of operational settings and changing environmental conditions such as illumination and weather. This limitation can be overcome by fusing and integrating computer vision information with data from other external sensors, in a process known as “sensor fusion.”