1.
Abstract
Like Intelligent IPCameras, Autonomous driving cars rely on built-in intelligence using video analytics. Unlike in IPCameras, a bad
decision from video analytics software in an automobiles could cause loss of life. Hence cars must overcome accuracy limitations to
improve detection accuracy for safety and avoid false alarms to maintain the driver experience. With the recent success in research
around several depth perception technologies, and their cost effective availability, surveillance cameras are starting to adopt these
highly accurate technologies. The paper sheds light into some of these technologies and how they improve overall analytics accuracy.
Problems with Monocular Vision
Binocular vision helps humans perceive depth. Still, humans are prone to illusions. Most security cameras are worse. They are
monocular and cannot sense depth, so security cameras cannot easily distinguish between a person and his/her shadow or distinguish
between two people standing behind each other (problem of occlusion). Monocular vision imposes stringent limits on the accuracy of
any video analytics.
The “turning lady illusion” is a good example to illustrate this problem. The mid-section of the lady in the video has no associated depth
information. In observing the illusion, humans subconsciously apply various availability and representative heuristics to make an
assumption that the mid-section of the lady is either part of the raised leg or part of the leg on the floor. Viewers who pursue a decision
path that associates the mid-section to the lady’s raised leg concludes that the lady is rotating clockwise while the rest would judge that
the lady is rotating counterclockwise.
Automotive Vision Technologies Improve Accuracy of Intelligent IPCameras
Jacob Jose,Product Manager, Embedded Processors, Texas Instruments
Depth sensing solutions for IPCameras
Processor solutions for intelligent IPCameras that use depth sensing need to incorporate power efficient signal processing as well as
peripheral interfaces that connect to various sensing devices. The processors need an optimal signal processing capability that
balances cost, processing power and heat dissipation. Integrated processor solutions hence need a combination of ARM and DSP
processors, vector engines and dedicated ASICs.
Texas Instruments DaVinci DM8127 integrates an inbuilt DSP engine and a vision co-processor for depth sensing and fusion, in
addition to an ARM processor and several ASIC engine blocks. DSPs are ideal for raw vision processing and also comes with the
programming flexibility that is usually associated with a general purpose processor. Compared to an ARM processor, a DSP consumes
much less power and dissipate less heat for the same MIPS consumed.
Structured Lighting using Digital Light Processing (DLP) technology
DLP technology relies on structured lighting to illuminate a grid on the scene and observe the distortions in the grid to perceive depth.
DLP offers significantly higher resolution than TOF technology, but has limited range.
Effective use of temporal information could increase the accuracy of
analytics in certain scenarios through background modelling. Video
Analytics algorithms offered with Texas Instruments DaVinci DMVA and
DM81xx processors offer accuracies that approach these theoretical
limits.
Use of Depth in Intelligent IPCameras
Use of depth perception to augment video analytics
algorithms will reduce false detections in Intelligent
IPCameras by 5-10x. Figure 2 below shows an example
where stereo sensors are used for depth perception. The
hand of the subject is clearly marked out by the analytics
engine despite the hand being part of the bigger subject
and within a crowd of details. It may also be noted that
fingers are further marked with a different color that
represents a higher proximity, augmenting the accuracy of
information.
Techniques like auto-calibration, noise cleanup and fusion
with redundant depth information from alternate sensors
are now highly refined in automotive applications.
Increased application of these technologies in video
surveillance will usher in a new generation of highly
intelligent IPCameras.
Depth sensing by self­driving cars
Smart and autonomous driving cars often have more than 10 cameras in a single automobile. However, as shown in the Table 1 below,
a combination of various non-visual sensors are often employed perceive depth accurately in different environments
For example, based on Table 1, a combination of stereo vision systems and a redundant mid-range radar can effectively detect
pedestrians, while long range radar is required for adaptive cruise control. Different technologies vary in accuracy, range and cost.
Hence a combination of diverse technologies are needed for different applications.
Figure 4 – Designing an optimal vision processing pipeline
Typical radar systems use either a CSI-2 or
JESD interface to transmit depth data to host
processor. The DM8127 processor also
integrates a MIPI interface. Some of the other
DSP processors from TI for the market
integrate a JESD interface, enabling
seamless connection to radar devices without
using additional glue logic.
Various depth sensing solutions, especially those that are commonly used in the automotive market, offer unique advantages to
different IPCamera applications. As with smart cars, a combination of technologies are required to solve the diverse product needs in
the intelligent camera market. For instance, Structured Lighting or Time of Flight sensors offer good accuracy in dimly lit indoor
environments, but may not be effective outdoors where solar radiation is abundant. On the other hand, stereo sensors that may
struggle indoors are highly effective outdoors due to the more beneficial lighting conditions.
Among the prominent depth sensing technologies, two technologies are explored below:
Time of Flight (TOF) Technology
Time of flight technology relies on flooding a scene with modulated pulses, often infrared or laser. The reflections of each of the pulses
are then timed to accurately calculate the distance from the object that the pulses have been reflected from. TOF technology can be
employed very cost effectively for near-field perception. Using laser or higher levels of illumination (more LEDs) will increase the depth
of field or resolution of the sensor, at a cost of increased power consumption.
Originally used for gesture recognition and accuracy assistance in automotive surround view systems, TOF technology is very effective
to perceive depth in dark environments, where stereo can be ineffective due to lack of illumination. TOF resolution is inversely
proportional to the distance from the sensor, making it an ideal technology for use in small rooms. On the other hand, TOF may not be
appropriate for outdoor cameras since solar radiation will negatively impact accuracy. Large warehouse rooms or parking lots that
require a large depth of field are also likely to need one of the alternate technologies.
Figure 5 – TOF Principle of Operation Figure 6 – Depth Resolution vs Distance
Texas Instruments offers DS325, a depth sense tool kit with an LED driver, an analog front end, a Time of Flight controller, sensor and
associated power components. A typical block diagram is in figure 7. More information can be found here:
http://www.ti.com/ww/en/analog/3dtof/index.shtml
Figure 8 – Principle of operation of DLP
structured lighting and application results
Texas Instruments offers DLP4500NIR Digital Micro-Mirror
Device (DMD) and controllers, DLPC200 and DLPC300 for this
market. An added advantage with DLP technology is that
investment in DLP technology can often be leveraged in
security, retail and machine vision markets, benefiting firms
with interests in more than one of these markets. More
information on DLP technology can be found here:
http://www.ti.com/solution/3d_machine_vision
A table comparing four of the depth sensing technologies is
outlined below. Common applications other than Intelligent
Cameras are also listed:
Intelligent IPCamera Reference Designs
Texas Instruments, in association with its partners, offers a
combination of reference design and turnkey solutions to help
TI’s Intelligent Camera customers get to market quickly using
some of the latest available technology.
Table 2 – Comparison of various depth sensing technologies
TI’s Intelligent IPCamera reference designs bundle 5 commonly used video analytics algorithms
within in a scalable software framework. These 5 algorithms are People counting, Trip zone,
Streaming metadata, Tamper detection and Motion detection. A set of performance optimized vision
API libraries - VLIB for DSPs and VCOPLIB for vision co-processors - offers a strong foundation for
vision software. The 5 bundled algorithms take advantage of the optimized libraries. TI's customers
further differentiate through designing their own analytics functions using APIs like discrepancy
mapping (for depth sensing) included in these libraries. More information on these reference
designs are available at www.ti.com/ipcamera.
Texas Instruments (TI) is a leader in embedded processor and analog technology for autonomous
driving cars. As the automotive technologies made available by TI become mainstream in the
differentiated intelligent camera market, early adopters stand to gain significantly.
Figure 9 ­
Figure 10 ­ Five high level video analytics algorithms bundled with TI DMVA processors Figure 11 ­ DMVA Analytics Software Framework