Addressing Encoding Challenges in Automotive Video with ROHM’s Compact Video Encoders and Decoders

As automotive infotainment and advanced driver assistance systems (ADAS) have become ubiquitous in modern vehicles, the demand for high-quality video processing capabilities has grown significantly. Incorporating advanced video encoding and decoding capabilities poses a challenge for engineers and designers to ensure support for diverse formats and interfaces and reliable operation in automotive environments, meeting size, power, and cost constraints, etc.

This white paper explores the key challenges associated with video encoding and decoding in automotive applications. It examines various implementations, considering factors such as flexibility, performance, power efficiency, and development complexity. It also introduces ROHM’s compact, high-performance video encoders and decoders for addressing the challenges of automotive video processing.

Automotive Video Encoding: Digital Video Formats and Interfaces

There are three of the most widely employed color formats for representing video data in automotive systems:

– Red, Green, Blue (RGB): A color space that represents individual pixels as a combination of red, green, and blue color components.
– YCbCr: Another color space that separates luminance (Y) and chrominance (Cb/Cr) information, enabling efficient compression and transmission of video data.
– BT.656: A digital video interface standard that defines the format for transmitting YCbCr 4:2:2 video data over a serial interface.

YCbCr color subsampling formats like 4:2:2 and 4:2:0 are commonly used in automotive video to reduce bandwidth requirements while maintaining acceptable quality.

Converting between these formats is a common requirement in automotive applications. For example, a camera sensor may output video data in RGB format, which needs to be converted to YCbCr for efficient transmission over a MIPI CSI-2 interface. Similarly, a display module may require video data in RGB, necessitating a conversion from YCbCr back to RGB.

In addition to these formats, automotive video systems must also support interface standards for transmitting video data between components. **Low-Voltage Differential Signaling (LVDS)** and **Camera Serial Interface (MIPI CSI-2)** are common interfaces utilized in automotive applications. LVDS is a high-speed, low-power interface that is typically used to transmit video between a camera and an electronic control unit (ECU). On the other hand, MIPI CSI-2 is a high-bandwidth interface for connecting camera sensors to processors.

Another key consideration in automotive video encoding is choosing between interlaced and progressive scanning methods. In interlaced scanning, each video frame is divided into two fields—one field containing the odd-numbered lines and the other containing even-numbered lines. The fields are then captured and displayed alternately, resulting in a higher frame rate. Progressive scanning, on the other hand, captures and displays each frame in entirety, with all lines being captured and displayed sequentially.

Interlaced scanning has been widely used in standard analog video systems, as it provides a way to reduce bandwidth requirements while maintaining acceptable video quality. However, interlaced video suffers from visual artifacts such as flickering and jagged edges, particularly when displaying fast-moving content or when viewed on large screens. Progressive scanning eliminates these artifacts and provides a higher-quality video experience but requires more bandwidth and processing power.

Making the choice between interlaced and progressive scanning depends on factors such as the available bandwidth, processing power, and the specific requirements of the application. For example, a rear-view camera system may use interlaced scanning to reduce bandwidth requirements and minimize the cost of the camera and associated components. Similarly, an infotainment system with a large, high-resolution display should require progressive scanning to provide the best possible video quality.

To support the use of interlaced video in progressive scanning systems, or vice versa, video encoders and decoders perform progressive to interlaced (P/I) or interlaced to progressive (I/P) conversion. P/I conversion generates interlaced video fields from a progressive video source, typically by discarding alternate lines or by using more advanced techniques such as motion-adaptive deinterlacing. Conversely, I/P conversion generates progressive video frame data from interlaced video fields by combining the fields or using advanced techniques, such as motion-compensated deinterlacing.

Encoding System Architectures

The choice of encoding system architecture is another essential consideration in automotive video applications. Encoding can be performed in a centralized manner, with a single ECU handling video encoding from multiple sources, or in distributed topography, with each video source having its own dedicated encoder. Centralized encoding provides benefits in terms of cost and power efficiency, by sharing encoding hardware, thus reducing the overall number of components in the system. Although it introduces latency and increases the complexity of the system because video data is routed from multiple sources to the central encoder, distributed encoding offers low latency and greater flexibility, with each source having a dedicated encoder optimized for its specific requirements.

Many automotive video systems support a combination of analog and digital video sources, requiring the use of hybrid architectures to accommodate both types of signals. For example, a system may utilize analog cameras for low-cost, legacy applications, while also supporting digital cameras for high-end applications that require better video quality and more advanced features. In such cases, the encoder must handle the necessary conversions and processing to ensure compatibility between the various system components.

Key Requirements and Challenges in Automotive Video Encoding

Automotive video encoding offers a unique set of challenges and requirements that must be carefully considered when designing video processing systems for vehicles. One of the main issues is meeting high video quality and processing performance while maintaining the strict size, power, and cost constraints of automotive applications.

In terms of video quality, modern automobiles require high resolutions, frame rates, and color depths to provide clear, detailed video used in various applications such as backup cameras, surround-view systems, driver monitoring, and more. For example, a backup camera system may require a resolution of at least 720p at 30 frames per second (fps), while most high-end surround-view systems require 1080p resolution at 60 fps. Meeting these requirements can be challenging, especially in systems that require encoding/decoding of multiple streams.

Many systems are also starting to require support for high dynamic range (HDR) to improve low-light performance and overall image quality. HDR allows the camera to capture a wider range of brightness levels, from deep shadows to bright highlights, in a single frame. This is important in situations where the camera needs to deal with challenging lighting conditions, such as bright sunlight, dark tunnels, or rapidly changing shadows. To support HDR, video encoders must be able to process and compress the increased amount of data captured by the HDR sensor, which can be up to 20 bits per pixel compared to 8-10 bits for standard dynamic range video. Moreover, the video decoder must be able to accurately reproduce the content on a display, while supporting HDR-specific metadata and tone mapping techniques.

Latency is another consideration for applications that require real-time video processing and display. For example, a backup camera must provide low-latency video for the safe operation of the vehicle. Similarly, a driver monitoring system must be able to quickly detect and respond to changes in a driver’s behavior, such as drowsiness or distraction. To minimize the latency, video encoding systems must be designed with careful consideration of the entire processing pipeline, from capture to display. This involves the use of specialized low-latency interfaces, such as MIPI CSI-2, as well as optimized pipelines that minimize the number of buffer stages and processing steps.

Automotive components must be designed to withstand the harsh environmental conditions typical under the hood of modern vehicles. This includes exposure to extreme temperatures, humidity, vibration, and EMI. For reliable operation under these conditions, automotive-grade components and design practices must be selected. Automotive-grade video encoders and decoders are qualified to meet the standards of the AEC-Q100, which specifies requirements for ICs in automotive systems, such as temperature range, humidity, and other stresses.

Automotive video processing components must also meet safety and reliability requirements. In particular, for systems used in safety-critical applications, such as backup cameras, driver monitoring systems, and autonomous driving, components may need to be designed to meet functional safety standards such as ISO 26262. The specific requirements will depend on the Automotive Safety Integrity Level (ASIL). For example, a backup camera might be assigned the ASIL-B designation, which requires a higher level of fault tolerance and redundancy than a non-safety-critical system.

Making Trade-offs: Size, Integration, and Power Constraints

Modern automobiles have limited space for electronic components and must integrate video encoders with components like displays, cameras, ECUs, etc. To meet space constraints and achieve easier integration, automotive video processing systems must utilize compact, highly integrated components that can be easily mounted on a PCB or integrated seamlessly into a larger module. For example, System-on-Chip (SoC) devices that integrate multiple functions, such as video encoding, image processing, and interface support, can minimize the size and complexity of the encoder. Similarly, compact connector and mounting solutions can simplify integration and reduce the overall size of the system. Flexibility and scalability are also key in automotive video encoding, as vehicles need support for multiple video sources and formats. To offer this flexibility, encoding systems can use programmable or configurable components that can be easily adapted to different video formats and resolutions.

Power usage is another key aspect of automotive video applications since automakers have limited power budgets and need to prioritize power for essential systems such as propulsion and safety. To minimize power consumption, automotive encoder/decoder systems must be designed using power-efficient components and techniques, such as low-power processing cores, clock gating, power islands, etc., to lower the power usage when not in use. Hardware accelerators and optimized software algorithms can also lower processing requirements, and subsequently, power consumption.

Evaluating Automotive Video Encoder Implementations

When designing automotive video encoder systems, engineers have several implementation options to choose from, each with its advantages and limitations. One approach is to perform the encoding in software running on a general-purpose application processor. This approach affords maximum flexibility,