Addressing Encoding Challenges in Automotive Video with ROHM’s Compact Video Encoders and Decoders

As automotive infotainment and advanced driver assistance systems (ADAS) have become ubiquitous in modern vehicles, the demand for high-quality video processing capabilities has grown significantly. Incorporating advanced video encoding and decoding capabilities poses a challenge for engineers and designers to ensure support for diverse formats and interface and reliable operation in automotive environments, meeting size, power, and cost constraints, etc.

This white paper explores the key challenges associated with video encoding and decoding in automotive applications. It examines various implementations, considering factors such as flexibility, performance, power efficiency, and development complexity. It also introduces ROHM’s compact, high-performance video encoders and decoders for addressing the challenges of automotive video processing. 

Automotive Video Encoding: Digital Video Formats and Interfaces 

There are three of the most widely employed color formats for representing video data in automotive systems: Red, Green, Blue, RGB, is a color space that represents individual pixels as a combination of red, green, and blue color components; YCbCr is another color space that separates luminance (Y) and chrominance (Cb/Cr) information, enabling efficient compression and transmission of video data; BT.656 is a digital video interface standard that defines the format for transmitting YCbCr 4:2:2 video data over a serial interface. YCbCr color subsampling formats like 4:2:2 and 4:2:0 are commonly used in automotive video to reduce bandwidth requirements while maintaining acceptable quality. 

Converting between these formats is a common requirement in automotive applications. For example, a camera sensor may output video data in RGB format, which needs to be converted to YCbCr for efficient transmission over a MIPI CSI-2 interface. Similarly, a display module may require video data in RGB, necessitating a conversion from YCbCr back to RGB.

In addition to these formats, automotive video systems must also support interface standards for transmitting video data between components. Low-Voltage Differential Signaling (LVDS), Camera Serial Interface (MIPI CSI-2), and Flat Panel Display Link (FPD-Link) are common interfaces utilized in automotive applications. LVDS is a high-speed, low-power interface that is typically used to transmit video between a camera and an electronic control unit (ECU). On the other hand, MIPI CSI-2 is a high-bandwidth interface for connecting camera sensors to processors; FPD-Link is typically used for transmitting video data between an ECU and a display module.

Another key consideration in automotive video encoding is choosing between interlaced and progressive scanning methods. In interlaced scanning, each video frame is divided into two fields—one field containing the odd-numbered lines and the other containing even-numbered lines. The fields are then captured and displayed alternately, resulting in a higher frame rate. Progressive scanning, on the other hand, captures and displays each frame in entirety, with all lines being captured and displayed sequentially.

Interlaced scanning has been widely used in standard analog video systems, as it provides a way to reduce bandwidth requirements while maintaining acceptable video quality. However, interlaced video suffers from visual artifacts such as flickering and jagged edges, particularly when displaying fast-moving content or when viewed on large screens. Progressive scanning eliminates these artifacts and provides a higher-quality video experience, but requires more bandwidth and processing power.

Making the choice between interlaced and progressive scanning depends on factors such as the available bandwidth, processing power, and the specific requirements of the application. For example, a rear-view camera system may use interlaced scanning to reduce bandwidth requirements and minimize the cost of the camera and associated components. Similarly, an infotainment system with a large, high-resolution display should require progressive scanning to provide the best possible video quality.

To support the use of interlaced video in progressive scanning systems, or vice versa, video encoders and decoders perform progressive to interlaced (P/I) or interlaced to progressive (I/P) conversion. P/I conversion generates interlaced video fields from a progressive video source, typically by discarding alternate lines or by using more advanced techniques such as motion-adaptive deinterlacing. Conversely, I/P conversion generates progressive video frame data from interlaced video fields by combining the fields or using advanced techniques, such as motion-compensated deinterlacing.

Encoding System Architectures

The choice of encoding system architecture is another essential consideration in automotive video applications. Encoding can be performed in a centralized manner, with a single ECU handling video encoding from multiple sources, or in distributed topography, with each video source having its own dedicated encoder. Centralized encoding provides benefits in terms of cost and power efficiency, by sharing encoding hardware, thus reducing the overall number of components in the system. Although it introduces latency and increases the complexity of the system because video data is routed from multiple sources to the central encoder, distributed encoding offers low latency and greater flexibility, with each source having a dedicated encoder optimized for its specific requirements. 

Many automotive video systems support a combination of analog and digital video sources, requiring the use of hybrid architectures to accommodate both types of signals. For example, a system may utilize analog cameras for low-cost, legacy applications, while also supporting digital cameras for high-end applications that require better video quality and more advanced features. In such cases, the encoder must handle the necessary conversions and processing to ensure compatibility between the various system components. 

Key Requirements and Challenges in Automotive Video Encoding 

Automotive video encoding offers a unique set of challenges and requirements that must be carefully considered when designing video processing systems for vehicles. One of the main issues is meeting high video quality and processing performance, while maintaining the strict size, power, and cost constraints of automotive applications. 

In terms of video quality, modern automobiles require high resolutions, frame rates, and color depths to provide clear, detailed video used in various applications such as backup cameras, surround-view systems, driver monitoring, and more. For example, a backup camera system may require a resolution of at least 720p at 30 frames per second (fps), while most high-end surround-view systems require 1080p resolution at 60 fps. Meeting these requirements can be challenging, especially in systems that require encoding/decoding of multiple streams. 

Many systems are also starting to require support for high dynamic range (HDR) to improve low-light performance and overall image quality. HDR allows the camera to capture a wider range of brightness levels, from deep shadows to bright highlights, in a single frame. This is important in situations where the camera needs to deal with challenging lighting conditions, such as bright sunlight, dark tunnels, or rapidly changing shadows. To support HDR, video encoders must be able to process and compress the increased amount of data captured by the HDR sensor, which can be up to 20 bits per pixel compared to 8-10 bits for standard dynamic range video. Moreover, the video decoder must be able to accurately reproduce the content on a display, while supporting HDR-specific metadata and tone mapping techniques. 

Latency is another consideration for applications that require real-time video processing and display. For example, a backup camera must provide low-latency video for safe operation of the vehicle. Similarly, a driver monitoring system must be able to quickly detect and respond to changes in a driver’s behavior, such as drowsiness or distraction. To minimize the latency, video encoding systems must be designed with careful consideration of the entire processing pipeline, from capture to display. This involves the use of specialized low-latency interfaces, such as MIPI CSI-2, as well as optimized pipelines that minimize the number of buffer stages and processing steps.

Automotive components must be designed to withstand the harsh environmental conditions typical under the hood of modern vehicles. This includes exposure to extreme temperatures, humidity, vibration, and EMI. For reliable operation under these conditions, automotive-grade components and design practices must be selected. Automotive-grade video encoders and decoders are qualified to meet the standards of the AEC-Q100, which specifies requirements for ICs in automotive systems, such as temperature range, humidity, and other stresses. 

Automotive video processing components must also meet safety and reliability requirements. In particular, for systems used in safety-critical applications, such as backup cameras, driver monitoring systems, and autonomous driving, components may need to be designed to meet functional safety standards such as ISO 26262. The specific requirements will depend on the Automotive Safety Integrity Level (ASIL). For example, a backup camera might be assigned the ASIL-B designation, which requires a higher level of fault tolerance and redundancy than a non-safety-critical system. 

Making Trade-offs: Size, Integration, and Power Constraints

Modern automobiles have limited space for electronic components and must integrate video encoders with components like displays, cameras, ECUs, etc. To meet space constraints and achieve easier integration, automotive video processing systems must utilize compact, highly integrated components that can be easily mounted on a PCB or integrated seamlessly into a larger module. For example, System-on-Chip (SoC) devices that integrate multiple functions, such as video encoding, image processing, and interface support, can minimize the size and complexity of the encoder. Similarly, compact connector and mounting solutions can simplify integration and reduce the overall size of the system. Flexibility and scalability are also key in automotive video encoding, as vehicles need support for multiple video sources and formats. To offer this flexibility, encoding systems can use programmable or configurable components that can be easily adapted to different video formats and resolutions.

Power usage is another key aspect of automotive video applications, since automakers have limited power budgets and need to prioritize power for essential systems such as propulsion and safety. To minimize power consumption, automotive encoder/decoder systems must be designed using power-efficient components and techniques, such as low-power processing cores, clock gating, power islands, etc., to lower the power usage when not in use. Hardware accelerators and optimized software algorithms can also lower processing requirements, and subsequently, power consumption. 

Evaluating Automotive Video Encoder Implementations 

When designing automotive video encoder systems, engineers have several implementation options to choose from, each with its advantages and limitations. One approach is to perform the encoding in software running on a general-purpose application processor. This approach affords maximum flexibility, as the encoding algorithms and parameters can be modified with software to support various video formats, resolutions, and quality settings. 

However, encoding on general-purpose processors affects performance and power efficiency as it is a computationally-intensive process that requires high processing power and memory bandwidth. Running encoding software on such processors consumes a large portion of its resources, leaving limited headroom for other tasks. Moreover, general-purpose processors are not designed for video encoding workloads, which can lead to inefficient use of resources and higher power consumption compared to using dedicated encoding hardware. 

To overcome the inefficiencies of software-based encoding on general-purpose processors, automotive video systems can utilize dedicated video encoding ASICs—fixed-function devices optimized for video encoding tasks. These ASICs provide high performance and efficiency in a compact, power-efficient package and incorporate specialized hardware accelerators and optimized memory architectures. The accelerators within these ASICs perform tasks such as motion estimation, transform and entropy coding much more efficiently than general-purpose processors or FPGAs. In addition, video encoding ASICs are designed with the requirements of automotive systems in mind, including support for automotive-grade temperature ranges, reliability, and EMI immunity. This simplifies the design and validation of video systems, since the ASIC can be integrated into the system without requiring major modifications. 

Compact Automotive Video Encoders and Decoders from ROHM 

ROHM Semiconductor has developed a family of compact, high-performance video encoders and decoders that are specifically optimized for automotive applications. A key product in this family is the ML86640TB, a highly integrated NTSC/PAL video encoder offering a single-chip solution for converting digital video signals to analog composite video output. 

Image credit: ROHM Semiconductor

The ML86640TB supports multiple digital video input formats, including 24-bit RGB/YCbCr, 16/8-bit YCbCr, and BT.656, providing flexibility and compatibility with a variety of sources. It also offers advanced video processing capabilities, including built-in color space conversion, P/I conversion, and video parameter adjustment. This ability allows the encoder to optimize quality and performance for applications, without requiring external components. 

Another product in ROHM’s video encoder/decoder family is the ML86112, a compact CVBS to MIPI CSI-2/LVTTL video decoder that is suitable for automotive camera applications. The ML86112 supports up to four composite video inputs, allowing it to interface with multiple cameras and convert their output to a digital MIPI CSI-2 or LVTTL interface for processing by an automotive SoC or image processor. It also features a high-quality 10-bit analog-to-digital converter (ADC) with multi-standard support for NTSC and PAL video formats, and advanced image enhancement features, including noise reduction, edge enhancement, and automatic white balance. This allows the product to deliver high-quality digital video output from analog camera sources, while providing flexibility and compatibility with a wide range of automotive SoCs and image processors. The ML86112 also uses a wettable flank package, allowing the solderability of the electrodes to be visually confirmed from the side of the package.

Image credit: ROHM Semiconductor

Both the ML86640TB and ML86112 are fully qualified to meet the rigorous standards of the automotive industry. This includes an AEC-Q100 Grade 2 qualification for operation in harsh automotive environments, as well as support for functional safety requirements up to ASIL-B. Both devices are available in compact, automotive-grade packages optimized for size and power efficiency–the ML86640TB is available in a 48-pin TQFP package, while the ML86112 comes in a 32-pin WQFN package. 

ROHM video encoders/decoders are also designed for compatibility with leading automotive SoCs and image processors, including those from leading vendors, including Renesas, NXP, and TI. This ensures interoperability across the automotive ecosystem, reducing integration and validation costs for automotive customers. 

Key Applications

ROHM’s video encoders and decoders are suitable for a range of automotive video systems, from infotainment to surround view and backup camera systems for ADAS.

High-end Infotainment Systems with Analog Video Interfaces

High-end automotive infotainment systems require the ability to interface with legacy sources of video like DVD players or gaming consoles. In these applications, the ML86640TB video encoder can be used to convert analog video output from these sources into a digital format to be processed and displayed by the infotainment system’s digital display and processing hardware. 

The compact size and low power consumption of the ML86640TB make it suitable for integration into space-constrained head units. With support for multiple video input formats and resolutions, the ML86640TB also interfaces with a range of media sources, from smartphones to gaming consoles. This flexibility enables automakers to offer advanced infotainment and connectivity without compromising on system design or power efficiency. 

Multi-camera Surround View Systems

Surround view camera systems in high-end vehicles afford drivers a 360-degree view of their surroundings for improved situational awareness and safety. These systems utilize cameras located around the vehicle that capture feeds that must be processed and “stitched” together to create a surround view image. The ML86112’s support for up to four analog camera inputs allows it to efficiently process multiple video streams for a smooth surround view experience. Its ability to decode high-resolution video at high frame rates ensures that the surround view image is smooth, detailed, and free of latency. The ML86112’s HDR processing capabilities also help to maintain visibility and clarity in low-light conditions, providing a clear view of the vehicle’s surroundings to enhance safety and convenience. 

Analog Backup Camera Systems with Dynamic Overlays

Backup camera systems, now required by law in many countries, have become ubiquitous, even in entry-level vehicles. While new models use digital cameras and interfaces, there are still many applications where an analog backup camera is preferred for cost or compatibility. In these applications, the ML86112 decoder can be used to convert analog video output from the backup camera to a digital format to be processed by an ADAS ECU. The ECU can then add dynamic overlays to the video feed, such as parking guidelines or object detection alerts, before sending the video to the vehicle’s infotainment display.

Conclusion

From backup camera systems to ADAS and autonomous driving, video information is playing an increasingly critical role in enabling features and capabilities that improve vehicle safety, convenience, and entertainment. However, implementing these advanced video features in automotive applications presents a number of challenges, from the need to support a broad range of video formats and interfaces to the harsh environmental and reliability requirements of the automotive industry. Automotive designers must also balance the need for high video quality and processing performance with the size, power, and cost constraints of automotive ECUs and other components. 

ROHM’s video encoder/decoders provide a robust solution for automotive video processing that is optimized for performance, power efficiency, and ease of integration. With products like the ML86640TB and ML86112, ROHM is enabling automakers to implement advanced video features in a wide range of applications.

For more information on ROHM video encoder/decoder solutions for automotive applications, please visit our website at www.rohm.com/products/audio-video/video-lsi#productFamily. 

For application-specific enquiries or assistance with your automotive video design, our team of experts can help. Please contact a local ROHM sales office or representative, or reach out to us directly at https://www.rohm.com/contactus.