The intelligence behind ADAS systems

Software and hardware developers are moving in largely uncharted territory as they develop the powerful systems needed to improve safety by providing ever-more sophisticated advanced driving assistance systems (ADAS).

Since its founding 20 years ago Xylon has become a leading provider of the intellectual property (IP) cores, advanced field programmable gate array (FPGA) application solutions and design services which hold the key to the success of ADAS.

Automotive Industries (AI) asked Davor Kovačec, Founder and CEO of Xylon, what technologies the company showcased at Embedded World 2016, which was hosted in Nuremberg, Germany during February.

Kovačec: We showcased our IPs for the production-line ready 3D 360° surround view, which provides quick and effortless automated vehicle calibration of multi-camera surround view systems; driver drowsiness detection with an in-cabin camera; multi-object detection (vehicles, pedestrians and bikes/motorbikes) with the front camera for collision avoidance; blind spot detection; lane departure warning and a rear-view camera with different modes. We also premiered our new Structure from Motion (SfM) IP that reconstructs the 3D space surrounding the moving vehicle using four optical cameras – one of the key enablers for semi-autonomous and autonomous vehicles.

AI: How has Xylon’s logiADAK evolved?

Kovačec: We are in the fourth generation of the logiADAK development kit, which when introduced in 2010 demonstrated our first 2D bird’s eye surround view with different rear-view camera views. Since then we systematically added new IP for ADAS applications and upgraded the hardware platform by introducing newer and more powerful Xilinx Zynq®-7000 All Programmable SoC. programmable devices and improved cameras. Through collaboration with partners like eVS Embedded Vision Systems, DDC and Visage Technologies we are able to offer applications with more features and to provide off-the-shelf products to our customers (major USA, Europe and Korea/Japan Tier1s and OEMs). The evolution of the logiADAK development kit illustrates one of the greatest programmable logic benefits, which is the relatively straightforward re-use, not only of the legacy software code, but of the hardware code as well. Code re-use decreases the need for total new designs and related non-recurring engineering costs due to changes in system platforms. It also enables manufacturers to better plan their inventories to reduce the risks of holding obsolete silicon components.

AI: Tell us about Xylon’s portfolio.

Kovačec: Xylon intellectual property IP cores are offered through the logicBRICKS library, which includes IP cores not exclusively designed for the automotive industry. The logicBRICKS IP cores are optimized for Xilinx® All Programmable devices and are fully compatible with the latest Xilinx Vivado® Design Suite implementation tools.

Xylon logicBRICKS IP cores enable our automotive customers to jump-start their new ADAS development and get to market faster. Some of the systems are in the production-ready stage or are already used in volume production automotive systems, while others enable ADAS developers to save months of precious development time. Flexible IP licensing and business models allow Xylon customers to get a quick grasp of the technology and to do further development with minimal or no assistance from Xylon. Trained users can start designing with logicBRICKS right away, with no need for any additional knowledge/techniques. Of course, Xylon also provides technical support throughout the whole SoC design process, and offers long term maintenance to support ADAS development teams.

To be able to properly demonstrate logicBRICKS IPs and enable our customers to start their own development on Xilinx All Programmable devices we have developed the logiADAK Automotive Driver Assistance and the logiRECORDER Multi-Channel Video Recorder programmable ADAS development platforms. The logiADAK kit comes with Xylon implementations of customized hardware accelerators implemented by Xylon logicBRICKS and partner IP cores in the Xilinx Zynq®-7000 All Programmable SoC.

The logiRECORDER kit, which is basically an advanced video data logger, enables synchronous recording of six uncompressed video camera inputs with metadata collected from the vehicle’s OBD-II diagnostic bus and an external GPS module: recording the vehicle’s speed, position, timestamps, etc. The recordings of real-world driving conditions and situations are used for ADAS development and algorithm refinements.

Xylon also supports the latest trend in programmable logic development, which is the use of software-defined environments that enable developers to define hardware SoC design by using familiar development environment and C/C++ coding.

AI: What other technologies can we expect to see this year from Xylon?

Kovačec: We will concentrate on the Structure from Motion reconstruction of the 3D space surrounding the vehicle with optical cameras that are already used for ADAS applications like surround view. Our plan is to achieve very high performance, i.e. 60 fps simultaneous processing of video from four cameras on small and low power Xilinx devices. This technology is one of the keys for semi-autonomous and autonomous vehicles. We plan to add auto-braking and auto-parking features.

In 2016 we also plan to upgrade the logiADAK to Xilinx’s new Zynq UltraScale+™ MPSoC device that integrates improved programmable logic with the real heterogeneous processing system that includes the 64-bit ARM Cortex®-A53 application processors, real-time ARM Cortex-R5 co-processors and the ARM Mali™-400MP GPU in a single device. The Xilinx MPSoC devices will enable us to pack more ADAS functions in a single unit.

Another major breakthrough we see is our Driver Drowsiness Detection solution that is currently undergoing a thorough performance check by a multidisciplinary scientific team. Our goal is to improve the precision of the driver drowsiness detection by matching the results acquired by video camera monitoring with biometric values measured by medical equipment worn by the test driver.

We will also be working on the partial reconfiguration of Xilinx programmable devices. By using this feature it is possible to design different hardware accelerators for ADAS applications and to use them to reprogram the Xilinx device. For example, the Xilinx SoC/MPSoC can work with one set of hardware accelerators while the driver parks the vehicle (active surround view and rear-camera view with pedestrians detection ADAS apps), and with another set while speeding on the motorway (active lane crossing warning, traffic signs recognition and other ADAS apps). The re-programmability feature enables ADAS designers to use smaller and less costly low-power Xilinx All Programmable devices.

AI: What is the biggest challenge facing ADAS developers?

Kovačec: ADAS designers in the automotive space are at present forced to select from just a handful of silicon devices,  which come from an even lower number of silicon manufacturers. The newest ADAS applications are in still uncharted territory, and need new video and other data processing algorithms. That’s why the development of the specialized hardware accelerators in forms of complete SoCs or SoC parts continuously stays behind the real needs of the ADAS designers.

Due to high costs incurred with the R&D and the silicon the most advanced ADAS solutions enter the high-end luxury cars first. This slows down ADAS validation through a real-life knowledge and experience gathering, which are invaluable for the development of self-driving vehicles. This explains why more and more ADAS teams are turning to programmable logic in an attempt to satisfy both the business and the technical objectives.

AI: Tell us more about the Xylon R&D team?

Kovačec: There are 30 developers working in our headquarters in Zagreb, Croatia. The team consists of highly capable professionals, with the combined experience of hundreds of years in FPGA programmable logic and with the heterogeneous expertise in vision processing, FPGA and Embedded SW development optimized for footprint and performance. Xylon works closely with Xilinx and industry partners who are ADAS intellectual property providers. The resulting synergies create highly efficient implementations on Xilinx All Programmable technologies.

Xylon also works closely with universities, especially with The University of Zagreb, and with several advanced groups focused on computer vision developments. Through this collaboration we are able to expand our internal research capacities and explore new algorithms that may potentially be adopted for use in the programmable logic. Since the inception of the partnership with Xilinx, Xylon has been at the forefront of their ecosystem and one of the first companies in the world in terms of early adoption of the most advanced Xilinx technologies. We were one of the pioneer companies in building the IP cores market for programmable logic and in the programmable logic evangelization of many automotive development teams in the world. In many ways Xylon paved the road to flawless design processes with third-party intellectual property and adoption of Xilinx FPGA/SoC designs within the automotive industry.