The car of the future will be an ACE: it will provide Automated driving, Connected services, and it will be Electrifying for sure. While we've been working toward this vision for multiple years; to achieve this goal, the foundation must be there—the software-defined vehicle. Ever since the victory of the smartphone, redefining the entire personality of an electronic device through software has become the norm. This personalized experience is what drivers and passengers are looking for as well; nobody wants to purchase a car anymore that stays the same for five or more years. Over-the-air updates are mandatory. Consumers want these updates for new convenience functions, as well as efficiency boosts like extending the range of an EV, and even for the latest innovative safety features. Safety and reliability are the highest priorities when working on advanced driver assistance systems (ADAS) or Autonomous Driving (AD) features, but they have three main challenges (Figure 1), which highlight the complexity and why it is so hard to bring trusted safety to the roads.

Many challenges exist to putting vehicles out on the road, as they need to be safer while using higher levels of automated driving. For example, a level 2 feature like automated emergency braking (AEB) can be found in many cars today. One of the key performance indicators for an AEB function is false activation. One of the main test methods is based on replaying a finite set of recorded scenarios in an open loop. The main parameter to consider is the change of distance between the ego vehicle and the target in front over time. This parameter makes the amount of data still reasonable, and the test complexity remains manageable.

But as we move toward higher levels of automation, safety features become more complex. Functions are now based on neural networks, so they do not only require thorough validation—they must be trained first. Therefore, an extra set of training artifacts must be created and then exercised by the neural network before the actual testing can happen. In addition, features for highly automated driving interact much more with the ego vehicle. They not only take control of the brakes, like in our AEB example, but they also plan and steer the trajectory of the ego vehicle. Hence, a tight closed-loop validation approach becomes a necessity and the number of scenarios for testing converges towards infinity. As the amount of both simulated and recorded data at Jaguar Land Rover is massive already, their clear focus is on maximizing the utilization of those datasets to further optimize time, effort, and cost of test.

First, developing and testing cars including automation features means that every OEM—and in fact every model—has a unique sensor configuration which needs to be considered. These configurations greatly vary in the number of sensors, the different sensor modalities like camera, radar, and lidar—just to name a few. On top of that, OEMs are combining sensors from multiple suppliers to get the best and most cost-efficient variants into their products. 

The second challenge is the move toward a more centralized compute architectures referred to as zonal or multi-domain controllers. Here, the industry is increasingly mimicking a centralized brain. Lastly, ADAS and AD functions based on Artificial Intelligence (AI) must be trained and tested against countless scenarios. Unknown unknowns can occur everywhere and everywhen, which creates an infinity problem.

The introduction of the eScooter (Figure 2) provides the perfect example of the infinite scenario challenge and why the software-defined vehicle makes the difference. Several years ago, eScooters did not exist. Therefore, their driving behavior and characteristics were unknown by mankind. The good thing is that human beings can learn quickly as they engage with eScooters not only while driving a car, but every day and night, as well as whenever they are encountered (also outside of traffic). The human being can turn these experiences into common sense and anticipate the aforementioned driving behavior and characteristics to handle eScooters also while driving a car in traffic.

When considering AI, the story becomes a bit different, as it only learns when it engages with a new road user. So therefore, when compared to the human being, the amount of time to learn to adapt to these new situations is already limited. Furthermore, the AI algorithm will start to compare and classify the eScooter against what it knows and what it has been trained against. The eScooter might be perceived as a cyclist, as the bicycle has wheels as well and it can move relatively fast. Still, the person is sitting on the bike and the legs of the bicyclist are moving most of the time, while the driver of the eScooter stands still, but is also moving quickly. The eScooter could also be misinterpreted as a pedestrian. Again, the person on the eScooter is standing still, yet moving fast and it has wheels—so this classification does not really make sense either. The only thing that can fix the problem of automated cars learning to handle new and unidentified obstacles and scenarios quickly is an over-the-air software update—and this is the true power of the software-defined vehicle.

Valeo is known as a leader and global Automotive Tier 1 supplier covering electrification, comfort and driving assistance, and smart connectivity. Valeo supplies cameras, radars, and lidar sensors to OEMs along with other components. But with the disruption of the automotive industry, they have further evolved to a tech company that provides full functions like 360-degree view and parking assist, all the way up to connected valet parking to OEMs. Valeo must deal with all the complexities and dependencies of a system of systems. After the completion of the design phase of the front- and back-end of the product, systems validation is the key to solving the previously mentioned challenges.

The toolchain architecture and the definition of a comprehensive validation plan become crucial elements to deliver according to Valeo’s quality and coverage standards. Because of the complexity of the systems under test, the test team at Valeo decided to split the validation process into three levels. The first level is to validate the system in a fully controlled environment in a deterministic way. The second one is to collect statistics around the system performance in open road testing, and the third one is to use AI and other alternative methods to generate synthetic data to cover any gap that is not covered in the previous two attempts.

Data acts as the bloodstream and flows through all the different stages of the product development lifecycle from the moment it is acquired at the vehicle all the way up to data archival. Most of the systems at hand are predominantly vision- or radar-based systems. NI test benches (Figure 3) are instrumental for the validation of the systems under test, especially when it comes down to resimulation and processing big data. Valeo operates eight hardware-in-the-loop (HIL) farms all over the globe, most of them equipped with NI equipment and supported by NI engineers.

The multi-year journey and strong alignment between Valeo and NI has resulted in a trustful collaboration. Speaking of results, working together with NI, Valeo was able to increase their Verification & Validation throughput by 6x for their camera portfolio. Moreover, Valeo utilizes NI test systems and software from design verification to validation and up to manufacturing test, enabling a true data- and software-connected test workflow.

The mission of Valeo’s systems is to make the vehicle safer for its occupants and for other road users. “And for me personally, the system validation job is done when I’m comfortable to putting my kids into a running vehicle or to have them crossing the road in front of a running vehicle safely, knowing that the system will do the job,” says Dr. Yahia—certainly something all of us can relate to as the ultimate litmus test.

To learn more about how test is ensuring the reliability of future autonomous vehicles, hear directly from Dr. Maymoon Yahia.