Quigley’s Corner: The challenges of testing increasingly complex systems

Small changes often go unnoticed

It is funny how being close to something, we miss the result of the myriad of small things. I like to refer to it as the actions we take, the learning and competencies developed over time, accrete, which is a geeky way of saying accumulate.

It is also funny how the longer you are in an industry, the smaller the world seems. I have recently had serendipitous connections and exchanges on product testing in the automotive industry, so it seemed like a good time to write a bit out product testing over time.

How things have changed

I have been in vehicle development and manufacturing for more than three decades, in a variety of capacities, including significant stints in test and verification. I recently read an article in the January/February 2021 issue of SAE Automotive Engineering magazine, Allison Builds A Testing Powerhouse and thought of reflection on starting points and present state.

Many OEMs (Original Equipment Manufacturers) have invested considerable time and dollars in developing their product testing capabilities. As the product grows in complexity, so too will the way we should test.

It was not so long ago – in my career lifetime – when most engines were largely mechanical and had little or no embedded components. Now the entire vehicle has embedded electronic systems distributed throughout that interact using a variety of serial data bus communication.

I have worked at two OEMs, both of which had specialized testing facilities. I enjoyed my time at both and watched the respective testing capabilities grow. This includes specialized track development, engine test cells, complete vehicle thermal chambers, dynamometers and other equipment.

To go along with that specialized equipment, and more important, the talent acquired and developed. In fact, I worked at one of the facilities discussed in the SAE Magazine article, PACCAR Technical Center (or PTC as we referred to it was known as in the olden days). A facility located in one of the most invigorating and beautiful parts of the world.

The Skagit Valley comes alive (home of PTC) with pops of color around this time of year with the Skagit Valley Tulip Festival.

I would not want to speculate how many rolls of film and how much time I spent taking pictures of these fields and the wide variety of tulips.

Wide angle, telephoto and macro shots, not a waste of time and no two that look alike.

I will one day recount how I ended up working there. For now, let’s explore the capabilities of these types of test facilities and how this must grow as we learn and vehicles become more complex.

Keep in mind this was the 90’s. Heavy vehicles have long wire harnesses carrying substantial current. Think about four batteries of roughly 1000 CCA (cold cranking amps) – that is a considerable potential energy to controlled.

Multiplexing reduces some of this wire content, but does not eliminate it. In the earlier part of my career, we tested the cab electrical system using thermal imaging.

We started with a plexiglass version of a generic vehicle that represented the external parts of the vehicle. This ease of visibility made it possible to use a thermal camera to view the wire harness placed within this cabin as it would be in the actual vehicle.

Then various failure modes were exercised. For example, wires were shorted to initiate a failure and the thermal camera captured the event. The expectation was that the system controls would prevent serious damage. As this virtual cab had no cloth material, it was not prone to spontaneous events which we must recognize, but shall not name. I will give a hint; it starts with F and rhymes with ire.

The thermal camera monitored the heat generated in order to identify hot spots that may lead to field problems. Areas that did not perform as expected, generated too high heat, were noted for further exploration and design changes to remediate. This was but one of the tests demonstrated at the PTC open house during the Tulip Festival.

At another vehicle OEM, I was part of the burgeoning vehicle telemetry systems development, and of course, testing.

Through a combination of real and virtual vehicle components, it was possible to test the systems electronics and controls.

For example, a vehicle can drive a circuit, recording the stimuli associated with this route, recording all the message exchanges over the various vehicle data buses. Back at the lab, the messages of all or some of the of specific electronic control units (ECUs), are used for testing ECUs under development.

This effectively enables a lab to mimic a version of the system without having to create a vehicle and put it on the road. We are able to ensure it is configured adequately, and take it on a trip to test the system interactions, without leaving the lab!

For example, selective messages can be played back simulating the route, exercising a new ECU (new hardware /software) plugged into the system as if it were in the vehicle. In this way the new unit is tested on a virtual drive cycle, responding, and interacting with other components.

This is not to suggest that this will be the end of the testing of the system, only that we will be able to test some measure of the completed vehicle before the actual vehicle is available. Moreover, even when the vehicle is completed, taking it out on the road to qualify it is a risky proposition. That is not where we start with system testing.

Functional Orientation

There are many new approaches to product development. Agile is one example.

These approaches are often a response to the limits of a functional orientation of some organizations, specifically communication and dependencies.

One way to surmount these difficulties is creating a team with all the competencies required for the project, ideally co-located.

However, as we can see, there are some benefits of a functionally oriented organization.

We can see the benefits in the examples of PTC and other OEMs. Gathering competencies and creating an environment where in the individuals can develop equipment and processes to adequately test the product has great benefit.

This sets the foundation for continuous improvement via growth in test capabilities.

This in turn will have a positive impact on the quality of the developed product.

This only becomes more necessary as OEMs are straddle the line between internal combustion and battery or hybrid vehicles.