Will Simulation End Physical Test?

If we can spend time designing and analyzing in the digital world, is it still necessary to spend time testing things in the physical world? While the need for physical test will never truly go away, advancements in modeling and simulation are making the shift towards digital engineering achievable and changing the reasons why we test and how we use test data.

The traditional approach to engineering, pioneered by Thomas Edison with his famous invention of the light bulb, included creating a robust design, building a physical prototype, and testing the prototype to see if requirements were met. Although this process is effective, it can be time consuming and require multiple iterations. Modern technology has greatly enhanced the design process by giving us the ability to test thousands (or even millions) of iterations without having to create and test physical versions. With the use of simulation, ultimately, we can prioritize finite test resources and use data to improve known weaknesses and less mature parts of a model and accelerate innovation.

In this episode of Test Talks—our video series that invites experts to weigh in on test, technology, and trends—Barron Stone, an engineer and aerospace specialist at NI, answers questions surrounding the move to digital engineering and walks us through the benefits and limitations of simulation.

To learn more about NI’s approach to test and how new systems such as computer processors, aircrafts, and self-driving cars are being developed today with digital technology, visit ni.com/solutions.

Transcript

Can digital engineering eliminate the need for test? Just think about how powerful computers have become and all the advancements we’ve made in modeling and simulation to where we can now virtually build and evaluate new systems with such incredible levels of fidelity. It’s almost as if they’re real. If we can do all that design and analysis in the digital world, do we still need to spend time testing things in the physical world? Short answer, yes.

The need for test isn’t going away. But the ongoing shift towards digital engineering will change the reasons why we test and how we use test data. Now, to really appreciate what this means, we need to first think about how products have traditionally been designed and tested.

Let’s consider a historic example by traveling back in time to the 1870s when Thomas Edison and his Menlo Park posse invented the first light bulb. They tried a lot of things along the way, and Edison famously remarked that, “I have not failed. I’ve just found 10,000 ways that won’t work.”

Think about that. Edison’s team came up with 10,000 different designs. They built 10,000 prototypes, and then tested each one until they found the winning recipe for a light bulb that burned sufficiently bright and long. I envision their laboratory had a wall of test racks filled with thousands of light bulbs and a small army of test engineers grasping clipboards meticulously recording the time each bulb burned out. Now, that quoted number of 10,000 is probably an exaggeration, but the point is Edison’s team relied on a traditional design-build-test approach to engineering.

You come up with the best design you can given the knowledge you have at that time. You build a physical prototype. And then test it to see if that thing you designed and built meets your requirements. If it does, great. If not, hopefully, you learned something from the process so you can come up with a better design next time.

Rinse and repeat. In that traditional engineering paradigm, the speed at which you can iterate is limited by how fast you can physically build and test each new design. Now, let’s imagine we have a time machine and transport all those Menlo Park engineers from the 1870s to today and give them access to all of our technological advances, except for light bulbs, which still need to be invented. So how would that team of engineers go about inventing the light bulb armed with our modern tools and technology? Well, for starters, rather than drawing out their ideas with pen and paper, they’ll develop digital models to represent each design.

Next, with computer modeling, we have computer simulation, which has come a long way since the 1800s. Our time-warped engineers would leverage high-fidelity physics simulations to analyze each of their model designs to determine if it meets requirements. Will that design burn bright enough? Will it last long enough? The beauty of a computer simulation is that it doesn’t have to run in real time.

You don’t have to wait hours, days, weeks to see when a light bulb burns out in a laboratory. Its entire lifespan can be simulated in fractions of a second. So now, instead of physically testing 10,000 light bulb designs, those engineers can virtually evaluate 10 million designs. And they can do so way faster and cheaper because they don’t have to build thousands of prototypes. And, in the end, after all those simulations converge on an optimal design, they ideally only need to build one type of light bulb, and that’s it.

Edison patents that design and makes tons of money. Obviously, time-traveling Thomas Edison is a fictional story, but it illustrates a very real shift in how new systems are being developed today with digital technology and the type of questions test engineers need to answer. In the traditional engineering approach, we use tests to determine how well does this thing I designed and built meet my requirements. In the modern digital approach, on the other hand, we relied on modeling and analysis to determine if a particular design meets requirements. But that raises a different question we now need to answer, and that’s how well does our digital model actually represent the real world?

After all, a model is only as good as the assumptions and data used to create it. So, once our team of engineers has digitally designed their perfect light bulb through analysis, they should still physically build it and collect real-world test data to validate the correctness of the model on which they base the design decisions. This change is the reason why we collect test data from simply validating that we’ve met a checklist of requirements to gathering relevant information to validate the correctness of a model. And that change, in purpose, can influence how we plan and prioritize test activities. Let’s say through all the prior experimentation and test data that went into developing those models in the first place, we’ve established a high level of confidence that our model can accurately predict how long a new light bulb design will last.

However, when it comes to predicting how bright that bulb will be, maybe those physics aren’t as well understood and the model is somewhat questionable. Does it make sense to spend a lot of time measuring how long it takes for bulbs to burn out? Probably not. If we know that part of the model is already pretty solid, we’ll be better served to prioritize our finite test resources to focus on the known weaknesses and less mature parts of our model and spend our time gathering data to improve those aspects of it. Now, you might be wondering, why should our time-traveling engineers even bother validating a model?

I mean, they use their modeling and analysis tools to find what’s supposedly the best design for a light bulb. Can’t they just check that it meets requirements? If it does, good enough. Be done with it and move on. In the case of a light bulb, uh, probably.

But the light bulb is a simple example. It’s only made up of a few parts, and the way a light bulb is used is pretty straightforward. So there’s only a handful of conditions you might need to test. Compare that simple light bulb with the scale and complexity of modern systems engineers are designing today. I’m talking about computer processors with billions of transistors and aircraft that can fly at hypersonic speeds and self-driving cars that can navigate busy streets on their own.

The complexity of those systems is overwhelming to the point that it becomes impossible to exhaustively test them in the traditional sense. We can’t physically send a self-driving car through the practically infinite number of scenarios it could encounter in any reasonable amount of time. Yet, we need to be confident that the car will safely navigate those situations when it gets out into the real world. The only way we can sufficiently test a system like that and build the confidence it will work is through modeling and simulation. Virtually driving that car millions of test miles.

But to be confident in the test results of that simulation, we need to first have confidence in the models behind it. So back to our initial question. Will digital engineering eliminate the need for test? No. We’ll always need test data from the physical world to enable the work we do in the digital one. And this will allow future innovations to have better designs and come faster than Edison could have ever imagined, which means a brighter future for everyone.