Leveraging Automated Test to Enable Digital Transformation


Many aerospace and defense (A&D) companies are pursuing digital transformation initiatives to deliver sophisticated systems and capabilities in an increasingly complex business environment. This trend includes the full spectrum of companies serving the A&D industry, from the largest primes to smaller companies throughout the supply chain in all major markets around the world. To be competitive, companies are looking to leverage the latest computing, connectivity, and analytics capabilities to create new business models, increase operational efficiency, and reduce schedule risk, among many other objectives.

Unfortunately, most companies have made little progress on these initiatives, and success has been rare.

~3%
About 3% of A&D companies consider their digital transformation initiatives successful.
29%
29% of A&D companies feel that their digital transformation efforts have failed.
69%
69% of A&D companies have settled for an intermediate result that fell short of their goal.
Source: Bain & Co.

In some cases, the investments are small and distributed, making it difficult to prove out value at scale. Other times, companies have kicked off and funded initiatives only to watch them flounder because they’re not sure where to focus their time and energy. It’s true that digital technologies can provide value to many different areas of business, but because the possibilities are seemingly endless, companies aren’t sure which digital technologies to focus on or implement first.

25%
Only a quarter of the A&D companies are currently using digital technologies and tools to access, manage, analyze, and leverage data from their digital assets to inform decision-making in real time.
84%
84% of A&D executives said they consider leveraging new digital technologies as key to market differentiation.
Source: Deloitte Insights

The Value of Test for the Digital Product Life Cycle

A key enabler of digital transformation is the data generated by a system or device in operation. This data reflects how the system performs in the real-world scenarios it was designed for. Connected aircraft, space vehicles, and many other systems generate huge amounts of data, such as the latest Pratt & Whitney aircraft engines. This data can yield many benefits, such as reduced downtime, new service models, and valuable insights that engineering teams can use for future design iterations.

Up to5000
Sensors
per engine
10 GB
Data generated
per second
844 TB
Data stored per
12-hour flight

A common goal in many digital transformation initiatives is the development of a fully digital system model, often referred to as a digital twin, that predicts how the system will operate in real life—and real-life operational data adds to the accuracy and knowledge embodied in the digital twin. The digital twin is part of a fully digital product life cycle; its backbone is a complete digital thread of all product life cycle data.

Along the journey to a fully digital product life cycle, companies must face the realities of designing, manufacturing, and assembling complex cyber-physical systems. Although simulation can vastly reduce the amount of physical testing required, it cannot completely eliminate the need to validate critical aspects of the design, assembly, and performance of physical systems.

With digital transformation, the focus isn’t on testing more but on testing more intelligently.

Test data offers unique insights about a system and its components. The most effective way that companies can leverage automated test for digital transformation is to tightly integrate their test process and test data with the critical procedures that make up the digital life cycle. With digital transformation, the focus isn’t on testing more but on testing more intelligently. A sophisticated, software-centric approach to automated test and data management is essential for realizing the added value of test integration.

Asset Performance and Health

TEST PROCESSES

  • Findings
  • Requirements
  • Design
  • Specs
  • Quality
  • Usage

TEST DATA

  • Measurements
  • Validation
  • Characterization
  • Raw Measurements
  • Diagnostics
Digital thread documents the entire product life from concept to reality

Integrating Test with Digital Engineering

Paramount to the digital product life cycle is an engineering process driven by modeling and simulation, using:

Model-based systems engineering (MBSE) for the top-down design specification and systems architecture

Model-based design (MBD) during the design phase for subsystems and components

Modeling and simulation aren’t new to test. However, simulation and analytics are becoming more extensive and accurate thanks to improvements in processing power and data storage capabilities.

Engineers rely on test data to diagnose differences between expected and actual behavior.

Test data plays an important role in supporting digital engineering. Unfortunately, information at the time of test isn’t always properly captured. Data often lacks adequate details, making it difficult to understand the operating context of the test. Also, test data is frequently mismanaged. It may exist in silos on computers that aren’t network accessible. Test systems must provide a way to access data securely through open connectivity and support industry standard data formats.

The ability to execute models from simulation directly on the test system provides another opportunity to tightly integrate test with the digital engineering process.

They must also connect to multiple test data sources and enterprise data architectures. The ability to execute models from simulation directly on the test system provides another opportunity to tightly integrate test with the digital engineering process. Hardware-in-the-loop (HIL) testing is a well-known approach used in embedded software test that intertwines validation test with model-based software development.

Embedded software and electronics are becoming increasingly complex, comprising networks of embedded devices, such as multiple LRUs in air and space craft. HIL testing of such systems is very complex. It requires a sophisticated architecture including software, I/O, and networks. HIL testing is just one phase of an overall development process that includes model-in-the-loop (MIL) and software-in-the-loop (SIL) testing, development phases that rely exclusively on software. The test software used for HIL testing must also be adaptable to MIL and SIL testing so that common models and test plans can be used throughout all phases of the process, even when physical hardware isn’t involved. This approach increases engineering efficiency through reuse and reduces the risk of errors when comparing simulated versus real operations.

Operational
Analysis
Validating the System
Operational
Test
System
Requirements
Verifying the System
System Test
System and
Architecture Design
Verifying the Subsystems
Integration Test
Detailed Design
Verifying the
Units/Devices
Unit Test
Implementation

Tightly integrating test with the digital engineering process accelerates the overall design process and contributes to the fully digital product life cycle.

It’s possible to integrate test systems with digital engineering more effectively by tying them to requirements and other artifacts associated with the product life cycle. Product life cycle management (PLM) systems are becoming the central repository for all data associated with the digital life cycle. Test systems can also interface with the PLM, linking test to the product processes and data managed by the PLM. By operating off a single source of product truth managed in the PLM system, test remains in sync with the overall process, increasing overall design efficiency, reducing design cycle time, and ultimately ensuring that programs stay on schedule and on budget.

Integrating Test into the Connected Factory

Like most industries, A&D companies believe digital transformation and the connected factory will usher in a wealth of benefits, including:

Improved visibility across all manufacturing operations

Enhanced asset utilization

Improved quality

Lowered cost

Reduced schedule risk

Automated test—and more specifically, data from the device under test (DUT) and the test equipment—can turn these expectations into reality.As a foundational step, companies must seamlessly integrate test systems into the overall operations of the connected factory.

A digital transformation-ready test system reliably and securely connects to enterprise systems and offers standard services to make its operation visible and manageable.

A test system that is ready for digital transformation is one that can reliably and securely connect to enterprise systems and offer standard services to make its operation visible and manageable. A connected test system provides visibility into its state, including software and hardware versioning and asset information. It’s essential for both quality and security to control test software configurations in a manufacturing or MRO environment.

Once available to the organization, test data can help identify problems well before they impact quality or reliability. Engineers can use test data to isolate contributors that determine the true cause for shifts in performance and provide insights to support process and product optimizations. As with engineering, poor data storage and management hampers the value of test data. It’s important to pay careful attention to data formats, ensuring that operational context, or metadata, is included with the data to enable relation to other data about the process and equipment on the factory floor.

Test data can deliver value to the manufacturing process more broadly by “instrumenting” the factory floor and incorporating data from other software, including monitoring and management (MOM) applications such as manufacturing execution systems (MES) and ERP systems. Insights gained from this broader set of data enable the next step along the path to digital transformation, which is integration across all manufacturing processes. This could mean:

Dynamic test scheduling to maximize asset utilization
Sharing manufacturing data with the upstream design process to change how a feature is built and reduce product defects
Dynamically optimizing test by changing the order of operations to identify failures earlier

Ensuring the Security of Test Systems and Data

The convergence of information technology (IT) and test systems is driving new requirements that impact test equipment used throughout the product life cycle. Historically, test systems have been somewhat immune to security risks because they weren’t connected to the broader network. As the sharing of test data becomes more important, these systems must connect to the network. This is driving more investment in the products and processes that are used to validate test systems.

In addition to system certifications, there are new governmental requirements for test systems in the works. For example, the US Department of Defense (DoD) is driving new organizational rules to protect sensitive information and to create secure supplier networks. In 2020, the DoD plans to release the Cybersecurity Maturity Model Certification (CMMC). CMMC evolves NIST 800-53 and 800-171 to provide a framework for any company providing services to the DoD, as contractors or subcontractors.

As the sharing of test data becomes more important, these systems must connect to the network.

Organizations must adapt and evolve their internal processes to comply with this new standard. These changes drive the need for design, test, and IT teams to collaborate in new ways. They’re also creating the need for new investments in systems that provide data visibility to stakeholders across the company while protecting valuable company assets. The A&D industry should anticipate major new innovations in security over the next few years as a result.

Putting Test on Your Digital Transformation Road Map

There are many opportunities for A&D companies to use automated test to enable digital transformation. As with any other venture, they should identify areas that are reasonably attainable and likely to generate a high ROI. Although it may be tempting to “go big,” it’s more advisable to take a pragmatic approach which includes milestones that add incremental value, build expertise, and lay the foundation for new capabilities in the future.

Software-Defined Automation Enables Optimization and Continuous Insight

LEVERAGE FIELD LEARNINGS INTO NEW DESIGNS
Integrated test with design
Generate optimal test strategy
Integrate test with MES
Perform predictive maintenance
  • Research
  • Development
  • Manufacturing
  • Operation
  • FINDINGS
  • REQUIREMENTS
  • DESIGN
  • SPECS
  • QUALITY
  • USAGE
  • Measurements
  • Validation
  • Characterization
  • Raw Measurements
  • Diagnostics

Asset Performance and Health

Digital thread documents the entire product life from concept to reality

The key to leveraging automated test for digital transformation is to tightly integrate the test process and test data into key processes that make up the digital life cycle.

Four Advantages of Using Test Data

Connectivity

Accelerate productivity with dynamic test scheduling.

Data and Systems Management

Access data faster for effective operations management.

Data-Driven Applications

Create applications using test data to enhance manufacturing practices.

Workflow Integration and Automation

Increase efficiency by tying test into higher level processes of engineering and manufacturing.

NI: Your Partner for Digitally Transforming Automated Test

For over 40 years, NI has helped A&D companies realize the value of embracing more digital and automated approaches. NI’s software-defined platform leverages data and software to reduce schedule risk, increase asset quality, improve mission readiness, and increase operational efficiency. From LabVIEW graphical development tools, to modular measurement hardware, to integration with widely-used software tools like Python and MATLAB®, NI is committed to helping customers adapt quickly to rapid changes in business and technology.

Recently, NI expanded its platform to include SystemLink™ software, which is designed to help companies manage increasingly connected systems and get more out of the vast amount of data generated by automated test and automated measurement systems. Incorporating SystemLink software is the first step in establishing the digital thread for measurement throughout a development workflow.

NI is a leading supplier of automated test solutions used throughout the product life cycle and across the entire spectrum of A&D markets and supply chain. NI’s approach to automated test and automated measurement helps A&D companies address demanding test challenges by leveraging modular software and hardware for adaptability, long-term compatibility, and maintainability. This approach also supports the openness and accessibility of data, which is essential to actualizing digital transformation initiatives.

In addition to advanced technical product capabilities, NI and its extensive network of Alliance Partners offer services and support to ensure that customers receive maximum return and value from their automated test investments.

Next Steps

To learn more about achieving the benefits of digital transforamtion throughout the product life cycle, visit ni.com/aerodef-digital-transformation or download our resource guide.

A National Instruments Alliance Partner is a business entity independent from National Instruments and has no agency, partnership, or joint-venture relationship with National Instruments.