Improve Test Coverage, Documentation and Traceability by Integrating Python Automation with HIL Testing

Danfoss Power Solutions is a world-class provider of mobile hydraulics for the construction, agriculture, and other off-highway vehicle markets. It is a segment of the Danfoss Group and provides the power our changing world needs to thrive. Danfoss Power Solutions is offering embedded control units as well as application software that is tailored to the customer’s requirements. As embedded electronics get more and more important and embedded software is increasing in complexity, we saw the need of extending our application software testing capabilities by creating an automated (programmable) and real-time-capable test system that is optimized for working with ECUs of our PLUS+1 product line. The project is internally named “MATS”: Modular Application Test System, emphasizing our demands for modularity.

Working with MATS

Being a hardware-in-the-loop system, MATS runs a simulation model of the target vehicle and aims to provide the same environment for embedded control units as on the real machine. Validating machine control algorithms in dynamic driving scenarios or monitoring the ECUs reaction to failure modes requires the creation of highly complex test sequences. MATS is the test platform for many application software projects and therefore the software and hardware configurations are changed multiple times per day. The MATS software framework has to be highly flexible, customizable and automatable to support these tasks.

Adding a Python interface to NI VeriStand

NI VeriStand can execute real-time test cases through stimulus profiles and real-time test sequences which are created using the NI VeriStand Stimulus Profile Editor. We realized that the Stimulus Profile Editor does not satisfy our demands of modularity, reusability and coding speed. In developing our test system we needed to add text based automation and scripting to speed up the development and deployment of real time test sequences.  Fortunately, VeriStand allows users to extend its built-in functionalities by providing a powerful .NET interface. Nearly any action that can be performed via the VeriStand Graphical User Interface (GUI) can also be performed programmatically using the .NET interface. This is a very important and valuable feature for us. We realized that we can use the .NET interface to create our own, customized environment for programming, generating and executing realtime sequences. The Python programming language is the core part of this environment. Python, specifically IronPython, is our language of choice because it is tightly integrated with .NET, is easy to learn and use, and yet it is incredibly powerful in the hands of advanced users. The main challenge was to make realtime programming feel like a natural, built-in feature of Python. For example, instead of writing a line of code like

RTSequence.Code.Main.AddStatement(Expression(“x = y + z”))

one can simply write

x := y + z

Here, := is a new operator, which we call the “realtime assignment operator”. It is one of multiple syntax extensions that we added to the Python language to make realtime programming more convenient. These syntax extensions are only “syntactic sugar” and are not strictly required for realtime programming because the core functionality is implemented as a class library, which is implemented as regular Python code. The library contains Python representations of realtime programing primitives like “If-Statement”, “While-Loop”, etc. It also includes classes for signal generation (Linear Ramp, Soft Ramp, Filters, etc.), CAN-Bus access, Timers and many more.

We use the .mpy extension for files that contain realtime code (MATS Python) and use the regular .py extension for normal Python code. The realtime sequence generation process is depicted below:

With the Python interface, global libraries and project specific libraries, test engineers can create complex test sequences with relatively few lines of code. A single test case typically consists of about 100 lines of Python code which generate a realtime sequence with about 10000 lines of code, 100 channel references and 200 internal variables.

Customized sequence deployment

During the compilation process, the MPY-Compiler usually places meta-information into the generated .nivsseq file. The meta-information describes how custom devices, simulation models or the unit(s) under test (UUT) need to be configured before the test is executed. Since the meta-information is not evaluated by the Stimulus Profile Editor, we wrote our own program that prepares the system according to the meta-information, deploys the realtime sequence and records test-results.

Testing the entire application through test plans

Individual test-cases (realtime sequences) are designed to test a certain feature of the application software. To test the entire application, test-cases are grouped into test plans. Test plans are generated through Python scripts that generate an Excel document (easily edited and readable). The scripts can perform a sequence of steps necessary to test the entire application. For example, a step could be to download the application under test to the target ECU or to deploy a realtime sequence. A custom made graphical interface for NI TestStand is used for running the generated test plans. The result of executing a test plan is an automatically generated test report, summarizing the performed tests with their respective results, by anyone with access to Microsoft Excel.

The Outcome

The Modular Application Test Systems are on their way of becoming the new standard for application software testing within Danfoss Power Solutions. Similar test systems have been built in multiple locations worldwide. Testing application software with programmed test cases offers many advantages like increased test coverage, time savings through easy test-repeatability and improved documentation and traceability. Being able to perform a thorough application test (including dynamic characteristics) without having access to the real vehicle drastically reduces customer startup time and leads to faster time to market. The NI Software products VeriStand, TestStand and LabVIEW form the backbone of our test systems. However, these products are mostly hidden from the user (test engineer) as their openness allowed us to create our own, customized test environment.

Author Information:

Dirk Runge
Danfoss Power Solutions GmbH & Co. OHG
Krokamp 35
Neumünster 24539
Germany
drunge@danfoss.com