Choosing the Appropriate Report Generation Strategy

Most automated test systems need to generate a report that logs information about each test run against the unit under test (UUT), including the test status, measurement results, test parameters, and diagnostic information collected during the test. Test systems also have system-level requirements and constraints—such as hardware platform, test throughput, tester up-time, and linkage to Manufacturing Execution Systems (MES) or other enterprise management systems—that can influence the report generation strategy the test system uses.

In most cases, the report generation strategy defined for one test system might not meet the requirements of another test system, even within the same organization. Test system architects must typically consider many or all of the following requirements when defining the appropriate report generation strategy for a particular test system:

  • Maximize test system throughput
  • Minimize report file size
  • Include all necessary report content
  • Interoperate with other processes and systems
  • Support post-failure information recovery
  • Generate reports immediately for the current UUT before testing the next UUT
  • Generate multiple reports for each UUT
  • Generate and view the report during test sequence execution

The topics in this section examine each of these requirements, recommend TestStand report generation settings and configurations that best meet each requirement in typical scenarios, and identify other requirements you might have to sacrifice to optimize a test system for the requirement. NI recommends that you use the information in this section as a starting point for designing a report generation strategy for a test system and iteratively measure and modify the test system to achieve the optimal balance among the requirements.

Consider the different scenarios in which you run a test system and which requirements are most important for each scenario. For example, when you first develop and debug a test system, throughput might not matter, but support for post-failure information recovery might be critical. When you move to production, throughput might be the most important requirement.

Note
  • In some cases, a test system might specify a unique set of requirements that the built-in report generators cannot satisfy. NI recommends that you choose a report format that most closely satisfies the set of requirements and modify it to fit the requirements. You can create custom result processing model plug-ins to generate such a report. For example, a test system might require the performance throughput of an ASCII report format, the interoperability of an XML report format, and the ability to log a small amount of data to the report. You can create a custom result processing model plug-in that generates an XML report that contains only the information you require.
  • You can use multiple results processing configurations to switch easily between settings based on test system execution scenarios.