Modernizing Your Lab with a Validation Framework Architecture

The first white paper in this series, “How a Modern Lab Approach Optimizes Post-Silicon Validation,” explored how device functionality testing becomes increasingly important as integrated circuit (IC) designs get more complex and require lower cost and shorter time to market. IC manufacturers must move to a highly automated and integrated validation framework to drive all validation activity, ensuring maximum reusability with quicker bring-up, and efficient measurement/test methodologies for faster releases to manufacturing (RTMs).

Here, we discuss the people, processes, and technology Soliton and NI recommend for successfully implementing such a framework. An important validation framework characteristic is a seamless device validation workflow. The validation team focuses on developing measurement required for device validation and collects data over various conditions. It’s critical to use the validation team’s core measurement-development time effectively. The framework’s different components should make it easy for developing reusable measurement IP to share across teams. A highly efficient framework must include the following components:

Figure 1. People, Processes, and Technologies Combine to Form a Comprehensive Validation Framework

People

• Collaboration between multiple silicon development lifecycle disciplines (design, verification, validation, and test) over specifications and measurement methodologies
• Access to data dashboards to track device validation progress
• A simple onboarding process with training and support to make the new validation engineer productive quickly

Process

• Bench setup and management (installation automation, bench replication, and instrument utilization)
• Both interactive debug (during wake-up for designer) and automated data collection for validation engineers
• Measurement IP development with reuse and sharing capabilitie

Technology

• A central data lake for storing measurement data and correlating with data from other verticals
• Multiple user-preferred options (Python, NI LabVIEW, .NET)
• Hardware abstraction to support different instrument types and models

Figure 2. An In-Depth View of Efficient Validation Framework Elements

The configuration management tool stores all of the framework components and reusable module packages in a central repository. The tool, which manages the repository at the server side, also manages the software and hardware details deployed in each lab station. Such infrastructure substantially reduces the time it takes to set up a bench and keep the software up to date.

Software Deployment and Update

You can use a central server to deploy core framework components such as reusable device communication, reusable measurement IP, and instrument drivers to stations—and you can do so remotely. When you improve measurement IP or fix any bug on a driver, you can update it to all stations with just a click from the central repository. You also can replicate bench setup automatically, including device-specific test configuration—avoiding the effort to manually copy all of the source code and edit the configurations, as well as eliminating introduced errors.

While you can custom-build station configuration management and software-deployment solutions in-house, commercially available tools can help reduce your development and maintenance burden. For example, the NI SystemLink™ Software Configuration Module provides a turnkey solution to centrally administer measurement systems so that you can remotely and efficiently automate patches and code-base updates. Furthermore, the NI SystemLink Asset Module improves equipment utilization and quality compliance by recording, tracking, controlling, and reporting on test assets—from procurement to disposal—in real time.

As soon as the first silicon reaches the lab, the validation engineer’s first step is to bring up the device and confirm that all basic functions are working. After all, it’s not just the validation engineer, but engineers from different teams such as design, verification systems, and applications, that come into the lab to work with the device. These engineers would not have software or automation expertise, so the framework should have easy-to-use wake-up, debug customer use-case, and correlation tools. Because engineers spend more than 50 percent of their time on these activities in the lab, it’s critical that the lab has easy-to-use and interactive tools for efficient debug.

Run Measurements Interactively with a UI

Engineers run and debug measurements through their preferred ADE or IDE. This poses a challenge when users are unfamiliar with the development tech stack and need to run measurements. The availability of a UI for measurements to run interactively enables engineers without development experience to leverage the same measurement IP used in the lab. It is more efficient when the measurement developer doesn’t need to spend a lot of effort to add UI to their measurement and make them easily shareable.

Engineers can write measurements using the UI in NI InstrumentStudio™ Professional, which can be interactively run from this software along with the instrument soft front panels and use NI TestStand for test automation. Measurements developed in text-based languages like Python can benefit from interactive UIs created graphically without additional programming.

Now, an application engineer can effectively use the measurements developed by a validation engineer to assist customers without the burden of learning the tech stack, making the process more efficient and user friendly.

Device Control (Register Peek and Poke)

IC designs are getting more complex, and each IC has huge register banks to configure and control different functionalities. DUTs have digital I/O to control different operations.

The engineer develops the protocol communication APIs using a chosen controller or digital pattern generator over multiple weeks. The communication module also should be able to control digital lines from the instrument to configure device modes.

The UI-based semiconductor register-configure utility helps establish device communication with a simple configuration for protocols like I2C, SPI, SPMI, I3C, and so on. The tool helps to understand device behavior and verify the measurement procedure interactively before the measurement is automated.

The other features are listed as follows:

• Import the entire register map (register and field level information) of the device into the intuitive GUI.
• Use the write/read register operations from the GUI to communicate with the device and configure the device registers for debugging a customer use case or basic working of the device.
• Record the read/write operation sequences as a macro to be used in automation later.
• Save the register map snapshot of the device for a particular measurement and recall the setup later in automation.

If you’re looking to implement standardized device control, consider an off-the-shelf turnkey solution such as the NI Semiconductor Device Control Add-On, which provides intuitive interactive device control capabilities that quickly scale to automated validation.

Instrument Control Panel

You can control bench instruments through a simple user interface and add new instrument control panels as required via the plug-in architecture. The off-the-shelf framework provides often-used instruments such as power supplies, SMUs, and oscilloscopes so that you don’t need to develop the software. These tools help you manually power up the hardware boards and DUT—and debug both hardware and DUT issues.

Instrument Configuration

You can configure the instrument from these interactive GUIs during debug. Store instrument settings as configuration and later load them to the instrument during measurement automation.

For example, with InstrumentStudio, you get an integrated instrument-configuration environment in which you can take quick, interactive measurements and export configuration data that simplifies the path to automation.

Mapping DUT Pin to Instrument Channel

Usually, engineers define test-procedure parameters from the DUT point of view. The DUT pin connection to the validation hardware board is embedded in the measurement code. The pin map utility defines the mapping between the DUT pins and the validation hardware PCB, making it easier to maintain the mapping information outside of the measurement code. When instruments and the DUT change, only the mapping is changed—not the code. A pin map abstracts specific instrument and DUT details from the core measurement IP and promotes measurement IP reuse across various products and teams without the need for manual editing during reuse.

Measurement IP/Library

A validation engineer’s core task is to develop the measurements required for the device. When developing a typical measurement, they often hard-code the input parameters within the measurement code. When another engineer or team validates a similar device, they must copy the code and customize the measurement code to fit the new device. This involves a lot of code change and requires complete code testing, adding time, effort, and cost.

Ideally, the framework would abstract the measurement IP development with input parameters and sweep parameters for reuse, whether in another device in the same family or a different device with the same silicon IP. Then, other engineers can use the reusable measurement IPs in the automation to run multiple measurements on the same device and sweep over different test conditions such as temperature and input supply. Reusable measurement IP reduces measurement development/debug time for a similar device or family, saving time and cost and shortening time to market.

Sweeping and Looping Conditions

A comprehensive device validation requires making measurements over many conditions to ensure that the device functions within specifications. So, the validation engineer should run measurements across many process, voltage, and temperature (PVT) conditions, requiring a flexible, robust automation framework. The validation engineer should be able to configure, change sweeping conditions on the fly, and run the automation effectively without any measurement IP code change. Within the framework, you can configure the parameters over which the measurements must be swept:

• Easily configure the range of values for each sweep parameter both individually and comprehensively in a nested sweep
• Quickly generate sweep values based on a distribution and visualize the conditions
• Automatically log the sweep conditions along with the measurement data

When the sweep parameters are abstracted from the measurement module, you can develop reusable measurement IP and promote collaboration between teams.

Test Sequencer

The test sequencer is the core part of automation and data collection. The validation engineer’s objective is to string together the measurement (from the measurement IP library) developed for the DUT and run the measurements over PVT test conditions. The validation engineer should be able to do this quickly without a lot of software development. The framework’s test sequencer is an easy-to-use tool with a low training barrier. Use the test sequencing environment to:

• Sequence tests using drag-and-drop actions and quickly configure input
• Create instrument and device configurations in the debug/ interactive environment
• Configure and save all PVT conditions in the integrated sweep condition manager
• Quickly run tests and get to data in the default CSV integrated data log module
• Accelerate data collection over multiple DUTs in parallel with multisite testing
• Customize development with advanced programming language features such as loops and statements

With TestStand test and measurement management software, you can develop, debug, and deploy measurement sequences in one standard off-the-shelf solution—develop multithreaded sequences, support any programming language, generate reports, and log to a database. Industry-leading validation organizations don’t waste valuable time and engineering resources developing and maintaining software solutions that don’t add value back to their product designs. Instead, working with a software expert like NI empowers teams to spend more time on measurement IP, analyzing device performance and providing design feedback with a turnkey test sequencer.

Hardware Abstraction and Measurement Abstraction

Often, validation labs use different instrument models at different sites or benches. In these cases, the measurement code developed at one bench cannot be reused in another. Each bench has its own measurement code; hence, test measurements are fragmented and not reusable. A hardware abstraction layer (HAL) solves this problem by abstracting the instrument information from the measurement code.

For example, Bench A might use an NI SMU to power up the DUT. But Bench B uses a benchtop power supply. Without HAL, each bench requires custom measurement code to power up the DUT, and the power-up sequence is nonreusable. With HAL, though, the measurement author just calls the top-level parent APIs in their implementation. The choice of hardware rests in an interactive configuration on the instrument pin mapping screen.

Figure 3. Test Sequence Abstraction Layers

Reuse Measurement IP

Developing measurement modules over HAL and digital communication modularizes the specific dependency. These abstractions promote modular and reusable measurement IP development, reducing development and testing effort.

​Standardized Data Logger

Because validation tests run over multiple hours or even days, you want to capture and save measurement data quickly to avoid data loss due to unexpected system or software issues. The framework’s standardized data logger constantly logs data after each loop into a local CSV file to prevent data loss. This local storage means that you can check the data right away for faster debugging.

Furthermore, this data log module tags the sweep conditions (PVT) set at the time of measurement. So, when the manager or designer reviews the data, all associated conditions are available along with the measurement data.

Figure 4. Constantly Logging Measurement Data with a Standardized Data Logger

Standardized data logger advantages include:

• Common implementation across both interactive bring-up and test automation workflows
• The measurement developer concentrates only on core measurement IP development, while the framework takes care of logging measurement data
• The developer passes the measurement data from the measurement module using the framework’s data logging APIs, and the framework tags all conditions automatically along with the measurement data
• You can load the standard data into your choice of analysis tool for review
• With a flexible plug-in architecture, you can add storage formats without affecting core measurement modules

Centralized Storage and Analysis

One of the biggest data analysis challenges is naming conventions: Because validation engineers develop logging capabilities and measurement code using their choice of names, each name requires a unique data review and analysis method for each device. With a structured and standardized data-logging format, you can visualize data quickly, making debugging less time consuming.

The data logging framework supports comma-separated values (CSV), Excel, TDM streaming, and Standard Test Data Format (STDF). CSV data logging can be wide or tall:

• In the wide format, each column represents a parameter and each row a measurement value.
• In the tall format, all the measurements are logged in three columns: test name, parameter name, and parameter value.

A standardized format with meta measurement data tags increases data-logging efficiency and helps plot it quickly for visualization. It’s crucial to have easy access to centralized validation data storage, specification information, and verification data throughout the validation lifecycle. The central database is key for developing tools such as:

• Automated specification compliance report generation (compare device measured parameters against the key design parameters)
• Data correlation for comparing design verification with silicon validation data

If you are interested in evaluating the impact a validation framework architecture could have on your lab, NI and Soliton offer decades of experience working together to help semiconductor customers improve efficiency and accelerate time to market. With more than 100 LabVIEW and TestStand engineers and more than 20 NI technical awards, Soliton provides highly differentiated engineering services that complement NI products and solutions. As an NI Semiconductor Specialty Partner for the Modern Lab, STS, and SystemLink™ software, Soliton specializes in developing purpose-built platform solutions for post-silicon validation that can substantially improve engineering productivity through powerful tools and standardization.

• Discover how a modern lab approach optimizes post-silicon validation
• Learn more about Soliton’s services for the modern lab
• Contact Soliton
• Learn how to accelerate the validation workflow with the Measurement Software Development Kit (SDK)
• Learn more about validation lab innovations