NI-DAQmx High-Speed Streaming to Disk
Overview
The TDMS file format is the NI recommended file format for saving time-based measurement data to disk for high performance, usability, and reuse. With NI-DAQmx 9.0, TDMS file writing is now directly integrated into the NI-DAQmx driver for the easiest and fastest way to stream data to disk reaching speeds up to 1.2 GB/s.
Contents
NI-DAQmx High-Speed Logging
NI-DAQmx 9.0 or later installs a new VI, DAQmx Configure Logging VI, which allows users to configure writing data to a TDMS file within a single VI on the DAQ palette. Not only does this make it simple to use and integrate into existing data acquisition applications, but with under the hood optimizations, it is the fastest way to write data to disk.
Figure 1. The DAQmx Configure Logging (TDMS) VI allows users to set up data logging to TDMS before even starting the task.
To optimize file writing speeds, NI-DAQmx can stream data directly from the device buffer to the hard disk bypassing the Windows, LabVIEW, and TDMS buffers. The Voltage -Continuos Input.vi also improves performance and reduces the disk footprint by writing raw data to the TDMS file, including the scaling information separately for use when reading back the TDMS file. These optimizations result in writing speeds up to 1.2 GB/s. The example can be found by selecting Help>>Find Examples... that opens the NI Example Finder from there select Hardware Input and Output>>DAQmx>>Analog Input>>Voltage - Continuous Input.vi to open the example.
Figure 2. NI Example Finder.
Once the Data has been stored to the TDMS file, you use the TDMS File API and TDMS File Viewer in LabVIEW, DIAdem, TDMS Excel Add-on, or other TDMS supported interfaces for reading and viewing the acquired data. Learn more about TDMS writing and reading interfaces.
Optional Data Compaction
The primary factors which affect high-speed streaming to disk in a PC-based data acquisition system are disk I/O bandwidth, CPU bandwidth, and bus bandwidth. Assuming the professional developer has sufficient bandwidth for both the disk I/O as well as the bus, NI-DAQmx data compaction can provide significant performance gains for high-speed streaming to disk applications.
NI-DAQmx primarily affects streaming-to-disk performance because it uses CPU bandwidth while moving and scaling data acquired by devices. Therefore, on systems where CPU bandwidth is a limiting factor, it is critical that NI-DAQmx efficiently uses the CPU.
NI-DAQmx implements an extremely simple form of data compaction. Bits at the end of samples are simply thrown out and the resulting samples are packed into memory. Developers configure NI-DAQmx to discard one or more of the least significant bits in a sample. For example, you can configure a channel with 24-bit resolution and a 32-bit sample size to return only the 20 highest-order bits. A total of 12 bits are discarded, including eight unused bits, which reduces the required disk space by 37.5 percent. In some cases, no data is “lossed” because the samples already contain unused bits. For example, a channel with 12-bit resolution and a 16-bit sample size contains four unused bits. You can discard the four bits with no loss while reducing the required disk space by 25 percent. NI-DAQmx does not implement any other lossless or lossy compression algorithms such as LZW or JPEG.
Figure 3. Use NI-DAQmx data compaction to improve high-speed streaming to disk performance.