Streaming Data To and From Disk

The following guidelines provide information about how to configure your system to continuously stream data to and from disk with your FlexRIO device.
Note These guidelines are only relevant when performing a direct memory access (DMA) to and from host memory using Host to Target DMA FIFOs and Target to Host DMA FIFOs. For information about peer-to-peer transfers directly between devices, refer to the NI Peer-to-Peer Streaming Help.

NI recommends using the TDMS (technical data management system) API method of streaming to disk. Refer to Streaming External Data to a TDMS File in the LabVIEW Help for information about using TDMS to stream data to disk.

Before You Begin

Refer to the following table for a list of recommended equipment to use to achieve maximum streaming rates.

Recommended Equipment Description
64-bit Windows operating system A 64-bit Windows operating system allows you to allocate large DMA buffers for multiple devices. Using a 32-bit Windows operating system can cause the kernel address space to become fragmented as processes allocate and free memory.
Embedded or remote controller that supports 256-byte PCIe payload sizes Most NI controllers support 256-byte PCIe payloads. Your computer's BIOS determines the maximum payload size.
Matched memory sticks in the embedded controller Two DIMMs (dual in-line memory modules) provide more memory bandwidth than a single DIMM and reduce jitter on PCIe bus writes.

Configuring Your Computer

Refer to the following list for information about how to configure your computer for maximum streaming rates.

  • Size the RAID array's volume for the streaming rate you need. Refer to the specifications for your RAID storage system for information about throughput as a function of capacity. Refer to the following figure for information about the RAID volume required for the desired streaming rate.

For example, a streaming rate of 2.0 GB/s requires a RAID array sized for approximately 15 TB. Beyond 15 TB, the stream to disk rate falls below 2.0 GB/s.

  • When writing multi-terabyte files to disk, manually wait for the TDMS Reserve File to complete. Wait 500 ms per terabyte.
  • Use TDMS Asynchronous Read and Write (Data Ref) for best performance. Use 4 to 8 asynchronous reads or writes.
  • Disable Windows services such as Application Experience, Windows Updates/Windows Module Installer, and Google Update, and close any other unnecessary applications.
Caution Placing a RAID controller card and the FlexRIO module on the same PXI backplane switch could limit throughput during continuous streaming. To avoid this issue, place the FlexRIO device and the RAID card in different backplane switches.

Configuring the DMA

Refer to the following list for tips about configuring the DMA to reduce streaming rates and prevent device underflow and overflow.

  • Use the Invoke Method function on a DMA FIFO to set the DMA buffer to a multiple of the read/write size you are using. The minimum DMA buffer size is 4,096 bytes. Setting the DMA buffer to a size that is not a multiple of the read/write size can drastically reduce streaming rates.
  • Set the DMA buffer size to a value between 64 MB to 1 GB, depending on the read/write size and data throughput, to prevent device underflow or overflow.
    Note The DMA buffer size must be an exact multiple of the read/write size you are using. The minimum multiple size is 4,096 bytes. NI recommends using large reads and writes (for example, 8 MB, 16 MB, or 32 MB).

Configuring LabVIEW

  • To reduce jitter, do not directly update LabVIEW front panel indicators from the same loop performing the DMA FIFO Read/Write. Use a separate loop to update front panel indicators.
  • Use a maximum PCIe payload size of 256 bytes.