LabVIEW FPGA Compile Worker Compile Time Benchmarks

NI R&D compiled a set of 14 FPGA compilation jobs to test a wide range of FPGA VI complexity and resource utilization patterns to give an accurate comparison of the different remote compile options for LabVIEW FPGA. It is important to note that all of these test compiles are run on dedicated computers with non-standard amounts of RAM and processing resources as specified below. This does not reflect the compile time reductions that you would see if you offload the compilation from your development machines, which NI has anecdotally seen to be significantly higher.

Figure 1. These FPGA compile worker benchmarks are executed on dedicated compile computers, therefore your compile time reductions will be even further for your project if you are currently compiling on your development computer.

For example, a number of early users of the Linux-based FPGA compile worker have seen on average a 30% reduction in compilation time when offloading a compile from their development computer to the LabVIEW FPGA Compile Cloud Service. You will see these larger compile time reductions when offloading to dedicated computers especially if:

1. 1. Your development machine is running 32-bit operating system
   2. You are multi-tasking on the computer while compiling
   3. You are using a laptop versus a desktop for compilation

Although it is hard to capture these additional performance improvements in predictable benchmarks, they are important factors to consider when evaluating how much time and money you could save through these compilation options.  See the Selecting an FPGA Compile Option section below that covers more of these considerations.

The compilation process for LabVIEW FPGA VIs consists of two stages. The first step in the compilation process is the generation of intermediate files. For this process, LabVIEW parses your block diagram and converts the code to a form the Xilinx compiler can consume. The logic is then handed off to the Xilinx compilation tools which through a series of four resource intensive steps take your requested logic and analyze it, synthesize it down to a lower-level representation, iterate to efficiently map it to the hardware resources, and then return to you the final bitfile.

Figure 2. The FPGA compile worker benchmark tests the Xilinx compilation tool time of execution, which is the majority of the overall LabVIEW FPGA compilation time.

These benchmarks only test the execution time of the Xilinx compile tools, the second stage of this process, which will be running on your FPGA compile worker. This second stage of the compilation consumes the majority of the overall compilation time so any time reduction in this stage will significantly impact you overall time savings.

Note: In LabVIEW 2011 FPGA Module, NI invested heavily in the edit-time responsiveness of the product, which included up to an 80% reduction in the first stage of compilation, intermediate file generation. 

To have a direct comparison between the onsite options of having Windows or Linux-based FPGA compile workers, NI used identical hardware and Xilinx compile tools as defined below. Since the LabVIEW FPGA Compile Cloud Service executes on one of a large set of potential server resources in the cloud NI cannot establish a one-to-one direct comparison. Instead, the equivalency definition below helps us to conservatively estimate the hardware resource allocation used for the compile benchmark.

Software installed on all workers is the Xilinx ISE 13.4 compilation tools which are shipped with the LabVIEW 2012 FPGA module.

FPGA Compile Worker Machine Specification

3.2GHz Intel Xeon W3565
12 GB of RAM
250GB HDD 7200rpm

One machine with 64-bit Windows 7
One machine with 64-bit RedHat Enterprise Linux 5.6

LabVIEW FPGA Cloud Compile Service Specification

Amazon Web Services High-Memory Extra Large Instance

6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each) equivalent to approximately 3.25 – 3.9 GHz 2007 Xeon processor**
17.1 GB of RAM
420 GB of instance storage
64-bit platform

**The Xilinx compile tools only can leverage one processor core per compile job and therefore the compile service only utilizes 3.25 EC2 Compute Units. According to Amazon “One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor” and therefore the instance has processing resources equivalent to a 3.25 – 3.9 GHz 2007 Xeon processor.

The graph below is a sum of the time of execution across the 3 compilation options for the 14 compile jobs. As noted in a previous section, this compile time only includes the execution of the Xilinx compile tools and not the initial Intermediate File Generation stage. The LabVIEW FPGA VIs that comprise the 14 compile jobs range in complexity and resource utilization and result in compile times from 30 minutes up to 145 minutes.

The high-level results are that the summed compile time for the LabVIEW FPGA Compile Cloud Service was a 19% reduction as compared to the dedicated Windows FPGA compile worker. The dedicated Linux FPGA compile worker had a 32% reduction in summed compile time as compared to the Windows FPGA compile worker.

Additionally, a large high memory resource utilization FPGA VI compilation, not shown on the graph below, was also executed on all three compile options. Due to the high RAM available with the LabVIEW FPGA Compile Cloud Service, it was the only option that was successful in this compilation.

Figure 3. Benchmarking the different technologies to offload you FPGA compiles shows that the LabVIEW FPGA Compile Cloud Service reduces compile time by 19% and the dedicated Linux compile worker reduces compile time by 32% as compared to a dedicated Windows computer.

Note: Generally you will see even larger reductions in compile times when comparing compiling on your development machine to these offloading options. See more details in the Compile Time Tests section above.

There are a number of considerations when selecting the appropriate FPGA compile option to offload the process from your development system. In order to obtain the compile time seen in these benchmarks for onsite systems it is vital that you setup and maintain appropriately specified servers. Some of the hardware factors that substantially impact compile times on a dedicated machine are:

1. Processor Clock Rate – Since the compile job can only be executed on a single core, the clock rate of the processor is more important than the number of cores.
   
   
2. RAM – Both the amount of RAM and the operating system, as 32-bit operating systems cannot address more than 4GB of RAM, impact your ability to compile FPGA VIs successfully.
   
   
3. Operating System – The Xilinx compile tools were originally built and optimized for the Linux operating system and as a result you will see the best performance by using Linux.

Based on the results of the compilation time benchmark and the major hardware factors defined above, if you are able to setup and maintain your own Linux based compile farm at your office you will see the greatest reduction in compilation time. However this system will require regular maintenance and upkeep to ensure that your compile computers are up to date with adequate hardware, latest Xilinx compilation tools, and user account system administration.

Once you have selected which dedicated LabVIEW FPGA Compile Worker you will use to offload your LabVIEW FPGA compilations, learn more information about how to set them up at:

• Setting Up a Linux-based FPGA Compile Worker
• Getting Started with the LabVIEW FPGA Compile Farm Toolkit