Creation of an Infrastructure for Test Data Management

Author(s):

Simon Foster - Jaguar Land Rover
Pablo Abad - Jaguar Land Rover

The Big Analog Data Problem at Jaguar Land Rover

Advanced design, engineering, and technology have all played a part in the success of Jaguar Land Rover (JLR) over the years. We invest more in research and development than any other manufacturing company in the UK, which has empowered our thousands of engineers to develop world-class innovations. With our investment in R&D, over 400 engineers in our powertrain calibration and controls department collect up to 500 GB of time-series data daily.

As a department, we had difficulty managing the amount of data we collected. We found that we often repeated tests because we could not find specific test results. We based our analysis routine off a manual process and by our estimate, took 20 times longer than with the automated process that we have since developed. We had multiple analysis tools, all of which required special scripting to implement algorithms. Nothing was standardized, not even our metadata or channel names. This process led to us analyzing only 10 percent of the data we were collecting.

Benchmarking Criteria to Find the Best Solution for Our Application

To address this Big Analog Data challenge, we created a dedicated team to benchmark nine tools to determine which platform would be the best for our application. Some of the criteria used to evaluate these tools include:

• Ability to automate data uploads—To make it as easy as possible for the 400+ users of this solution, we needed to be able to upload any new data sets automatically whenever a computer was connected to a network and move those files to a centralized server.
• Manage metadata—This included combining metadata from multiple sources, adding metadata that was not originally saved to file, and being able to search files from their metadata.
• Interactive tool to perform analysis—Our engineers did not want to learn another programming language, so our solution needed to feature interactive analysis. This also empowered the engineers to perform their own analysis instead of relying on a handful of analysis experts.
• Parallel analysis run as a batch process—This would help us execute simultaneous processes on potentially thousands of files.
• Reporting templates—We wanted to produce consistent, reliable reports to aid decision making.
• Flexible platform—We wanted to integrate into existing data acquisition processes and potentially expand to other departments in the future.

Why We Chose a Solution Based on DIAdem and DataFinder Server Edition

After reviewing the results from the nine tools we benchmarked, we chose to build our solution off DIAdem software and now called SystemLink TDM DataFinder Module. In addition to meeting the criteria listed above, we chose the NI Technical Data Management Platform for the following reasons:

• DataFinder Server Edition is an off-the-shelf database solution to index metadata to which we can send queries to find specific test results.
• DIAdem can read in and load over 1,000 file formats, which meant none of our current data acquisition processes had to change and should they change in the future, we do not have to worry about compatibility with our data analysis program.
• Using DIAdem, we can interactively create analytics dashboards without programming.
• We can selectively load data. Sometimes our files contained over 200 channels, but we only wanted to analyze five of the channels. We could use DataFinder to perform a query to narrow down the data we wanted to analyze and then selectively load what data we wanted from one or more files for analysis in the DIAdem client.
• The technical support offered by NI was invaluable when we ran into problems or had questions.

The Workflow of a Fully Automated Analysis Process

When testing a car, we collect data in a variety of ways. One way is through data loggers equipped to capture information from sensors during a test drive. Once the test is completed, the data is transferred over WiFi or 3G to a dump area for preprocessing. We also collect data by directly connecting laptops to the vehicle to collect vehicle network data including CAN, MOST, FlexRay or ECU protocols, CCP, XCP, or ETK. Once we complete the test and have a reliable connection to the JLR network, the data automatically transfers to the dump area. No action is required by the user to kick off this process.

Once data is collected, we always check to see if the proper metadata is present. We do this by cross-referencing the file identifiers/tags to other internal databases (vehicle database, engine calibration database) or decode network data for signals like CAN logs. We calculate some parameters (such as average temperature, speed, MPG, and more) and save that metadata in our server based on DataFinder Server Edition. Any engineer can query the data and return all test results that meet the specified parameters.

From here, any engineer can execute his or her own analysis routine. We can perform predefined routines or custom ad-hoc analysis on data that requires closer inspection. We can then choose to run the analysis on select files, or we can run a batch process we send to our server solution. This can sometimes run on thousands of files, which made it important to have multicore processing capability. The user can also use existing analysis tools and algorithms, which helps with internally adopting this new data management and analysis solution.

After running the analysis, we receive a comprehensive report based off a template that helps us iterate on our designs and make data-driven decisions faster. The benefit of using a template in an automated report is that we get information presented in the same way (graphs with same axis or zoom area, tables with same subsets of data and calculations, and more), which makes it easier to compare and decide confidently.

Results to Date

Within one year of developing and implementing this solution, we estimate we are now analyzing up to 95 percent of our data and have reduced our test cost and number of annual tests because we do not have to rerun tests.

Another benefit is that even though our systems increase with complexity, we can now find and address more issues before we pass our final product to our customers. Our customer satisfaction rating is higher than before because our product is more robust than in the past.

As we look toward the future and expanding this data management and analysis solution beyond the powertrain calibration and controls department, DIAdem, DataFinder Server Edition, and our partnership with NI will be critical in moving this application forward.

Author Information:

Simon Foster
Jaguar Land Rover
sfoste31@jaguarlandrover.com