Earle Jamieson, ReSolve Research Engineering Ltd
Creating an accurate, portable, and automated tool to test the manual dexterity and cognitive skills of 13,500 children as part of a large study to identify and help those with difficulties.
Developing the flexible software platform Kinelab, which allows the user to perform a variety of cognitive-motor tests on a tablet PC, and using the comprehensive collection of libraries and specialist toolkits in LabVIEW to rapidly implement an application that presents visual stimuli, captures movement, performs statistical analyses, and automatically generates feedback reports.
Earle Jamieson - ReSolve Research Engineering Ltd
James Chandler - ReSolve Research Engineering Ltd
Faisal Mushtaq - School of Psychology, University of Leeds
Dan Mason - Bradford Institute for Health Research
Ann Barratt - Bradford Institute for Health Research
Liam Hill - School of Psychology, University of Leeds
Amanda Waterman - School of Psychology, University of Leeds
Richard Allen - School of Psychology, University of Leeds
John Wright - Bradford Institute for Health Research
Peter Culmer - School of Mechanical Engineering, University of Leeds
Mark Mon-Williams - School of Psychology, University of Leeds
At ReSolve Research Engineering we specialize in the design, development, and maintenance of bespoke research technologies, delivering the engineering solutions our clients need to efficiently meet their research objectives. Working closely with scientists in academia and industry, we pride ourselves on our ability to act as a dynamic, effective, and reliable extension to the research team.
The Born in Bradford (BiB) project, nested within the UK’s National Health Service, is one of the world’s largest scientific studies.
The project involves tracking the lives of 13,500 children from birth to adulthood to understand the childhood influences that shape health and wellbeing. An important part of the study requires the assessment of manual dexterity and cognitive capacity —two critical aspects of a child’s development—to understand how a child’s motor skills affect the ability to carry out essential, everyday tasks (for example, handwriting) and the impact that related deficits have on a child’s social and emotional wellbeing.
Previous techniques for measuring cognitive-motor function ranged from basic and time-consuming pen and paper techniques, to the use of accurate but complicated and costly laboratory equipment. The key requirements for Kinelab were to provide an easily configurable, rugged, and portable platform to collect kinematic measurements rapidly during the presentation of interactive visual stimuli to children aged 4-12 years.
In 2009, University of Leeds researchers Dr. Peter Culmer and Professor Mark Mon-Williams used LabVIEW to develop the first version of Kinelab, which, for the first time, combined the portability of traditional methods with the speed and accuracy of high-tech lab equipment.
The original developers considered using C++ or .NET to integrate with Windows, but the overhead required to do so made this unviable as it was far easier to use the common platform offered by LabVIEW.
As the BiB project grew, the team asked ReSolve Research Engineering to address the challenges involved with the increasing demands of data collection. ReSolve needed to refresh the original software to accommodate and integrate recent innovations and standards in modern computing. Because the software is built on tried-and-tested scalable LabVIEW design patterns, we can quickly and easily make gradual improvements to ensure the application continues to run smoothly, saving thousands of hours over the testing period.
Kinelab operates on tablet computers as a digital equivalent to pen and paper. A child uses a stylus to interact with 2D objects that appear on-screen and stylus input is captured and analyzed to measure performance.
We built the Kinelab architecture around the delivery of kinematic assessment trials in which visual objects are coordinated with the position of the handheld stylus. We define a trial as a group of visual objects—each with properties such as size, graphical appearance, and location. The developers used the LabVIEW built-in XML functions to define, create, save, and edit custom configuration files containing all object properties within a ‘game.’ An object can trigger events when definable criteria are met, such as moving the stylus over an object. Events can alter the properties of an object, as well as activate other operations such as ending a trial.
The initial developers used LabVIEW graphical design to implement a trial designer—an intuitive graphical user interface that non-programmers can use to quickly and easily generate full experimental procedures without needing to access low-level code, making significant time and cost savings.
Visual Feedback: The LabVIEW 3D picture control and OpenGL provided an ideal platform to readily implement the visual feedback needed for Kinelab, realised as a series of 2D sprites. Incorporating OpenGL meant we could utilise the graphical processing unit and ensure its capabilities are used efficiently, freeing up CPU resources to deal with timing and event-triggered game logic. To optimize the application further, we programmed LabVIEW to allocate CPU resources for high priority and time-critical functions—maximizing the smooth running of Kinelab.
Data Processing: We capture and analyse stylus position within the context of each trial to derive performance measures. We calculate spatial and temporal metrics on each trial using the mathematical functions within LabVIEW and store the data locally.
Since its conception, we have worked with our colleagues to make a number of updates to Kinelab to increase overall functionality and integrate it seamlessly into the BiB testing procedure.
Statistical Analyses and Report Generation: We combine and process the raw metrics obtained from Kinelab to create summary scores for each task. We used the statistical analysis functions in LabVIEW to compare each child’s scores against all other children through a database. Storing raw and processed results makes it possible for researchers to revisit the data and carry out further analyses.
It is imperative that teachers and pupils receive immediate, non-technical feedback to help identify areas for improvement of the pupil’s skills. We use the LabVIEW Report Generation Toolkit (included with LabVIEW Professional) to populate a custom Microsoft Excel template automatically; thus, generating a printable feedback PDF report for each child immediately after testing.
Eliminating Operator Error: To protect the personal data of 13,500 children, the operator does not have direct access to the database. This could be problematic if errors are made when entering data into the user interface. We needed a robust method of cross-checking user interface fields against the database. We solved this by ensuring that all fields populate automatically when a unique pupil number and date of birth match the database. After confirming a child’s details, the operator can start the trial by pressing ‘play.’ This simple yet effective method safeguards against the possibility of entering incorrect data. Since implementing the database cross-referencing functionality there have been no erroneous inputs into the front panel, which has potentially saved hundreds of hours of post-processing time to correct operator errors manually and match data to individual children.
Auxiliary Third-Party Software Interface: To complement the comprehensive capabilities of Kinelab, we can now call executables developed in other environments such as Python in Kinelab using the LabVIEW System Exec function. This allows for quick and easy integration of third-party tasks by passing data seamlessly from LabVIEW to the external applications and vice versa to significantly reduce the development time needed to recreate the same games in LabVIEW.
Kinelab is a novel tool that makes possible the objective assessment of cognitive-motor abilities in children, and other areas where it is important to provide precise and reliable measures of human behaviour. The tool has been a huge success producing substantial research output, with 16 peer-reviewed articles published to date, as well as receiving overwhelmingly positive comments from schools across Bradford.
The application was originally developed for Windows XP and early generations of tablets, but we have performed a number of updates that were made simpler due to the scalable LabVIEW architecture. Currently, we are extending Kinelab’s functionality to work with a variety of hardware platforms and acquire additional data such as contact pressure of the stylus on the screen.
The intuitive graphical interface and established scalable design architectures of LabVIEW make it easy to maintain the system with minimum development times. The collection of libraries and specialist toolkits for LabVIEW have empowered us to develop the extensive functionality we needed for Kinelab within the given timeframe.
Thus far, we have used Kinelab with thousands of school children to help identify individuals with difficulties and to define useful intervention strategies to improve school grades and quality of life. The Born in Bradford project will continue to run for many years and Kinelab will play a vital role in the data collection and management.
We are now exploring new application areas for Kinelab, such as monitoring the cognitive-motor skills of older adults, rehabilitation of neurologically injured patients, and assessing potential surgery and dentistry trainees. This research continues to push the boundaries of our understanding of human cognitive-motor control and development.
Earle Jamieson
ReSolve Research Engineering Ltd
Leeds Innovation Centre, 103 Clarendon Road
Leeds LS2 3DF
United Kingdom
e.jamieson@resolve-re.co.uk