Developing a Motion Analysis System for Aerial Sports by Inertial Sensors

Heike Katrein Brock

PhD Candidate

Graduate School of Media and Governance

Intention

One of the most interesting objectives for the provision of technological support in sports is to obtain additional performance feedback for the enhancement of motor skill acquisition and training. With ubiquitous tools for performance monitoring, more and more training and activity surveillance applications that provide data insights or recommendations on future activities appeared over the last years in leisure, health and recreation. They alter and revolutionize human activity habits in everyday life. In contrast to the increasing technological supply of the leisure market, performance monitoring tools are however still less common and often also less developed in competitive and professional sports so far. This is due to the fact that the motion data is more complex on the one hand, while on the other hand generally also more accurate and complex motion information for processing is required. For an automated, technology-supported training environment it can therefore be useful to create meaningful machine knowledge on the base of relevant motion information.

During the last year, I investigated how to evolve and adapt common machine learning methods, so that motion can be automatically processed and analyzed. Here, I particularly considered aerial sports for an establishment of new flight monitoring system for training and competition. Aerial sports still remain relatively unexplored and are generally subject to direct supervision by coaches. Motion feedback is given to the athletes by visual or video data analysis and generally no quantitative data is used to affirm the visual impression. The development of a flight performance measurement system is consequently not only important to understand the motion, but also to improve safety issues in aerial sports.

Research details

For my research, the development of techniques that provide motion information on an easy-to-use common basis like wearable sensor devices was a crucial landmark. Especially in sports with extremely large capture volumes as aerial sports and difficult natural weather and daylight conditions, cheap, small and light devices such as inertial measurement units (IMUs) are very beneficial. They capture human motion in a direct way and do not depend on outer capture conditions so that they can also be used over a wide area. However, they only measure angular velocities, acceleration and gravity of designated body parts and can generally not display all information necessary for a complete motion analysis. Therefore, the main question is to find specific motion knowledge from the sensor data, so that useful information can be extracted and provided to users like coaches and athletes. Gradually enhancing sensor hardware as well as processing methods that determine characteristics and kinematic performance parameters like angular changes and body positions, the first step for tuning sensor-based training and performance monitoring tools into more sophisticated devices for future applications has been made in the previous years. Now it is time to use the methods and algorithms for meaningful applications.

The crucial part for this research was to conduct experiments under real conditions to obtain sufficient aerial motion data for the development of the intended innovative motion analysis algorithms. I chose ski jumping for the creation of a main motion data set. In two experiments in summer with several training sessions and training days, I could collect sufficient fundamental data for an analysis (more than 180 jumps in total). Figure 1 gives an impression of the experiments.

Figure 1: Experiment in summer at Myoko Kogen ski jump stadium. Sensor are attached to the athlete’s body and skis and the athlete’s body movements during jump captured.

The winner of a ski jump competition is decided on a scoring system based on distance, style points awarded by five judges and inrun length and wind conditions to compensate for the variable outdoor conditions. To obtain a measure for different jump qualities and to evaluate the following feature selection strategies, I additionally took note of the jump lengths of every jump as well as their point scores awarded by an experienced ski jumping judge. The judge scores were collected on paper in real-time during the data capture sessions and under real judging circumstances from the judge's tower. They consequently conform to score results as they are obtained in real competitions. After data acquisition, all score sheets were digitized to be used as ground truth in the machine learning and testing step. Figure 2 shows a sample judging score sheet with point deductions acquired in my experiments.

Figure 2: Sample judge score sheet for the evaluation of flight performances acquired during the data experiments.

Results

Data accuracy and usability:

The screening of the captured data showed that data could be captured well in general, whereas not all acquired data could be used in the next step. One problem was that the wireless data transmission between computer (software) and sensors did not work flawlessly in every capture process and the transmission of the start signal sent from the computer could not always reach all sensors. Another problem were concussions and random noise of the gyro rate sensor modules within the measurement units attached to the skis as a result of high landing impact. This noise could be observed as peculiarity and linear deviation from the zero sensor reading occurring immediately data after a hard landing. Those concussions of the gyro rate sensor modules led to bias in the sensor data for generally 30-40 minutes or the rest of a capture session until battery depletion or the manual turning off the sensor status.

Incomplete jumps with missing or distorted sensor data, have been not considered within the use and building of the analysis methods, so that from the 180 captured jumps, approximately 100 jumps have been then selected to form a ski jump data set. All of the selected ski jumps have then been further investigated with the processing methods developed in an earlier stage of my research, whereas 60% of the jumps were used as a training data and the remaining 40% were chosen to be used to test the results of the newly implemented algorithms.

As a first step, I computed all positions and orientations for the jumps in the training data to analyze, compare and evaluate the flight performances in various motion information retrieval tasks. Looking at the general orientation data, I could observe huge variations in the heading values among all jumps in the test data set. Initial heading differences were especially present in dependence on the sensors used during every specific capture session, the body segment they were attached to and the date of the data acquisition. Such differences and errors appeared randomly within the complete data set and the different capture sessions and seemed to be influenced by the following properties: different sensitivities in the sensor's magnetometers (every sensor measures a slightly different magnetic field at the same position of the ski jump hill), closeness to the inrun slope and jump hill (sensors attached to the arms are less variant than sensors attached to the thigh and for example close to the magnetic start gate) as well as differences in the magnetic field with changing outer weather conditions.

I concluded that variances in the magnetic field are responsible for the registered heading variations - especially as ski jumping hills are built from a lot of ferromagnetic materials like steel. As research showed that differences already occur in less electric and magnetic environments, magnetic bias is likely to be even larger in artificial, man-made environments with jumping ramps, slopes and technical equipment and the magnetic measurements cannot be trusted blindly. Consequently, any processing system should be made invariant to magnetic differences and variations first. For this, I introduced a new magnetic bias compensation method, which can drastically reduce the variation within the magnetic heading to stable and reasonable values.

Figure 3: Visualization of the differences in heading before (top) and after (bottom) magnetic bias compensation at various sensor locations. Heading angles are of positive values 0 toπon the right side of the circle and of negative values 0 toπon the left side of the circle, whereas 0 is at the bottom and ±pi at the top of the circle plot. Red pies represent the angular areas in which initial heading angles occur.

Application in motion analysis and evaluation:

After verifying the robustness and accuracy of the estimates, the data could then be used for the development of motion analysis methods. For this, it was necessary to transform the kinematic motion data into data representations that describe well the motion performance. Ski jumping is subject to two kinds of motion descriptors: technical determinants of a motion and the aesthetic overall impression of flight, landing and outrun. A combination of both has been used in this research as motion descriptors for feature extraction.

The flight characteristics were described by certain style criteria in the jump phases flight, landing and outrun (as they are also shown in Figure 2 at the filled judging score). Particularly important specifications for the flight phase are the active use of air pressure and aerodynamic conditions by a bold and aggressive move at the take-off with a rapid and smooth transition to the optimal flight position, characterized by steady ski positioning, good balance and symmetric body and ski positioning. For maximum landing points (or no point deduction), the landing should take place in a standardized landing in a squat-like position with one leg slightly shifted in front of the other, the so-called Telemark, and no other body parts touching the ground. Remaining in the stable Telemark position for approximately 10 to 15 meters and a stable outrun until the fall line are the main indicators for little or no point deduction in the outrun. Faults and deviations are punished in dependence on their severity and time of occurrence during flight.

For this research, I derived five performance determinants A1-A5 for the aerial phase and five performance determinants L1-L5 for the landing phase. After a segmentation into the two jump phases on the base of the raw sensor signals, I calculated the body kinematics for every jump with respect to the ten style criteria and transformed the motion descriptions into feature matrices that contain the information on every jump criteria’s specialties. A feature matrix for the flight phase then for example contained several single features like the v-opening angle of the skis, the forward lean angle of the upper body and the ski-attack angle (ski elevation angle), the feature matrix for landing the bending angle of the knees, the positional distance between the two ski boots and the orientation of the arms, as shown in Figure 4.

Figure 4: Visualization of ski jump style characteristics. The purple angles and distances display indicators of flight style quality and can be used as feature extractors.

As a next step, I set up an algorithm to evaluate the jumps in the test database. Reference feature matrices have been built for all style criteria from all jump files in the training data set for the two cases faulty execution and non-faulty execution. With the averaged reference motion features, I then investigated how the similarity between a test jump to the trained reference features of either error or no error could be measured. In concrete, a framewise distance for single feature vectors within the reference matrix and the incoming matrix of the test jump was computed. The distances were then summed up under the principles of Dynamic Time Warping (DTW), yielding a final accumulated cost as the similarity measure per every style criteria. I then labeled an error as existent, if the similarity measure between M and the faulty reference feature was smaller than the similarity measure between M and the non-faulty reference matrix and vice versa. Such labeling process was repeated for every style criteria and the annotations then compared with the annotations made by the human judges. Every correct and incorrect labeling was counted and the final count used to determine the precision and recall over all faulty and non-faulty jumps within the test data set. The distribution of precision and recall values for all C is shown in Figure 5. As one can see, precision and recall are of relatively good value, with recall ranging between 70 and 85% probability of a successful evaluation.

Figure 5: Precision and recall values for the different style criteria and the probability border of 50% (red line).

Future work

Although precision and recall show an overall satisfying accuracy, it is not possible to further distinguish within different errors so far. To really evaluate a performance in a meaningful way, it is now necessary to train further knowledge that can award point deductions within the different style categories with respect to the gravity of every occurring error. In future work, respective learning algorithms shall be implemented that retrieve the amount of deviation from a non-erroneous motion pattern and therewith create powerful performance knowledge. Then, it can become possible to numerically analyze a jump performance - and the framework be used in competition to increase measurability and objectivity as well in training to gain more reliable insights into motion executions, to improve performance or to recruit talent.

Next, the algorithms have to be tested with other aerial sports than ski jumping that might be easier to access and verify, but that might also be more difficult to evaluate (depending on the certain characteristics of related sports).

Conference Presentations

As outcome of my research, I have given an oral presentations at the annual conference of the International Association of Computer Science in Sports last autumn in Loughborough, Great Britain.

Heike Brock, Yuji Ohgi. Towards better measurability - IMU-based feature extractors for motion performance evaluation. 10th International Symposium on Computer Science in Sport, Loughborough, UK, 2015.

Two journal papers have been submitted for publication in scientific journals in 2015 and have already been revised several times. Currently, I am now awaiting their final acceptance. Furthermore, I am in the process of finishing a third journal paper by the end of this month, and have planned to write a fourth paper in March and April this year to sum up my thesis work.