Page Comparison

• Melissa Boswell
• Hannah O'Day

Description

...

Project Video

Description

Inertial Measurement Unit (IMU) sensors are electronic devices that measure and report a body's specific force, angular rate, and sometimes the magnetic field surrounding the body, using a combination of accelerometers, gyroscopes and magnetometers. Most people interact with these devices daily, via the IMU in their smartphone that allows them accurate navigating ability. Recent advances in wearable IMU technology, however, are allowing users to capture and store hours of  kinematic data. Traditionally, kinematic data was only attainable in motion capture laboratories equipped with multiple cameras that tracked a subject's motion via reflective markers worn on the body.  With wearable, wireless IMU sensors, we have a unique opportunity - the ability to capture human motion in "natural" settings (outside of the laboratory) lasting multiple hours.  One such IMU company, Xsens, uses a 3D accelerometer, 3D gyroscope, and 3D magnetometer in each of their IMU sensors to capture full 6 degree-of-freedom tracking of body segments.

...

1. Can knee joint angle be calculated accurately with two IMU's?
2. Can the IMU data be input into OpenSim to reconstruct movement kinematics?
3. Can movements kinematics be reconstructed in OpenSim with IMU measurements in real-time?

...

Instructions for creating an OpenSim motion file from XSens .mnvx file in Matlab (See README_JointAngles.txt)

1. Run read_jointAngles_lowerbody_write_motion_file.m (Matlab)

a. This runs main_mvnx.m (this is code modified from Xsens Toolkit, Xsens North America Inc.)

This file loads two figures of the “first segment position” - the pelvis rotation in 3D and the pelvis 3D translation

...

       i. main_mvnx.m uses the load_mvnx.m function (Xsens Toolkit)

...

       ii. Outputs a data tree with all the biomechanical model segment and IMU sensor data (joint angles, joint velocities, joint accelerations etc.)

b. Creates a table of all joints and joint angles from IMU data at every time point (jointTable)

c. Outputs selected joint angles to a matrix with time as first row (allAnglesMatrix)

d. Creates motion file named “mvnxfilename_Xsens_jointangle_q.mot” of joint angles to input in Open Sim from Xsens IMU Data

This file has required OpenSim headings and formats data correctly for use with OpenSim model (but only has pelvis_tilt pelvis_tx pelvis_ty hip_flexion_r knee_angle_r ankle_angle_r)

2. Open your .osim model in OpenSim (the .osim models we have included correspond to each .mvnx file, and all start in the default position determined by the subject’s physical body position at the beginning of the experiment).

For example, if you used the LegSwing_40bpm_5-26-17.mvnx then you should load the gait10dof18musc_40.osim model

3. Input created .mot file into OpenSim through 'File -> Load Motion' and pick the mvnxfilename_Xsens_jointangle_q.mot filemot file.

~5 deg	~90 deg
Examples of the reconstructed seated knee flexion motion in the OpenSim model.

IV. Reconstructing motion in OpenSim from Inverse Kinematics using Xsens IMU orientation data (see README_OpenSIMIK.txt)

1. Output IMU orientation data as quaternions in .mvnx file (default setting for Xsens Awinda sensors using MVN Studio)
2. Run ??Run rotationMatricesForOsimIK.m
   1. This code transforms quaternions from .mvnx file into a rotation matrix
3. Run C++ code
4. This code takes the determined rotation matrices and applies them to each respective segment
The code then runs an inverse kinematic simulation and outputs an OpenSim motion fileThe model used in this code must have the initial conditions to that of the experimental set-up
1. This can be done by opening the OpenSim model (gait10dof18musc.osim), and setting the initial condition of the joints to that of the subject at time = 0, and saving the pose as the "Default Pose"
5. Open the OpenSim motion file to observe the reconstructed motion and plot the knee joint angle.

Results

40 BPM (Walking)	80 BPM	120 BPM (Running)
Xsens Image Removed	Xsens Image Removed	Xsens Image Removed
OpenSim Image Removed	OpenSim Image Removed	OpenSim Image Removed
Image Removed	Image Removed	Image Removed

Root mean square error (RMSE) between Xsens joint angle and ground truth (2D video marker tracking) and OpenSim IK joint angle and ground truth (2D video marker tracking):

*Compare with literature RMSE of 5.09 deg for knee flexion using Xsens sensors while walking (Zhang et al. 2013)

Comparing Xsens vs. OpenSim Inverse Kinematics joint estimates

In the above plots, the knee angle estimation that utilized Xsens IMU sensor orientations in combination with OpenSim Inverse Kinematics is shown in blue, the knee angle estimation from the Xsens biomechanical model is shown in dark orange, and the ground truth knee angle determined from 2D video marker tracking is shown in yellow. 

In each plot, we see that the Xsens estimate is closer to the ground truth knee angle than the OpenSim_IK estimate. This is also reflected in the root mean square error values for each method: averaged across all three speeds, Xsens and OpenSim IK had an RMSE of 3.65 deg and  7.16 deg, respectively. These RMSEs are comparable with literature values of Xsens IMU sensor validation with camera-based motion capture, where they saw RMSE's of 5.09 degrees for knee flexion during walking [4].

The Xsens system is unique in its approach to estimate body segment orientation and position changes with a combination of the integrated inertial data as well as continuous update using a biomechanical model of the human body. By facilitating the constraints of the model, notably that the segments are connected by joints, the kinematics of body segments are corrected for drift and other errors. 

We think that Xsens has a better knee angle estimate because of their sensor fusion scheme with both a prediction and correction step. In the prediction step (pictured on left below), they integrate acceleration and angular velocities on all sensors. This is followed by prediction of segment kinematics using a known sensor to body alignment and a 23 segment biomechanical model of the body. However, skin and soft tissue artifacts influence the IMU measurements, much like traditional reflective marker systems due to contracting muscles and skin/fat motion [1]. In the correction and joint measurement update step (right), these artifacts are reduced by using the knowledge that the two segments are on average connected but with a statistical uncertainty. Detecting contact points of the body with the external world limit the boundless integration error of the assembled body model in the global frame and constrains the global position and velocity. For example, if a person is sitting but moving around (e.g. on a bike), then you could include a constraint that the pelvis is fixed. After the correction step, these updated kinematics are fed back to the sensor fusion algorithms and segment kinematic step to be used in the next time frame [1]. 

Image RemovedImage RemovedImage Removed

Xsens used a scaled model based on the subject's body measurements; however, our OpenSim model was not scaled. Thus, we were not able to incorporate position data and the Inverse Kinematics simulation was only based on orientation data, which makes the joint estimation less accurate. Moreover, with only orientation driven motion "capture", it is not possible to analyze the clearance of both feet (such as in running or jumping) or determine displacement of body with respect to a coordinate system not fixed to the body. Therefore, in the future we want to do real-time estimation of body segment orientation and position changes using raw IMU data and an Opensim model, which we can scale based on the subject's body measurements. 

Why Xsens joint estimation is not perfect

...

1. 1. . This has been done in the following files for the 40, 80 and 120 BPM trials respectively: gait10dof18musc_40.osim, gait10dof18musc_80.osim, gait10dof18musc_120.osim
2. Build and run futureOrientationInverseKinematics.cpp, (you'll need to add these to your directory where the C++ code is saved CMakeLists.txt, futureOrientationInverseKinematics.trc)
   
   1. This code takes the determined rotation matrices and applies them to each respective segment

   • The code then runs an inverse kinematic simulation and outputs OpenSim kinematics files:
     futureOrientationInverseKinematics_Kinematics_q.sto

     futureOrientationInverseKinematics_Kinematics_u.sto

     futureOrientationInverseKinematics_Kinematics_dudt.sto

     It is helpful to rename these based on the trial you are analyzing e.g. futureOrientationInverseKinematics_Kinematics_q_40.sto
3. Load the motion, the OpenSim joint angles file (e.g. futureOrientationInverseKinematics_Kinematics_q_40.sto) to observe the reconstructed motion and plot the knee joint angle in OpenSim.

Results

40 BPM (Walking)	80 BPM	120 BPM (Running)
Xsens Image Added	Xsens Image Added	Xsens Image Added
OpenSim Image Added	OpenSim Image Added	OpenSim Image Added
Image Added	Image Added	Image Added

Root mean square error (RMSE) between Xsens joint angle and ground truth (2D video marker tracking) and OpenSim IK joint angle and ground truth (2D video marker tracking):

*Compare with literature RMSE of 5.09 deg for knee flexion using Xsens sensors while walking (Zhang et al. 2013)

Comparing Xsens vs. OpenSim Inverse Kinematics joint estimates

In the above plots, the knee angle estimation that utilized Xsens IMU sensor orientations in combination with OpenSim Inverse Kinematics is shown in blue, the knee angle estimation from the Xsens biomechanical model is shown in dark orange, and the ground truth knee angle determined from 2D video marker tracking is shown in yellow. 

In each plot, we see that the Xsens estimate is closer to the ground truth knee angle than the OpenSim_IK estimate. This is also reflected in the root mean square error values for each method: averaged across all three speeds, Xsens and OpenSim IK had an RMSE of 3.65 deg and  7.16 deg, respectively. These RMSEs are comparable with literature values of Xsens IMU sensor validation with camera-based motion capture, where they saw RMSE's of 5.09 degrees for knee flexion during walking [4].

The Xsens system is unique in its approach to estimate body segment orientation and position changes with a combination of the integrated inertial data as well as continuous update using a biomechanical model of the human body. By facilitating the constraints of the model, notably that the segments are connected by joints, the kinematics of body segments are corrected for drift and other errors. 

We think that Xsens has a better knee angle estimate because of their sensor fusion scheme with both a prediction and correction step. In the prediction step (pictured on left below), they integrate acceleration and angular velocities on all sensors. This is followed by prediction of segment kinematics using a known sensor to body alignment and a 23 segment biomechanical model of the body. However, skin and soft tissue artifacts influence the IMU measurements, much like traditional reflective marker systems due to contracting muscles and skin/fat motion [1]. In the correction and joint measurement update step (right), these artifacts are reduced by using the knowledge that the two segments are on average connected but with a statistical uncertainty. Detecting contact points of the body with the external world limit the boundless integration error of the assembled body model in the global frame and constrains the global position and velocity. For example, if a person is sitting but moving around (e.g. on a bike), then you could include a constraint that the pelvis is fixed. After the correction step, these updated kinematics are fed back to the sensor fusion algorithms and segment kinematic step to be used in the next time frame [1]. 

Image AddedImage AddedImage Added

Xsens used a scaled model based on the subject's body measurements; however, our OpenSim model was not scaled. Thus, we were not able to incorporate position data and the Inverse Kinematics simulation was only based on orientation data, which makes the joint estimation less accurate. Moreover, with only orientation driven motion "capture", it is not possible to analyze the clearance of both feet (such as in running or jumping) or determine displacement of body with respect to a coordinate system not fixed to the body. Therefore, in the future we want to do real-time estimation of body segment orientation and position changes using raw IMU data and an Opensim model, which we can scale based on the subject's body measurements. 

Why Xsens joint estimation is not perfect

There are a number of reasons why the Xsens joint estimation is not equal to the ground truth knee angles. As mentioned in the experimental set-up, the subject must stand in a neutral pose, so that the rotation from sensor to body segment can be determined by matching the orientation of the sensor in the global frame with the known orientation of each segment in this pose. However, if the subject's sensors are placed incorrectly, (e.g. the thigh sensor is put on horizontally instead of vertically) or even shifted slightly, then the model will be expecting the sensor to be placed one way and will calibrate based on this incorrect conformation. Thus, the resulting joint angles will have slight errors. In a similar manner, if the subject does not assume the correct neutral pose, the sensors will be calibrated incorrectly and there will be error incurred. In addition, if there are magnetic disturbances, the magnetometer readings will be erroneous and this will affect the orientation estimation and thereby the joint angle estimation. Note that this would affect both the Xsens estimation and the OpenSim based method. 

...

• Real-time estimation of body segment orientation and position changes using raw IMU data and update from a scaled OpenSim biomechanical model (instead of using Xsens' biomechanical model, can we translate from the raw IMU data to OpenSim directly?)
• Determining a minimal sensor set required to obtain similar accuracy in kinematics estimated using full 17-sensor set (e.g. could we get reproduce gait kinematics with only 2 IMU sensors, one on each ankle? Or perhaps one on the waist, and one on each foot?)
  • If using the full 17 sensor suit, then the wireless update sampling rate is capped at 60 Hz (the rate for more than 10 sensors), but if using less than 6 sensors, you can sample as high as 120 Hz which might be a necessary resolution for some desired activity. Thus, having a minimal sensor set could be advantageous in cases such as these.
• Simpler or no "formal" calibration of sensors (e.g. calibrating sensors while in some 'static' pose, but not necessarily needing subject to stand in predetermined static pose)
• Collecting real-world kinematic data:

Characterizing Parkinsonian Gait

Though Parkinsonian motor function has been studied heavily in clinics and research labs, there is far less known about how patients function at home or in daily life, let alone movement classifiers for such activities. IMUs that allow characterization of body kinematics outside the lab could be worn by a Parkinson’s patient to validate past motion capture experiments that concluded that Parkinson’s patients have decreased range of motion in their hips, knees and ankles during straight forward walking as compared to healthy age-matched control subjects (Morrris et al. 2005). Furthermore, they could be used to inform closed loop neurostimulation for deep brain stimulation therapy or sensory cueing to improve gait and other movement.

Characterizing Running Gait during Fatigue

Many studies have shown that fatigue alters running kinematics.  A review of  incidence and associated potential risk factors of lower extremity running injuries in long distance runners found that long distance training increases injury risk, with knee injuries being the most common [6].  While some studies have been able to observe these changes, motion capture limits the studies to treadmill running and most of the studies have only studied fatigue in less than 30 minutes.  IMUs can be used outside of the lab to characterize running kinematics of fatigue and the resulting injuries.  Additionally, the change in kinematics while running on different terrains can be studied which could also help prevent injury. Though this has been tried by Reenalda et al. in 2016 using Xsens IMU sensors, there were limitations with data acquisition and battery life [7]. Rapid improvements in this technology will allow for effective exploration of running kinematics and fatigue. 

...

• • ), but if using less than 6 sensors, you can sample as high as 120 Hz which might be a necessary resolution for some desired activity. Thus, having a minimal sensor set could be advantageous in cases such as these.
• Simpler or no "formal" calibration of sensors (e.g. calibrating sensors while in some 'static' pose, but not necessarily needing subject to stand in predetermined static pose)
• Collecting real-world kinematic data:

Characterizing Parkinsonian Gait

Though Parkinsonian motor function has been studied heavily in clinics and research labs, there is far less known about how patients function at home or in daily life, let alone movement classifiers for such activities. IMUs that allow characterization of body kinematics outside the lab could be worn by a Parkinson’s patient to validate past motion capture experiments that concluded that Parkinson’s patients have decreased range of motion in their hips, knees and ankles during straight forward walking as compared to healthy age-matched control subjects (Morrris et al. 2005). Furthermore, they could be used to inform closed loop neurostimulation for deep brain stimulation therapy or sensory cueing to improve gait and other movement.

Characterizing Running Gait during Fatigue

Many studies have shown that fatigue alters running kinematics.  A review of  incidence and associated potential risk factors of lower extremity running injuries in long distance runners found that long distance training increases injury risk, with knee injuries being the most common [6].  While some studies have been able to observe these changes, motion capture limits the studies to treadmill running and most of the studies have only studied fatigue in less than 30 minutes.  IMUs can be used outside of the lab to characterize running kinematics of fatigue and the resulting injuries.  Additionally, the change in kinematics while running on different terrains can be studied which could also help prevent injury. Though this has been tried by Reenalda et al. in 2016 using Xsens IMU sensors, there were limitations with data acquisition and battery life [7]. Rapid improvements in this technology will allow for effective exploration of running kinematics and fatigue. 

Conclusions

1. Knee angle can be measured accurately using two IMUs with the Xsens pipeline.
2. Knee joint angle can be input into OpenSim to reconstruct the movement.
3. Orientations can be exported prior to feedback from the Xsens system and input into OpenSim to run an Inverse Kinematics simulation and knee joint angle can be determined.
4. Obtaining knee joint angle from an IK simulation in OpenSim is less accurate, but can be improved with feedback from the OpenSim model to update orientation and position.

Though we were able to answer our first 2 research questions, we leave our last research question for the future work.:

Can movements kinematics be reconstructed in OpenSim with IMU measurements in real-time to use OpenSim as the biomechanical model for orientation and position feedback?

Progress

•  Collect IMU measurements of knee flexion/extension with Xsens MVN Studio software
•  Input Xsens joint angle estimates into OpenSim to reconstruct movement
•  Do simultaneous IMU data collection with 2D video + goniometer to validate joint angles obtained with IMUs 
•  Determine knee joint angles from Xsens IMU orientation data from thigh and shank sensors 
  •  Get orientation from thigh and shank sensors
  •  Input into OpenSim inverse kinematics script
  •  Load motion in OpenSim, and apply to gait10dof18musc.osim model
  •  Compare with Xsens joint estimation, and 2D video + goniometer data 
•  Read through Xsens literature to better understand their biomechanical model

...

1. Xsens Technologies (2013). Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors.

2. Gait 2392 and 2354 Models - OpenSim Documentation

3. Ferrari A, Cutti A G and Cappello A 2010a A new formulation of the coefficient of multiple correlation to assess the similarity of waveforms measured synchronously by different motion analysis protocols Gait Posture 31 540–2 

4. Zhang, J. T., Novak, A. C., Brouwer, B., & Li, Q. (2013). Concurrent validation of Xsens MVN measurement of lower limb joint angular kinematics. Physiological measurement, 34(8), N63.

5. Morris, M., Iansek, R., McGinley, J., Matyas, T. and Huxham, F. (2005), Three-dimensional gait biomechanics in Parkinson's disease: Evidence for a centrally mediated amplitude regulation disorder. Mov. Disord., 20: 40–50. doi:10.1002/mds.20278

6. van Gent, R.N. et al. (2007). Incidence and determinants of lower extremity running injuries in long distance runners: A systematic review. British Journal of Sports Medicine.

7. Reenalda, J., Maartens, E., Homan, L., & Buurke, J. J. (2016). Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics. Journal of Biomechanics, 49(14), 3362-3367.