A study, published on Sept. 23 in npj Parkinson’s Disease, found that Apple’s augmented reality technology could detect reduced facial expression, called hypomimia, often described as facial masking in Parkinson’s patients. Hypomimia can make it seem like they show no emotion, even though the opposite is true when they try to convey normal emotions. Someone with Parkinson’s may appear to smile noticeably less or blink less frequently than someone without it. The research team at the University of Bonn and the German Center for Neurodegenerative Disease used an iPad Pro and its TrueDepth camera, and a custom application called ExpressionTracker, collecting 261 measurements from the movements of 70 participants: 34 with Parkinson’s and 36 healthy controls in less than 10 minutes. Using machine learning, the model was able to distinguish those with Parkinson’s from those without, with a 79% accuracy.
ARKit can use the TrueDepth camera to create and track a three-dimensional representation of a person’s face, allowing it to measure changes in the movements of the eyebrows, cheeks, lips, and mouth in real time using what Apple calls “blendshapes.” These are numerical representations of individual movements created when someone raises their eyebrow, moves their mouth, or closes an eye.
The results of the study identified six facial features that differed significantly between the control group and those with the disease. People with Parkinson’s showed a significantly reduced range of lower-lip movement and greater asymmetry in the corners of the mouth. Movement of the right eye also differed in how it interacted with movement of the right lower lip. This means the iPad wasn’t just detecting facial movements; it was detecting where movements were reduced, whether the two sides of the face behaved differently, and how separate regions of the face moved together.
To take the test even further, researchers fed the facial movements into eight different machine learning models and asked whether they could distinguish the healthy controls from the Parkinson’s patients. The best-performing of these models scored an AUC of 0.834, meaning it could distinguish Parkinson’s patients from healthy controls 83.4% of the time, with other models correctly identifying 25 of the 34 people with Parkinson’s and 30 of the 36 people who did not. Nine People with Parkinson’s were missed, and six healthy people were incorrectly identified as having the disease. When researchers looked only at how much the face moved, the model wasn’t as good at telling the two groups apart. The same was true when they looked only at a person’s age and sex. So while great leaps in using the iPad were made, it’s still a work in progress for the researchers.
Comparing an iPad With the MDS-UPDRS
Software developments like these further the argument that the MDS-UPDRS needs to be updated to encompass the scope of technology when it comes to Parkinson’s disease and diagnosis. The MDS-UPDRS, or Movement Disorder Society-Unified Parkinson’s Disease Rating Scale, is a series of tests neurologists use to measure the emotional and physical effects of Parkinson’s. For facial recognition in section 3.2 of the test, clinicians evaluate blinking, movement around the mouth, spontaneous smiling, and whether the lips remain parted.
With ExpressionTracker, the software produces a numerical value for the specific movements it tracks, similar to what a neurologist sees during a visit, putting the TrueDepth camera’s results more in line with observations made by a clinician during a typical neurological exam. It explains about 42% of the variation in MDS-UPDRS facial-expression scores. Right-sided smiling alone accounted for about 38% of the Facial Parkinson’s Test, making the two approaches measure the same problem at very different levels of resolution.
Why Apple’s TrueDepth Camera Matters
Researchers point out that the TrueDepth system uses what’s called depth sensing, making measurements less dependent on ambient light and skin texture. ARKit adjusts its measurements to conform to each person’s face it scans, allowing researchers to compare facial movements from one person to another more consistently, making the technology easier to use in larger studies. This means people would be able to do this from their neurologist’s office or the comfort of their home with a compatible iPad Pro during a telehealth visit.
It’s Not a Parkinson’s App, Yet
ExpressionTracker is a research app only right now, so you can’t download it from the App Store. The research team still has a lot of hurdles to clear if they ever decide to. To release it commercially, they’d have to build and validate the measurement system, get FDA approval, provide Apple with evidence that it works, follow the process for declaring regulated medical-device status in the U.S., and meet Apple’s standard privacy and technical requirements.
The study provided another proof of concept that existing software and hardware can help diagnose and treat patients, and it has a strong head start.
Apple Limitations
ARKit is proprietary, closed-source software developed primarily for consumer facial tracking and animation, not clinical measurement. Researchers cannot look under the hood to see how the software was put together or how well it was designed to work with people with Parkinson’s. However, with Apple’s focus on health and the Apple Watch, many potential applications could one day offer apps to medical professionals and make them part of routine checkups, telehealth visits, and early detection. researchers in this study say a system built specifically for this kind of testing, rather than relying on Apple’s technology, could eventually be a better option.
You can read the full study and get details here
Discover more from
Subscribe to get the latest posts sent to your email.
