First Results of our Mobile Health Innovation Project

Last year we received an innovation grant from INNOLABS, for our project H2A2 – A Healthy Heart with Automated Assistance – to create an unobtrusive health monitoring tool. With an innovative technique, called remote photoplethysmography (remote PPG), heart rate can be detected from the face. This functionality is already available in FaceReader. Since this technique requires high quality recordings, we wanted to test whether it is also accurate when the camera of a mobile device was used. Together with our partner PLUX, a Portuguese company specialized in advanced biosignals monitoring platforms, and a Portuguese telecommunication company IT, we collected physiological ground truth data and video recordings from a tablet. This data can validate the heart rate assessment and emotion classification on a mobile device.

Screenshot from a FaceReader analysis window from a mobile device.
Scatter plot of raw heart rate data (measured with ECG) and computer estimated heart rate (RPPG) from all 2 sessions of the analysed participants.


Accurate Heart Rate Assessments from a Mobile Device

The experiments are still ongoing, but we are happy to report the preliminary results. The first heart rate analyses are successful. We can accurately assess heart rate from a video recording of a mobile device. Of all the video’s 82% of the heart rate assessments were within 10 beats of the ECG heart rate (Figure 1). The detailed analysis of a few participants shows this can even be accurately assessed over short time intervals (Figure 2). The unsuccessful analyses gives us further knowledge to hopefully reduce error rates in the future. In addition, from the FaceReader emotion analysis we see there are some differences in resting state emotional expressiveness between high risk anxious and low risk anxious people. A lot of work has to be done, but the fact differences exist, is a promising first step for a rule-based and machine learning stress prediction approach.


Figure 2. Recordings from two different participants with ground truth recording from the ECG and estimated heart rate with RPPG from FaceReader (note, the discrepancy on the right graph is actually due to noise in the ECG).

Healthcare Demo Days in Oslo

Carrying out this study gave us many useful insights (e.g. best practices for recruitment, lighting and video setup, software synchronization, etc.). Eventually this valuable dataset will help us bring our heart rate assessment to the next level, make it possible to design a new machine learning based stress prediction algorithm, and extend the possibilities of our software to mobile platforms.

Thursday will be the final INNOLABS event in Oslo that features many innovative solutions for patient centric healthcare and remote care delivery. Here, we will showcase our innovation, together with 60 other successful SMEs. This will be a nice opportunity for additional insights into our innovation and for finding interesting collaborations. In the future, we hope to further develop our application as a research and eHealth application. This way we hope to help address mental health issues.