This report introduces VitalLens, an app that estimates vital signs such as heart rate and respiration rate from selfie video in real time. VitalLens uses a computer vision model trained on a diverse dataset of video and physiological sensor data. We benchmark performance on several diverse datasets, including VV-Medium, which consists of 289 unique participants. VitalLens outperforms several existing methods including POS and MTTS-CAN on all datasets while maintaining a fast inference speed. On VV-Medium, VitalLens achieves mean absolute errors of 0.71 bpm for heart rate estimation, and 0.76 bpm for respiratory rate estimation.
Respiration is a critical vital sign for infants, and continuous respiratory monitoring is particularly important for newborns. However, neonates are sensitive and contact-based sensors present challenges in comfort, hygiene, and skin health, especially for preterm babies. As a step toward fully automatic, continuous, and contactless respiratory monitoring, we develop a deep-learning method for estimating respiratory rate and waveform from plain video footage in natural settings. Our automated infant respiration flow-based network (AIRFlowNet) combines video-extracted optical flow input and spatiotemporal convolutional processing tuned to the infant domain. We support our model with the first public annotated infant respiration dataset with 125 videos (AIR-125), drawn from eight infant subjects, set varied pose, lighting, and camera conditions. We include manual respiration annotations and optimize AIRFlowNet training on them using a novel spectral bandpass loss function. When trained and tested on the AIR-125 infant data, our method significantly outperforms other state-of-the-art methods in respiratory rate estimation, achieving a mean absolute error of $\sim$2.9 breaths per minute, compared to $\sim$4.7--6.2 for other public models designed for adult subjects and more uniform environments.
Respiratory rate (RR) is an important biomarker as RR changes can reflect severe medical events such as heart disease, lung disease, and sleep disorders. Unfortunately, standard manual RR counting is prone to human error and cannot be performed continuously. This study proposes a method for continuously estimating RR, RRWaveNet. The method is a compact end-to-end deep learning model which does not require feature engineering and can use low-cost raw photoplethysmography (PPG) as input signal. RRWaveNet was tested subject-independently and compared to baseline in four datasets (BIDMC, CapnoBase, WESAD, and SensAI) and using three window sizes (16, 32, and 64 seconds). RRWaveNet outperformed current state-of-the-art methods with mean absolute errors at optimal window size of 1.66 \pm 1.01, 1.59 \pm 1.08, 1.92 \pm 0.96 and 1.23 \pm 0.61 breaths per minute for each dataset. In remote monitoring settings, such as in the WESAD and SensAI datasets, we apply transfer learning to improve the performance using two other ICU datasets as pretraining datasets, reducing the MAE by up to 21$\%$. This shows that this model allows accurate and practical estimation of RR on affordable and wearable devices. Our study also shows feasibility of remote RR monitoring in the context of telemedicine and at home.
Estimating human vital signs in a contactless non-invasive method using radar provides a convenient method in the medical field to conduct several health checkups easily and quickly. In addition to monitoring while sitting and sleeping, the standing position has aroused interest for both the industrial and medical fields. However, it is more challenging due to the micro motions induced by the body for balancing that may cause false respiratory rate estimation. In this work, we focus on the measurement of the respiratory rate of a standing person accurately with the capability of heavy breath detection and estimation. Multiple estimation approaches are presented and compared, including spectral estimation, deep-learning-based approaches, and adaptive peak selection with Kalman filtering. The latest technique is showing the best performance with an absolute error rate of 1.5 bpm, when compared to a Vernier Go Direct\textsuperscript{\textregistered} respiration belt.
Heart rate and respiratory rate measurement is a vital step for diagnosing many diseases. Non-contact camera based physiological measurement is more accessible and convenient in Telehealth nowadays than contact instruments such as fingertip oximeters since non-contact methods reduce risk of infection. However, remote physiological signal measurement is challenging due to environment illumination variations, head motion, facial expression, etc. It's also desirable to have a unified network which could estimate both heart rate and respiratory rate to reduce system complexity and latency. We propose a convolutional neural network which leverages spatial attention and channel attention, which we call it dual attention network (DAN) to jointly estimate heart rate and respiratory rate with camera video as input. Extensive experiments demonstrate that our proposed system significantly improves heart rate and respiratory rate measurement accuracy.
This paper presents a method that estimates the respiratory rate based on the frame capturing of wireless local area networks. The method uses beamforming feedback matrices (BFMs) contained in the captured frames, which is a rotation matrix of channel state information (CSI). BFMs are transmitted unencrypted and easily obtained using frame capturing, requiring no specific firmware or WiFi chipsets, unlike the methods that use CSI. Such properties of BFMs allow us to apply frame capturing to various sensing tasks, e.g., vital sensing. In the proposed method, principal component analysis is applied to BFMs to isolate the effect of the chest movement of the subject, and then, discrete Fourier transform is performed to extract respiratory rates in a frequency domain. Experimental evaluation results confirm that the frame-capture-based respiratory rate estimation can achieve estimation error lower than 3.2 breaths/minute.
Respiratory rate (RR) is a clinical sign representing ventilation. An abnormal change in RR is often the first sign of health deterioration as the body attempts to maintain oxygen delivery to its tissues. There has been a growing interest in remotely monitoring of RR in everyday settings which has made photoplethysmography (PPG) monitoring wearable devices an attractive choice. PPG signals are useful sources for RR extraction due to the presence of respiration-induced modulations in them. The existing PPG-based RR estimation methods mainly rely on hand-crafted rules and manual parameters tuning. An end-to-end deep learning approach was recently proposed, however, despite its automatic nature, the performance of this method is not ideal using the real world data. In this paper, we present an end-to-end and accurate pipeline for RR estimation using Cycle Generative Adversarial Networks (CycleGAN) to reconstruct respiratory signals from raw PPG signals. Our results demonstrate a higher RR estimation accuracy of up to 2$\times$ (mean absolute error of 1.9$\pm$0.3 using five fold cross validation) compared to the state-of-th-art using a identical publicly available dataset. Our results suggest that CycleGAN can be a valuable method for RR estimation from raw PPG signals.
We develop and evaluate a respiratory rate estimation algorithm that utilizes data from pressure-sensitive mat (PSM) technology for continuous patient monitoring in neonatal intensive care units (NICU). An analysis of the random effect of drift and systematic effect of creep in the PSM data is presented, showing that these are essentially dependent on the applied load and contact surface. Uncertainty measurements are pivotal when estimating physiologic parameters. The standard uncertainty in the PSM data is here represented by the percent drift. Next, we evaluate the applicability of PSM technology to estimate RR in neonatal patient simulator trials under five mixed effects including internally and externally induced motion, mattress type, grunting, laying position, and different breathing rates. We analyze the limits of agreement on the mixed effects model to derive the uncertainty in the estimated RR obtained through two estimation techniques. In comparison with the gold standard RR values, we achieved a mean bias of 0.56 breaths per minute (bpm) with an error bounded by a 95% confidence interval of [-2.26, 3.37] bpm. These results meet the clinical accuracy requirements of RR within +/-5 bpm.
Vital signs, such as heart rate (HR), heart rate variability (HRV), respiratory rate (RR), are important indicators for a person's health. Vital signs are traditionally measured with contact sensors, and may be inconvenient and cause discomfort during continuous monitoring. Commercial cameras are promising contact-free sensors, and remote photoplethysmography (rPPG) have been studied to remotely monitor heart rate from face videos. For remote RR measurement, most prior art was based on small periodical motions of chest regions caused by breathing cycles, which are vulnerable to subjects' voluntary movements. This paper explores remote RR measurement based on rPPG obtained from face videos. The paper employs motion compensation, two-phase temporal filtering, and signal pruning to capture signals with high quality. The experimental results demonstrate that the proposed framework can obtain accurate RR results and can provide HR, HRV and RR measurement synergistically in one framework.