rPPG Remote Heart Rate Detection

Traditional heart rate monitoring requires physical contact with the skin (e.g., pulse oximeters, ECG electrodes). Remote photoplethysmography (rPPG) aims to extract the blood volume pulse signal from subtle color changes in facial skin captured by a standard camera. The key challenges are separating the weak rPPG signal from motion artifacts, illumination changes, and skin tone variation.

The model was trained and evaluated on the UBFC-rPPG dataset, which contains 42 video recordings of subjects performing a cognitive task while their ground-truth heart rate was recorded with a CMS50E pulse oximeter. Videos are recorded at 30 fps in uncompressed format. Additional evaluation was performed on the MAHNOB-HCI dataset for cross-dataset generalization.

The pipeline consists of three stages: (1) face detection and ROI extraction using a lightweight RetinaFace detector, (2) spatial averaging of RGB channels over the facial ROI to produce a raw color signal, and (3) a 1D convolutional neural network (PhysNet) that maps the raw color signal to a clean BVP (blood volume pulse) waveform. Heart rate is estimated by finding the dominant frequency in the BVP signal via FFT.

graph LR
  A[Webcam Video] --> B[Face Detection\nRetinaFace]
  B --> C[ROI Extraction\nForehead + Cheeks]
  C --> D[Spatial Averaging\nRGB Signal]
  D --> E[PhysNet\n1D CNN]
  E --> F[BVP Waveform]
  F --> G[FFT Peak Detection]
  G --> H[Heart Rate BPM]

PhysNet was trained end-to-end using a negative Pearson correlation loss between the predicted and ground-truth BVP signals. Training used the Adam optimizer (lr = 1e-4) with a batch size of 4 video clips of 300 frames each. Data augmentation included random temporal cropping, horizontal flipping, and brightness/contrast jitter to improve robustness to illumination changes.

On the UBFC-rPPG test set, the system achieves a Mean Absolute Error (MAE) of 2.1 BPM and a Root Mean Square Error (RMSE) of 3.4 BPM, outperforming traditional signal processing baselines (CHROM, POS) by 35%. Cross-dataset evaluation on MAHNOB-HCI yields an MAE of 5.8 BPM, demonstrating reasonable generalization. Real-time inference runs at 28 fps on a standard laptop CPU.