A recurrent neural network and parallel hidden Markov model algorithm to segment and detect heart murmurs in phonocardiograms

Fig 1. Parallel HSMM algorithm structure. A heart sound recording is transformed into log-spectrogram features, which are then input to an RNN. The RNN probabilities are then used to produce four segmentations using four parallel HSMMs. The confidence in these segmentations is then compared to give the most probable segmentation, murmur prediction, and a signal quality estimate.

Fig 2. Neural network prediction of per-time heart sound categories. The heart sound recording (top) is transformed to a normalised spectrogram (middle) that is then passed to a RNN that predicts the state corresponding to each time window (bottom). The RNN is able to use the spectrogram to correctly distinguish between S1, S2, and the systolic murmur. The RNN also predicts systolic and diastolic states which are omitted from the bottom figure for clarity.

Fig 3. CatBoost algorithm structure to predict clinical outcome. The parallel HSMM algorithm described in Fig 1 is applied to each recording to generate murmur likelihood C(M−N) and signal quality C(ω^) metrics. These metrics are then combined with additional patient information and input to a CatBoost decision tree to predict the patient’s final clinical outcome.

Fig 4. Separation of murmur, normal, and poor signal quality signals using HSMM confidences. The difference between the normal HSMM confidence, C(N), and the most confident murmur HSMM, C(M), is used to predict murmur likelihood C(M−N), whilst the most confident overall HSMM C(ω^) is used as an indication of signal quality. The horizontal and vertical lines show thresholds that have been chosen to separate the different classes, optimised for the challenge task. The marker type indicates the ground-truth labels.

Fig 5. Four example recordings analysed by the algorithm. Three-second windows are shown for each recording, with their database ID shown in the top right. Recording (a) contains a strong systolic murmur that is confidently detected by the algorithm (C(M−N) = 0.25). (b) is a healthy signal that is correctly identified (C(M−N) = −0.10). (c) contains significant talking and other noise, and is marked as ‘Unknown’ by the clinician. The algorithm correspondingly rejects the signal with a low confidence of C(ω^)=0.49. (d) contains a lower amplitude signal with some noise that is marked as ‘Unknown’ by the clinician, but is segmented by the algorithm with a very high confidence of C(ω^)=0.91.

Fig 6. Receiver operating characteristic curve for murmur detection algorithm. The sensitivity and specificity of the parallel HSMM model is shown as the threshold confidence difference C(M−N) varies. The normal operating point of the algorithm (i.e. decide murmur if C(M−N) > 0) is also shown. A 95% confidence interval (CI) in the curve has been calculated through bootstrap resampling with 1000 iterations. Patients with an ‘unknown’ murmur label have been removed.

Fig 7. Reliability diagram for prediction of murmurs using HSMM confidence difference. As the confidence difference C(M−N) increases, so does the relative frequency of murmurs with an almost linear relationship (dashed line). This suggests good calibration of the murmur likelihood output.

Fig 8. Performance of algorithms and clinician on clinical outcomes task. A receiver operating characteristic (ROC) curve for the CatBoost algorithm at predicting clinical outcome is shown. Also shown are operating points for the clinician (using the murmur label) and the parallel HSMM murmur detection algorithm. The colourmap shows how sensitivity and specificity relate to the PhysioNet challenge cost score. On the training set, an optimum random classifier can achieve a cost score of 12581.

Fig 9. Training and test performance of all the official entries on the clinical outcomes task. A lower score is better. Many algorithms had significantly better performance on the training set, indicating overfitting has taken place. The diagonal line shows ideal performance where training and test performance are equal. The parallel HSMM algorithm achieves the best score and shows little evidence of overfitting. For context, the performance of a random classifier at an optimal operating point is also shown.