ViViEchoformer: Deep Video Regressor Predicting Ejection Fraction

Figures

• 

  Fig. 1. Comparative distribution of EF values in the training dataset before and after data augmentation and down-sampling techniques. The initial dataset (a) consisted of 7462 samples, while the augmented dataset (b) expanded to 30,787 samples, illustrating the effect of augmentation and balancing strategies on the EF value distribution. c, d The spread of EF before and after augmentation in the training dataset

• 

  Fig. 2. Model evaluation. a Comparison of ViViEchoformer predicted, and EchoNet dataset reported ejection fractions (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n=1288$$\end{document}n=1288). b The violin plot showcasing the model error distribution. c Error distribution histogram. d Error values across samples. e Autocorrelation plot of residuals. f Partial autocorrelation of the residuals

• 

  Fig. 3. Sequential visualization of echocardiogram frame preprocessing. The top row (a–k) displays the first 11 original frames from the echocardiogram video, demonstrating the raw imaging data. After applying our preprocessing steps, the bottom row (a–k) illustrates the corresponding frames: median frame calculation, frame-wise multiplication, histogram equalization, and median blur filtering. The processed frames reveal a marked enhancement in the definition and contrast of cardiac structures, providing a clear visual distinction from the original frames and underscoring the efficacy of the preprocessing technique

• 

  Fig. 4. The layered structure of a neural network model. The diagram showcases the arrangement of various layers, including multi-head attention, encoder, and dense layers, illustrating the flow of information within the model

• 

  Fig. 5. The model pipeline for video regression. The Tubelet embedding technique extracts and linearly embeds nonoverlapping tubelets across the spatiotemporal input volume. Using spatial–temporal attention, the transformer encoder forwards all spatiotemporal tokens extracted from the video

• 

  Fig. 6. The graph illustrates the model’s loss over epochs for training (blue) and validation (orange) datasets

• 

  Fig. 7. Classification accuracy of 85 and AUC of 0.83 using a standard threshold of less than 50% EF. a ROC curve. b Confusion matrix where HFrEF, labeled as 0 and non-HFrEF labeled 1

• 

  Fig. 8. Scatter plot of true vs. predicted LVEF values by the regression model, illustrating classification accuracy relative to the 50% threshold. The green area represents correct classifications, while the orange area signifies incorrect classifications. The highlighted square around the 50% line delineates the zone of uncertainty, emphasizing the model’s challenge in making near-threshold predictions