A Neuro-Symbolic Spatiotemporal Vision Transformer for Zero-Shot Safety Behavior Recognition

This study evaluates video-level behavior recognition on the seven-class Construction Meta Action (CMA) benchmark using a neuro-symbolic spatiotemporal Vision Transformer. An MViTv2-S backbone feeds a neural classification head and eight weakly supervised predicate variables, which are scored against fixed class prototypes; a temporal extension uses six log-domain operators over four ordered segments. On 540 official test videos, the fused model achieves 93.35% Macro-AP, compared with 93.23% for the neural branch. Zero-shot recognition is evaluated as leave-one-class-out transfer within CMA: the target class is excluded from task-specific head training and model selection, while Kinetics pretraining and target-class prototypes remain available. Mean held-out AP is 47.66% for the rule branch and 44.69% for the neural branch, without a statistically established difference. Log-domain computation maintains finite outputs and gradients under the reported stress conditions. The measured offline cost is 199.36 ms/video for single-clip file processing and approximately 1.17 s/video for four-segment extraction on an RTX 3080 Ti laptop GPU. The findings support inspection of prototype-based score construction and numerical behavior within the evaluated benchmark. Predicate semantics are not independently annotated, predictions are not attributed to individual workers, and cross-site generalization and live-stream performance have not been validated.