Humanizing “multimodal” digital phenotyping of serious mental illness.

for video/audio analysis = 109/912 and 120/2,200, respectively), but most features failed to show acceptable reliability. Videos were behaviorally rated for alogia and blunted vocal and facial affect. Relationships between nonverbal features and human ratings were dependent on verbal tone, but only when verbal tone was positive in valence. For example, behaviorally rated alogia was associated with longer pauses, but primarily when language had a positive tone. These results support a relatively novel approach to multimodal digital phenotyping, one that emphasizes using multimodal features to provide "context" in the way that humans likely interpret and integrate multimodal streams of information. (PsycInfo Database Record (c) 2026 APA, all rights reserved).