Large-scale cross-sectional and longitudinal validation of a digital speech-based measure of cognition

Gabriela M. Stegmann, Shira Hahn, Julie Liss, Visar Berisha, Kimberly D. Mueller

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


BACKGROUND: Cognitive decline is associated with deficits in attention to tasks and attention to relevant details. We developed a metric, semantic relevance (SemR), which is algorithmically extracted from speech and measures overlap between a picture's content and the words used to describe the picture. In this study, we validate it in a sample that was not used when developing it. We automatically extract SemR from transcripts of Cookie Theft (BDAE) and evaluate its cross-sectional and longitudinal clinical validity using four groups: Normal Cognition (NC), Early MCI (EMCI), MCI, and Dementia (D). METHOD: Dementia Bank and Wisconsin Registry for Alzheimer's Prevention (WRAP) were combined, and participants (average age 63.7) included: NC (N = 918; 647 F), EMCI (n = 180; 110 F), MCI (n = 26, 9 F), and D (n = 195, 126 F). Participants provided Cookie Theft descriptions and Mini-Mental State Exam (MMSE) assessments on an average of 2.1 occasions/participant, assessed on average 2.4 years apart (total n=2,717). SemR was algorithmically computed from each picture description transcript. Transcripts were also hand-coded for "content information units" as a ground-truth comparison with automated SemR. Cross-sectionally, we used a mixed-effects model to calculate relationships between SemR and ground truth, and between SemR and MMSE. We estimated within-speaker SemR longitudinal trajectories using growth curve models (GCM) for each group. RESULT: (Figure 1) Automatic and hand-coded SemR were strongly correlated (r = .85, p<.05). (Figure 2) SemR was significantly related to MMSE (b = .002, p<.05), such that decrease in MMSE resulted in decrease in SemR. (Figure 3) Longitudinal GCMs showed that SemR declined with age for all groups. The decline was slowest for NCs, steepened for the EMCI and MCI groups, and then slowed again for D, who had the lowest scores. Figure 3 displays SemR trajectories and confidence bands for age ranges with the most data for each group. SemR has a standard error of measurement (SEM) of .05. CONCLUSION: SemR is reliable, shows convergent validity with MMSE, and correlates strongly with manual hand-counts. SemR declines with age and severity of cognitive impairment, with the speed of decline differing by level of impairment.

Original languageEnglish (US)
Pages (from-to)e056199
JournalAlzheimer's and Dementia
StatePublished - Dec 1 2021

ASJC Scopus subject areas

  • Clinical Neurology
  • Geriatrics and Gerontology
  • Psychiatry and Mental health
  • Cellular and Molecular Neuroscience
  • Health Policy
  • Developmental Neuroscience
  • Epidemiology


Dive into the research topics of 'Large-scale cross-sectional and longitudinal validation of a digital speech-based measure of cognition'. Together they form a unique fingerprint.

Cite this