Decorrelating Language Model Embeddings for Speech-Based Prediction of Cognitive Impairment

Lingfeng Xu, Kimberly D. Mueller, Julie Liss, Visar Berisha

Research output: Contribution to journalConference articlepeer-review


Training robust clinical speech-based models that generalize requires large sample sizes because speech is variable and high-dimensional. Researchers have turned to foundational models, such as the Bidirectional Encoder Representations from Transformers (BERT), to generate lower-dimensional embeddings, and then finetuned the models for a specific down-stream clinical task. While there is empirical evidence that this approach is helpful, a recent study reveals that the embeddings generated by BERT models tend to be highly correlated, which makes the downstream models difficult to fine-tune, particularly in the small sample size regime. In this work, we propose a new regularization scheme to penalize correlated embeddings during fine tuning of BERT and apply the approach to speech-based assessment of cognitive impairment. Compared to existing methods, the proposed method yields lower estimation errors and smaller false alarm rates in a Mini-Mental State Examination (MMSE) score regression task.


  • clinical speech analytics
  • decorrelated features
  • Language modeling

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering


Dive into the research topics of 'Decorrelating Language Model Embeddings for Speech-Based Prediction of Cognitive Impairment'. Together they form a unique fingerprint.

Cite this