Addressing imaging accessibility by cross-modality transfer learning

Zhiyang Zheng, Yi Su, Kewei Chen, David A. Weidman, Teresa Wu, Ben Lo, Fleming Lure, Jing Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Multi-modality images usually exist for diagnosis/prognosis of a disease, such as Alzheimer's Disease (AD), but with different levels of accessibility and accuracy. MRI is used in the standard of care, thus having high accessibility to patients. On the other hand, imaging of pathologic hallmarks of AD such as amyloid-PET and tau-PET has low accessibility due to cost and other practical constraints, even though they are expected to provide higher diagnostic/prognostic accuracy than standard clinical MRI. We proposed Cross-Modality Transfer Learning (CMTL) for accurate diagnosis/prognosis based on standard imaging modality with high accessibility (mod-HA), with a novel training strategy of using not only data of mod-HA but also knowledge transferred from the model based on advanced imaging modality with low accessibility (mod-LA). We applied CMTL to predict conversion of individuals with Mild Cognitive Impairment (MCI) to AD using the Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets, demonstrating improved performance of the MRI (mod-HA)-based model by leveraging the knowledge transferred from the model based on tau-PET (mod-LA).

Original languageEnglish (US)
Title of host publicationMedical Imaging 2022
Subtitle of host publicationComputer-Aided Diagnosis
EditorsKaren Drukker, Khan M. Iftekharuddin
ISBN (Electronic)9781510649415
StatePublished - 2022
EventMedical Imaging 2022: Computer-Aided Diagnosis - Virtual, Online
Duration: Mar 21 2022Mar 27 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE


ConferenceMedical Imaging 2022: Computer-Aided Diagnosis
CityVirtual, Online


  • Alzheimer s disease
  • Knowledge distillation
  • Mild cognitive impairment
  • Multi-modality images
  • Transfer learning

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Addressing imaging accessibility by cross-modality transfer learning'. Together they form a unique fingerprint.

Cite this