Benchmark of DNN Model Search at Deployment Time

Lixi Zhou, Arindam Jain, Zijie Wang, Amitabh Das, Yingzhen Yang, Jia Zou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep learning has become the most popular direction in machine learning and artificial intelligence. However, the preparation of training data, as well as model training, are often time-consuming and become the bottleneck of the end-to-end machine learning lifecycle. Reusing models for inferring a dataset can avoid the costs of retraining. However, when there are multiple candidate models, it is challenging to discover the right model for reuse. Although there exist a number of model sharing platforms such as ModelDB, TensorFlow Hub, PyTorch Hub, and DLHub, most of these systems require model uploaders to manually specify the details of each model and model downloaders to screen keyword search results for selecting a model. We are lacking a highly productive model search tool that selects models for deployment without the need for any manual inspection and/or labeled data from the target domain. This paper proposes multiple model search strategies including various similarity-based approaches and non-similarity-based approaches. We design, implement and evaluate these approaches on multiple model inference scenarios, including activity recognition, image recognition, text classification, natural language processing, and entity matching. The experimental evaluation showed that our proposed asymmetric similarity-based measurement, adaptivity, outperformed symmetric similarity-based measurements and non-similarity-based measurements in most of the workloads.

Original languageEnglish (US)
Title of host publicationScientific and Statistical Database Management - 34th International Conference, SSDBM 2022 - Proceedings
EditorsElaheh Pourabbas, Yongluan Zhou, Yuchen Li, Bin Yang
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450396677
DOIs
StatePublished - Jul 6 2022
Event34th International Conference on Scientific and Statistical Database Management, SSDBM 2022 -
Duration: Jul 6 2022 → …

Publication series

NameACM International Conference Proceeding Series

Conference

Conference34th International Conference on Scientific and Statistical Database Management, SSDBM 2022
Period7/6/22 → …

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint

Dive into the research topics of 'Benchmark of DNN Model Search at Deployment Time'. Together they form a unique fingerprint.

Cite this