A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences

Shahab Asoodeh; Jiachun Liao; Flavio P. Calmon; Oliver Kosut; Lalitha Sankar

doi:10.1109/ISIT44484.2020.9174015

A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences

Shahab Asoodeh, Jiachun Liao, Flavio P. Calmon, Oliver Kosut, Lalitha Sankar

Engineering, Ira A. Fulton Schools of (IAFSE)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

15 Scopus citations

Abstract

We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Renyí differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Renyi variations of differential privacy. We apply our result tó the moments accountant framework for characterizing privacy guarantees of stochastic gradient descent. When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget.

Original language	English (US)
Title of host publication	2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	920-925
Number of pages	6
ISBN (Electronic)	9781728164328
DOIs	https://doi.org/10.1109/ISIT44484.2020.9174015
State	Published - Jun 2020
Event	2020 IEEE International Symposium on Information Theory, ISIT 2020 - Los Angeles, United States Duration: Jul 21 2020 → Jul 26 2020

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
Volume	2020-June

Conference

Conference	2020 IEEE International Symposium on Information Theory, ISIT 2020
Country/Territory	United States
City	Los Angeles
Period	7/21/20 → 7/26/20

ASJC Scopus subject areas

Theoretical Computer Science
Information Systems
Modeling and Simulation
Applied Mathematics

Access to Document

10.1109/ISIT44484.2020.9174015

Cite this

Asoodeh, S., Liao, J., Calmon, F. P., Kosut, O., & Sankar, L. (2020). A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences. In 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings (pp. 920-925). Article 9174015 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2020-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT44484.2020.9174015

A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences. / Asoodeh, Shahab; Liao, Jiachun; Calmon, Flavio P. et al.
2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. p. 920-925 9174015 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2020-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Asoodeh, S, Liao, J, Calmon, FP, Kosut, O & Sankar, L 2020, A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences. in 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings., 9174015, IEEE International Symposium on Information Theory - Proceedings, vol. 2020-June, Institute of Electrical and Electronics Engineers Inc., pp. 920-925, 2020 IEEE International Symposium on Information Theory, ISIT 2020, Los Angeles, United States, 7/21/20. https://doi.org/10.1109/ISIT44484.2020.9174015

Asoodeh S, Liao J, Calmon FP, Kosut O , Sankar L. A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences. In 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2020. p. 920-925. 9174015. (IEEE International Symposium on Information Theory - Proceedings). doi: 10.1109/ISIT44484.2020.9174015

Asoodeh, Shahab ; Liao, Jiachun ; Calmon, Flavio P. et al. / A Better Bound Gives a Hundred Rounds : Enhanced Privacy Guarantees via f-Divergences. 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 920-925 (IEEE International Symposium on Information Theory - Proceedings).

@inproceedings{14e09fef20604501aba130399e0a123d,

title = "A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences",

abstract = "We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Reny{\'i} differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Renyi variations of differential privacy. We apply our result t{\'o} the moments accountant framework for characterizing privacy guarantees of stochastic gradient descent. When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget.",

author = "Shahab Asoodeh and Jiachun Liao and Calmon, {Flavio P.} and Oliver Kosut and Lalitha Sankar",

note = "Funding Information: This work was supported in part by NSF under grants CIF 1922971, 1815361, 1742836, 1900750, and CIF CAREER 1845852. Publisher Copyright: {\textcopyright} 2020 IEEE.; 2020 IEEE International Symposium on Information Theory, ISIT 2020 ; Conference date: 21-07-2020 Through 26-07-2020",

year = "2020",

month = jun,

doi = "10.1109/ISIT44484.2020.9174015",

language = "English (US)",

series = "IEEE International Symposium on Information Theory - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "920--925",

booktitle = "2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings",

}

TY - GEN

T1 - A Better Bound Gives a Hundred Rounds

T2 - 2020 IEEE International Symposium on Information Theory, ISIT 2020

AU - Asoodeh, Shahab

AU - Liao, Jiachun

AU - Calmon, Flavio P.

AU - Kosut, Oliver

AU - Sankar, Lalitha

PY - 2020/6

Y1 - 2020/6

N2 - We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Renyí differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Renyi variations of differential privacy. We apply our result tó the moments accountant framework for characterizing privacy guarantees of stochastic gradient descent. When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget.

AB - We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Renyí differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Renyi variations of differential privacy. We apply our result tó the moments accountant framework for characterizing privacy guarantees of stochastic gradient descent. When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget.

UR - http://www.scopus.com/inward/record.url?scp=85090406130&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090406130&partnerID=8YFLogxK

U2 - 10.1109/ISIT44484.2020.9174015

DO - 10.1109/ISIT44484.2020.9174015

M3 - Conference contribution

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 920

EP - 925

BT - 2020 IEEE International Symposium on Information Theory, ISIT 2020 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 21 July 2020 through 26 July 2020

ER -

A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this