Federated
learning is a distributed machine learning technique that trains a global model
by exchanging model parameters or intermediate results among multiple data
sources. Although federated learning achieves physical isolation of data, the
local data of federated learning clients are still at risk of leakage under the attack of malicious individuals. For this
reason, combining data protection techniques (e.g., differential privacy
techniques) with federated learning is a sure way to further improve the
data security of federated learning models. In this survey, we review recent
advances in the research of differentially-private federated learning models.
First, we introduce the workflow of federated learning and the theoretical
basis of differential privacy. Then, we
review three differentially-private federated learning paradigms: central
differential privacy, local differential privacy, and distributed differential
privacy. After this, we review the algorithmic optimization and communication
cost optimization of federated learning models with differential privacy.
Finally, we review the applications of federated learning models with
differential privacy in various domains. By systematically summarizing the
existing research, we propose future research opportunities.
References
[1]
Bishop, C.M. and Nasrabadi, N.M. (2006) Pattern Recognition and Machine Learning. Springer, New York.
[2]
Nadkarni, P.M., Ohno-Machado, L. and Chapman, W.W. (2011) Natural Language Processing: An Introduction. Journal of the American Medical Informatics Association, 18, 544-551. https://doi.org/10.1136/amiajnl-2011-000464
[3]
Jarvis, R.A. (1983) A Perspective on Range Finding Techniques for Computer Vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5, 122-139. https://doi.org/10.1109/TPAMI.1983.4767365
[4]
Fatima, M. and Pasha, M. (2017) Survey of Machine Learning Algorithms for Disease Diagnostic. Journal of Intelligent Learning Systems and Applications, 9, 1-16. https://doi.org/10.4236/jilsa.2017.91001
[5]
Bolton, R.J. and Hand, D.J. (2002) Statistical Fraud Detection: A Review. Statistical Science, 17, 235-255. https://doi.org/10.1214/ss/1042727940
[6]
Zhao, W., Chellappa, R., Phillips, P.J. and Rosenfeld, A. (2003) Face Recognition: A Literature Survey. ACM Computing Surveys, 35, 399-458. https://doi.org/10.1145/954339.954342
[7]
Reddy, D.R. (1976) Speech Recognition by Machine: A Review. Proceedings of the IEEE, 64, 501-531. https://doi.org/10.1109/PROC.1976.10158
[8]
Ji, Z., Lipton, Z.C. and Elkan, C. (2014) Differential Privacy and Machine Learning: A Survey and Review.
[9]
El Ouadrhiri, A. and Abdelhadi, A. (2022) Differential Privacy for Deep and Federated Learning: A Survey. IEEE Access, 10, 22359-22380. https://doi.org/10.1109/ACCESS.2022.3151670
[10]
McMahan, B., Moore, E., Ramage, D., Hampson, S. and Arcas, B.A. (2017) Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, 20-22 April 2017, 1273-1282.
[11]
Ziller, A., Trask, A., Lopardo, A., et al. (2021) Pysyft: A Library for Easy Federated Learning. In: ur Rehman, M.H. and Gaber, M.M., Eds., Federated Learning Systems, Springer, Berlin, 111-139. https://doi.org/10.1007/978-3-030-70604-3_5
[12]
Liu, Y., Fan, T., Chen, T., Xu, Q. and Yang, Q. (2021) FATE: An Industrial Grade Platform for Collaborative Learning with Data Protection. Journal of Machine Learning Research, 22, 1-6.
[13]
Li, Q., Wen, Z., Wu, Z., et al. (2021) A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/TKDE.2021.3124599
[14]
Nasr, M., Shokri, R. and Houmansadr, A. (2019) Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-Box Inference Attacks against Centralized and Federated Learning. 2019 IEEE Symposium on Security and Privacy, San Francisco, 19-23 May 2019, 739-753. https://doi.org/10.1109/SP.2019.00065
[15]
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A. and Smith, V. (2020) Federated Optimization in Heterogeneous Networks. Proceedings of Machine Learning and Systems, Vol. 2, 429-450.
[16]
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A. and Pedarsani, R. (2020) Fedpaq: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization. International Conference on Artificial Intelligence and Statistics, 26-28 August 2020, 2021-2031.
[17]
So, J., Géler, B. and Avestimehr, A.S. (2021) Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning. IEEE Journal on Selected Areas in Information Theory, 2, 479-489. https://doi.org/10.1109/JSAIT.2021.3054610
[18]
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D. and Khazaeni, Y. (2020) Federated Learning with Matched Averaging.
[19]
Liu, L., Zhang, J., Song, S.H. and Letaief, K.B. (2020) Client-Edge-Cloud Hierarchical Federated Learning. ICC 2020 IEEE International Conference on Communications, Dublin, 7-11 June 2020, 1-6. https://doi.org/10.1109/ICC40277.2020.9148862
[20]
Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T. and Yu, H. (2019) Federated Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 13, Springer, Berlin, 1-207. https://doi.org/10.1007/978-3-031-01585-4
[21]
Ur Rehman, M.H. and Gaber, M.M. (2021) Federated Learning Systems: Towards Next-Generation AI. Springer Nature, Berlin. https://doi.org/10.1007/978-3-030-70604-3
[22]
Ludwig, H. and Baracaldo, N. (2022) Federated Learning: A Comprehensive Overview of Methods and Applications. Springer Nature, Berlin. https://doi.org/10.1007/978-3-030-96896-0
[23]
Dwork, C., McSherry, F., Nissim, K. and Smith, A. (2006) Calibrating Noise to Sensitivity in Private Data Analysis. In: Theory of Cryptography Conference, Springer, Berlin, 265-284. https://doi.org/10.1007/11681878_14
[24]
McSherry, F. and Talwar, K. (2007) Mechanism Design via Differential Privacy. 48th Annual IEEE Symposium on Foundations of Computer Science, Providence, 21-23 October 2007, 94-103. https://doi.org/10.1109/FOCS.2007.66
[25]
Nikolov, A., Talwar, K. and Zhang, L. (2013) The Geometry of Differential Privacy: The Sparse and Approximate Cases. Proceedings of the 45th Annual ACM Symposium on Theory of Computing, Palo Alto, 2-4 June 2013, 351-360. https://doi.org/10.1145/2488608.2488652
[26]
Dwork, C. (2008) Differential Privacy: A Survey of Results. International Conference on Theory and Applications of Models of Computation, Xi’an, 25-29 April 2008, 1-19. https://doi.org/10.1007/978-3-540-79228-4_1
[27]
Dwork, C. and Roth, A. (2014) The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science, 9, 211-407. https://doi.org/10.1561/0400000042
[28]
Geyer, R.C., Klein, T. and Nabi, M. (2017) Differentially Private Federated Learning: A Client Level Perspective.
[29]
Triastcyn, A. and Faltings, B. (2019) Federated Learning with Bayesian Differential Privacy. 2019 IEEE International Conference on Big Data, Los Angeles, 9-12 December 2019, 2587-2596. https://doi.org/10.1109/BigData47090.2019.9005465
[30]
Wei, K., Li, J., Ding, M., Ma, C., et al. (2020) Federated Learning with Differential Privacy: Algorithms and Performance Analysis. IEEE Transactions on Information Forensics and Security, 15, 3454-3469. https://doi.org/10.1109/TIFS.2020.2988575
[31]
Bernau, D., Robl, J., Grassal, P.W., Schneider, S. and Kerschbaum, F. (2021) Comparing Local and Central Differential Privacy Using Membership Inference Attacks. IFIP Annual Conference on Data and Applications Security and Privacy, Calgary, 19-20 July 2021, 22-42. https://doi.org/10.1007/978-3-030-81242-3_2
[32]
Zhang, X., Chen, X., Hong, M., Wu, S. and Yi, J. (2022) Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy. International Conference on Machine Learning, Baltimore, 17-23 July 2022, 26048-26067.
[33]
Hu, R., Gong, Y. and Guo, Y. (2022) Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy.
[34]
Li, Z., Huang, Z., Chen, C. and Hong, C. (2019) Quantification of the Leakage in Federated Learning.
[35]
Melis, L., Song, C., De Cristofaro, E. and Shmatikov, V. (2019) Exploiting Unintended Feature Leakage in Collaborative Learning. 2019 IEEE Symposium on Security and Privacy, San Francisco, 19-23 May 2019, 691-706. https://doi.org/10.1109/SP.2019.00029
[36]
Kasiviswanathan, S.P., Lee, H.K., Nissim, K., Raskhodnikova, S. and Smith, A. (2011) What Can We Learn Privately? SIAM Journal on Computing, 40, 793-826. https://doi.org/10.1137/090756090
[37]
Erlingsson, ú., Pihur, V., Korolova, A. (2014) Rappor: Randomized Aggregatable Privacy-Preserving Ordinal Response. Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, Scottsdale, 3-7 November 2014, 1054-1067. https://doi.org/10.1145/2660267.2660348
[38]
Liu, F. (2018) Generalized Gaussian Mechanism for Differential Privacy. IEEE Transactions on Knowledge and Data Engineering, 31, 747-756. https://doi.org/10.1109/TKDE.2018.2845388
[39]
Truex, S., Liu, L., Chow, K.H., Gursoy, M.E. and Wei, W. (2020) LDP-Fed: Federated Learning with Local Differential Privacy. Proceedings of the 3rd ACM International Workshop on Edge Systems, Analytics and Networking, Heraklion, 27 April 2020, 61-66. https://doi.org/10.1145/3378679.3394533
[40]
Sun, Z., Kairouz, P., Suresh, A.T. and McMahan, H.B. (2019) Can You Really Backdoor Federated Learning?
[41]
Xu, M., Ding, B., Wang, T. and Zhou, J. (2020) Collecting and Analyzing Data Jointly from Multiple Services under Local Differential Privacy. Proceedings of the VLDB Endowment, 13, 2760-2772. https://doi.org/10.14778/3407790.3407859
[42]
Naseri, M., Hayes, J. and De Cristofaro, E. (2020) Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy.
[43]
Wang, Y., Tong, Y. and Shi, D. (2020) Federated Latent Dirichlet Allocation: A Local Differential Privacy based Framework. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 6283-6290. https://doi.org/10.1609/aaai.v34i04.6096
[44]
Girgis, A., Data, D. and Diggavi, S. (2021) Renyi Differential Privacy of the Subsampled Shuffle Model in Distributed Learning. Advances in Neural Information Processing Systems, Vol. 34, 29181-29192. https://doi.org/10.1145/3460120.3484794
[45]
Wei, K., Li, J., Ding, M., Ma, C., Su, H., Zhang, B. and Poor, H.V. (2022) User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization. IEEE Transactions on Mobile Computing, 21, 3388-3401. https://doi.org/10.1109/TMC.2021.3056991
[46]
Zhou, H., Yang, G., Dai, H. and Liu, G. (2022) PFLF: Privacy-Preserving Federated Learning Framework for Edge Computing. IEEE Transactions on Information Forensics and Security, 17, 1905-1918. https://doi.org/10.1109/TIFS.2022.3174394
[47]
Thapa, C., Arachchige, P.C.M., Camtepe, S. and Sun, L. (2022) Splitfed: When Federated Learning Meets Split Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 8485-8493. https://doi.org/10.1609/aaai.v36i8.20825
[48]
Wu, C., Wu, F., Lyu, L., Qi, T., Huang, Y. and Xie, X. (2022) A Federated Graph Neural Network Framework for Privacy-Preserving Personalization. Nature Communications, 13, Article No. 3091. https://doi.org/10.1038/s41467-022-30714-9
[49]
Wang, C., Wu, X., Liu, G., Deng, T., Peng, K. and Wan, S. (2022) Safeguarding Cross-Silo Federated Learning with Local Differential Privacy. Digital Communications and Networks, 8, 446-454. https://doi.org/10.1016/j.dcan.2021.11.006
[50]
Zheng, Q., Chen, S., Long, Q. and Su, W. (2021) Federated F-Differential Privacy. International Conference on Artificial Intelligence and Statistics, 13-15 April 2021, 2251-2259.
[51]
Bonawitz, K., Ivanov, V., Kreuter, B., et al. (2017) Practical Secure Aggregation for Privacy-Preserving Machine Learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, 30 October-3 November 2017, 1175-1191. https://doi.org/10.1145/3133956.3133982
[52]
Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I. and Naor, M. (2006) Our Data, Ourselves: Privacy via Distributed Noise Generation. Annual International Conference on the Theory and Applications of Cryptographic Techniques, St. Petersburg, 28 May-1 June 2006, 486-503. https://doi.org/10.1007/11761679_29
[53]
Agarwal, N., Suresh, A.T., Yu, F.X.X., Kumar, S. and McMahan, B. (2018) cpSGD: Communication-Efficient and Differentially-Private Distributed SGD. Advances in Neural Information Processing Systems, Vol. 31, 1-12.
[54]
Canonne, C.L., Kamath, G. and Steinke, T. (2020) The Discrete Gaussian for Differential Privacy. Advances in Neural Information Processing Systems, Vol. 33, 15676-15688.
[55]
Kairouz, P., Liu, Z. and Steinke, T. (2021) The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation. International Conference on Machine Learning, 18-24 July 2021, 5201-5212.
[56]
Agarwal, N., Kairouz, P. and Liu, Z. (2021) The Skellam Mechanism for Differentially Private Federated Learning. Advances in Neural Information Processing Systems, Vol. 34, 5052-5064.
[57]
Bao, E., Zhu, Y., Xiao, X., Yang, Y., Ooi, B.C., Tan, B.H.M. and Aung, K.M.M. (2022) Skellam Mixture Mechanism: A Novel Approach to Federated Learning with Differential Privacy. Proceedings of the VLDB Endowment, 15, 2348-2360. https://doi.org/10.14778/3551793.3551798
[58]
Chen, W.N., Ozgur, A. and Kairouz, P. (2022) The Poisson Binomial Mechanism for Unbiased Federated Learning with Secure Aggregation. International Conference on Machine Learning, Baltimore, 17-23 July 2022, 3490-3506.
[59]
Chen, W.N., Choo, C.A.C., Kairouz, P. and Suresh, A.T. (2022) The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning. International Conference on Machine Learning, Baltimore, 17-23 July 2022, 3056-3089.
[60]
Cheu, A., Smith, A., Ullman, J., Zeber, D. and Zhilyaev, M. (2019) Distributed Differential Privacy via Shuffling. Annual International Conference on the Theory and Applications of Cryptographic Techniques, Darmstadt, 19-23 May 2019, 375-403. https://doi.org/10.1007/978-3-030-17653-2_13
[61]
Jiang, Z., Wang, W. and Chen, R. (2022) Taming Client Dropout for Distributed Differential Privacy in Federated Learning.
[62]
Zhou, Y. and Tang, S. (2020) Differentially Private Distributed Learning. INFORMS Journal on Computing, 32, 779-789. https://doi.org/10.1287/ijoc.2019.0912
[63]
Hu, R., Guo, Y., Li, H., Pei, Q. and Gong, Y. (2020) Personalized Federated Learning with Differential Privacy. IEEE Internet of Things Journal, 7, 9530-9539. https://doi.org/10.1109/JIOT.2020.2991416
[64]
Van Dijk, M., Nguyen, N.V., Nguyen, T.N., Nguyen, L.M., Tran-Dinh, Q. and Nguyen, P.H. (2020) Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise.
[65]
Girgis, A., Data, D., Diggavi, S., Kairouz, P. and Suresh, A.T. (2021) Shuffled Model of Differential Privacy in Federated Learning. International Conference on Artificial Intelligence and Statistics, 13-15 April 2021, 2521-2529.
[66]
Zhang, L., Zhu, T., Xiong, P., Zhou, W. and Yu, P. (2022) A Robust Game-Theoretical Federated Learning Framework with Joint Differential Privacy. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/TKDE.2021.3140131
[67]
Denisov, S., McMahan, B., Rush, K., Smith, A. and Thakurta, A. (2022) Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams. Advances in Neural Information Processing Systems.
[68]
Lian, Z., Wang, W. and Su, C. (2021) COFEL: Communication-efficient and Optimized Federated Learning with Local Differential Privacy. ICC 2021-IEEE International Conference on Communications, Montreal, 14-23 June 2021, 1-6. https://doi.org/10.1109/ICC42927.2021.9500632
[69]
Amiri, S., Belloum, A., Klous, S. and Gommans, L. (2021) Compressive Differentially Private Federated Learning through Universal Vector Quantization. AAAI Workshop on Privacy-Preserving Artificial Intelligence, 2-9 February 2021, 1-5.
[70]
Liu, J., Lou, J., Xiong, L., Liu, J. and Meng, X. (2021) Projected Federated Averaging with Heterogeneous Differential Privacy. Proceedings of the VLDB Endowment, 15, 828-840. https://doi.org/10.14778/3503585.3503592
[71]
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R. and Zhou, Y. (2019) A Hybrid Approach to Privacy-Preserving Federated Learning. Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, London, 15 November 2019, 1-11. https://doi.org/10.1145/3338501.3357370
[72]
Andr’s, M.E., Bordenabe, N.E., Chatzikokolakis, K. and Palamidessi, C. (2013) Geo-Indistinguishability: Differential Privacy for Location-Based Systems. Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, Berlin, 4-8 November 2013, 901-914. https://doi.org/10.1145/2508859.2516735
[73]
Wang, S., Huang, L., Nie, Y., Zhang, X., Wang, P., Xu, H. and Yang, W. (2019) Local Differential Private Data Aggregation for Discrete Distribution Estimation. IEEE Transactions on Parallel and Distributed Systems, 30, 2046-2059. https://doi.org/10.1109/TPDS.2019.2899097
[74]
Zhao, Y., Zhao, J., Yang, M., et al. (2020) Local Differential Privacy-Based Federated Learning for Internet of Things. IEEE Internet of Things Journal, 8, 8836-8853. https://doi.org/10.1109/JIOT.2020.3037194
[75]
Cao, H., Liu, S., Zhao, R. and Xiong, X. (2020) IFed: A Novel Federated Learning Framework for Local Differential Privacy in Power Internet of Things. International Journal of Distributed Sensor Networks, 16. https://doi.org/10.1177/1550147720919698
[76]
Jia, B., Zhang, X., Liu, J., Zhang, Y., Huang, K. and Liang, Y. (2021) Blockchain-Enabled Federated Learning Data Protection Aggregation Scheme with Differential Privacy and Homomorphic Encryption in IIoT. IEEE Transactions on Industrial Informatics, 18, 4049-4058. https://doi.org/10.1109/TII.2021.3085960
[77]
Olowononi, F.O., Rawat, D.B. and Liu, C. (2021) Federated Learning with Differential Privacy for Resilient Vehicular Cyber Physical Systems. 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, 9-12 January 2021, 1-5. https://doi.org/10.1109/CCNC49032.2021.9369480
[78]
Liu, Q., Chen, C., Qin, J., Dou, Q. and Heng, P.A. (2021) Feddg: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 1013-1023. https://doi.org/10.1109/CVPR46437.2021.00107
[79]
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., et al. (2021) End-to-End Privacy Preserving Deep Learning on Multi-Institutional Medical Imaging. Nature Machine Intelligence, 3, 473-484. https://doi.org/10.1038/s42256-021-00337-8
[80]
Adnan, M., Kalra, S., Cresswell, J.C., Taylor, G.W. and Tizhoosh, H.R. (2022) Federated Learning and Differential Privacy for Medical Image Analysis. Scientific Reports, 12, Article No. 1953. https://doi.org/10.1038/s41598-022-05539-7
[81]
Zhang, Z., Zhang, L., Li, Q., Wang, K., He, N. and Gao, T. (2022) Privacy-Enhanced Momentum Federated Learning via Differential Privacy and Chaotic System in Industrial Cyber-Physical Systems. ISA Transactions, 128, 17-31. https://doi.org/10.1016/j.isatra.2021.09.007
[82]
Liu, W., Cheng, J., Wang, X., Lu, X. and Yin, J. (2022) Hybrid Differential Privacy based Federated Learning for Internet of Things. Journal of Systems Architecture, 124, Article ID: 102418. https://doi.org/10.1016/j.sysarc.2022.102418
[83]
Cormode, G., Jha, S., Kulkarni, T., Li, N., Srivastava, D. and Wang, T. (2018) Privacy at Scale: Local Differential Privacy in Practice. Proceedings of the 2018 International Conference on Management of Data, Houston, 10-15 June 2018, 1655-1658. https://doi.org/10.1145/3183713.3197390
