This
paper aims to reduce the communication cost of the distributed learning
algorithm for stochastic configuration networks (SCNs), in which information
exchange between the learning agents is conducted only at a trigger time. For this purpose, we propose the
communication-censored distributed learning algorithm for SCN, namely
ADMMM-SCN-ET, by introducing the event-triggered communication mechanism to the
alternating direction method of multipliers (ADMM).To avoid
unnecessary information transmissions, each learning agent is equipped with a
trigger function. Only if the event-trigger error exceeds a specified threshold
and meets the trigger condition, the agent will transmit the variable
information to its neighbors and update its state in time. The simulation
results show that the proposed algorithm can effectively reduce the
communication cost for training decentralized SCNs and save communication
resources.
References
[1]
Bekkerman, R., Bilenko, M. and Langford, J. (2011) Scaling Up Machine Learning: Parallel and Distributed Approaches. Cambridge University Press, Cambridge.
https://doi.org/10.1145/2107736.2107740
[2]
Bi, X., Zhao, X., Wang, G., Zhang, P. and Wang, C. (2015) Distributed Extreme Learning Machine with Kernels Based on Mapreduce. Neurocomputing, 149, 456-463.
https://doi.org/10.1016/j.neucom.2014.01.070
[3]
Georgopoulos, L. and Hasler, M. (2014) Distributed Machine Learning in Networks by Consensus. Neurocomputing, 124, 2-12.
https://doi.org/10.1016/j.neucom.2012.12.055
[4]
Brandolese, A., Brun, A. and Portioli-Staudacher, A. (2000) A Multi-Agent Approach for the Capacity Allocation Problem. International Journal of Production Economics, 66, 269-285. https://doi.org/10.1016/S0925-5273(00)00004-9
[5]
Scardapane, S. and Di Lorenzo, P. (2017) A Framework for Parallel and Distributed Training of Neural Networks. Neural Networks, 91, 42-54.
https://doi.org/10.1016/j.neunet.2017.04.004
[6]
Shnayder, V., Hempstead, M., Chen, B.R., Werner-Allen, G. and Welsh, M. (2004) Simulating the Power Consumption of Large-Scale Sensor Network Applications. Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems, Baltimore, 3-5 November 2004, 188-200.
https://doi.org/10.1145/1031495.1031518
[7]
Heemels, W.P., Johansson, K.H. and Tabuada, P. (2012) An Introduction to Event-Triggered and Self-Triggered Control. 2012 IEEE 51st Conference on Decision and Control (CDC), Maui, 10-13 December 2012, 3270-3285.
https://doi.org/10.1109/CDC.2012.6425820
[8]
Liu, Q., Wang, Z., He, X. and Zhou, D. (2014) A Survey of Event-Based Strategies on Control and Estimation. Systems Science & Control Engineering: An Open Access Journal, 2, 90-97. https://doi.org/10.1080/21642583.2014.880387
[9]
Nowzari, C., Garcia, E. and Cortés, J. (2019) Event-Triggered Communication and Control of Networked Systems for Multi-Agent Consensus. Automatica, 105, 1-27.
https://doi.org/10.1016/j.automatica.2019.03.009
[10]
Yang, J., Xiao, F. and Ma, J. (2018) Model-Based Edge-Event-Triggered Containment Control under Directed Topologies. IEEE Transactions on Cybernetics, 49, 2556-2567. https://doi.org/10.1109/TCYB.2018.2828645
[11]
Ge, X., Han, Q.L., Ding, L., Wang, Y.L. and Zhang, X.M. (2020) Dynamic Event-Triggered Distributed Coordination Control and Its Applications: A Survey of Trends and Techniques. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50, 3112-3125. https://doi.org/10.1109/TSMC.2020.3010825
[12]
Dimarogonas, D.V. and Johansson, K.H. (2009) Event-Triggered Control for Multi-Agent Systems. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) Held Jointly with 2009 28th Chinese Control Conference, Shanghai, 15-18 December 2009, 7131-7136. https://doi.org/10.1109/CDC.2009.5399776
[13]
Dimarogonas, D.V., Frazzoli, E. and Johansson, K.H. (2011) Distributed Event-Triggered Control for Multi-Agent Systems. IEEE Transactions on Automatic Control, 57, 1291-1297. https://doi.org/10.1109/TAC.2011.2174666
[14]
Wang, D. and Li, M. (2017) Stochastic Configuration Networks: Fundamentals and Algorithms. IEEE Transactions on Cybernetics, 47, 3466-3479.
https://doi.org/10.1109/TCYB.2017.2734043
[15]
Bazaraa, M.S. and Goode, J.J. (1973) On Symmetric Duality in Nonlinear Programming. Operations Research, 21, 1-9. https://doi.org/10.1287/opre.21.1.1
[16]
Tyukin, I.Y. and Prokhorov, D.V. (2009) Feasibility of Random Basis Function Approximators for Modeling and Control. 2009 IEEE Control Applications, (CCA) & Intelligent Control, (ISIC), St. Petersburg, 8-10 July 2009, 1391-1396.
https://doi.org/10.1109/CCA.2009.5281061
