Distributed Learning over Unreliable Networks

Most of today's distributed machine learning systems assume {\em reliable networks}: whenever two machines exchange information (e.g., gradients or models), the network should guarantee the delivery of the message. At the same time, recent work exhibits the impressive tolerance of machine learning algorithms to errors or noise arising from relaxed communication or synchronization. In this paper, we connect these two trends, and consider the following question: {\em Can we design machine learning systems that are tolerant to network unreliability during training?} With this motivation, we focus on a theoretical problem of independent interest---given a standard distributed parameter server architecture, if every communication between the worker and the server has a non-zero probability $p$ of being dropped, does there exist an algorithm that still converges, and at what speed? The technical contribution of this paper is a novel theoretical analysis proving that distributed learning over unreliable network can achieve comparable convergence rate to centralized or distributed learning over reliable networks. Further, we prove that the influence of the packet drop rate diminishes with the growth of the number of \textcolor{black}{parameter servers}. We map this theoretical result onto a real-world scenario, training deep neural networks over an unreliable network layer, and conduct network simulation to validate the system improvement by allowing the networks to be unreliable

非貴金属系の多金属電気化学触媒の開発及び水分解反応への応用

九州工業大学博士（工学）1. Introduction||2. Bimetal heterojunction FexSy/WS2 nanosheets electrocatalyst for energy-saving HER and water-splitting||3. Development bifunctional quinary NiFeCoMnCu high entropy alloy electrocatalyst for alkalic OER and HER||4. Bifunctional NiFeCoMnCu HEAs for electrochemical alkaline seawater splittingHydrogen evolution from electrochemical water-splitting is recognized as an effective and sustainable approach to producing hydrogen. Up to now, Ru and Pt-based electrocatalysts display superior oxygen evolution reaction (OER) and hydrogen evolution reaction (HER) activities and long-term stability. However, the high price and rare reserve on the earth limit their large-scale usage. In this thesis, we developed a series of non-noble transition multimetallic electrocatalysts. As we expected, the prepared electrocatalysts exhibited excellent both HER and OER activities. In addition, as the bifunctional electrocatalysts, the prepared electrocatalysts displayed outstanding electrochemical water-splitting activity and excellent electrocatalytic stability. In chapter 1, the mechanism of OER and HER during the electrochemical water-splitting process, and the current development of transition metal electrocatalysts for OER and HER were introduced. In chapter 2, a novel bimetallic heterojunction FexSy/WS2 nanosheets (FexSy/WS2 Ns) were prepared through interface engineering technology. The prepared FexSy/WS2 Ns had an 8 nm thickness and a large diameter of ~1 μm. The FexSy/WS2 Ns exhibited excellent HER activity with an overpotential of 118 mV at 10 mA cm-2 and a low Tafel slope of 87 mV dec-1. In addition, by replacing OER with the urea oxidation reaction (UOR), the FexSy/WS2 Ns as a bifunctional electrocatalyst could achieve energy-saving water-splitting process, in which the applied voltage could be decreased with147 mV. In chapter 3, to further increase the electrocatalytic activity, we developed a series of nonnoble transition metal electrocatalysts of unary Ni, binary NiFe, ternary NiFeCo, quaternary NiFeCoMn, and quinary NiFeCoMnCu by a facile critic acid chelating method. All the electrocatalysts had been used as the bifunctional electrocatalysts to test the OER and HER activities. As the result, the NiFeCoMnCu high entropy alloys (NiFeCoMnCu HEAs) displayed excellent OER and HER activity with an overpotential of 240 mV and 165 mV at 10 mA cm-2 respectively. In chapter 4, a high-efficiency overall electrochemical water-splitting process was enabled by the prepared NiFeCoMnCu HEAs electrocatalyst through a two-electrode system. During the water-splitting process, the NiFeCoMnCu HEAs electrocatalyst only required 1.53 V to reach 10 mA cm-2. Meanwhile, almost 10 h continuous water-splitting process confirmed that the NiFeCoMnCu HEAs had an excellent stability. In addition, an electrochemical seawater-splitting reaction could also be enabled by the NiFeCoMnCu HEAs electrocatalyst with a current density of 10.5 mA cm-2 at 1.8 V for a continuous 10 h reaction process. Finally, the general conclusions of this thesis and the further prospects were summarized. The strategy of mixing different active metal elements into one single electrocatalyst would provide a better solution for developing more multifunctional electrocatalysts. It is also the point we will focus on in the future.九州工業大学博士学位論文 学位記番号：生工博甲第445号 学位授与年月日：令和4年9月26日令和4年度doctoral thesi