Continual Local Training for Better Initialization of Federated Models

Federated learning (FL) refers to the learning paradigm that trains machine learning models directly in the decentralized systems consisting of smart edge devices without transmitting the raw data, which avoids the heavy communication costs and privacy concerns. Given the typical heterogeneous data distributions in such situations, the popular FL algorithm \emph{Federated Averaging} (FedAvg) suffers from weight divergence and thus cannot achieve a competitive performance for the global model (denoted as the \emph{initial performance} in FL) compared to centralized methods. In this paper, we propose the local continual training strategy to address this problem. Importance weights are evaluated on a small proxy dataset on the central server and then used to constrain the local training. With this additional term, we alleviate the weight divergence and continually integrate the knowledge on different local clients into the global model, which ensures a better generalization ability. Experiments on various FL settings demonstrate that our method significantly improves the initial performance of federated models with few extra communication costs.Comment: This paper has been accepted to 2020 IEEE International Conference on Image Processing (ICIP 2020

Effects of Long-Range Interactions on Magnetic Excitations and Phase Transition on a Magnetically Frustrated Square Lattice

We investigate the effects of long-range interaction on the magnetic excitations and the competition between magnetic phases on a frustrated square lattice. Applying the spin wave theory and assisted with symmetry analysis, we obtain analytical expression for spin wave spectrum of competing Neel and (pi, 0) stripe states of systems containing any-order long-range interactions. In the specific case of long-range interactions with power-law decay, we found surprisingly that staggered long-range interaction suppresses quantum fluctuation and enlarges the ordered moment, especially in the Neel state, and thus extends its phase boundary to the stripe state. Our findings only illustrate the rich possibilities of the roles of long-range interactions, and advocate future investigations in other magnetic systems with different structures of interactions.Comment: 9 pages, 9 figure

Algorithm Portfolio for Individual-based Surrogate-Assisted Evolutionary Algorithms

Surrogate-assisted evolutionary algorithms (SAEAs) are powerful optimisation tools for computationally expensive problems (CEPs). However, a randomly selected algorithm may fail in solving unknown problems due to no free lunch theorems, and it will cause more computational resource if we re-run the algorithm or try other algorithms to get a much solution, which is more serious in CEPs. In this paper, we consider an algorithm portfolio for SAEAs to reduce the risk of choosing an inappropriate algorithm for CEPs. We propose two portfolio frameworks for very expensive problems in which the maximal number of fitness evaluations is only 5 times of the problem's dimension. One framework named Par-IBSAEA runs all algorithm candidates in parallel and a more sophisticated framework named UCB-IBSAEA employs the Upper Confidence Bound (UCB) policy from reinforcement learning to help select the most appropriate algorithm at each iteration. An effective reward definition is proposed for the UCB policy. We consider three state-of-the-art individual-based SAEAs on different problems and compare them to the portfolios built from their instances on several benchmark problems given limited computation budgets. Our experimental studies demonstrate that our proposed portfolio frameworks significantly outperform any single algorithm on the set of benchmark problems