Distributed Training Large-Scale Deep Architectures

Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training. Via lessons learned from our routine benchmarking effort, we first identify bottlenecks and overheads that hinter data parallelism. We then devise guidelines that help practitioners to configure an effective system and fine-tune parameters to achieve desired speedup. Specifically, we develop a procedure for setting minibatch size and choosing computation algorithms. We also derive lemmas for determining the quantity of key components such as the number of GPUs and parameter servers. Experiments and examples show that these guidelines help effectively speed up large-scale deep learning training

Progressive amorphization of GeSbTe phase-change material under electron beam irradiation

Fast and reversible phase transitions in chalcogenide phase-change materials (PCMs), in particular, Ge-Sb-Te compounds, are not only of fundamental interests, but also make PCMs based random access memory (PRAM) a leading candidate for non-volatile memory and neuromorphic computing devices. To RESET the memory cell, crystalline Ge-Sb-Te has to undergo phase transitions firstly to a liquid state and then to an amorphous state, corresponding to an abrupt change in electrical resistance. In this work, we demonstrate a progressive amorphization process in GeSb2Te4 thin films under electron beam irradiation on transmission electron microscope (TEM). Melting is shown to be completely absent by the in situ TEM experiments. The progressive amorphization process resembles closely the cumulative crystallization process that accompanies a continuous change in electrical resistance. Our work suggests that if displacement forces can be implemented properly, it should be possible to emulate symmetric neuronal dynamics by using PCMs

The effect of subgroup homogeneity of efficacy on contribution in public good dilemmas

open access articleThis paper examines how to maximize contribution in public good dilemmas by arranging people into homogeneous or heterogeneous subgroups. Past studies on the effect of homo- geneity of efficacy have exclusively manipulated group composition in their experimental designs, which might have imposed a limit on ecological validity because group membership may not be easily changed in reality. In this study, we maintained the same group composi- tion but varied the subgroup composition. We developed a public good dilemmas paradigm in which participants were assigned to one of the four conditions (high- vs. low-efficacy; homogeneous vs. heterogeneous subgroup) to produce their endowments and then to decide how much to contribute. We found that individuals in homogeneous and heteroge- neous subgroups produced a similar amount and proportion of contribution, which was due to the two mediating effects that counteracted each other, namely (a) perceived efficacy rel- ative to subgroup and (b) expectation of contribution of other subgroup members. This paper demonstrates both the pros and cons of arranging people into homogeneous and het- erogeneous subgroups of efficacy

Sixteen years post radiotherapy of nasopharyngeal carcinoma elicited multi-dysfunction along PTX and chronic kidney disease with microcytic anemia

BACKGROUND: The hypothalamic–pituitary (h-p) unit is a particularly radiosensitive region in the central nervous system. As a consequence, radiation-induced irreversible, progressively chronic onset hypopituitarism (RIH) commonly develops after radiation treatments and can result in variably impaired pituitary function, which is frequently associated with increased morbidity and mortality. CASE PRESENTATION: A 38-year-old male subject, previously having received radiotherapy for treatment of nasopharygeal carcinoma (NPCA) 16 years ago, appeared at OPD complaining about his failure in penile erection, loss of pubic hair, atrophy of external genitalia: testicles reduced to 2×1.5 cm; penile size shrunk to only 4 cm long. Characteristically, he showed extremely lowered human growth hormone, (HGH, 0.115 ng/mL), testosterone (<0.1 ng/mL), total thyroxine (tT4: 4.740 g/mL), free T4 (fT4, 0.410 ng/mL), cortisol (2.34 g/dL); lowered LH (1.37 mIU/mL) and estradiol (22 pg/mL); highly elevated TSH (7.12 IU/mL). As contrast, he had low end normal ACTH, FSH, total T3, free T3, and estriol; high end normal prolactin (11.71 ng/mL), distinctly implicating hypopituitarism-induced hypothyroidism and hypogonadism. serologically, he showed severely lowered Hb (10.6 g/dL), HCT (32.7%), MCV (77.6 fL), MCH (25.3 pg), MCHC (32.6 g/dL), and platelet count (139×103/L) with extraordinarily elevated RDW (18.2%), together with severely lowered ferritin (23.6 ng/mL) and serum iron levels; highly elevated total iron binding capacity (TIBC, 509 g/dL) and transferrin (363.4 mg/dL), suggesting microcytic anemia. Severely reduced estimated glomerular filtration rate (e-GFR) (89 mL/mim/1.73 m2) pointed to CKD2. Hypocortisolemia with hyponatremia indicated secondary adrenal insufficiency. Replacement therapy using androgen, cortisol, and Ringer’s solution has shown beneficial in improving life quality. CONCLUSIONS: To our believe, we are the first group who report such complicate PTX dysfunction with adrenal cortisol insufficiency concomitantly occurring in a single patient

Conformal Field Theory Ground States as Critical Points of an Entropy Function

We derive an entropy formula satisfied by the ground states of 1+1D conformal field theories. The formula implies that the ground state is the critical point of an entropy function. We conjecture that this formula may serve as an information-theoretic criterion for conformal field theories, which differs from the conventional algebraic definition. In addition to these findings, we use the same proof method to extract the six global conformal generators of the conformal field theory from its ground state. We validate our results by testing them on different critical lattice models with excellent agreement

Sodium vanadate combined with l-ascorbic acid delays disease progression, enhances motor performance, and ameliorates muscle atrophy and weakness in mice with spinal muscular atrophy

BACKGROUND: Proximal spinal muscular atrophy (SMA), a neurodegenerative disorder that causes infant mortality, has no effective treatment. Sodium vanadate has shown potential for the treatment of SMA; however, vanadate-induced toxicity in vivo remains an obstacle for its clinical application. We evaluated the therapeutic potential of sodium vanadate combined with a vanadium detoxification agent, L-ascorbic acid, in a SMA mouse model. METHODS: Sodium vanadate (200 μM), L-ascorbic acid (400 μM), or sodium vanadate combined with L-ascorbic acid (combined treatment) were applied to motor neuron-like NSC34 cells and fibroblasts derived from a healthy donor and a type II SMA patient to evaluate the cellular viability and the efficacy of each treatment in vitro. For the in vivo studies, sodium vanadate (20 mg/kg once daily) and L-ascorbic acid (40 mg/kg once daily) alone or in combination were orally administered daily on postnatal days 1 to 30. Motor performance, pathological studies, and the effects of each treatment (vehicle, L-ascorbic acid, sodium vanadate, and combined treatment) were assessed and compared on postnatal days (PNDs) 30 and 90. The Kaplan-Meier method was used to evaluate the survival rate, with P < 0.05 indicating significance. For other studies, one-way analysis of variance (ANOVA) and Student's t test for paired variables were used to measure significant differences (P < 0.05) between values. RESULTS: Combined treatment protected cells against vanadate-induced cell death with decreasing B cell lymphoma 2-associated X protein (Bax) levels. A month of combined treatment in mice with late-onset SMA beginning on postnatal day 1 delayed disease progression, improved motor performance in adulthood, enhanced survival motor neuron (SMN) levels and motor neuron numbers, reduced muscle atrophy, and decreased Bax levels in the spinal cord. Most importantly, combined treatment preserved hepatic and renal function and substantially decreased vanadium accumulation in these organs. CONCLUSIONS: Combined treatment beginning at birth and continuing for 1 month conferred protection against neuromuscular damage in mice with milder types of SMA. Further, these mice exhibited enhanced motor performance in adulthood. Therefore, combined treatment could present a feasible treatment option for patients with late-onset SMA