FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Self-organizing, two-temperature Ising model describing human segregation

A two-temperature Ising-Schelling model is introduced and studied for describing human segregation. The self-organized Ising model with Glauber kinetics simulated by M\"uller et al. exhibits a phase transition between segregated and mixed phases mimicking the change of tolerance (local temperature) of individuals. The effect of external noise is considered here as a second temperature added to the decision of individuals who consider change of accommodation. A numerical evidence is presented for a discontinuous phase transition of the magnetization.Comment: 5 pages, 4 page

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

Reconfigurable self-sufficient traps for ultracold atoms based on a superconducting square

We report on the trapping of ultracold atoms in the magnetic field formed entirely by persistent supercurrents induced in a thin film type-II superconducting square. The supercurrents are carried by vortices induced in the 2D structure by applying two magnetic field pulses of varying amplitude perpendicular to its surface. This results in a self-sufficient quadrupole trap that does not require any externally applied fields. We investigate the trapping parameters for different supercurrent distributions. Furthermore, to demonstrate possible applications of these types of supercurrent traps we show how a central quadrupole trap can be split into four traps by the use of a bias field.Comment: 5 pages, 7 figure

Asymptotic silence-breaking singularities

We discuss three complementary aspects of scalar curvature singularities: asymptotic causal properties, asymptotic Ricci and Weyl curvature, and asymptotic spatial properties. We divide scalar curvature singularities into two classes: so-called asymptotically silent singularities and non-generic singularities that break asymptotic silence. The emphasis in this paper is on the latter class which have not been previously discussed. We illustrate the above aspects and concepts by describing the singularities of a number of representative explicit perfect fluid solutions.Comment: 25 pages, 6 figure

Excessive growth hormone expression in male GH transgenic mice adversely alters bone architecture and mechanical strength

Patients with acromegaly have a higher prevalence of vertebral fractures despite normal bone mineral density (BMD), suggesting that GH overexpression has adverse effects on skeletal architecture and strength. We used giant bovine GH (bGH) transgenic mice to analyze the effects of high serum GH levels on BMD, architecture, and mechanical strength. Five-month-old hemizygous male bGH mice were compared with age- and sex-matched nontransgenic littermates controls (NT; n=16/group). Bone architecture and BMD were analyzed in tibia and lumbar vertebrae using microcomputed tomography. Femora were tested to failure using three-point bending and bone cellular activity determined by bone histomorphometry. bGH transgenic mice displayed significant increases in body weight and bone lengths. bGH tibia showed decreases in trabecular bone volume fraction, thickness, and number compared with NT ones, whereas trabecular pattern factor and structure model index were significantly increased, indicating deterioration in bone structure. Although cortical tissue perimeter was increased in transgenic mice, cortical thickness was reduced. bGH mice showed similar trabecular BMD but reduced trabecular thickness in lumbar vertebra relative to controls. Cortical BMD and thickness were significantly reduced in bGH lumbar vertebra. Mechanical testing of femora confirmed that bGH femora have decreased intrinsic mechanical properties compared with NT ones. Bone turnover is increased in favor of bone resorption in bGH tibia and vertebra compared with controls, and serum PTH levels is also enhanced in bGH mice. These data collectively suggest that high serum GH levels negatively affect bone architecture and quality at multiple skeletal sites