Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach

Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be "compressed" to much smaller representations. The purpose of this paper is to connect these two empirical observations. Our main technical result is a generalization bound for compressed networks based on the compressed size. Combined with off-the-shelf compression algorithms, the bound leads to state of the art generalization guarantees; in particular, we provide the first non-vacuous generalization guarantees for realistic architectures applied to the ImageNet classification problem. As additional evidence connecting compression and generalization, we show that compressibility of models that tend to overfit is limited: We establish an absolute limit on expected compressibility as a function of expected generalization error, where the expectations are over the random choice of training examples. The bounds are complemented by empirical results that show an increase in overfitting implies an increase in the number of bits required to describe a trained network.Comment: 16 pages, 1 figure. Accepted at ICLR 201

Onset of Sediment Transport Is a Continuous Transition Driven by Fluid Shear and Granular Creep

Fluid-sheared granular transport sculpts landscapes and undermines infrastructure, yet predicting the onset of sediment transport remains notoriously unreliable. For almost a century, this onset has been treated as a discontinuous transition at which hydrodynamic forces overcome gravity-loaded grain–grain friction. Using a custom laminar-shear flume to image slow granular dynamics deep into the bed, here we find that the onset is instead a continuous transition from creeping to granular flow. This transition occurs inside the dense granular bed at a critical viscous number, similar to granular flows and colloidal suspensions and inconsistent with hydrodynamic frameworks. We propose a new phase diagram for sediment transport, where ‘bed load’ is a dense granular flow bounded by creep below and suspension above. Creep is characteristic of disordered solids and reminiscent of soil diffusion on hillslopes. Results provide new predictions for the onset and dynamics of sediment transport that challenge existing model

River-bed armouring as a granular segregation phenomenon

River bed-load transport is a kind of dense granular flow, and such flows are known to segregate grains. While gravel-river beds typically have an “armoured” layer of coarse grains on the surface, which acts to protect finer particles underneath from erosion, the contribution of granular physics to river-bed armouring has not yet been investigated. Here we examine these connections in a laboratory river with bimodal sediment size, by tracking the motion of particles from the surface to deep inside the bed, and find that armour develops by two distinct mechanisms. Bed-load transport in the near-surface layer drives rapid, shear ratedependent advective segregation. Creeping grains beneath the bed-load layer give rise to slow but persistent diffusion-dominated segregation. We verify these findings with a continuum phenomenological model and discrete element method simulations. Our experiments suggest that some river-bed armouring may be due to granular segregation from below— rather than fluid-driven sorting from above—while also providing new insights on the mechanics of segregation that are relevant to a wide range of granular flows

Non-vacuous Generalization Bounds at the ImageNet Scale: a PAC-Bayesian Compression Approach

Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be “compressed” to much smaller representations. The purpose of this paper is to connect these two empirical observations. Our main technical result is a generalization bound for compressed networks based on the compressed size that, combined with off-theshelf compression algorithms, leads to state-of-the-art generalization guarantees. In particular, we provide the first non-vacuous generalization guarantees for realistic architectures applied to the ImageNet classification problem. Additionally, we show that compressibility of models that tend to overfit is limited. Empirical results show that an increase in overfitting increases the number of bits required to describe a trained network

Shared motivations, goals and values in the practice of personal science: A community perspective on self-tracking for empirical knowledge

Some individuals do not limit their self-tracking efforts to passively collecting and observing gathered data about themselves, but rather develop it into forms of self-research and self-experimentation, also called “personal science”. This type of N-of-1 research is relevant to the fields of personal informatics, patient-led research and social studies of science, but as a knowledge generation practice is still poorly understood. To fill this gap, we conducted 22 semi-structured interviews to investigate the intrinsic and extrinsic motivations of individuals engaging in personal science activities, as well as shared goals and values present in self-research communities. Our analysis is based on a conceptual framework that integrates previous approaches in self-research, as well as in connection with citizen science, the scientific ethos and cooperation in peer production. We identify how self-researchers seek to go beyond personal metrics about their health and wellbeing regarding data provided by wearables, are engaged over time by individual involvement in technology and scientific-related activity, and collaborate following similar goals and values when learning and sharing empirical knowledge with peers. In this sense, personal science can be understood as a specific type of citizen science and an example of a more participatory and inclusive scientific culture driven by self-reflection, critical thinking and openness

Rheology of Sediment Transported by a Laminar Flow

Understanding the dynamics of fluid-driven sediment transport remains challenging, as it occurs at the interface between a granular material and a fluid flow. Boyer, Guazzelli, and Pouliquen [Phys. Rev. Lett.107, 188301 (2011)] proposed a local rheology unifying dense dry-granular and viscous-suspension flows, but it has been validated only for neutrally buoyant particles in a confined and homogeneous system. Here we generalize the Boyer, Guazzelli, and Pouliquen model to account for the weight of a particle by addition of a pressure P0 and test the ability of this model to describe sediment transport in an idealized laboratory river. We subject a bed of settling plastic particles to a laminar-shear flow from above, and use refractive-index-matching to track particles\u27 motion and determine local rheology—from the fluid-granular interface to deep in the granular bed. Data from all experiments collapse onto a single curve of friction μ as a function of the viscous number Iv over the range 3 × 10−5≤ Iv ≤ 2, validating the local rheology model. For Iv \u3c 3 × 10−5, however, data do not collapse. Instead of undergoing a jamming transition with μ → μs as expected, particles transition to a creeping regime where we observe a continuous decay of the friction coefficient μ ≤ μs as Iv decreases. The rheology of this creep regime cannot be described by the local model, and more work is needed to determine whether a nonlocal rheology model can be modified to account for our findings

On the morphology of the astragalus and calcaneus of the amphicyonids (Carnivora, Mammalia) from the Paleogene of Europe: implications for the ecology of the European bear-dogs

The Paleogene mammals of Europe are rarely known from partial or complete skeletons. As a result, their systematics and ecology are often solely based on dental characters and postcranial remains, when available, are usually neglected. This explains why the locomotion of mammals of the Eocene-Oligocene transition, the “Grande Coupure”, is poorly known. The aim of this study is to describe the tarsal bones (astragali and calcanei) and characterize the locomotion of amphicyonid carnivorans, one of the most abundant mammalian predator groups from the Phosphorites du Quercy (France) sites. The identification of taxa and the characterization of both posture and locomotion were carried out using four criteria: relative abundance (in comparison with dental data), morphology, size, and body mass. Seven morphotypes, four among astragali and three among calcanei, are identified as Amphicyonidae and show various postures: plantigrade, semi-digitigrade, and digitigrade. One morphotype of the astragalus and one of the calcaneus are identified as Cynodictis lacustrisGervais, 1852, which exhibits a digitigrade posture. The study of postcranial bones, such as tarsals, allows for a better understanding of the ecology of these animals and deserves more interest in future morphological and phylogenetic studies

Signature of attochemical quantum interference upon ionization and excitation of an electronic wavepacket in fluoro-benzene

Ultrashort pulses can excite or ionize molecules and populate coherent electronic wavepackets, inducing complex dynamics. In this work, we simulate the coupled electron-nuclear dynamics upon ionization to different electronic wavepackets of (deuterated) benzene and fluoro-benzene molecules, quantum mechanically and in full dimensionality. In fluoro-benzene, the calculations unravel both inter-state and intra-state quantum interferences that leave clear signatures of attochemistry and charge-directed reactivity in the shape of the autocorrelation function. The latter are in agreement with experimental high harmonic spectroscopy measurements of benzenes and fluoro-benzene

The upper Eocene-Oligocene carnivorous mammals from the Quercy Phosphorites (France) housed in Belgian collections

The Quercy Phosphorites Formation in France is world famous for its Eocene to Miocene faunas, especially those from the upper Eocene to lower Oligocene, the richest of all. The latter particularly helped to understand the ‘Grande Coupure’, a dramatic faunal turnover event that occurred in Europe during the Eocene-Oligocene transition. Fossils from the Quercy Phosphorites were excavated from the middle 19th century until the early 20th century in a series of sites and became subsequently dispersed over several research institutions, while often losing the temporal and geographical information in the process. In this contribution, we provide an overview and reassess the taxonomy of these barely known collections housed in three Belgian institutions: the Université de Liège, KU Leuven, and the Royal Belgian Institute of Natural Sciences. We focus our efforts on the carnivorous mammals (Hyaenodonta and Carnivoramorpha) and assess the stratigraphic intervals covered by each collection. These fossils are derived from upper Eocene (Priabonian), lower Oligocene (Rupelian), and upper Oligocene (Chattian) deposits in the Quercy area. The richness of the three collections (e.g., the presence of numerous postcranial elements in the Liège collection), the presence of types and figured specimens in the Leuven collection, and some identified localities in the RBINS collection make these collections of great interest for further studies on systematics and the evolution of mammals around the ‘Grande Coupure’

Modulation of TCR signalling components occurs prior to positive selection and lineage commitment in iNKT cells

iNKT cells play a critical role in controlling the strength and character of adaptive and innate immune responses. Their unique functional characteristics are induced by a transcriptional program initiated by positive selection mediated by CD1d expressed by CD4+CD8+ (double positive, DP) thymocytes. Here, using a novel Vα14 TCR transgenic strain bearing greatly expanded numbers of CD24hiCD44loNKT cells, we examined transcriptional events in four immature thymic iNKT cell subsets. A transcriptional regulatory network approach identified transcriptional changes in proximal components of the TCR signalling cascade in DP NKT cells. Subsequently, positive and negative selection, and lineage commitment, occurred at the transition from DP NKT to CD4 NKT. Thus, this study introduces previously unrecognised steps in early NKT cell development and separates, the events associated with modulation of the T cell signalling cascade prior to changes associated with positive selection and lineage commitment