Reconciling modern machine learning practice and the bias-variance trade-off

Breakthroughs in machine learning are rapidly changing science and society, yet our fundamental understanding of this technology has lagged far behind. Indeed, one of the central tenets of the field, the bias-variance trade-off, appears to be at odds with the observed behavior of methods used in the modern machine learning practice. The bias-variance trade-off implies that a model should balance under-fitting and over-fitting: rich enough to express underlying structure in data, simple enough to avoid fitting spurious patterns. However, in the modern practice, very rich models such as neural networks are trained to exactly fit (i.e., interpolate) the data. Classically, such models would be considered over-fit, and yet they often obtain high accuracy on test data. This apparent contradiction has raised questions about the mathematical foundations of machine learning and their relevance to practitioners. In this paper, we reconcile the classical understanding and the modern practice within a unified performance curve. This "double descent" curve subsumes the textbook U-shaped bias-variance trade-off curve by showing how increasing model capacity beyond the point of interpolation results in improved performance. We provide evidence for the existence and ubiquity of double descent for a wide spectrum of models and datasets, and we posit a mechanism for its emergence. This connection between the performance and the structure of machine learning models delineates the limits of classical analyses, and has implications for both the theory and practice of machine learning

Transparent dense sodium

Under pressure, metals exhibit increasingly shorter interatomic distances. Intuitively, this response is expected to be accompanied by an increase in the widths of the valence and conduction bands and hence a more pronounced free-electron-like behaviour. But at the densities that can now be achieved experimentally, compression can be so substantial that core electrons overlap. This effect dramatically alters electronic properties from those typically associated with simple free-electron metals such as lithium and sodium, leading in turn to structurally complex phases and superconductivity with a high critical temperature. But the most intriguing prediction - that the seemingly simple metals Li and Na will transform under pressure into insulating states, owing to pairing of alkali atoms - has yet to be experimentally confirmed. Here we report experimental observations of a pressure-induced transformation of Na into an optically transparent phase at 200 GPa (corresponding to 5.0-fold compression). Experimental and computational data identify the new phase as a wide bandgap dielectric with a six-coordinated, highly distorted double-hexagonal close-packed structure. We attribute the emergence of this dense insulating state not to atom pairing, but to p-d hybridizations of valence electrons and their repulsion by core electrons into the lattice interstices. We expect that such insulating states may also form in other elements and compounds when compression is sufficiently strong that atomic cores start to overlap strongly.Comment: Published in Nature 458, 182-185 (2009

Cosmological perturbations: a new gauge-invariant approach

A new gauge-invariant approach for describing cosmological perturbations is developed. It is based on a physically motivated splitting of the stress-energy tensor of the perturbation into two parts - the bare perturbation and the complementary perturbation associated with stresses in the background gravitational field induced by the introduction of the bare perturbation. The complementary perturbation of the stress-energy tensor is explicitly singled out and taken to the left side of the perturbed Einstein equations so that the bare stress-energy tensor is the sole source for the perturbation of the metric tensor and both sides of these equations are gauge invariant with respect to infinitesimal coordinate transformations. For simplicity we analyze the perturbations of the spatially-flat Friedmann-Lemaitre-Robertson-Walker dust model. A cosmological gauge can be chosen such that the equations for the perturbations of the metric tensor are completely decoupled for the h_{00}, h_{0i}, and h_{ij} metric components and explicitly solvable in terms of retarded integrals.Comment: 10 pages, corrected proofs, published in PL

Individually addressable arrays of replica microbial cultures enabled by splitting SlipChips

Isolating microbes carrying genes of interest from environmental samples is important for applications in biology and medicine. However, this involves the use of genetic assays that often require lysis of microbial cells, which is not compatible with the goal of obtaining live cells for isolation and culture. This paper describes the design, fabrication, biological validation, and underlying physics of a microfluidic SlipChip device that addresses this challenge. The device is composed of two conjoined plates containing 1000 microcompartments, each comprising two juxtaposed wells, one on each opposing plate. Single microbial cells are stochastically confined and subsequently cultured within the microcompartments. Then, we split each microcompartment into two replica droplets, both containing microbial culture, and then controllably separate the two plates while retaining each droplet within each well. We experimentally describe the droplet retention as a function of capillary pressure, viscous pressure, and viscosity of the aqueous phase. Within each pair of replicas, one can be used for genetic analysis, and the other preserves live cells for growth. This microfluidic approach provides a facile way to cultivate anaerobes from complex communities. We validate this method by targeting, isolating, and culturing Bacteroides vulgatus, a core gut anaerobe, from a clinical sample. To date, this methodology has enabled isolation of a novel microbial taxon, representing a new genus. This approach could also be extended to the study of other microorganisms and even mammalian systems, and may enable targeted retrieval of solutions in applications including digital PCR, sequencing, single cell analysis, and protein crystallization

Neutral and cationic di(tert-butyl) cyclopentadienyl titanium, zirconium and hafnium complexes. Dynamic NMR study of the ligand-free cations [M(1,3-tBu2-η5-C5H3)(η5-f5H5)(CH3)]+(M=Zr, Hf)

Group 4 metal complexes containing the di(tert-butyl)cyclopentadienyl ligand (l,3-tBu2-r/5-CsH3) have been synthesized. The\ud reaction of a mixture of 1,3- and 1,4-di(tert-butyl)cyclopentadiene isomers with KH in THF at -78°C gives the salt K+[(1,3 -\ud tBu2CsH3)]-(THF)I_3 2 as a white solid. Treatment of 2 with chlorotrimethylsilane in a 1:1 molar ratio gives the air-stable\ud trimethylsilylcyclopentadienyl derivative Si(1,3-tBu2C5H3XCH3)3 3. The silyl derivative 3 is an excellent precursor for monocyclopentadienyl\ud trichlorotitanium and zirconium compounds M(1,3 -t Bu 2-r/5-C 5 H 3)C13 [M = Ti (4), Zr (5)]. Addition of a stoichiometric amount\ud of water in the presence of NEt 3 to a toluene solution of 4 affords the oxo trimer compound [Ti(1,3-tBu2-~75 - CsH3)CI( p,-O)] 3 6. The\ud reaction of 4 with 2 equiv, of LiMe affords the chloro dimethyl derivative Ti(1,3-tBu2-'r/5-CsH3)CI(CH3)2 7. The mixed dicyclopentadienyl\ud compounds M(1,3-tBu2-r/5-CsH3XCsHs)CI2 [M = Ti (8); Zr (9)] were prepared by reaction of complexes 4 and 5 respectively with\ud TI(CsHs). Treatment of complexes (8) and (9) with the appropriate alkylating reagent and molar ratio, in hexane at -78 °C, gives the\ud chloro alkyl derivatives M(1,3-tBu2-@-C5H3XCsHs)CIR [M = Ti, R = Me (10); M = Zr, R = Me (11), Bz (12)] or the dialkyl\ud complexes M(1,3-tBu2-@-CsH3)(CsHs)Rz [M = Ti, R = Me (13); M = Zr, R = Me (14), Bz (15), Nf (16)]. When 8 reacts with 2 equiv.\ud of MgBz2(THF) 2 or LiCH2CMe2Ph the metallacyclic complexes Ti(1-tBu-3-CMe2CH~-r/5-C~Ha)(CsHs)R [R = Bz (17); Nf (18)] were\ud isolated as red oils at room temperature, with the elimination of toluene or ten-butyl benzene respectively. The previously reported\ud cationic mono 1,3-di(tert-butyl)cyclopentadienyl dibenzyl zirconium species [Zr(1,3 -t Bu 2-'05-C 5 H 3 XCH 2 Ph) 2 ] + (19) can be stabilized\ud by reaction with tBuNC or PMe 3, in CD2C12 at -78°C, and the formation of the new cationic species [Zr(1,3-tBu2-r/5-\ud CsHa)(L)(CH2Ph)2] + [L=tBuNC (20); PMe 3 (21)] was identified by NMR spectroscopy. The reaction of B(CrFs) 3 with the\ud monocyclopentadienyl trimethyl derivatives M(1,3-tBu2-r/5-CsH3XCH3)3 [M = Ti (22), Zr (23)], in the presence of PMe 3, gives the\ud cationic species [M(I,3-tBu2-@-C~H3)(PMe3)2(CH3)2] + [M = Ti (24); Zr (25)], obtained as orange-yellow solids, stable at room\ud temperature. The reaction of B(C6Fs) 3 with the metallocene dimethyl derivatives M(1,3-tBu2-r/5-CsHa)(@-CsHs)(CH3)z [M = Zr (14);\ud Hf (26)], in a 1:1 molar ratio and in hydrocarbon solvents gives the cationic derivatives [M(1,3-tBu2-@-CsH3)(@ -\ud CsHsXCH3)]+[(CH3)B(CrFs)3] - [M = Zr (27); Hf (28)] as yellow oils which can be stored for weeks under an inert atmosphere. When\ud the same reactions of (14) and (26) with B(C6Fs) 3 are carried out in a 2:1 molar ratio at room temperature, the complexes\ud {[M(1,3-tBu2-@-CsH3X@-CsH5)Me]2(/.L-Me)}[MeB(C6Fs)3] [M =Zr (29), Hf (30)] can be obtained as a mixture of syn- and\ud anti-isomers as shown by NMR spectroscopic observations. The formation of (29) and (30) implies the stabilization of the 14-electron\ud cationic intermediate by interaction with one methyl group of the neutral complexes (14) and (26). Complexes (27) and (28) undergo\ud heterolytic dissociation of the Metal-MeB(C6Fs) 3 bonds, leading to the formation of the free [M(I,3-tBu2-r/5-CsH3)(r/5-CsHs)(CH3)] +\ud 14-electron species, verified by 1H DNMR spectroscopy. When compound (27) was heated at 50°C the metallacyclic cation\ud [Zr(1-tBu-3-CMezCH2-@-C5H3)(@-CsHs)] + (31) was formed. The alkyl derivatives synthesized and reported herein, activated with MAO, B(C6Fs) 3 or [Ph3C][B(C6Fs)4], polymerize ethylene with very low activity. The molecular structure of [Ti(1,3-tBu2-r/5-\ud C5H3)C1(/x-O)] 3 6 has been determined by X-ray diffraction methods.Financial support for this research by DGICYT (Project PB92-0178C) is gratefully acknowledged. J.I.A.\ud acknowledges Repsol Petróleo S.A. for a fellowship. A.M. is grateful to Consejeria Educaci6n (CAM) for a fellowship