Understanding Self-Supervised Learning of Speech Representation via Invariance and Redundancy Reduction

Self-supervised learning (SSL) has emerged as a promising paradigm for learning flexible speech representations from unlabeled data. By designing pretext tasks that exploit statistical regularities, SSL models can capture useful representations that are transferable to downstream tasks. Barlow Twins (BTs) is an SSL technique inspired by theories of redundancy reduction in human perception. In downstream tasks, BTs representations accelerate learning and transfer this learning across applications. This study applies BTs to speech data and evaluates the obtained representations on several downstream tasks, showing the applicability of the approach. However, limitations exist in disentangling key explanatory factors, with redundancy reduction and invariance alone being insufficient for factorization of learned latents into modular, compact, and informative codes. Our ablation study isolated gains from invariance constraints, but the gains were context-dependent. Overall, this work substantiates the potential of Barlow Twins for sample-efficient speech encoding. However, challenges remain in achieving fully hierarchical representations. The analysis methodology and insights presented in this paper pave a path for extensions incorporating further inductive priors and perceptual principles to further enhance the BTs self-supervision framework

Understanding Self-Supervised Learning of Speech Representation via Invariance and Redundancy Reduction

Self-supervised learning (SSL) has emerged as a promising paradigm for learning flexible speech representations from unlabeled data. By designing pretext tasks that exploit statistical regularities, SSL models can capture useful representations that are transferable to downstream tasks. This study provides an empirical analysis of Barlow Twins (BT), an SSL technique inspired by theories of redundancy reduction in human perception. On downstream tasks, BT representations accelerated learning and transferred across domains. However, limitations exist in disentangling key explanatory factors, with redundancy reduction and invariance alone insufficient for factorization of learned latents into modular, compact, and informative codes. Our ablations study isolated gains from invariance constraints, but the gains were context-dependent. Overall, this work substantiates the potential of Barlow Twins for sample-efficient speech encoding. However, challenges remain in achieving fully hierarchical representations. The analysis methodology and insights pave a path for extensions incorporating further inductive priors and perceptual principles to further enhance the BT self-supervision framework.Comment: 13 pages, 5 figures, in submission to MDPI Informatio

Learning Disentangled Audio Representations through Controlled Synthesis

This paper tackles the scarcity of benchmarking data in disentangled auditory representation learning. We introduce SynTone, a synthetic dataset with explicit ground truth explanatory factors for evaluating disentanglement techniques. Benchmarking state-of-the-art methods on SynTone highlights its utility for method evaluation. Our results underscore strengths and limitations in audio disentanglement, motivating future research.Comment: 12 pages, 12 figures, accepted as a Tiny paper at ICLR 202

Operational experience with the GEM detector assembly lines for the CMS forward muon upgrade

The CMS Collaboration has been developing large-area triple-gas electron multiplier (GEM) detectors to be installed in the muon Endcap regions of the CMS experiment in 2019 to maintain forward muon trigger and tracking performance at the High-Luminosity upgrade of the Large Hadron Collider (LHC); 10 preproduction detectors were built at CERN to commission the first assembly line and the quality controls (QCs). These were installed in the CMS detector in early 2017 and participated in the 2017 LHC run. The collaboration has prepared several additional assembly and QC lines for distributed mass production of 160 GEM detectors at various sites worldwide. In 2017, these additional production sites have optimized construction techniques and QC procedures and validated them against common specifications by constructing additional preproduction detectors. Using the specific experience from one production site as an example, we discuss how the QCs make use of independent hardware and trained personnel to ensure fast and reliable production. Preliminary results on the construction status of CMS GEM detectors are presented with details of the assembly sites involvement

Conflicting results of prenatal FISH with different probes for Down's Syndrome critical regions associated with mosaicism for a de novo del(21)(q22) characterised by molecular karyotyping: Case report

For the rapid detection of common aneuploidies either PCR or Fluorescence in situ hybridisation (FISH) on uncultured amniotic fluid cells are widely used. There are different commercial suppliers providing FISH assays for the detection of trisomies affecting the Down's syndrome critical regions (DSCR) in 21q22. We present a case in which rapid FISH screening with different commercial probes for the DSCR yielded conflicting results. Chromosome analysis revealed a deletion of one chromosome 21 in q22 which explained the findings. Prenatally an additional small supernumerary marker chromosome (sSMC) was discovered as well, which could not be characterised. Postnatal chromosome analysis in lymphocytes of the infant revealed complex mosaicism with four cell lines. By arrayCGH the sSMC was provisionally described as derivative chromosome 21 which was confirmed by targeted FISH experiments

Severe early onset preeclampsia: short and long term clinical, psychosocial and biochemical aspects

Preeclampsia is a pregnancy specific disorder commonly defined as de novo hypertension and proteinuria after 20 weeks gestational age. It occurs in approximately 3-5% of pregnancies and it is still a major cause of both foetal and maternal morbidity and mortality worldwide1. As extensive research has not yet elucidated the aetiology of preeclampsia, there are no rational preventive or therapeutic interventions available. The only rational treatment is delivery, which benefits the mother but is not in the interest of the foetus, if remote from term. Early onset preeclampsia (<32 weeks’ gestational age) occurs in less than 1% of pregnancies. It is, however often associated with maternal morbidity as the risk of progression to severe maternal disease is inversely related with gestational age at onset2. Resulting prematurity is therefore the main cause of neonatal mortality and morbidity in patients with severe preeclampsia3. Although the discussion is ongoing, perinatal survival is suggested to be increased in patients with preterm preeclampsia by expectant, non-interventional management. This temporising treatment option to lengthen pregnancy includes the use of antihypertensive medication to control hypertension, magnesium sulphate to prevent eclampsia and corticosteroids to enhance foetal lung maturity4. With optimal maternal haemodynamic status and reassuring foetal condition this results on average in an extension of 2 weeks. Prolongation of these pregnancies is a great challenge for clinicians to balance between potential maternal risks on one the eve hand and possible foetal benefits on the other. Clinical controversies regarding prolongation of preterm preeclamptic pregnancies still exist – also taking into account that preeclampsia is the leading cause of maternal mortality in the Netherlands5 - a debate which is even more pronounced in very preterm pregnancies with questionable foetal viability6-9. Do maternal risks of prolongation of these very early pregnancies outweigh the chances of neonatal survival? Counselling of women with very early onset preeclampsia not only comprises of knowledge of the outcome of those particular pregnancies, but also knowledge of outcomes of future pregnancies of these women is of major clinical importance. This thesis opens with a review of the literature on identifiable risk factors of preeclampsia

Impacts of the Tropical Pacific/Indian Oceans on the Seasonal Cycle of the West African Monsoon

The current consensus is that drought has developed in the Sahel during the second half of the twentieth century as a result of remote effects of oceanic anomalies amplified by local land–atmosphere interactions. This paper focuses on the impacts of oceanic anomalies upon West African climate and specifically aims to identify those from SST anomalies in the Pacific/Indian Oceans during spring and summer seasons, when they were significant. Idealized sensitivity experiments are performed with four atmospheric general circulation models (AGCMs). The prescribed SST patterns used in the AGCMs are based on the leading mode of covariability between SST anomalies over the Pacific/Indian Oceans and summer rainfall over West Africa. The results show that such oceanic anomalies in the Pacific/Indian Ocean lead to a northward shift of an anomalous dry belt from the Gulf of Guinea to the Sahel as the season advances. In the Sahel, the magnitude of rainfall anomalies is comparable to that obtained by other authors using SST anomalies confined to the proximity of the Atlantic Ocean. The mechanism connecting the Pacific/Indian SST anomalies with West African rainfall has a strong seasonal cycle. In spring (May and June), anomalous subsidence develops over both the Maritime Continent and the equatorial Atlantic in response to the enhanced equatorial heating. Precipitation increases over continental West Africa in association with stronger zonal convergence of moisture. In addition, precipitation decreases over the Gulf of Guinea. During the monsoon peak (July and August), the SST anomalies move westward over the equatorial Pacific and the two regions where subsidence occurred earlier in the seasons merge over West Africa. The monsoon weakens and rainfall decreases over the Sahel, especially in August.Peer reviewe

Measurements of the pp → ZZ production cross section and the Z → 4ℓ branching fraction, and constraints on anomalous triple gauge couplings at √s = 13 TeV

Four-lepton production in proton-proton collisions, pp -> (Z/gamma*)(Z/gamma*) -> 4l, where l = e or mu, is studied at a center-of-mass energy of 13 TeV with the CMS detector at the LHC. The data sample corresponds to an integrated luminosity of 35.9 fb(-1). The ZZ production cross section, sigma(pp -> ZZ) = 17.2 +/- 0.5 (stat) +/- 0.7 (syst) +/- 0.4 (theo) +/- 0.4 (lumi) pb, measured using events with two opposite-sign, same-flavor lepton pairs produced in the mass region 60 4l) = 4.83(-0.22)(+0.23) (stat)(-0.29)(+0.32) (syst) +/- 0.08 (theo) +/- 0.12(lumi) x 10(-6) for events with a four-lepton invariant mass in the range 80 4GeV for all opposite-sign, same-flavor lepton pairs. The results agree with standard model predictions. The invariant mass distribution of the four-lepton system is used to set limits on anomalous ZZZ and ZZ. couplings at 95% confidence level: -0.0012 < f(4)(Z) < 0.0010, -0.0010 < f(5)(Z) < 0.0013, -0.0012 < f(4)(gamma) < 0.0013, -0.0012 < f(5)(gamma) < 0.0013

Penilaian Kinerja Keuangan Koperasi di Kabupaten Pelalawan

This paper describe development and financial performance of cooperative in District Pelalawan among 2007 - 2008. Studies on primary and secondary cooperative in 12 sub-districts. Method in this stady use performance measuring of productivity, efficiency, growth, liquidity, and solvability of cooperative. Productivity of cooperative in Pelalawan was highly but efficiency still low. Profit and income were highly, even liquidity of cooperative very high, and solvability was good