Neural Collapse in Multi-label Learning with Pick-all-label Loss

We study deep neural networks for the multi-label classification (MLab) task through the lens of neural collapse (NC). Previous works have been restricted to the multi-class classification setting and discovered a prevalent NC phenomenon comprising of the following properties for the last-layer features: (i) the variability of features within every class collapses to zero, (ii) the set of feature means form an equi-angular tight frame (ETF), and (iii) the last layer classifiers collapse to the feature mean upon some scaling. We generalize the study to multi-label learning, and prove for the first time that a generalized NC phenomenon holds with the "pick-all-label" formulation, which we term as MLab NC. While the ETF geometry remains consistent for features with a single label, multi-label scenarios introduce a unique combinatorial aspect we term the "tag-wise average" property, where the means of features with multiple labels are the scaled averages of means for single-label instances. Theoretically, under proper assumptions on the features, we establish that the only global optimizer of the pick-all-label cross-entropy loss satisfy the multi-label NC. In practice, we demonstrate that our findings can lead to better test performance with more efficient training techniques for MLab learning

Revisiting O(N)O(N) σ\sigma model at unphysical pion masses and high temperatures

Roy-equation analyses on lattice data of $\pi\pi$ scattering phase shifts at $m_\pi=391$MeV reveals that the lowest $f_0$ meson becomes a bound state under this condition. In addition, there is a pair of complex poles below threshold generated by crossing symmetry [X.-H. Cao et al., Phys. Rev. D 108, 034009 (2023)]. We use the $N/D$ method to partially recover crossing symmetry of the $O(N)$ $\sigma$ model amplitude at leading order of $1/N$ expansion, and qualitatively reproduce the pole structure and pole trajectories with varying pion masses as revealed by Roy-equation analyses. The $\sigma$ pole trajectory with varying temperature is also discussed and found to be similar to its properties when varying $m_\pi$. As the temperature increases, the complex $\sigma$ poles firstly move from the second Riemann sheet to the real axis becoming two virtual state poles, and then one virtual state pole moves to the first sheet turning into a bound state pole and finally tends to the pion pole position at high temperature which is as expected from the chiral symmetry restoration. Our results provide further evidences that the lowest $f_0$ state extracted from experiments and lattice data plays the role of $\sigma$ meson in the spontaneous breaking of chiral symmetry. Finally, we also briefly discuss the problems of the effective potential in the situation when $m_\pi$ and temperature get large.Comment: 13 pages, 5 figures, 1 table; to match the published versio

Are All Losses Created Equal: A Neural Collapse Perspective

While cross entropy (CE) is the most commonly used loss to train deep neural networks for classification tasks, many alternative losses have been developed to obtain better empirical performance. Among them, which one is the best to use is still a mystery, because there seem to be multiple factors affecting the answer, such as properties of the dataset, the choice of network architecture, and so on. This paper studies the choice of loss function by examining the last-layer features of deep networks, drawing inspiration from a recent line work showing that the global optimal solution of CE and mean-square-error (MSE) losses exhibits a Neural Collapse phenomenon. That is, for sufficiently large networks trained until convergence, (i) all features of the same class collapse to the corresponding class mean and (ii) the means associated with different classes are in a configuration where their pairwise distances are all equal and maximized. We extend such results and show through global solution and landscape analyses that a broad family of loss functions including commonly used label smoothing (LS) and focal loss (FL) exhibits Neural Collapse. Hence, all relevant losses(i.e., CE, LS, FL, MSE) produce equivalent features on training data. Based on the unconstrained feature model assumption, we provide either the global landscape analysis for LS loss or the local landscape analysis for FL loss and show that the (only!) global minimizers are neural collapse solutions, while all other critical points are strict saddles whose Hessian exhibit negative curvature directions either in the global scope for LS loss or in the local scope for FL loss near the optimal solution. The experiments further show that Neural Collapse features obtained from all relevant losses lead to largely identical performance on test data as well, provided that the network is sufficiently large and trained until convergence.Comment: 32 page, 10 figures, NeurIPS 202

Lack of association between the GRP78 polymorphisms in the promoter and 3' UTR and susceptibility to chronic HBV infection in a Chinese Han population

<p>Abstract</p> <p>Background</p> <p>Hepatitis B virus (HBV) infection causes large amount of unfolding or false-folding protein accumulation in the endoplasmic reticulum (ER), which in turn induces the expression of glucose-regulated protein 78 (GRP78). The aim in the present study was to analyse the potential association between GRP78 single-nucleotide polymorphisms (SNPs) and the risk of HBV infection.</p> <p>Methods</p> <p>The associations between seven common <it>GRP78 </it>polymorphisms in the promoter (rs391957, rs17840762, rs17840761, rs11355458) and in the 3' untranslated region (UTR) (rs16927997, rs1140763, rs12009) and possible risk of chronic HBV infection were assessed in a case-control study. 496 cases and 539 individually matched healthy controls were genotyped.</p> <p>Results</p> <p>Overall, no associations were observed in genotypic analyses. In addition, haplotypes and diplotypes combining those SNPs in the promoter or in the 3' UTR in high linkage disequilibrium (LD) were also not associated with HBV risk.</p> <p>Conclusion</p> <p>These observations do not support a role for <it>GRP78 </it>polymorphisms in HBV infection in a predominantly Chinese Han population.</p

Effect of acetone extract of Rumex japonicas Houtt on hydrogen peroxide-induced apoptosis in rat myocardial cells

Purpose: To investigate the protective effect of the acetone extract of Rumex japonicas Houtt. (AER) on rat myocardial cells.Methods: R. japonicas was extracted with 75 % aqueous ethanol by reflux to afford total extract (TER). TER was suspended in water and then extracted with acetone to afford acetone fraction of R. japonicas (AER). High performance liquid chromatography (HPLC) combined with standard substances was carried out to analyze the major constituents of AER. Apoptosis in myocardial H9c2 cell line was induced by H2O2 (100 μmol/L). The cells were treated with AER (50, 100 and 200 μg/mL, and cell viability was evaluated by the 3-（4,5-dimethyl-2-thiazolyl)-2,5-diphenyl-2-H-tetrazolium bromide (MTT) assay, while oxidative stress level in H9c2 cells was evaluated by determining levels of lactate dehydrogenase (LDH), malondialdehyde (MDA), creatinine kinase (CK), superoxide dismutase (SOD), and catalase (CAT). Furthermore, apoptotic proteins (caspase-3, Bax and Bcl-2) in H9c2 cells were analyzed by using western blot assay.Results: Results revealed that the main components of AER are aloe-emodin, rhein, emodin, chrysophanol and physcion. AER (50, 100 and 200 μg/mL) inhibited the cell viability reduction of the H9c2 cells induced by H2O2 (p &lt; 0.05, p &lt; 0.01, p &lt; 0.01, respectively). AER (50, 100 and 200 μg/mL) decreased LDH and CK contents of H9c2 cells (p &lt; 0.01). The levels of SOD (p&lt;0.01) and CAT (p &lt; 0.01) were increased by AER treatments (100 and 200 μg/mL); in addition, AER (50, 100 and 200 μg/mL) decreased MDA levels (p &lt; 0.01). Besides, the present results also revealed that AER could down-regulate caspase-3 and Bax, but up-regulated Bcl-2.Conclusion: AER alleviates apoptosis induced by H2O2 in myocardial H9c2 cells via inhibition of oxidative stress and mitochondria-mediated apoptosis. This finding suggests that AER can potentially be developed for the treatment of myocardial apoptosis.Keywords: Rumex japonicas Houtt., Myocardial cells, Apoptosis, H9c2 cell, Oxidative stres