A Universal Trust-Region Method for Convex and Nonconvex Optimization

This paper presents a universal trust-region method simultaneously incorporating quadratic regularization and the ball constraint. We introduce a novel mechanism to set the parameters in the proposed method that unifies the analysis for convex and nonconvex optimization. Our method exhibits an iteration complexity of $\tilde O(\epsilon^{-3/2})$ to find an approximate second-order stationary point for nonconvex optimization. Meanwhile, the analysis reveals that the universal method attains an $O(\epsilon^{-1/2})$ complexity bound for convex optimization and can be accelerated. These results are complementary to the existing literature as the trust-region method was historically conceived for nonconvex optimization. Finally, we develop an adaptive universal method to address practical implementations. The numerical results show the effectiveness of our method in both nonconvex and convex problems

Trust Region Methods For Nonconvex Stochastic Optimization Beyond Lipschitz Smoothness

In many important machine learning applications, the standard assumption of having a globally Lipschitz continuous gradient may fail to hold. This paper delves into a more general $(L_0, L_1)$-smoothness setting, which gains particular significance within the realms of deep neural networks and distributionally robust optimization (DRO). We demonstrate the significant advantage of trust region methods for stochastic nonconvex optimization under such generalized smoothness assumption. We show that first-order trust region methods can recover the normalized and clipped stochastic gradient as special cases and then provide a unified analysis to show their convergence to first-order stationary conditions. Motivated by the important application of DRO, we propose a generalized high-order smoothness condition, under which second-order trust region methods can achieve a complexity of $\mathcal{O}(\epsilon^{-3.5})$ for convergence to second-order stationary points. By incorporating variance reduction, the second-order trust region method obtains an even better complexity of $\mathcal{O}(\epsilon^{-3})$, matching the optimal bound for standard smooth optimization. To our best knowledge, this is the first work to show convergence beyond the first-order stationary condition for generalized smooth optimization. Preliminary experiments show that our proposed algorithms perform favorably compared with existing methods

Homogeneous Second-Order Descent Framework: A Fast Alternative to Newton-Type Methods

This paper proposes a homogeneous second-order descent framework (HSODF) for nonconvex and convex optimization based on the generalized homogeneous model (GHM). In comparison to the Newton steps, the GHM can be solved by extremal symmetric eigenvalue procedures and thus grant an advantage in ill-conditioned problems. Moreover, GHM extends the ordinary homogeneous model (OHM) to allow adaptiveness in the construction of the aggregated matrix. Consequently, HSODF is able to recover some well-known second-order methods, such as trust-region methods and gradient regularized methods, while maintaining comparable iteration complexity bounds. We also study two specific realizations of HSODF. One is adaptive HSODM, which has a parameter-free $O(\epsilon^{-3/2})$ global complexity bound for nonconvex second-order Lipschitz continuous objective functions. The other one is homotopy HSODM, which is proven to have a global linear rate of convergence without strong convexity. The efficiency of our approach to ill-conditioned and high-dimensional problems is justified by some preliminary numerical results.Comment: improved writin

A Homogenization Approach for Gradient-Dominated Stochastic Optimization

Gradient dominance property is a condition weaker than strong convexity, yet it sufficiently ensures global convergence for first-order methods even in non-convex optimization. This property finds application in various machine learning domains, including matrix decomposition, linear neural networks, and policy-based reinforcement learning (RL). In this paper, we study the stochastic homogeneous second-order descent method (SHSODM) for gradient-dominated optimization with $\alpha \in [1, 2]$ based on a recently proposed homogenization approach. Theoretically, we show that SHSODM achieves a sample complexity of $O(\epsilon^{-7/(2 \alpha) +1})$ for $\alpha \in [1, 3/2)$ and $\tilde{O}(\epsilon^{-2/\alpha})$ for $\alpha \in [3/2, 2]$. We further provide a SHSODM with a variance reduction technique enjoying an improved sample complexity of $O( \epsilon ^{-( 7-3\alpha ) /( 2\alpha )})$ for $\alpha \in [1,3/2)$. Our results match the state-of-the-art sample complexity bounds for stochastic gradient-dominated optimization without \emph{cubic regularization}. Since the homogenization approach only relies on solving extremal eigenvector problems instead of Newton-type systems, our methods gain the advantage of cheaper iterations and robustness in ill-conditioned problems. Numerical experiments on several RL tasks demonstrate the efficiency of SHSODM compared to other off-the-shelf methods

A Homogeneous Second-Order Descent Method for Nonconvex Optimization

In this paper, we introduce a Homogeneous Second-Order Descent Method (HSODM) using the homogenized quadratic approximation to the original function. The merit of homogenization is that only the leftmost eigenvector of a gradient-Hessian integrated matrix is computed at each iteration. Therefore, the algorithm is a single-loop method that does not need to switch to other sophisticated algorithms and is easy to implement. We show that HSODM has a global convergence rate of $O(\epsilon^{-3/2})$ to find an $\epsilon$-approximate second-order stationary point, and has a local quadratic convergence rate under the standard assumptions. The numerical results demonstrate the advantage of the proposed method over other second-order methods.Comment: Add inexactness, significantly improve the pape

cuPDLP-C: A Strengthened Implementation of cuPDLP for Linear Programming by C language

A recent GPU implementation of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programming was proposed in Lu and Yang (2023). Its computational results demonstrate the significant computational advantages of the GPU-based first-order algorithm on certain large-scale problems. The average performance also achieves a level close to commercial solvers for the first time in history. However, due to limitations in experimental hardware and the disadvantage of implementing the algorithm in Julia compared to C language, neither the commercial solver nor cuPDLP reached their maximum efficiency. Therefore, in this report, we have re-implemented and optimized cuPDLP in C language. Utilizing state-of-the-art CPU and GPU hardware, we extensively compare cuPDLP with the best commercial solvers. The experiments further highlight its substantial computational advantages and potential for solving large-scale linear programming problems. We also discuss the profound impact this breakthrough may have on mathematical programming research and the entire operations research community.Comment: fix typos, update numerical result

An Enhanced ADMM-based Interior Point Method for Linear and Conic Optimization

The ADMM-based interior point (ABIP, Lin et al. 2021) method is a hybrid algorithm that effectively combines interior point method (IPM) and first-order methods to achieve a performance boost in large-scale linear optimization. Different from traditional IPM that relies on computationally intensive Newton steps, the ABIP method applies the alternating direction method of multipliers (ADMM) to approximately solve the barrier penalized problem. However, similar to other first-order methods, this technique remains sensitive to condition number and inverse precision. In this paper, we provide an enhanced ABIP method with multiple improvements. Firstly, we develop an ABIP method to solve the general linear conic optimization and establish the associated iteration complexity. Secondly, inspired by some existing methods, we develop different implementation strategies for ABIP method, which substantially improve its performance in linear optimization. Finally, we conduct extensive numerical experiments in both synthetic and real-world datasets to demonstrate the empirical advantage of our developments. In particular, the enhanced ABIP method achieves a 5.8x reduction in the geometric mean of run time on $105$ selected LP instances from Netlib, and it exhibits advantages in certain structured problems such as SVM and PageRank. However, the enhanced ABIP method still falls behind commercial solvers in many benchmarks, especially when high accuracy is desired. We posit that it can serve as a complementary tool alongside well-established solvers

Diversity of Trichoderma species associated with the black rot disease of Gastrodia elata, including four new species

IntroductionTrichoderma species establish symbiotic relationships with plants through both parasitic and mutualistic mechanisms. While some Trichoderma species act as plant pathogenic fungi, others utilize various strategies to protect and enhance plant growth.MethodsPhylogenetic positions of new species of Trichoderma were determined through multi-gene analysis relying on the internal transcribed spacer (ITS) regions of the ribosomal DNA, the translation elongation factor 1-α (tef1-α) gene, and the RNA polymerase II (rpb2) gene. Additionally, pathogenicity experiments were conducted, and the aggressiveness of each isolate was evaluated based on the area of the cross-section of the infected site.ResultsIn this study, 13 Trichoderma species, including 9 known species and 4 new species, namely, T. delicatum, T. robustum, T. perfasciculatum, and T. subulatum were isolated from the diseased tubers of Gastrodia elata in Yunnan, China. Among the known species, T. hamatum had the highest frequency. T. delicatum belonged to the Koningii clade. T. robustum and T. perfasciculatum were assigned to the Virens clade. T. subulatum emerged as a new member of the Spirale clade. Pathogenicity experiments were conducted on the new species T. robustum, T. delicatum, and T. perfasciculatum, as well as the known species T. hamatum, T. atroviride, and T. harzianum. The infective abilities of different Trichoderma species on G. elata varied, indicating that Trichoderma was a pathogenic fungus causing black rot disease in G. elata.DiscussionThis study provided the morphological characteristics of new species and discussed the morphological differences with phylogenetically proximate species, laying the foundation for research aimed at preventing and managing diseases that affect G. elata

Analyses of a chromosome-scale genome assembly reveal the origin and evolution of cultivated chrysanthemum

DATA AVAILABILITY : The raw sequencing data generated in this study have been deposited in the NCBI under accession PRJNA796762 and PRJNA895586 The chloroplast andmitochondrial genome were also available at GenBank under the accession number OP104251 and OP104742 respectively. The assembled genome sequences and annotations are available at Figshare [https://doi.org/10.6084/m9.figshare.21655364.v2]. The Arabidopsis ABCE and chrysanthemum CYC2 genes were used as query sequences for gene family identification, which are available at Figshare [https://doi.org/10.6084/m9.figshare.21610305]. Source data are provided with this paper.Chrysanthemum (Chrysanthemum morifolium Ramat.) is a globally important ornamental plant with great economic, cultural, and symbolic value. However, research on chrysanthemum is challenging due to its complex genetic background. Here, we report a near-complete assembly and annotation for C. morifolium comprising 27 pseudochromosomes (8.15 Gb; scaffold N50 of 303.69Mb). Comparative and evolutionary analyses reveal a whole-genome triplication (WGT) event shared by Chrysanthemum species approximately 6 million years ago (Mya) and the possible lineage-specific polyploidization of C. morifolium approximately 3 Mya. Multilevel evidence suggests that C. morifolium is likely a segmental allopolyploid. Furthermore, a combination of genomics and transcriptomics approaches demonstrate the C. morifolium genome can be used to identify genes underlying key ornamental traits. Phylogenetic analysis of CmCCD4a traces the flower colour breeding history of cultivated chrysanthemum. Genomic resources generated from this study could help to accelerate chrysanthemum genetic improvement.The National Natural Science Foundation of China, the Natural Science Fund of Jiangsu Province, China Agriculture Research System, the National Key Research and Development Program of China, the “JBGS” Project of Seed Industry Revitalisation in Jiangsu Province, the European Union’s Horizon 2020 research and innovation program from European Research Council, the Methusalem funding from Ghent University, and a project funded by the Priority Academic Program Development of Jiangsu Higher Education Institution.https://www.nature.com/ncomms/am2024BiochemistryGeneticsMicrobiology and Plant PathologySDG-15:Life on lan

Two new Trichoderma species (Hypocreales, Hypocreaceae) isolated from decaying tubers of Gastrodia elate

Species of Trichoderma are widely distributed around the world. In this study, two new species in Trichoderma, named as T. albidum and T. variegatum, were introduced and illustrated. These species were isolated from diseased tubers of Gastrodia elata in China and identified based on morphological characteristics and multi-gene sequence analyses of three loci that is the internal transcribed spacer regions of the ribosomal DNA (ITS), the translation elongation factor 1-α encoding gene (tef1-α) and the gene encoding the second largest nuclear RNA polymerase subunit (rpb2). Distinctions between the new species and their close relatives were discussed. According to results of the phylogenetic analyses, T. albidum belonged to the Harzianum clade and T. variegatum are grouped with species of the Spirale clade. The expansion of two clades provided research foundations for the prevention and control of tuber diseases in G. elata