Data-Driven Sparse Structure Selection for Deep Neural Networks

Deep convolutional neural networks have liberated its extraordinary power on various tasks. However, it is still very challenging to deploy state-of-the-art models into real-world applications due to their high computational complexity. How can we design a compact and effective network without massive experiments and expert knowledge? In this paper, we propose a simple and effective framework to learn and prune deep models in an end-to-end manner. In our framework, a new type of parameter -- scaling factor is first introduced to scale the outputs of specific structures, such as neurons, groups or residual blocks. Then we add sparsity regularizations on these factors, and solve this optimization problem by a modified stochastic Accelerated Proximal Gradient (APG) method. By forcing some of the factors to zero, we can safely remove the corresponding structures, thus prune the unimportant parts of a CNN. Comparing with other structure selection methods that may need thousands of trials or iterative fine-tuning, our method is trained fully end-to-end in one training pass without bells and whistles. We evaluate our method, Sparse Structure Selection with several state-of-the-art CNNs, and demonstrate very promising results with adaptive depth and width selection.Comment: ECCV Camera ready versio

More About the Tetrahedral Unstructured Software System

TetrUSS is a comprehensive suite of computational fluid dynamics (CFD) programs that won the Software of the Year award in 1996 and has found increasing use in government, academia, and industry for solving realistic flow problems (especially in aerodynamics and aeroelastics of aircraft having complex shapes). TetrUSS includes not only programs for solving basic equations of flow but also programs that afford capabilities for efficient generation and utilization of computational grids and for graphical representation of computed flows (see figure). The 2004 version of the Tetrahedral Unstructured Software System (TetrUSS), which is one of two software systems reported in "NASA s 2004 Software of the Year," NASA Tech Briefs, Vol. 28, No. 10 (October 2004), page 18, has been improved greatly since 1996. These improvements include (1) capabilities to simulate viscous flow by solving the Navier-Stokes equations on unstructured grids, (2) portability to personal computers from diverse manufacturers, (3) advanced models of turbulence, (4) a parallel-processing version of one of the unstructured-grid Navier-Stokes-equation-solving programs, and (5) advanced programs for generating unstructured grids

Status of VGRID/USM3D aero analysis system

The topics are presented in viewgraph form and include the following: grid generation; flow solver; graphic postprocessing; dissemination; customer applications; and plans

An unstructured-grid software system for solving complex aerodynamic problems

A coordinated effort has been underway over the past four years to elevate unstructured-grid methodology to a mature level. The goal of this endeavor is to provide a validated capability to non-expert users for performing rapid aerodynamic analysis and design of complex configurations. The Euler component of the system is well developed, and is impacting a broad spectrum of engineering needs with capabilities such as rapid grid generation and inviscid flow analysis, inverse design, interactive boundary layers, and propulsion effects. Progress is also being made in the more tenuous Navier-Stokes component of the system. A robust grid generator is under development for constructing quality thin-layer tetrahedral grids, along with a companion Navier-Stokes flow solver. This paper presents an overview of this effort, along with a perspective on the present and future status of the methodology

Intuition: Myth or a Decision-making Tool?

Faced with today’s ill-structured business environment of fast-paced change and rising uncertainty, organizations have been searching for management tools that will perform satisfactorily under such ambiguous conditions. In the arena of managerial decision making, one of the approaches being assessed is the use of intuition. Based on our definition of intuition as a non-sequential information-processing mode, which comprises both cognitive and affective elements and results in direct knowing without any use of conscious reasoning, we develop a testable model of integrated analytical and intuitive decision making and propose ways to measure the use of intuition

Multiple novel prostate cancer susceptibility signals identified by fine-mapping of known risk loci among Europeans

Genome-wide association studies (GWAS) have identified numerous common prostate cancer (PrCa) susceptibility loci. We have fine-mapped 64 GWAS regions known at the conclusion of the iCOGS study using large-scale genotyping and imputation in 25 723 PrCa cases and 26 274 controls of European ancestry. We detected evidence for multiple independent signals at 16 regions, 12 of which contained additional newly identified significant associations. A single signal comprising a spectrum of correlated variation was observed at 39 regions; 35 of which are now described by a novel more significantly associated lead SNP, while the originally reported variant remained as the lead SNP only in 4 regions. We also confirmed two association signals in Europeans that had been previously reported only in East-Asian GWAS. Based on statistical evidence and linkage disequilibrium (LD) structure, we have curated and narrowed down the list of the most likely candidate causal variants for each region. Functional annotation using data from ENCODE filtered for PrCa cell lines and eQTL analysis demonstrated significant enrichment for overlap with bio-features within this set. By incorporating the novel risk variants identified here alongside the refined data for existing association signals, we estimate that these loci now explain ∼38.9% of the familial relative risk of PrCa, an 8.9% improvement over the previously reported GWAS tag SNPs. This suggests that a significant fraction of the heritability of PrCa may have been hidden during the discovery phase of GWAS, in particular due to the presence of multiple independent signals within the same regio

Association between TNF Receptors and KIM-1 with Kidney Outcomes in Early-Stage Diabetic Kidney Disease

Background and objectives Clinical trials in nephrology are enriched for patients with micro-or macroalbuminuria to enroll patients at risk of kidney failure. However, patients with normoalbuminuria can also progress to kidney failure. TNF receptor-1, TNF receptor-2, and kidney injury marker-1 (KIM-1) are known to be associated with kidney disease progression in patients with micro-or macroalbuminuria. We assessed the value of TNF receptor-1, TNF receptor-2, and KIM-1 as prognostic biomarkers for CKD progression in patients with type 2 diabetes and normoalbuminuria. Design, setting, participants, & measurements TNF receptor-1, TNF receptor-2, and KIM-1 were measured using immunoassays in plasma samples from patients with type 2 diabetes at high cardiovascular risk participating in the Canagliflozin Cardiovascular Assessment Study trial. We used multivariable adjusted Cox proportional hazards analyses to estimate hazard ratios per doubling of each biomarker for the kidney outcome, stratified the population by the fourth quartile of each biomarker distribution, and assessed the number of events and event rates. Results In patients with normoalbuminuria (n=2553), 51 kidney outcomes were recorded during a median follow-up of 6.1 (interquartile range, 5.8–6.4) years (event rate, 3.5; 95% confidence interval, 2.6 to 4.6 per 1000 patient-years). Each doubling of baseline TNF receptor-1 (hazard ratio, 4.2; 95% confidence interval, 1.8 to 9.6) and TNF receptor-2 (hazard ratio, 2.3; 95% confidence interval, 1.5 to 3.6) was associated with a higher risk for the kidney outcome. Baseline KIM-1, urinary albumin-creatinine ratio, and eGFR were not associated with kidney outcomes. The event rates in the highest quartile of TNF receptor-1 (≥2992 ng/ml) and TNF receptor-2 (≥11,394 ng/ml) were 5.6 and 7.0 events per 1000 patient-years, respectively, compared with 2.8 and 2.3, respectively, in the lower three quartiles. Conclusions TNF receptor-1 and TNF receptor-2 are associated with kidney outcomes in patients with type 2 diabetes and normoalbuminuria

Principal Component Analysis Using Structural Similarity Index for Images

Despite the advances of deep learning in specific tasks using images, the principled assessment of image fidelity and similarity is still a critical ability to develop. As it has been shown that Mean Squared Error (MSE) is insufficient for this task, other measures have been developed with one of the most effective being Structural Similarity Index (SSIM). Such measures can be used for subspace learning but existing methods in machine learning, such as Principal Component Analysis (PCA), are based on Euclidean distance or MSE and thus cannot properly capture the structural features of images. In this paper, we define an image structure subspace which discriminates different types of image distortions. We propose Image Structural Component Analysis (ISCA) and also kernel ISCA by using SSIM, rather than Euclidean distance, in the formulation of PCA. This paper provides a bridge between image quality assessment and manifold learning opening a broad new area for future research.Comment: Paper for the methods named "Image Structural Component Analysis (ISCA)" and "Kernel Image Structural Component Analysis (Kernel ISCA)

Timing and Predictors of Recanalization After Anticoagulation in Cerebral Venous Thrombosis.

BACKGROUND AND PURPOSE: Vessel recanalization after cerebral venous thrombosis (CVT) is associated with favorable outcomes and lower mortality. Several studies examined the timing and predictors of recanalization after CVT with mixed results. We aimed to investigate predictors and timing of recanalization after CVT. METHODS: We used data from the multicenter, international AntiCoagulaTION in the Treatment of Cerebral Venous Thrombosis (ACTION-CVT) study of consecutive patients with CVT from January 2015 to December 2020. Our analysis included patients that had undergone repeat venous neuroimaging more than 30 days after initiation of anticoagulation treatment. Prespecified variables were included in univariate and multivariable analyses to identify independent predictors of failure to recanalize. RESULTS: Among the 551 patients (mean age, 44.4±16.2 years, 66.2% women) that met inclusion criteria, 486 (88.2%) had complete or partial, and 65 (11.8%) had no recanalization. The median time to first follow-up imaging study was 110 days (interquartile range, 60-187). In multivariable analysis, older age (odds ratio [OR], 1.05; 95% confidence interval [CI], 1.03-1.07), male sex (OR, 0.44; 95% CI, 0.24-0.80), and lack of parenchymal changes on baseline imaging (OR, 0.53; 95% CI, 0.29-0.96) were associated with no recanalization. The majority of improvement in recanalization (71.1%) occurred before 3 months from initial diagnosis. A high percentage of complete recanalization (59.0%) took place within the first 3 months after CVT diagnosis. CONCLUSION: Older age, male sex, and lack of parenchymal changes were associated with no recanalization after CVT. The majority recanalization occurred early in the disease course suggesting limited further recanalization with anticoagulation beyond 3 months. Large prospective studies are needed to confirm our findings