1,202 research outputs found

    Cooperation Does Matter: Exploring Multi-Order Bilateral Relations for Audio-Visual Segmentation

    Full text link
    Recently, an audio-visual segmentation (AVS) task has been introduced, aiming to group pixels with sounding objects within a given video. This task necessitates a first-ever audio-driven pixel-level understanding of the scene, posing significant challenges. In this paper, we propose an innovative audio-visual transformer framework, termed COMBO, an acronym for COoperation of Multi-order Bilateral relatiOns. For the first time, our framework explores three types of bilateral entanglements within AVS: pixel entanglement, modality entanglement, and temporal entanglement. Regarding pixel entanglement, we employ a Siam-Encoder Module (SEM) that leverages prior knowledge to generate more precise visual features from the foundational model. For modality entanglement, we design a Bilateral-Fusion Module (BFM), enabling COMBO to align corresponding visual and auditory signals bi-directionally. As for temporal entanglement, we introduce an innovative adaptive inter-frame consistency loss according to the inherent rules of temporal. Comprehensive experiments and ablation studies on AVSBench-object (84.7 mIoU on S4, 59.2 mIou on MS3) and AVSBench-semantic (42.1 mIoU on AVSS) datasets demonstrate that COMBO surpasses previous state-of-the-art methods. Code and more results will be publicly available at https://yannqi.github.io/AVS-COMBO/.Comment: CVPR 2024 Highlight. 13 pages, 10 figure

    Mitochondrial Membrane Remodeling

    Get PDF
    Mitochondria are key regulators of many important cellular processes and their dysfunction has been implicated in a large number of human disorders. Importantly, mitochondrial function is tightly linked to their ultrastructure, which possesses an intricate membrane architecture defining specific submitochondrial compartments. In particular, the mitochondrial inner membrane is highly folded into membrane invaginations that are essential for oxidative phosphorylation. Furthermore, mitochondrial membranes are highly dynamic and undergo constant membrane remodeling during mitochondrial fusion and fission. It has remained enigmatic how these membrane curvatures are generated and maintained, and specific factors involved in these processes are largely unknown. This review focuses on the current understanding of the molecular mechanism of mitochondrial membrane architectural organization and factors critical for mitochondrial morphogenesis, as well as their functional link to human diseases.Peer reviewe

    Permanent disappearance and seasonal fluctuation of urban lake area in Wuhan, China monitored with long time series remotely sensed images from 1987 to 2016

    Get PDF
    Lakes are important to the healthy functioning of the urban ecosystem. The urban lakes in Wuhan, China, which is known as ‘city of hundreds of lakes’, are facing substantial threats mainly due to rapid urbanization. This paper focused on detecting the spatial and temporal change of urban lakes in Wuhan, using a long time series of Landsat and HJ-1A remotely sensed data from 1987 to 2016. The permanent disappearance and seasonal fluctuation of 28 main urban lakes were analysed, and their relationships with climatic change and human activities were discussed. The results show that most lakes in Wuhan had shrunk over the past 30 years resulting in a permanent change from water to land. The shrinkage was also most apparent in the central region of the city. Seasonal fluctuations of lake area were evident for most lakes but the relative important driving variable of lake area change varied between sub-periods of time for different lakes. The explanatory power of impervious surface to five-year permanent water change is 91.75%, suggesting that urbanization – as increasing impervious surface – had led to the shrinkage of urban lakes in Wuhan. In all, 128.28 km2 five-year permanent water disappeared from 1987 to 2016

    Mitochondrial protein dysfunction in pathogenesis of neurological diseases

    Get PDF
    Mitochondria are essential organelles for neuronal function and cell survival. Besides the well-known bioenergetics, additional mitochondrial roles in calcium signaling, lipid biogenesis, regulation of reactive oxygen species, and apoptosis are pivotal in diverse cellular processes. The mitochondrial proteome encompasses about 1,500 proteins encoded by both the nuclear DNA and the maternally inherited mitochondrial DNA. Mutations in the nuclear or mitochondrial genome, or combinations of both, can result in mitochondrial protein deficiencies and mitochondrial malfunction. Therefore, mitochondrial quality control by proteins involved in various surveillance mechanisms is critical for neuronal integrity and viability. Abnormal proteins involved in mitochondrial bioenergetics, dynamics, mitophagy, import machinery, ion channels, and mitochondrial DNA maintenance have been linked to the pathogenesis of a number of neurological diseases. The goal of this review is to give an overview of these pathways and to summarize the interconnections between mitochondrial protein dysfunction and neurological diseases.Peer reviewe

    Single-cell multiomics of the human retina reveals hierarchical transcription factor collaboration in mediating cell type-specific effects of genetic variants on gene regulation

    Get PDF
    BACKGROUND: Systematic characterization of how genetic variation modulates gene regulation in a cell type-specific context is essential for understanding complex traits. To address this question, we profile gene expression and chromatin accessibility in cells from healthy retinae of 20 human donors through single-cell multiomics and genomic sequencing. RESULTS: We map eQTL, caQTL, allelic-specific expression, and allelic-specific chromatin accessibility in major retinal cell types. By integrating these results, we identify and characterize regulatory elements and genetic variants effective on gene regulation in individual cell types. The majority of identified sc-eQTLs and sc-caQTLs display cell type-specific effects, while the cis-elements containing genetic variants with cell type-specific effects are often accessible in multiple cell types. Furthermore, the transcription factors whose binding sites are perturbed by genetic variants tend to have higher expression levels in the cell types where the variants exert their effects, compared to the cell types where the variants have no impact. We further validate our findings with high-throughput reporter assays. Lastly, we identify the enriched cell types, candidate causal variants and genes, and cell type-specific regulatory mechanism underlying GWAS loci. CONCLUSIONS: Overall, genetic effects on gene regulation are highly context dependent. Our results suggest that cell type-dependent genetic effect is driven by precise modulation of both trans-factor expression and chromatin accessibility of cis-elements. Our findings indicate hierarchical collaboration among transcription factors plays a crucial role in mediating cell type-specific effects of genetic variants on gene regulation

    Change Detection Methods for Remote Sensing in the Last Decade: A Comprehensive Review

    Get PDF
    Change detection is an essential and widely utilized task in remote sensing that aims to detect and analyze changes occurring in the same geographical area over time, which has broad applications in urban development, agricultural surveys, and land cover monitoring. Detecting changes in remote sensing images is a complex challenge due to various factors, including variations in image quality, noise, registration errors, illumination changes, complex landscapes, and spatial heterogeneity. In recent years, deep learning has emerged as a powerful tool for feature extraction and addressing these challenges. Its versatility has resulted in its widespread adoption for numerous image-processing tasks. This paper presents a comprehensive survey of significant advancements in change detection for remote sensing images over the past decade. We first introduce some preliminary knowledge for the change detection task, such as problem definition, datasets, evaluation metrics, and transformer basics, as well as provide a detailed taxonomy of existing algorithms from three different perspectives: algorithm granularity, supervision modes, and frameworks in the Methodology section. This survey enables readers to gain systematic knowledge of change detection tasks from various angles. We then summarize the state-of-the-art performance on several dominant change detection datasets, providing insights into the strengths and limitations of existing algorithms. Based on our survey, some future research directions for change detection in remote sensing are well identified. This survey paper sheds some light the topic for the community and will inspire further research efforts in the change detection task.</jats:p

    Pruning random resistive memory for optimizing analogue AI

    Full text link
    The rapid advancement of artificial intelligence (AI) has been marked by the large language models exhibiting human-like intelligence. However, these models also present unprecedented challenges to energy consumption and environmental sustainability. One promising solution is to revisit analogue computing, a technique that predates digital computing and exploits emerging analogue electronic devices, such as resistive memory, which features in-memory computing, high scalability, and nonvolatility. However, analogue computing still faces the same challenges as before: programming nonidealities and expensive programming due to the underlying devices physics. Here, we report a universal solution, software-hardware co-design using structural plasticity-inspired edge pruning to optimize the topology of a randomly weighted analogue resistive memory neural network. Software-wise, the topology of a randomly weighted neural network is optimized by pruning connections rather than precisely tuning resistive memory weights. Hardware-wise, we reveal the physical origin of the programming stochasticity using transmission electron microscopy, which is leveraged for large-scale and low-cost implementation of an overparameterized random neural network containing high-performance sub-networks. We implemented the co-design on a 40nm 256K resistive memory macro, observing 17.3% and 19.9% accuracy improvements in image and audio classification on FashionMNIST and Spoken digits datasets, as well as 9.8% (2%) improvement in PR (ROC) in image segmentation on DRIVE datasets, respectively. This is accompanied by 82.1%, 51.2%, and 99.8% improvement in energy efficiency thanks to analogue in-memory computing. By embracing the intrinsic stochasticity and in-memory computing, this work may solve the biggest obstacle of analogue computing systems and thus unleash their immense potential for next-generation AI hardware
    corecore