Learning to Become an Expert: Deep Networks Applied To Super-Resolution Microscopy

With super-resolution optical microscopy, it is now possible to observe molecular interactions in living cells. The obtained images have a very high spatial precision but their overall quality can vary a lot depending on the structure of interest and the imaging parameters. Moreover, evaluating this quality is often difficult for non-expert users. In this work, we tackle the problem of learning the quality function of super- resolution images from scores provided by experts. More specifically, we are proposing a system based on a deep neural network that can provide a quantitative quality measure of a STED image of neuronal structures given as input. We conduct a user study in order to evaluate the quality of the predictions of the neural network against those of a human expert. Results show the potential while highlighting some of the limits of the proposed approach.Comment: Accepted to the Thirtieth Innovative Applications of Artificial Intelligence Conference (IAAI), 201

Filtering Pixel Latent Variables for Unmixing Noisy and Undersampled Volumetric Images

The development of robust signal unmixing algorithms is essential for leveraging multimodal datasets acquired through a wide array of scientific imaging technologies, including hyperspectral or time-resolved acquisitions. In experimental physics, enhancing the spatio-temporal resolution or expanding the number of detection channels often leads to diminished sampling rate and signal-to-noise ratio, significantly affecting the efficacy of signal unmixing algorithms. We propose applying band-pass filters to the latent space of a multi-dimensional convolutional neural network to disentangle overlapping signal components, enabling the isolation and quantification of their individual contributions. Using multi-dimensional convolution kernels to process all dimensions simultaneously enhances the network's ability to extract information from adjacent pixels, time- or spectral-bins. This approach enables more effective separation of components in cases where individual pixels do not provide clear, well-resolved information. We showcase the method's practical use in experimental physics through two test cases that highlight the versatility of our approach: fluorescence lifetime microscopy and mode decomposition in optical fibers. The latent unmixing method extracts valuable information from complex signals that cannot be resolved by standard methods. Application of latent unmixing to real FLIM experiments will increase the number of distinguishable fluorescent markers. It will also open new possibilities in optics and photonics for multichannel separations at increased sampling rate.Comment: 16 pages, 8 figures (main paper) + 18 pages, 9 figures (supplementary material

Gold nanoparticle-assisted all optical localized stimulation and monitoring of Ca2+ signaling in neurons

Light-assisted manipulation of cells to control membrane activity or intracellular signaling has become a major avenue in life sciences. However, the ability to perform subcellular light stimulation to investigate localized signaling has been limited. Here, we introduce an all optical method for the stimulation and the monitoring of localized Ca2+ signaling in neurons that takes advantage of plasmonic excitation of gold nanoparticles (AuNPs). We show with confocal microscopy that 800 nm laser pulse application onto a neuron decorated with a few AuNPs triggers a transient increase in free Ca2+, measured optically with GCaMP6s. We show that action potentials, measured electrophysiologically, can be induced with this approach. We demonstrate activation of local Ca2+ transients and Ca2+ signaling via CaMKII in dendritic domains, by illuminating a single or few functionalized AuNPs specifically targeting genetically-modified neurons. This NP-Assisted Localized Optical Stimulation (NALOS) provides a new complement to light-dependent methods for controlling neuronal activity and cell signaling

A New Twist in the Photophysics of the GFP Chromophore: A Volume-Conserving Molecular Torsion Couple

The simple structure of the chromophore of the green fluorescent protein (GFP), a phenol and an imidazolone ring linked by a methyne bridge, supports an exceptionally diverse range of excited state phenomena. Here we describe experimentally and theoretically the photochemistry of a novel sterically crowded nonplanar derivative of the GFP chromophore. It undergoes an excited state isomerization reaction accompanied by an exceptionally fast (sub 100 fs) excited state decay. The decay dynamics are essentially independent of solvent polarity and viscosity. Excited state structural dynamics are probed by high level quantum chemical calculations revealing that the fast decay is due to a conical intersection characterized by a twist of the rings and pyramidalization of the methyne bridge carbon. The intersection can be accessed without a barrier from the pre-twisted Franck-Condon structure, and the lack of viscosity dependence is due to the fact that the rings twist in the same direction, giving rise to a volume-conserving decay coordinate. Moreover, the rotation of the phenyl, methyl and imidazolone groups are coupled in the sterically crowded structure, with the methyl group translating the rotation of one ring to the next. As a consequence, the excited state dynamics can be viewed as a torsional couple, where the absorbed photon energy leads to conversion of the out-of-plane orientation from one ring to the other in a volume conserving fashion. A similar modification of the range of methyne dyes may provide a new family of devices for molecular machines, specifically torsional couples

Author response

Despite remarkable developments in diffraction unlimited super-resolution microscopy, in vivo nanoscopy of tissues and model organisms is still not satisfactorily established and rarely realized. RESOLFT nanoscopy is particularly suited for live cell imaging because it requires relatively low light levels to overcome the diffraction barrier. Previously, we introduced the reversibly switchable fluorescent protein rsEGFP2, which facilitated fast RESOLFT nanoscopy (Grotjohann et al., 2012). In that study, as in most other nanoscopy studies, only cultivated single cells were analyzed. Here, we report on the use of rsEGFP2 for live-cell RESOLFT nanoscopy of sub-cellular structures of intact Drosophila melanogaster larvae and of resected tissues. We generated flies expressing fusion proteins of alpha-tubulin and rsEGFP2 highlighting the microtubule cytoskeleton in all cells. By focusing through the intact larval cuticle, we achieved lateral resolution of <60 nm. RESOLFT nanoscopy enabled time-lapse recordings comprising 40 images and facilitated recordings 40 µm deep within fly tissues

Understanding the nervous system: Lessons from Frontiers in Neurophotonics

The Frontiers in Neurophotonics Symposium is a biennial event that brings together neurobiologists and physicists/engineers who share interest in the development of leading-edge photonics-based approaches to understand and manipulate the nervous system, from its individual molecular components to complex networks in the intact brain. In this Community paper, we highlight several topics that have been featured at the symposium that took place in October 2022 in Québec City, Canada

Annotation Cost-Sensitive Deep Active Learning with Limited Data (Student Abstract)

Deep learning is a promising avenue to automate tedious analysis tasks in biomedical imaging. However, its application in such a context is limited by the large amount of labeled data required to train deep learning models. While active learning may be used to reduce the amount of labeling data, many approaches do not consider the cost of annotating, which is often significant in a biomedical imaging setting. In this work we show how annotation cost can be considered and learned during active learning on a classification task on the MNIST dataset

Annotation Cost-Sensitive Deep Active Learning with Limited Data (Student Abstract)

Deep learning is a promising avenue to automate tedious analysis tasks in biomedical imaging. However, its application in such a context is limited by the large amount of labeled data required to train deep learning models. While active learning may be used to reduce the amount of labeling data, many approaches do not consider the cost of annotating, which is often significant in a biomedical imaging setting. In this work we show how annotation cost can be considered and learned during active learning on a classification task on the MNIST dataset.</jats:p