SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation

Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous learned tasks. In this paper we present a method to overcome catastrophic forgetting on convolutional neural networks, that learns new tasks and preserves the performance on old tasks without accessing the data of the original model, by selective network augmentation. The experiment results showed that SeNA-CNN, in some scenarios, outperforms the state-of-art Learning without Forgetting algorithm. Results also showed that in some situations it is better to use SeNA-CNN instead of training a neural network using isolated learning.info:eu-repo/semantics/publishedVersio

Molecular and cellular limits to somatosensory specificity

Animals detect environmental changes through sensory neural mechanisms that enable them to differentiate the quality, intensity and temporal characteristics of stimuli. The 'doctrine of specific nervous energies' postulates that the different sensory modalities experienced by humans result of the activation of specific nervous pathways. Identification of functional classes of sensory receptors provided scientific support to the concept that somatosensory modalities (touch, pain, temperature, kinesthesis) are subserved by separate populations of sensory receptor neurons specialized in detecting innocuous and injurious stimuli of different quality (mechanical forces, temperature, chemical compounds). The identification of receptor proteins activated by different physicochemical stimuli, in particular ion channels of the Transient Receptor Potential (TRP) superfamily, has put forward the concept that specificity of peripheral sensory receptor neurons is determined by their expression of a particular "molecular sensor" that confers to each functional type its selectivity to respond with a discharge of nerve impulses to stimuli of a given quality. Nonetheless, recent experimental data suggest that the various molecular sensors proposed as specific transducer molecules for stimuli of different quality are not as neatly associated with the distinct functional types of sensory receptors as originally proposed. First, many ion channel molecules initially associated to the transduction of only one particular form of energy are also activated by stimuli of different quality, implying a limited degree of specificity in their transducing capacities. Second, molecular sensors associated with a stimulus quality and hence to a sensory receptor type and ultimately to a sensory modality may be concomitantly expressed in sensory receptor neurons functionally defined as specific for another stimulus quality. Finally, activation of voltage gated channels involved primarily in nerve impulse generation can also influence the gating of transducing channels, dramatically modifying their activation profile. Thus, we propose that the capacity exhibited by the different functional types of somatosensory receptor neurons to preferentially detect and encode specific stimuli into a discharge of nerve impulses, appears to result of a characteristic combinatorial expression of different ion channels in each neuronal type that finally determines their transduction and impulse firing properties. Transduction channels don't operate in isolation and their cellular context should also be taken into consideration to fully understand their function. Moreover, the inhomogeneous distribution of transduction and voltage-gated channels at soma, axonal branches and peripheral endings of primary sensory neurons influences the characteristics of the propagated impulse discharge that encodes the properties of the stimulus. Alteration of this concerted operation of ion channels in pathological conditions may underlie the changes in excitability accompanying peripheral sensory neuron injuries

A Comprehensive Workflow for General-Purpose Neural Modeling with Highly Configurable Neuromorphic Hardware Systems

In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware-experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware mapping process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware-software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware mapping software. The functionality and flexibility of the latter is proven with a variety of experimental results

New insights into the classification and nomenclature of cortical GABAergic interneurons.

A systematic classification and accepted nomenclature of neuron types is much needed but is currently lacking. This article describes a possible taxonomical solution for classifying GABAergic interneurons of the cerebral cortex based on a novel, web-based interactive system that allows experts to classify neurons with pre-determined criteria. Using Bayesian analysis and clustering algorithms on the resulting data, we investigated the suitability of several anatomical terms and neuron names for cortical GABAergic interneurons. Moreover, we show that supervised classification models could automatically categorize interneurons in agreement with experts' assignments. These results demonstrate a practical and objective approach to the naming, characterization and classification of neurons based on community consensus

Can computational efficiency alone drive the evolution of modularity in neural networks?

Some biologists have abandoned the idea that computational efficiency in processing multipart tasks or input sets alone drives the evolution of modularity in biological networks. A recent study confirmed that small modular (neural) networks are relatively computationally-inefficient but large modular networks are slightly more efficient than non-modular ones. The present study determines whether these efficiency advantages with network size can drive the evolution of modularity in networks whose connective architecture can evolve. The answer is no, but the reason why is interesting. All simulations (run in a wide variety of parameter states) involving gradualistic connective evolution end in non-modular local attractors. Thus while a high performance modular attractor exists, such regions cannot be reached by gradualistic evolution. Non-gradualistic evolutionary simulations in which multi-modularity is obtained through duplication of existing architecture appear viable. Fundamentally, this study indicates that computational efficiency alone does not drive the evolution of modularity, even in large biological networks, but it may still be a viable mechanism when networks evolve by non-gradualistic means

A framework for the first‑person internal sensation of visual perception in mammals and a comparable circuitry for olfactory perception in Drosophila

Perception is a first-person internal sensation induced within the nervous system at the time of arrival of sensory stimuli from objects in the environment. Lack of access to the first-person properties has limited viewing perception as an emergent property and it is currently being studied using third-person observed findings from various levels. One feasible approach to understand its mechanism is to build a hypothesis for the specific conditions and required circuit features of the nodal points where the mechanistic operation of perception take place for one type of sensation in one species and to verify it for the presence of comparable circuit properties for perceiving a different sensation in a different species. The present work explains visual perception in mammalian nervous system from a first-person frame of reference and provides explanations for the homogeneity of perception of visual stimuli above flicker fusion frequency, the perception of objects at locations different from their actual position, the smooth pursuit and saccadic eye movements, the perception of object borders, and perception of pressure phosphenes. Using results from temporal resolution studies and the known details of visual cortical circuitry, explanations are provided for (a) the perception of rapidly changing visual stimuli, (b) how the perception of objects occurs in the correct orientation even though, according to the third-person view, activity from the visual stimulus reaches the cortices in an inverted manner and (c) the functional significance of well-conserved columnar organization of the visual cortex. A comparable circuitry detected in a different nervous system in a remote species-the olfactory circuitry of the fruit fly Drosophila melanogaster-provides an opportunity to explore circuit functions using genetic manipulations, which, along with high-resolution microscopic techniques and lipid membrane interaction studies, will be able to verify the structure-function details of the presented mechanism of perception

Factors Affecting Frequency Discrimination of Vibrotactile Stimuli: Implications for Cortical Encoding

BACKGROUND: Measuring perceptual judgments about stimuli while manipulating their physical characteristics can uncover the neural algorithms underlying sensory processing. We carried out psychophysical experiments to examine how humans discriminate vibrotactile stimuli. METHODOLOGY/PRINCIPAL FINDINGS: Subjects compared the frequencies of two sinusoidal vibrations applied sequentially to one fingertip. Performance was reduced when (1) the root mean square velocity (or energy) of the vibrations was equated by adjusting their amplitudes, and (2) the vibrations were noisy (their temporal structure was irregular). These effects were super-additive when subjects compared noisy vibrations that had equal velocity, indicating that frequency judgments became more dependent on the vibrations' temporal structure when differential information about velocity was eliminated. To investigate which areas of the somatosensory system use information about velocity and temporal structure, we required subjects to compare vibrations applied sequentially to opposite hands. This paradigm exploits the fact that tactile input to neurons at early levels (e.g., the primary somatosensory cortex, SI) is largely confined to the contralateral side of the body, so these neurons are less able to contribute to vibration comparisons between hands. The subjects' performance was still sensitive to differences in vibration velocity, but became less sensitive to noise. CONCLUSIONS/SIGNIFICANCE: We conclude that vibration frequency is represented in different ways by different mechanisms distributed across multiple cortical regions. Which mechanisms support the “readout” of frequency varies according to the information present in the vibration. Overall, the present findings are consistent with a model in which information about vibration velocity is coded in regions beyond SI. While adaptive processes within SI also contribute to the representation of frequency, this adaptation is influenced by the temporal regularity of the vibration

Invariant computations in local cortical networks with balanced excitation and inhibition

[Abstract] Cortical computations critically involve local neuronal circuits. The computations are often invariant across a cortical area yet are carried out by networks that can vary widely within an area according to its functional architecture. Here we demonstrate a mechanism by which orientation selectivity is computed invariantly in cat primary visual cortex across an orientation preference map that provides a wide diversity of local circuits. Visually evoked excitatory and inhibitory synaptic conductances are balanced exquisitely in cortical neurons and thus keep the spike response sharply tuned at all map locations. This functional balance derives from spatially isotropic local connectivity of both excitatory and inhibitory cells. Modeling results demonstrate that such covariation is a signature of recurrent rather than purely feed-forward processing and that the observed isotropic local circuit is sufficient to generate invariant spike tuning

An anatomic gene expression atlas of the adult mouse brain

Studying gene expression provides a powerful means of understanding structure-function relationships in the nervous system. The availability of genome-scale in situ hybridization datasets enables new possibilities for understanding brain organization based on gene expression patterns. The Anatomic Gene Expression Atlas (AGEA) is a new relational atlas revealing the genetic architecture of the adult C57Bl/6J mouse brain based on spatial correlations across expression data for thousands of genes in the Allen Brain Atlas (ABA). The AGEA includes three discovery tools for examining neuroanatomical relationships and boundaries: (1) three-dimensional expression-based correlation maps, (2) a hierarchical transcriptome-based parcellation of the brain and (3) a facility to retrieve from the ABA specific genes showing enriched expression in local correlated domains. The utility of this atlas is illustrated by analysis of genetic organization in the thalamus, striatum and cerebral cortex. The AGEA is a publicly accessible online computational tool integrated with the ABA (http://mouse.brain-map.org/agea)

New technologies for examining neuronal ensembles in drug addiction and fear

Correlational data suggest that learned associations are encoded within neuronal ensembles. However, it has been difficult to prove that neuronal ensembles mediate learned behaviours because traditional pharmacological and lesion methods, and even newer cell type-specific methods, affect both activated and non-activated neurons. Additionally, previous studies on synaptic and molecular alterations induced by learning did not distinguish between behaviourally activated and non-activated neurons. Here, we describe three new approaches—Daun02 inactivation, FACS sorting of activated neurons and c-fos-GFP transgenic rats — that have been used to selectively target and study activated neuronal ensembles in models of conditioned drug effects and relapse. We also describe two new tools — c-fos-tTA mice and inactivation of CREB-overexpressing neurons — that have been used to study the role of neuronal ensembles in conditioned fear