Deep Neural Networks - A Brief History

Introduction to deep neural networks and their history.Comment: 14 pages, 14 figure

Foci of orientation plasticity in visual cortex

[Abstract] Cortical areas are generally assumed to be uniform in their capacity for adaptive changes or plasticity1, 2, 3, 4. Here we demonstrate, however, that neurons in the cat striate cortex (V1) show pronounced adaptation-induced short-term plasticity of orientation tuning primarily at specific foci. V1 neurons are clustered according to their orientation preference in iso-orientation domains5 that converge at singularities or pinwheel centres6, 7. Although neurons in pinwheel centres have similar orientation tuning and responses to those in iso-orientation domains, we find that they differ markedly in their capacity for adaptive changes. Adaptation with an oriented drifting grating stimulus alters responses of neurons located at and near pinwheel centres to a broad range of orientations, causing repulsive shifts in orientation preference and changes in response magnitude. In contrast, neurons located in iso-orientation domains show minimal changes in their tuning properties after adaptation. The anisotropy of adaptation-induced orientation plasticity is probably mediated by inhomogeneities in local intracortical interactions that are overlaid on the map of orientation preference in V1

Provably scale-covariant networks from oriented quasi quadrature measures in cascade

This article presents a continuous model for hierarchical networks based on a combination of mathematically derived models of receptive fields and biologically inspired computations. Based on a functional model of complex cells in terms of an oriented quasi quadrature combination of first- and second-order directional Gaussian derivatives, we couple such primitive computations in cascade over combinatorial expansions over image orientations. Scale-space properties of the computational primitives are analysed and it is shown that the resulting representation allows for provable scale and rotation covariance. A prototype application to texture analysis is developed and it is demonstrated that a simplified mean-reduced representation of the resulting QuasiQuadNet leads to promising experimental results on three texture datasets.Comment: 12 pages, 3 figures, 1 tabl

Intrinsic activity in the fly brain gates visual information during behavioral choices

The small insect brain is often described as an input/output system that executes reflex-like behaviors. It can also initiate neural activity and behaviors intrinsically, seen as spontaneous behaviors, different arousal states and sleep. However, less is known about how intrinsic activity in neural circuits affects sensory information processing in the insect brain and variability in behavior. Here, by simultaneously monitoring Drosophila's behavioral choices and brain activity in a flight simulator system, we identify intrinsic activity that is associated with the act of selecting between visual stimuli. We recorded neural output (multiunit action potentials and local field potentials) in the left and right optic lobes of a tethered flying Drosophila, while its attempts to follow visual motion (yaw torque) were measured by a torque meter. We show that when facing competing motion stimuli on its left and right, Drosophila typically generate large torque responses that flip from side to side. The delayed onset (0.1-1 s) and spontaneous switch-like dynamics of these responses, and the fact that the flies sometimes oppose the stimuli by flying straight, make this behavior different from the classic steering reflexes. Drosophila, thus, seem to choose one stimulus at a time and attempt to rotate toward its direction. With this behavior, the neural output of the optic lobes alternates; being augmented on the side chosen for body rotation and suppressed on the opposite side, even though the visual input to the fly eyes stays the same. Thus, the flow of information from the fly eyes is gated intrinsically. Such modulation can be noise-induced or intentional; with one possibility being that the fly brain highlights chosen information while ignoring the irrelevant, similar to what we know to occur in higher animals

Migraine aura: retracting particle-like waves in weakly susceptible cortex

Cortical spreading depression (SD) has been suggested to underlie migraine aura. Despite a precise match in speed, the spatio-temporal patterns of SD and aura symptoms on the cortical surface ordinarily differ in aspects of size and shape. We show that this mismatch is reconciled by utilizing that both pattern types bifurcate from an instability point of generic reaction-diffusion models. To classify these spatio-temporal pattern we suggest a susceptibility scale having the value [sigma]=1 at the instability point. We predict that human cortex is only weakly susceptible to SD ([sigma]<1), and support this prediction by directly matching visual aura symptoms with anatomical landmarks using fMRI retinotopic mapping. We discuss the increased dynamical repertoire of cortical tissue close to [sigma]=1, in particular, the resulting implications on migraine pharmacology that is hitherto tested in the regime ([sigma]>>1), and potentially silent aura occurring below a second bifurcation point at [sigma]=0 on the susceptible scale

Coverage, Continuity and Visual Cortical Architecture

The primary visual cortex of many mammals contains a continuous representation of visual space, with a roughly repetitive aperiodic map of orientation preferences superimposed. It was recently found that orientation preference maps (OPMs) obey statistical laws which are apparently invariant among species widely separated in eutherian evolution. Here, we examine whether one of the most prominent models for the optimization of cortical maps, the elastic net (EN) model, can reproduce this common design. The EN model generates representations which optimally trade of stimulus space coverage and map continuity. While this model has been used in numerous studies, no analytical results about the precise layout of the predicted OPMs have been obtained so far. We present a mathematical approach to analytically calculate the cortical representations predicted by the EN model for the joint mapping of stimulus position and orientation. We find that in all previously studied regimes, predicted OPM layouts are perfectly periodic. An unbiased search through the EN parameter space identifies a novel regime of aperiodic OPMs with pinwheel densities lower than found in experiments. In an extreme limit, aperiodic OPMs quantitatively resembling experimental observations emerge. Stabilization of these layouts results from strong nonlocal interactions rather than from a coverage-continuity-compromise. Our results demonstrate that optimization models for stimulus representations dominated by nonlocal suppressive interactions are in principle capable of correctly predicting the common OPM design. They question that visual cortical feature representations can be explained by a coverage-continuity-compromise.Comment: 100 pages, including an Appendix, 21 + 7 figure

Cortical depth dependent functional responses in humans at 7T: improved specificity with 3D GRASE

Ultra high fields (7T and above) allow functional imaging with high contrast-to-noise ratios and improved spatial resolution. This, along with improved hardware and imaging techniques, allow investigating columnar and laminar functional responses. Using gradient-echo (GE) (T2* weighted) based sequences, layer specific responses have been recorded from human (and animal) primary visual areas. However, their increased sensitivity to large surface veins potentially clouds detecting and interpreting layer specific responses. Conversely, spin-echo (SE) (T2 weighted) sequences are less sensitive to large veins and have been used to map cortical columns in humans. T2 weighted 3D GRASE with inner volume selection provides high isotropic resolution over extended volumes, overcoming some of the many technical limitations of conventional 2D SE-EPI, whereby making layer specific investigations feasible. Further, the demonstration of columnar level specificity with 3D GRASE, despite contributions from both stimulated echoes and conventional T2 contrast, has made it an attractive alternative over 2D SE-EPI. Here, we assess the spatial specificity of cortical depth dependent 3D GRASE functional responses in human V1 and hMT by comparing it to GE responses. In doing so we demonstrate that 3D GRASE is less sensitive to contributions from large veins in superficial layers, while showing increased specificity (functional tuning) throughout the cortex compared to GE

Rapid motion adaptation reveals the temporal dynamics of spatiotemporal correlation between ON and OFF pathways

At the early stages of visual processing, information is processed by two major thalamic pathways encoding brightness increments (ON) and decrements (OFF). Accumulating evidence suggests that these pathways interact and merge as early as in primary visual cortex. Using regular and reverse-phi motion in a rapid adaptation paradigm, we investigated the temporal dynamics of within and across pathway mechanisms for motion processing. When the adaptation duration was short (188 ms), reverse-phi and regular motion led to similar adaptation effects, suggesting that the information from the two pathways are combined efficiently at early-stages of motion processing. However, as the adaption duration was increased to 752 ms, reverse-phi and regular motion showed distinct adaptation effects depending on the test pattern used, either engaging spatiotemporal correlation between the same or opposite contrast polarities. Overall, these findings indicate that spatiotemporal correlation within and across ON-OFF pathways for motion processing can be selectively adapted, and support those models that integrate within and across pathway mechanisms for motion processing

Coordinated optimization of visual cortical maps (I) Symmetry-based analysis

In the primary visual cortex of primates and carnivores, functional architecture can be characterized by maps of various stimulus features such as orientation preference (OP), ocular dominance (OD), and spatial frequency. It is a long-standing question in theoretical neuroscience whether the observed maps should be interpreted as optima of a specific energy functional that summarizes the design principles of cortical functional architecture. A rigorous evaluation of this optimization hypothesis is particularly demanded by recent evidence that the functional architecture of OP columns precisely follows species invariant quantitative laws. Because it would be desirable to infer the form of such an optimization principle from the biological data, the optimization approach to explain cortical functional architecture raises the following questions: i) What are the genuine ground states of candidate energy functionals and how can they be calculated with precision and rigor? ii) How do differences in candidate optimization principles impact on the predicted map structure and conversely what can be learned about an hypothetical underlying optimization principle from observations on map structure? iii) Is there a way to analyze the coordinated organization of cortical maps predicted by optimization principles in general? To answer these questions we developed a general dynamical systems approach to the combined optimization of visual cortical maps of OP and another scalar feature such as OD or spatial frequency preference.Comment: 90 pages, 16 figure