Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

In recent years the field of neuromorphic low-power systems that consume orders of magnitude less power gained significant momentum. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, efficient processing of temporal sequences or variable length-inputs remain difficult. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of ~17 uW

Genetic Suppression of Basement Membrane Defects in Caenorhabditis elegans by Gain of Function in Extracellular Matrix and Cell-Matrix Attachment Genes.

Basement membranes are extracellular matrices essential for embryonic development in animals. Peroxidasins are extracellular peroxidases implicated in the unique sulfilimine cross-links between type IV basement membrane collagens. Loss of function in the Caenorhabditis elegans peroxidasin PXN-2 results in fully penetrant embryonic or larval lethality. Using genetic suppressor screening, we find that the requirement for PXN-2 in development can be bypassed by gain of function in multiple genes encoding other basement membrane components, or proteins implicated in cell-matrix attachment. We identify multiple alleles of let-805, encoding the transmembrane protein myotactin, which suppress phenotypes of pxn-2 null mutants and of other basement membrane mutants such as F-spondin/spon-1 These let-805 suppressor alleles cause missense alterations in two pairs of FNIII repeats in the extracellular domain; they act dominantly and have no detectable phenotypes alone, suggesting they cause gain of function. We also identify suppressor missense mutations affecting basement membrane components type IV collagen (emb-9, let-2) and perlecan (unc-52), as well as a mutation affecting spectraplakin (vab-10), a component of the epidermal cytoskeleton. These suppressor alleles do not bypass the developmental requirement for core structural proteins of the basement membrane such as laminin or type IV collagen. In conclusion, putative gain-of-function alterations in matrix proteins or in cell-matrix receptors can overcome the requirement for certain basement membrane proteins in embryonic development, revealing previously unknown plasticity in the genetic requirements for the extracellular matrix

Tweets as impact indicators: Examining the implications of automated bot accounts on Twitter

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific papers deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. We present preliminary findings that automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose differentiating between different levels of engagement from tweeting only bibliographic information to discussing or commenting on the content of a paper.Comment: 9 pages, 4 figures, 1 tabl

Gibbs Sampling with Low-Power Spiking Digital Neurons

Restricted Boltzmann Machines and Deep Belief Networks have been successfully used in a wide variety of applications including image classification and speech recognition. Inference and learning in these algorithms uses a Markov Chain Monte Carlo procedure called Gibbs sampling. A sigmoidal function forms the kernel of this sampler which can be realized from the firing statistics of noisy integrate-and-fire neurons on a neuromorphic VLSI substrate. This paper demonstrates such an implementation on an array of digital spiking neurons with stochastic leak and threshold properties for inference tasks and presents some key performance metrics for such a hardware-based sampler in both the generative and discriminative contexts.Comment: Accepted at ISCAS 201

The role of perceived source location in auditory stream segregation: separation affects sound organization, common fate does not

The human auditory system is capable of grouping sounds originating from different sound sources into coherent auditory streams, a process termed auditory stream segregation. Several cues can influence auditory stream segregation, but the full set of cues and the way in which they are integrated is still unknown. In the current study, we tested whether auditory motion can serve as a cue for segregating sequences of tones. Our hypothesis was that, following the principle of common fate, sounds emitted by sources moving together in space along similar trajectories will be more likely to be grouped into a single auditory stream, while sounds emitted by independently moving sources will more often be heard as two streams. Stimuli were derived from sound recordings in which the sound source motion was induced by walking humans. Although the results showed a clear effect of spatial separation, auditory motion had a negligible influence on stream segregation. Hence, auditory motion may not be used as a primitive cue in auditory stream segregation

The impact of rainfall and school break time policies on physical activity in 9-10 year old British children: a repeated measures study.

BACKGROUND: The weather may be a driver of seasonal patterns in children's physical activity (PA). A better understanding of the relationships between weather and PA may help increase children's PA. This study aims to examine the association between PA and rainfall in 9-10 year old children, and how it may be modified by school policies. METHODS: 1794 participants in the SPEEDY study in Norfolk, UK recorded PA using ActiGraph accelerometers over up to six days in the summer term of 2007. Multilevel regression models were used to determine the day-by-day association between rainfall and minutes spent sedentary, in moderate-to-vigorous PA (MVPA), and average counts per minute (cpm) over the whole day (07:00-21:00) and the lunchtime period (12:00-14:00). School policies for break times in bad weather were fitted as interaction terms with rainfall. RESULTS: Relative to days with no rain, children spent 9.4 minutes (95%CI 7.0 to 11.9) fewer in MVPA, were sedentary for 13.6 minutes (8.8 to 18.4) more, and accumulated 85.9 cpm (66.2 to 105.5) fewer over the whole day on the wettest days. Children allowed to play outside in wet weather showed the lowest lunchtime PA levels on the wettest days, undertaking 9.8 minutes (6.2 to 13.5) fewer MVPA, 16.1 minutes (10.3 to 21.9) more sedentary, and accumulating 408.0 cpm (250.9 to 565.1) fewer than those allowed to be active indoors. CONCLUSIONS: Rainfall is negatively associated with PA in primary school children, but providing indoor physical activities in wet weather may help children maintain physical activity levels irrespective of rainfall.RIGHTS : This article is licensed under the BioMed Central licence at http://www.biomedcentral.com/about/license which is similar to the 'Creative Commons Attribution Licence'. In brief you may : copy, distribute, and display the work; make derivative works; or make commercial use of the work - under the following conditions: the original author must be given credit; for any reuse or distribution, it must be made clear to others what the license terms of this work are

Habitual intake of flavonoid subclasses and risk of colorectal cancer in two large prospective cohorts

Background: Flavonoids inhibit the growth of colon cancer cells in vitro. In a secondary analysis of a randomized controlled trial, the Polyp Prevention Trial, a higher intake of one sub-class, flavonols, was significantly associated with reduced risk of recurrent advanced adenoma. Most previous prospective studies on colorectal cancer evaluated only a limited number of flavonoid sub-classes and intake ranges, yielding inconsistent results.  Objective: To examine whether higher habitual dietary intakes of flavonoid subclasses (flavonols, flavones, flavanones, flavan-3-ols and anthocyanins) are associated with lower risk of colorectal cancer.  Design: Using data from validated food frequency questionnaires administered every four years and an updated flavonoid food composition database flavonoid intakes were calculated for 42,478 male participants from the Health Professionals Follow-up Study and for 76,364 female participants from the Nurses’ Health Study.  Results: During up to 26 years of follow-up, 2,519 colorectal cancer cases (1,061 in men, 1,458 in women) were documented. Intakes of flavonoid subclasses were not associated with risk of colorectal cancer in either cohort. Pooled multivariable adjusted relative risks (95% confidence interval) comparing the highest with the lowest quintile were 1.04 (0.91, 1.18) for flavonols; 1.01 (0.89, 1.15) for flavones; 0.96 (0.84, 1.10) for flavanones; 1.07 (0.95, 1.21) for flavan-3-ols; and 0.98 (0.81, 1.19) for anthocyanins (all p-values for heterogeneity by sex >0.19). In subsite analyses, flavonoid intake was also not associated with colon or rectal cancer risk.  Conclusion: Our findings do not support the hypothesis that a higher habitual intake of any flavonoid sub-class decreases the risk of colorectal cancer

Highly unusual triangular crystals of theophylline: The influence of solvent on the growth rates of polar crystal faces

A noteworthy feature of the compound theophylline is that it forms crystals with a triangular habit, an extremely rare phenomenon for an organic molecule. Here, we investigate the formation of these crystals, comprised of the polymorph Form II (Pna21), and demonstrate that the triangles are obtained from solvents which are highly hydrophobic, or which have a hydrogen bond acceptor group and no hydrogen bond donor group. The formation of the triangular crystal habit is rationalized on the basis of the way such solvents interact with the inequivalent (001) and (00-1) polar crystal faces of Form II. Interactions are significantly stronger at one face than the other, inhibiting growth in one direction and limiting crystal growth to a single, triangle shaped, growth sector. This rationalization also enabled interesting surface features observed by atomic force microscopy to be interpreted. Furthermore, we report a second, previously unreported, type of triangular crystal of theophylline for which the angle at the tip of the triangle is obtuse rather than acute. These crystals are proposed, with the aid of transmission electron microscopy and crystal structure prediction, to be a new polymorphic form of theophyllin