Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods

Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods

Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors

The impressive performance of deep convolutional neural networks in single-view 3D reconstruction suggests that these models perform non-trivial reasoning about the 3D structure of the output space. However, recent work has challenged this belief, showing that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines or simple linear decoder models that exploit large amounts of per category data in standard benchmarks. On the other hand settings where 3D shape must be inferred for new categories with few examples are more natural and require models that generalize about shapes. In this work we demonstrate experimentally that naive baselines do not apply when the goal is to learn to reconstruct novel objects using very few examples, and that in a \emph{few-shot} learning setting, the network must learn concepts that can be applied to new categories, avoiding rote memorization. To address deficiencies in existing approaches to this problem, we propose three approaches that efficiently integrate a class prior into a 3D reconstruction model, allowing to account for intra-class variability and imposing an implicit compositional structure that the model should learn. Experiments on the popular ShapeNet database demonstrate that our method significantly outperform existing baselines on this task in the few-shot setting

Unusual Thermodynamics on the Fuzzy 2-Sphere

Higher spin Dirac operators on both the continuum sphere($S^2$) and its fuzzy analog($S^2_F$) come paired with anticommuting chirality operators. A consequence of this is seen in the fermion-like spectrum of these operators which is especially true even for the case of integer-spin Dirac operators. Motivated by this feature of the spectrum of a spin 1 Dirac operator on $S_F^2$, we assume the spin 1 particles obey Fermi-Dirac statistics. This choice is inspite of the lack of a well defined spin-statistics relation on a compact surface such as $S^2$. The specific heats are computed in the cases of the spin $\frac{1}{2}$ and spin 1 Dirac operators. Remarkably the specific heat for a system of spin $\frac{1}{2}$ particles is more than that of the spin 1 case, though the number of degrees of freedom is more in the case of spin 1 particles. The reason for this is inferred through a study of the spectrums of the Dirac operators in both the cases. The zero modes of the spin 1 Dirac operator is studied as a function of the cut-off angular momentum $L$ and is found to follow a simple power law. This number is such that the number of states with positive energy for the spin 1 and spin $\frac{1}{2}$ system become comparable. Remarks are made about the spectrums of higher spin Dirac operators as well through a study of their zero-modes and the variation of their spectrum with degeneracy. The mean energy as a function of temperature is studied in both the spin $\frac{1}{2}$ and spin 1 cases. They are found to deviate from the standard ideal gas law in 2+1 dimensions.Comment: 19 pages, 7 figures. The paper has been significantly modified. Main results are unchange

Bayesian Networks for Max-linear Models

We study Bayesian networks based on max-linear structural equations as introduced in Gissibl and Kl\"uppelberg [16] and provide a summary of their independence properties. In particular we emphasize that distributions for such networks are generally not faithful to the independence model determined by their associated directed acyclic graph. In addition, we consider some of the basic issues of estimation and discuss generalized maximum likelihood estimation of the coefficients, using the concept of a generalized likelihood ratio for non-dominated families as introduced by Kiefer and Wolfowitz [21]. Finally we argue that the structure of a minimal network asymptotically can be identified completely from observational data.Comment: 18 page

Decentralized Estimation over Orthogonal Multiple-access Fading Channels in Wireless Sensor Networks - Optimal and Suboptimal Estimators

Optimal and suboptimal decentralized estimators in wireless sensor networks (WSNs) over orthogonal multiple-access fading channels are studied in this paper. Considering multiple-bit quantization before digital transmission, we develop maximum likelihood estimators (MLEs) with both known and unknown channel state information (CSI). When training symbols are available, we derive a MLE that is a special case of the MLE with unknown CSI. It implicitly uses the training symbols to estimate the channel coefficients and exploits the estimated CSI in an optimal way. To reduce the computational complexity, we propose suboptimal estimators. These estimators exploit both signal and data level redundant information to improve the estimation performance. The proposed MLEs reduce to traditional fusion based or diversity based estimators when communications or observations are perfect. By introducing a general message function, the proposed estimators can be applied when various analog or digital transmission schemes are used. The simulations show that the estimators using digital communications with multiple-bit quantization outperform the estimator using analog-and-forwarding transmission in fading channels. When considering the total bandwidth and energy constraints, the MLE using multiple-bit quantization is superior to that using binary quantization at medium and high observation signal-to-noise ratio levels

Cross-language differences in the brain network subserving intelligible speech

SIGNIFICANCE: Language processing is generally left hemisphere dominant. However, whether the interactions among the typical left hemispheric language regions differ across different languages is largely unknown. An ideal method to address this question is modeling cortical interactions across language groups, but this is usually constrained by the model space with the prior hypothesis due to massive computation demands. With cloud-computing, we used functional MRI dynamic causal modeling analysis to compare more than 4,000 models of cortical dynamics among critical language regions in the temporal and frontal cortex, established the bias-free information flow maps that were shared or specific for processing intelligible speech in Chinese and English, and revealed the neural dynamics between the left and right hemispheres in Chinese speech comprehension. ABSTRACT: How is language processed in the brain by native speakers of different languages? Is there one brain system for all languages or are different languages subserved by different brain systems? The first view emphasizes commonality, whereas the second emphasizes specificity. We investigated the cortical dynamics involved in processing two very diverse languages: a tonal language (Chinese) and a nontonal language (English). We used functional MRI and dynamic causal modeling analysis to compute and compare brain network models exhaustively with all possible connections among nodes of language regions in temporal and frontal cortex and found that the information flow from the posterior to anterior portions of the temporal cortex was commonly shared by Chinese and English speakers during speech comprehension, whereas the inferior frontal gyrus received neural signals from the left posterior portion of the temporal cortex in English speakers and from the bilateral anterior portion of the temporal cortex in Chinese speakers. Our results revealed that, although speech processing is largely carried out in the common left hemisphere classical language areas (Broca’s and Wernicke’s areas) and anterior temporal cortex, speech comprehension across different language groups depends on how these brain regions interact with each other. Moreover, the right anterior temporal cortex, which is crucial for tone processing, is equally important as its left homolog, the left anterior temporal cortex, in modulating the cortical dynamics in tone language comprehension. The current study pinpoints the importance of the bilateral anterior temporal cortex in language comprehension that is downplayed or even ignored by popular contemporary models of speech comprehension

Factors associated with failed treatment : an analysis of 121,744 women embarking on their first IVF cycles

Peer reviewedPublisher PD

Simulating quantum statistics with entangled photons: a continuous transition from bosons to fermions

In contrast to classical physics, quantum mechanics divides particles into two classes-bosons and fermions-whose exchange statistics dictate the dynamics of systems at a fundamental level. In two dimensions quasi-particles known as 'anyons' exhibit fractional exchange statistics intermediate between these two classes. The ability to simulate and observe behaviour associated to fundamentally different quantum particles is important for simulating complex quantum systems. Here we use the symmetry and quantum correlations of entangled photons subjected to multiple copies of a quantum process to directly simulate quantum interference of fermions, bosons and a continuum of fractional behaviour exhibited by anyons. We observe an average similarity of 93.6\pm0.2% between an ideal model and experimental observation. The approach generalises to an arbitrary number of particles and is independent of the statistics of the particles used, indicating application with other quantum systems and large scale application.Comment: 10 pages, 5 figure

A Self-Reference False Memory Effect in the DRM Paradigm: Evidence from Eastern and Western Samples

It is well established that processing information in relation to oneself (i.e., selfreferencing) leads to better memory for that information than processing that same information in relation to others (i.e., other-referencing). However, it is unknown whether self-referencing also leads to more false memories than other-referencing. In the current two experiments with European and East Asian samples, we presented participants the Deese-Roediger/McDermott (DRM) lists together with their own name or other people’s name (i.e., “Trump” in Experiment 1 and “Li Ming” in Experiment 2). We found consistent results across the two experiments; that is, in the self-reference condition, participants had higher true and false memory rates compared to those in the other-reference condition. Moreover, we found that selfreferencing did not exhibit superior mnemonic advantage in terms of net accuracy compared to other-referencing and neutral conditions. These findings are discussed in terms of theoretical frameworks such as spreading activation theories and the fuzzytrace theory. We propose that our results reflect the adaptive nature of memory in the sense that cognitive processes that increase mnemonic efficiency may also increase susceptibility to associative false memories