Real-to-Virtual Domain Unification for End-to-End Autonomous Driving

In the spectrum of vision-based autonomous driving, vanilla end-to-end models are not interpretable and suboptimal in performance, while mediated perception models require additional intermediate representations such as segmentation masks or detection bounding boxes, whose annotation can be prohibitively expensive as we move to a larger scale. More critically, all prior works fail to deal with the notorious domain shift if we were to merge data collected from different sources, which greatly hinders the model generalization ability. In this work, we address the above limitations by taking advantage of virtual data collected from driving simulators, and present DU-drive, an unsupervised real-to-virtual domain unification framework for end-to-end autonomous driving. It first transforms real driving data to its less complex counterpart in the virtual domain and then predicts vehicle control commands from the generated virtual image. Our framework has three unique advantages: 1) it maps driving data collected from a variety of source distributions into a unified domain, effectively eliminating domain shift; 2) the learned virtual representation is simpler than the input real image and closer in form to the "minimum sufficient statistic" for the prediction task, which relieves the burden of the compression phase while optimizing the information bottleneck tradeoff and leads to superior prediction performance; 3) it takes advantage of annotated virtual data which is unlimited and free to obtain. Extensive experiments on two public driving datasets and two driving simulators demonstrate the performance superiority and interpretive capability of DU-drive

Summarizing First-Person Videos from Third Persons' Points of Views

Video highlight or summarization is among interesting topics in computer vision, which benefits a variety of applications like viewing, searching, or storage. However, most existing studies rely on training data of third-person videos, which cannot easily generalize to highlight the first-person ones. With the goal of deriving an effective model to summarize first-person videos, we propose a novel deep neural network architecture for describing and discriminating vital spatiotemporal information across videos with different points of view. Our proposed model is realized in a semi-supervised setting, in which fully annotated third-person videos, unlabeled first-person videos, and a small number of annotated first-person ones are presented during training. In our experiments, qualitative and quantitative evaluations on both benchmarks and our collected first-person video datasets are presented.Comment: 16+10 pages, ECCV 201

MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes

Attribute recognition, particularly facial, extracts many labels for each image. While some multi-task vision problems can be decomposed into separate tasks and stages, e.g., training independent models for each task, for a growing set of problems joint optimization across all tasks has been shown to improve performance. We show that for deep convolutional neural network (DCNN) facial attribute extraction, multi-task optimization is better. Unfortunately, it can be difficult to apply joint optimization to DCNNs when training data is imbalanced, and re-balancing multi-label data directly is structurally infeasible, since adding/removing data to balance one label will change the sampling of the other labels. This paper addresses the multi-label imbalance problem by introducing a novel mixed objective optimization network (MOON) with a loss function that mixes multiple task objectives with domain adaptive re-weighting of propagated loss. Experiments demonstrate that not only does MOON advance the state of the art in facial attribute recognition, but it also outperforms independently trained DCNNs using the same data. When using facial attributes for the LFW face recognition task, we show that our balanced (domain adapted) network outperforms the unbalanced trained network.Comment: Post-print of manuscript accepted to the European Conference on Computer Vision (ECCV) 2016 http://link.springer.com/chapter/10.1007%2F978-3-319-46454-1_

Representation learning for cross-modality classification

Differences in scanning parameters or modalities can complicate image analysis based on supervised classification. This paper presents two representation learning approaches, based on autoencoders, that address this problem by learning representations that are similar across domains. Both approaches use, next to the data representation objective, a similarity objective to minimise the difference between representations of corresponding patches from each domain. We evaluated the methods in transfer learning experiments on multi-modal brain MRI data and on synthetic data. After transforming training and test data from different modalities to the common representations learned by our methods, we trained classifiers for each of pair of modalities. We found that adding the similarity term to the standard objective can produce representations that are more similar and can give a higher accuracy in these cross-modality classification experiments

Prediction of sarcomere mutations in subclinical hypertrophic cardiomyopathy.

BACKGROUND: Sarcomere protein mutations in hypertrophic cardiomyopathy induce subtle cardiac structural changes before the development of left ventricular hypertrophy (LVH). We have proposed that myocardial crypts are part of this phenotype and independently associated with the presence of sarcomere gene mutations. We tested this hypothesis in genetic hypertrophic cardiomyopathy pre-LVH (genotype positive, LVH negative [G+LVH-]). METHODS AND RESULTS: A multicenter case-control study investigated crypts and 22 other cardiovascular magnetic resonance parameters in subclinical hypertrophic cardiomyopathy to determine their strength of association with sarcomere gene mutation carriage. The G+LVH- sample (n=73) was 29 ± 13 years old and 51% were men. Crypts were related to the presence of sarcomere mutations (for ≥1 crypt, β=2.5; 95% confidence interval [CI], 0.5-4.4; P=0.014 and for ≥2 crypts, β=3.0; 95% CI, 0.8-7.9; P=0.004). In combination with 3 other parameters: anterior mitral valve leaflet elongation (β=2.1; 95% CI, 1.7-3.1; P<0.001), abnormal LV apical trabeculae (β=1.6; 95% CI, 0.8-2.5; P<0.001), and smaller LV end-systolic volumes (β=1.4; 95% CI, 0.5-2.3; P=0.001), multiple crypts indicated the presence of sarcomere gene mutations with 80% accuracy and an area under the curve of 0.85 (95% CI, 0.8-0.9). In this G+LVH- population, cardiac myosin-binding protein C mutation carriers had twice the prevalence of crypts when compared with the other combined mutations (47 versus 23%; odds ratio, 2.9; 95% CI, 1.1-7.9; P=0.045). CONCLUSIONS: The subclinical hypertrophic cardiomyopathy phenotype measured by cardiovascular magnetic resonance in a multicenter environment and consisting of crypts (particularly multiple), anterior mitral valve leaflet elongation, abnormal trabeculae, and smaller LV systolic cavity is indicative of the presence of sarcomere gene mutations and highlights the need for further study

Top pair Asymmetries at Hadron colliders with general Z′Z' couplings

Recently it has been shown that measurement of charge asymmetry of top pair production at LHC excludes any flavor violating $Z'$ vector gauge boson that could explain Tevatron forward-backward asymmetry (FBA). We consider the general form of a $Z'$ gauge boson including left-handed, right-handed vector and tensor couplings to examine FBA and charge asymmetry. To evaluate top pair asymmetries at Tevatron and LHC, we consider $B^0_q$ mixing constraints on flavor changing $Z'$ couplings and show that this model still explain forward-backward asymmetry at Tevatron and charge asymmetry can not exclude it in part of parameters space.Comment: 18 pages, 7 figure

LHC diphoton Higgs signal and top quark forward-backward asymmetry in quasi-inert Higgs doublet model

In the quasi-inert Higgs doublet model, we study the LHC diphoton rate for a standard model-like Higgs boson and the top quark forward-backward asymmetry at Tevatron. Taking into account the constraints from the vacuum stability, unitarity, electroweak precision tests, flavor physics and the related experimental data of top quark, we find that compared with the standard model prediction, the diphoton rate of Higgs boson at LHC can be enhanced due to the light charged Higgs contributions, while the measurement of the top quark forward-backward asymmetry at Tevatron can be explained to within $1\sigma$ due to the non-standard model neutral Higgs bosons contributions. Finally, the correlations between the two observables are discussed.Comment: 14 pages, 5 figues. Version to appear in JHEP, some references adde

Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation

Unsupervised domain adaptation has caught appealing attentions as it facilitates the unlabeled target learning by borrowing existing well-established source domain knowledge. Recent practice on domain adaptation manages to extract effective features by incorporating the pseudo labels for the target domain to better solve cross-domain distribution divergences. However, existing approaches separate target label optimization and domain-invariant feature learning as different steps. To address that issue, we develop a novel Graph Adaptive Knowledge Transfer (GAKT) model to jointly optimize target labels and domain-free features in a unified framework. Specifically, semi-supervised knowledge adaptation and label propagation on target data are coupled to benefit each other, and hence the marginal and conditional disparities across different domains will be better alleviated. Experimental evaluation on two cross-domain visual datasets demonstrates the effectiveness of our designed approach on facilitating the unlabeled target task learning, compared to the state-of-the-art domain adaptation approaches

Limits on scalar leptoquark interactions and consequences for GUTs

A colored weak singlet scalar state with hypercharge 4/3 is one of the possible candidates for the explanation of the unexpectedly large forward-backward asymmetry in t tbar production as measured by the CDF and D0 experiments. We investigate the role of this state in a plethora of flavor changing neutral current processes and precision observables of down-quarks and charged leptons. Our analysis includes tree- and loop-level mediated observables in the K and B systems, the charged lepton sector, as well as the Z to b bbar decay width. We perform a global fit of the relevant scalar couplings. This approach can explain the (g-2)_mu anomaly while tensions among the CP violating observables in the quark sector, most notably the nonstandard CP phase (and width difference) in the Bs system cannot be fully relaxed. The results are interpreted in a class of grand unified models which allow for a light colored scalar with a mass below 1TeV. We find that the renormalizable SU(5) scenario is not compatible with our global fit, while in the SO(10) case the viability requires the presence of both the 126- and 120-dimensional representations.Comment: 26 pages, 7 figures; version as publishe