VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation

Rich and dense human labeled datasets are among the main enabling factors for the recent advance on vision-language understanding. Many seemingly distant annotations (e.g., semantic segmentation and visual question answering (VQA)) are inherently connected in that they reveal different levels and perspectives of human understandings about the same visual scenes --- and even the same set of images (e.g., of COCO). The popularity of COCO correlates those annotations and tasks. Explicitly linking them up may significantly benefit both individual tasks and the unified vision and language modeling. We present the preliminary work of linking the instance segmentations provided by COCO to the questions and answers (QAs) in the VQA dataset, and name the collected links visual questions and segmentation answers (VQS). They transfer human supervision between the previously separate tasks, offer more effective leverage to existing problems, and also open the door for new research problems and models. We study two applications of the VQS data in this paper: supervised attention for VQA and a novel question-focused semantic segmentation task. For the former, we obtain state-of-the-art results on the VQA real multiple-choice task by simply augmenting the multilayer perceptrons with some attention features that are learned using the segmentation-QA links as explicit supervision. To put the latter in perspective, we study two plausible methods and compare them to an oracle method assuming that the instance segmentations are given at the test stage.Comment: To appear on ICCV 201

Automatic Concept Discovery from Parallel Text and Visual Corpora

Humans connect language and vision to perceive the world. How to build a similar connection for computers? One possible way is via visual concepts, which are text terms that relate to visually discriminative entities. We propose an automatic visual concept discovery algorithm using parallel text and visual corpora; it filters text terms based on the visual discriminative power of the associated images, and groups them into concepts using visual and semantic similarities. We illustrate the applications of the discovered concepts using bidirectional image and sentence retrieval task and image tagging task, and show that the discovered concepts not only outperform several large sets of manually selected concepts significantly, but also achieves the state-of-the-art performance in the retrieval task.Comment: To appear in ICCV 201

Improved Light-cone QCD Sum Rule Analysis Of The Rare Decays Λb→Λγ\Lambda_b\rightarrow\Lambda\gamma And Λb→Λl+l−\Lambda_b\rightarrow\Lambda l^+l^-

We present a systematic light-cone QCD sum rule study of the exclusive rare radiative decay $\Lambda_b\rightarrow\Lambda\gamma$ and rare semileptonic decay $\Lambda_b\rightarrow\Lambda l^+l^-$ within the framework of the standard model. Although some LCSR studies on these rare processes can be found in different literatures, it is necessary to reanalyze them systematically for the reason that either the baryonic distribution amplitudes are improved or different interpolating currents for the $\Lambda_b$ baryon may lead to quite different results. In addition, the rare process $\Lambda_b\rightarrow\Lambda\gamma$ has not yet been analyzed by LCSR with the Ioffe-type current. Taking all these reasons into account, we perform LCSR calculations of both the processes with two types of interpolating currents. Our calculations show that the choice of the interpolating current for the $\Lambda_b$ baryon can affect the predictions significantly, especially for the rare radiative decay process.Comment: 21 pages,4 figure