Research in interactive scene analysis

Cooperative (man-machine) scene analysis techniques were developed whereby humans can provide a computer with guidance when completely automated processing is infeasible. An interactive approach promises significant near-term payoffs in analyzing various types of high volume satellite imagery, as well as vehicle-based imagery used in robot planetary exploration. This report summarizes the work accomplished over the duration of the project and describes in detail three major accomplishments: (1) the interactive design of texture classifiers; (2) a new approach for integrating the segmentation and interpretation phases of scene analysis; and (3) the application of interactive scene analysis techniques to cartography

Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders

Generative models that learn disentangled representations for different factors of variation in an image can be very useful for targeted data augmentation. By sampling from the disentangled latent subspace of interest, we can efficiently generate new data necessary for a particular task. Learning disentangled representations is a challenging problem, especially when certain factors of variation are difficult to label. In this paper, we introduce a novel architecture that disentangles the latent space into two complementary subspaces by using only weak supervision in form of pairwise similarity labels. Inspired by the recent success of cycle-consistent adversarial architectures, we use cycle-consistency in a variational auto-encoder framework. Our non-adversarial approach is in contrast with the recent works that combine adversarial training with auto-encoders to disentangle representations. We show compelling results of disentangled latent subspaces on three datasets and compare with recent works that leverage adversarial training

Research in interactive scene analysis

An interactive scene interpretation system (ISIS) was developed as a tool for constructing and experimenting with man-machine and automatic scene analysis methods tailored for particular image domains. A recently developed region analysis subsystem based on the paradigm of Brice and Fennema is described. Using this subsystem a series of experiments was conducted to determine good criteria for initially partitioning a scene into atomic regions and for merging these regions into a final partition of the scene along object boundaries. Semantic (problem-dependent) knowledge is essential for complete, correct partitions of complex real-world scenes. An interactive approach to semantic scene segmentation was developed and demonstrated on both landscape and indoor scenes. This approach provides a reasonable methodology for segmenting scenes that cannot be processed completely automatically, and is a promising basis for a future automatic system. A program is described that can automatically generate strategies for finding specific objects in a scene based on manually designated pictorial examples

DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs

A major difficulty of solving continuous POMDPs is to infer the multi-modal distribution of the unobserved true states and to make the planning algorithm dependent on the perceived uncertainty. We cast POMDP filtering and planning problems as two closely related Sequential Monte Carlo (SMC) processes, one over the real states and the other over the future optimal trajectories, and combine the merits of these two parts in a new model named the DualSMC network. In particular, we first introduce an adversarial particle filter that leverages the adversarial relationship between its internal components. Based on the filtering results, we then propose a planning algorithm that extends the previous SMC planning approach [Piche et al., 2018] to continuous POMDPs with an uncertainty-dependent policy. Crucially, not only can DualSMC handle complex observations such as image input but also it remains highly interpretable. It is shown to be effective in three continuous POMDP domains: the floor positioning domain, the 3D light-dark navigation domain, and a modified Reacher domain.Comment: IJCAI 202

Superpixel Convolutional Networks using Bilateral Inceptions

In this paper we propose a CNN architecture for semantic image segmentation. We introduce a new 'bilateral inception' module that can be inserted in existing CNN architectures and performs bilateral filtering, at multiple feature-scales, between superpixels in an image. The feature spaces for bilateral filtering and other parameters of the module are learned end-to-end using standard backpropagation techniques. The bilateral inception module addresses two issues that arise with general CNN segmentation architectures. First, this module propagates information between (super) pixels while respecting image edges, thus using the structured information of the problem for improved results. Second, the layer recovers a full resolution segmentation result from the lower resolution solution of a CNN. In the experiments, we modify several existing CNN architectures by inserting our inception module between the last CNN (1x1 convolution) layers. Empirical results on three different datasets show reliable improvements not only in comparison to the baseline networks, but also in comparison to several dense-pixel prediction techniques such as CRFs, while being competitive in time.Comment: European Conference on Computer Vision (ECCV), 201

ExplainIt! -- A declarative root-cause analysis engine for time series data (extended version)

We present ExplainIt!, a declarative, unsupervised root-cause analysis engine that uses time series monitoring data from large complex systems such as data centres. ExplainIt! empowers operators to succinctly specify a large number of causal hypotheses to search for causes of interesting events. ExplainIt! then ranks these hypotheses, reducing the number of causal dependencies from hundreds of thousands to a handful for human understanding. We show how a declarative language, such as SQL, can be effective in declaratively enumerating hypotheses that probe the structure of an unknown probabilistic graphical causal model of the underlying system. Our thesis is that databases are in a unique position to enable users to rapidly explore the possible causal mechanisms in data collected from diverse sources. We empirically demonstrate how ExplainIt! had helped us resolve over 30 performance issues in a commercial product since late 2014, of which we discuss a few cases in detail.Comment: SIGMOD Industry Track 201

Modulated Floquet Topological Insulators

Floquet topological insulators are topological phases of matter generated by the application of time-periodic perturbations on otherwise conventional insulators. We demonstrate that spatial variations in the time-periodic potential lead to localized quasi-stationary states in two-dimensional systems. These states include one-dimensional interface modes at the nodes of the external potential, and fractionalized excitations at vortices of the external potential. We also propose a setup by which light can induce currents in these systems. We explain these results by showing a close analogy to px+ipy superconductors

Comparative Spectra of Oxygen-Rich vs. Carbon-Rich Circumstellar Shells: VY Canis Majoris and IRC+10216 at 215-285 GHz

A sensitive (1{\sigma} rms at 1 MHz resolution ~3 mK) 1 mm spectral line survey (214.5-285.5 GHz) of VY Canis Majoris (VY CMa) and IRC+10216 has been conducted to compare the chemistries of oxygen and carbon-rich circumstellar envelopes. This study was carried out using the Submillimeter Telescope (SMT) of the Arizona Radio Observatory (ARO) with a new ALMA-type receiver. This survey is the first to chemically characterize an O-rich circumstellar shell at millimeter wavelengths. In VY CMa, 128 emission features were detected arising from 18 different molecules, and in IRC+10216, 720 lines were observed, assigned to 32 different species. The 1 mm spectrum of VY CMa is dominated by SO2 and SiS; in IRC +10216, C4H and SiC2 are the most recurrent species. Ten molecules were common to both sources: CO, SiS, SiO, CS, CN, HCN, HNC, NaCl, PN, and HCO+. Sulfur plays an important role in VY CMa, but saturated/unsaturated carbon dominates the molecular content of IRC+10216, producing CH2NH, for example. Although the molecular complexity of IRC+10216 is greater, VY CMa supports a unique "inorganic" chemistry leading to the oxides PO, AlO, and AlOH. Only diatomic and triatomic compounds were observed in VY CMa, while species with 4 or more atoms are common in IRC+10216, reflecting carbon's ability to form strong multiple bonds, unlike oxygen. In VY CMa, a new water maser (v_2=2) has been found, as well as vibrationally-excited NaCl. Toward IRC+10216, vibrationally-excited CCH was detected for the first time.Comment: 21 pages, 3 figures, accepted for publication in Astrophysical Journal Letter