Evaluation of CNN-based Single-Image Depth Estimation Methods

While an increasing interest in deep models for single-image depth estimation methods can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art single-image depth estimation approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol

iPose: Instance-Aware 6D Pose Estimation of Partly Occluded Objects

We address the task of 6D pose estimation of known rigid objects from single input images in scenarios where the objects are partly occluded. Recent RGB-D-based methods are robust to moderate degrees of occlusion. For RGB inputs, no previous method works well for partly occluded objects. Our main contribution is to present the first deep learning-based system that estimates accurate poses for partly occluded objects from RGB-D and RGB input. We achieve this with a new instance-aware pipeline that decomposes 6D object pose estimation into a sequence of simpler steps, where each step removes specific aspects of the problem. The first step localizes all known objects in the image using an instance segmentation network, and hence eliminates surrounding clutter and occluders. The second step densely maps pixels to 3D object surface positions, so called object coordinates, using an encoder-decoder network, and hence eliminates object appearance. The third, and final, step predicts the 6D pose using geometric optimization. We demonstrate that we significantly outperform the state-of-the-art for pose estimation of partly occluded objects for both RGB and RGB-D input

Cosmological Density and Power Spectrum from Peculiar Velocities: Nonlinear Corrections and PCA

We allow for nonlinear effects in the likelihood analysis of galaxy peculiar velocities, and obtain ~35%-lower values for the cosmological density parameter Om and the amplitude of mass-density fluctuations. The power spectrum in the linear regime is assumed to be a flat LCDM model (h=0.65, n=1, COBE) with only Om as a free parameter. Since the likelihood is driven by the nonlinear regime, we "break" the power spectrum at k_b=0.2 h/Mpc and fit a power law at k>k_b. This allows for independent matching of the nonlinear behavior and an unbiased fit in the linear regime. The analysis assumes Gaussian fluctuations and errors, and a linear relation between velocity and density. Tests using proper mock catalogs demonstrate a reduced bias and a better fit. We find for the Mark3 and SFI data Om_m=0.32+-0.06 and 0.37+-0.09 respectively, with sigma_8*Om^0.6 = 0.49+-0.06 and 0.63+-0.08, in agreement with constraints from other data. The quoted 90% errors include cosmic variance. The improvement in likelihood due to the nonlinear correction is very significant for Mark3 and moderately so for SFI. When allowing deviations from LCDM, we find an indication for a wiggle in the power spectrum: an excess near k=0.05 and a deficiency at k=0.1 (cold flow). This may be related to the wiggle seen in the power spectrum from redshift surveys and the second peak in the CMB anisotropy. A chi^2 test applied to modes of a Principal Component Analysis (PCA) shows that the nonlinear procedure improves the goodness of fit and reduces a spatial gradient of concern in the linear analysis. The PCA allows addressing spatial features of the data and fine-tuning the theoretical and error models. It shows that the models used are appropriate for the cosmological parameter estimation performed. We address the potential for optimal data compression using PCA.Comment: 18 pages, LaTex, uses emulateapj.sty, ApJ in press (August 10, 2001), improvements to text and figures, updated reference

Accurate and linear time pose estimation from points and lines

The final publication is available at link.springer.comThe Perspective-n-Point (PnP) problem seeks to estimate the pose of a calibrated camera from n 3Dto-2D point correspondences. There are situations, though, where PnP solutions are prone to fail because feature point correspondences cannot be reliably estimated (e.g. scenes with repetitive patterns or with low texture). In such scenarios, one can still exploit alternative geometric entities, such as lines, yielding the so-called Perspective-n-Line (PnL) algorithms. Unfortunately, existing PnL solutions are not as accurate and efficient as their point-based counterparts. In this paper we propose a novel approach to introduce 3D-to-2D line correspondences into a PnP formulation, allowing to simultaneously process points and lines. For this purpose we introduce an algebraic line error that can be formulated as linear constraints on the line endpoints, even when these are not directly observable. These constraints can then be naturally integrated within the linear formulations of two state-of-the-art point-based algorithms, the OPnP and the EPnP, allowing them to indistinctly handle points, lines, or a combination of them. Exhaustive experiments show that the proposed formulation brings remarkable boost in performance compared to only point or only line based solutions, with a negligible computational overhead compared to the original OPnP and EPnP.Peer ReviewedPostprint (author's final draft

Estimating Depth from RGB and Sparse Sensing

We present a deep model that can accurately produce dense depth maps given an RGB image with known depth at a very sparse set of pixels. The model works simultaneously for both indoor/outdoor scenes and produces state-of-the-art dense depth maps at nearly real-time speeds on both the NYUv2 and KITTI datasets. We surpass the state-of-the-art for monocular depth estimation even with depth values for only 1 out of every ~10000 image pixels, and we outperform other sparse-to-dense depth methods at all sparsity levels. With depth values for 1/256 of the image pixels, we achieve a mean absolute error of less than 1% of actual depth on indoor scenes, comparable to the performance of consumer-grade depth sensor hardware. Our experiments demonstrate that it would indeed be possible to efficiently transform sparse depth measurements obtained using e.g. lower-power depth sensors or SLAM systems into high-quality dense depth maps.Comment: European Conference on Computer Vision (ECCV) 2018. Updated to camera-ready version with additional experiment

Some open questions in "wave chaos"

The subject area referred to as "wave chaos", "quantum chaos" or "quantum chaology" has been investigated mostly by the theoretical physics community in the last 30 years. The questions it raises have more recently also attracted the attention of mathematicians and mathematical physicists, due to connections with number theory, graph theory, Riemannian, hyperbolic or complex geometry, classical dynamical systems, probability etc. After giving a rough account on "what is quantum chaos?", I intend to list some pending questions, some of them having been raised a long time ago, some others more recent

A finite model of two-dimensional ideal hydrodynamics

A finite-dimensional su($N$) Lie algebra equation is discussed that in the infinite $N$ limit (giving the area preserving diffeomorphism group) tends to the two-dimensional, inviscid vorticity equation on the torus. The equation is numerically integrated, for various values of $N$, and the time evolution of an (interpolated) stream function is compared with that obtained from a simple mode truncation of the continuum equation. The time averaged vorticity moments and correlation functions are compared with canonical ensemble averages.Comment: (25 p., 7 figures, not included. MUTP/92/1

Massively distributed authorship of academic papers

Wiki-like or crowdsourcing models of collaboration can provide a number of benefits to academic work. These techniques may engage expertise from different disciplines, and potentially increase productivity. This paper presents a model of massively distributed collaborative authorship of academic papers. This model, developed by a collective of thirty authors, identifies key tools and techniques that would be necessary or useful to the writing process. The process of collaboratively writing this paper was used to discover, negotiate, and document issues in massively authored scholarship. Our work provides the first extensive discussion of the experiential aspects of large-scale collaborative researc

Thinking outside the box: Generation of unconstrained 3D room layouts

We propose a method for room layout estimation that does not rely on the typical box approximation or Manhattan world assumption. Instead, we reformulate the geometry inference problem as an instance detection task, which we solve by directly regressing 3D planes using an R-CNN. We then use a variant of probabilistic clustering to combine the 3D planes regressed at each frame in a video sequence, with their respective camera poses, into a single global 3D room layout estimate. Finally, we showcase results which make no assumptions about perpendicular alignment, so can deal effectively with walls in any alignment

The devil is in the decoder

Many machine vision applications require predictions for every pixel of the input image (for example semantic segmentation, boundary detection). Models for such problems usually consist of encoders which decreases spatial resolution while learning a high-dimensional representation, followed by decoders who recover the original input resolution and result in low-dimensional predictions. While encoders have been studied rigorously, relatively few studies address the decoder side. Therefore this paper presents an extensive comparison of a variety of decoders for a variety of pixel-wise prediction tasks. Our contributions are: (1) Decoders matter: we observe significant variance in results between different types of decoders on various problems. (2) We introduce a novel decoder: bilinear additive upsampling. (3) We introduce new residual-like connections for decoders. (4) We identify two decoder types which give a consistently high performance