Occlusions and Their Role in Object Detection in Video

Occlusions and disocclusions are essential cues for human perception in understanding the layout of a scene. By analyzing how some parts of the scene go out of the sight (occluded) and new parts appear (disoccluded), one can infer the topology of the objects in it. Since the scene geometry and its dynamics induce this phenomena, they are fundamental cues in computer vision and video processing tasks such as visual exploration, object recognition, activity recognition, tracking and video compression. In this thesis, we first introduce three methods to detect occlusions in an image sequence: (1) a motion segmentation algorithm which partitions an oversegmented image into two parts: a region on which optical flow is expressed with a piecewise-constant field and occluded regions where flow is not defined, (2) an optical flow estimation method which additionally detects occlusions modeling them as sparse subset of the image domain, and (3) a saliency detection algorithm which detects the parts of the image domain whose motion is inconsistent with the camera motion. In the second part of the thesis, we show that the problem of object detection in a video can be cast as an unsupervised segmentation scheme using occlusion cues and solved using convex optimization for an unknown number and geometry of objects in the scene. We further extend this approach by incorporating semantic priors for object categories that are learned from object recognition datasets. This enables the detection algorithm to segment and categorize objects jointly

Joint Segmentation-Registration of Organs Using Geometric Models

In this paper, we present a novel method for the segmentation of the organs found in CT and MR images. The proposed algorithm utilizes the shape model of the target organ to gain robustness in the case where the objective organ is surrounded by other organs or tissue with the similar intensity profile. The algorithm labels the image based on the graph-cuts technique and incorporates the shape prior using a technique based on level-sets. The method requires proper registration of the shape template for an accurate segmentation, and we propose a unified registration-segmentation framework to solve this problem. Furthermore, to reduce the computational cost, the algorithm is designed to run on watershed regions instead of voxels. The accuracy of the algorithm is shown on the medical examples

Motion Segmentation with Occlusions on the Superpixel Graph

We present a motion segmentation algorithm that partitions the image plane into disjoint regions based on their parametric motion. It relies on a finer partitioning of the image domain into regions of uniform photometric properties, with motion segments made of unions of such “superpixels.” We exploit recent advances in combinatorial graph optimization that yield computationally efficient estimates. The energy functional is built on a superpixel graph, and is iteratively minimized by computing a parametric motion model in closed-form, followed by a graph cut of the superpixel adjacency graph. It generalizes naturally to multilabel partitions that can handle multiple motions. 1

Actionable Saliency Detection: Independent Motion Detection Without Independent Motion Estimation

We present a model and an algorithm to detect salient regions in video taken from a moving camera. In particular, we are interested in capturing small objects that move independently in the scene, such as vehicles and people as seen from aerial or ground vehicles. Many of the scenarios of interest challenge existing schemes based on background subtraction (background motion too complex), multi-body motion estimation (insufficient parallax), and occlusion detection (uniformly textured background regions). We adopt a robust statistical inference approach to simultaneously estimate a maximally reduced regressor, and select regions that violate the null hypothesis (co-visibility under an epipolar domain deformation) as “salient”. We show that our algorithm can perform even in the absence of camera calibration information: while the resulting motion estimates would be incorrect, the partition of the domain into salient vs. non-salient is unaffected. We demonstrate our algorithm on video footage from helicopters, airplanes, and ground vehicles. Figure 1: Detecting salient regions under camera motion: Left: Tracked feature points (blue) are classified as inliers (green) or outliers (red). Right: Estimated salient point density obtained by our algorithm. 1