Detector adaptation by maximising agreement between independent data sources

Traditional methods for creating classifiers have two main disadvantages. Firstly, it is time consuming to acquire, or manually annotate, the training collection. Secondly, the data on which the classifier is trained may be over-generalised or too specific. This paper presents our investigations into overcoming both of these drawbacks simultaneously, by providing example applications where two data sources train each other. This removes both the need for supervised annotation or feedback, and allows rapid adaptation of the classifier to different data. Two applications are presented: one using thermal infrared and visual imagery to robustly learn changing skin models, and another using changes in saturation and luminance to learn shadow appearance parameters

An improved spatiogram similarity measure for robust object localisation

Spatiograms were introduced as a generalisation of the commonly used histogram, providing the flexibility of adding spatial context information to the feature distribution information of a histogram. The originally proposed spatiogram comparison measure has significant disadvantages that we detail here. We propose an improved measure based on deriving the Bhattacharyya coefficient for an infinite number of spatial-feature bins. Its advantages over the previous measure and over histogram-based matching are demonstrated in object tracking scenarios

Multispectral object segmentation and retrieval in surveillance video

This paper describes a system for object segmentation and feature extraction for surveillance video. Segmentation is performed by a dynamic vision system that fuses information from thermal infrared video with standard CCTV video in order to detect and track objects. Separate background modelling in each modality and dynamic mutual information based thresholding are used to provide initial foreground candidates for tracking. The belief in the validity of these candidates is ascertained using knowledge of foreground pixels and temporal linking of candidates. The transferable belief model is used to combine these sources of information and segment objects. Extracted objects are subsequently tracked using adaptive thermo-visual appearance models. In order to facilitate search and classification of objects in large archives, retrieval features from both modalities are extracted for tracked objects. Overall system performance is demonstrated in a simple retrieval scenari

TennisSense: a platform for extracting semantic information from multi-camera tennis data

In this paper, we introduce TennisSense, a technology platform for the digital capture, analysis and retrieval of tennis training and matches. Our algorithms for extracting useful metadata from the overhead court camera are described and evaluated. We track the tennis ball using motion images for ball candidate detection and then link ball candidates into locally linear tracks. From these tracks we can infer when serves and rallies take place. Using background subtraction and hysteresis-type blob tracking, we track the tennis players positions. The performance of both modules is evaluated using ground-truthed data. The extracted metadata provides valuable information for indexing and efficient browsing of hours of multi-camera tennis footage and we briefly illustrative how this data is used by our tennis-coach playback interface

A hybrid method for indoor user localisation

In this work we describe an approach to indoor user localisation by combining image-based and RF-based methods and compare this new approach to prior work. This paper details a new algorithm for indoor user localisation, demonstrating more effective user localisation than prior approaches and therefore presents the next step in combining two different technologies for localisation in indoor type environments

Comparison of fusion methods for thermo-visual surveillance tracking

In this paper, we evaluate the appearance tracking performance of multiple fusion schemes that combine information from standard CCTV and thermal infrared spectrum video for the tracking of surveillance objects, such as people, faces, bicycles and vehicles. We show results on numerous real world multimodal surveillance sequences, tracking challenging objects whose appearance changes rapidly. Based on these results we can determine the most promising fusion scheme

Human motion reconstruction using wearable accelerometers

We address the problem of capturing human motion in scenarios where the use of a traditional optical motion capture system is impractical. Such scenarios are relatively commonplace, such as in large spaces, outdoors or at competitive sporting events, where the limitations of such systems are apparent: the small physical area where motion capture can be done and the lack of robustness to lighting changes and occlusions. In this paper, we advocate the use of body-worn wearable wireless accelerometers for reconstructing human motion and to this end we outline a system that is more portable than traditional optical motion capture systems, whilst producing naturalistic motion. Additionally, if information on the person's root position is available, an extended version of our algorithm can use this information to correct positional drift

Organising a daily visual diary using multifeature clustering

The SenseCam is a prototype device from Microsoft that facilitates automatic capture of images of a person's life by integrating a colour camera, storage media and multiple sensors into a small wearable device. However, efficient search methods are required to reduce the user's burden of sifting through the thousands of images that are captured per day. In this paper, we describe experiments using colour spatiogram and block-based cross-correlation image features in conjunction with accelerometer sensor readings to cluster a day's worth of data into meaningful events, allowing the user to quickly browse a day's captured images. Two different low-complexity algorithms are detailed and evaluated for SenseCam image clustering

Anti-social behavior detection in audio-visual surveillance systems

In this paper we propose a general purpose framework for detection of unusual events. The proposed system is based on the unsupervised method for unusual scene detection in web{cam images that was introduced in [1]. We extend their algorithm to accommodate data from different modalities and introduce the concept of time-space blocks. In addition, we evaluate early and late fusion techniques for our audio-visual data features. The experimental results on 192 hours of data show that data fusion of audio and video outperforms using a single modality

Image processing for smart browsing of ocean colour data products and subsequent incorporation into a multi-modal sensing framework

Ocean colour is defined as the water hue due to the presence of tiny plants containing the pigment chlorophyll, sediments and coloured dissolved organic material and so water colour can provide valuable information on coastal ecosystems. The ‘Ocean Colour project’ collects data from various satellites (e.g. MERIS, MODIS) and makes this data available online. One method of searching the Ocean Colour project data is to visually browse level 1 and level 2 data. Users can search via location (regions), time and data type. They are presented with images which cover chlorophyll, quasi-true colour and sea surface temperature (11 μ) and links to the source data. However it is often preferable for users to search such a complex and large dataset by event and analyse the distribution of colour in an image before examination of the source data. This will allow users to browse and search ocean colour data more efficiently and to include this information more seamlessly into a framework that incorporates sensor information from a variety of modalities. This paper presents a system for more efficient management and analysis of ocean colour data and suggests how this information can be incorporated into a multi-modal sensing framework for a smarter, more adaptive environmental sensor network