A sound approach: using large language models to generate audio descriptions for egocentric text-audio retrieval

Video databases from the internet are a valuable source of text-audio retrieval datasets. However, given that sound and vision streams represent different "views" of the data, treating visual descriptions as audio descriptions is far from optimal. Even if audio class labels are present, they commonly are not very detailed, making them unsuited for text-audio retrieval. To exploit relevant audio information from video-text datasets, we introduce a methodology for generating audio-centric descriptions using Large Language Models (LLMs). In this work, we consider the egocentric video setting and propose three new text-audio retrieval benchmarks based on the EpicMIR and EgoMCQ tasks, and on the EpicSounds dataset. Our approach for obtaining audio-centric descriptions gives significantly higher zero-shot performance than using the original visual-centric descriptions. Furthermore, we show that using the same prompts, we can successfully employ LLMs to improve the retrieval on EpicSounds, compared to using the original audio class labels of the dataset. Finally, we confirm that LLMs can be used to determine the difficulty of identifying the action associated with a sound

Phenotypic characterization and 16S rDNA identification of culturable non-obligate halophilic bacterial communities from a hypersaline lake, La Sal del Rey, in extreme South Texas (USA)

Background: La Sal del Rey ( the King’s Salt”) is one of several naturally-occurring salt lakes in Hidalgo County, Texas and is part of the Lower Rio Grande Valley National Wildlife Refuge. The research objective was to isolate and characterize halophilic microorganisms from La Sal del Rey. Water samples were collected from the lake and a small creek that feeds into the lake. Soil samples were collected from land adjacent to the water sample locations. Sample salinity was determined using a refractometer. Samples were diluted and cultured on a synthetic saline medium to grow halophilic bacteria. The density of halophiles was estimated by viable plate counts. A collection of isolates was selected, gram-stained, tested for catalase, and characterized using API 20E® test strips. Isolates were putatively identified by sequencing the 16S rDNA. Carbon source utilization by the microbial community from each sample site was examined using EcoPlate™ assays and the carbon utilization total activity of the community was determined. Results: Results showed that salinity ranged from 4 parts per thousand (ppt) at the lake water source to 420 ppt in water samples taken just along the lake shore. The density of halophilic bacteria in water samples ranged from 1.2 × 102 - 5.2 × 103 colony forming units per ml (cfu ml-1) whereas the density in soil samples ranged from 4.0 × 105 - 2.5 × 106 colony forming units per gram (cfu g-1). In general, as salinity increased the density of the bacterial community decreased. Microbial communities from water and soil samples were able to utilize 12 - 31 carbon substrates. The greatest number of substrates utilized was by water-borne communities compared to soil-based communities, especially at lower salinities. The majority of bacteria isolated were gram-negative, catalase-positive, rods. Biochemical profiles constructed from API 20E® test strips showed that bacterial isolates from low-salinity water samples (4 ppt) showed the greatest phenotypic diversity with regards to the types and number of positive tests from the strip. Isolates taken from water samples at the highest salinity (420 ppt) tended to be less diverse and have only a limited number of positive tests. Sequencing of 16S DNA displayed the presence of members of bacterial genera Bacillus, Halomonas, Pseudomonas, Exiguobacterium and others. The genus Bacillus was most commonly identified. None of the isolates were members of the Archaea probably due to dilution of salts in the samples. Conclusions: The La Sal del Rey ecosystem supports a robust and diverse bacterial community despite the high salinity of the lake and soil. However, salinity does appear to a limiting factor with

Geokinematics of Central Europe: New insights from the CERGOP-2/Environment Project

The Central European Geodynamics Project CERGOP/2, funded by the European Union from 2003to 2006 under the 5th Framework Programme, benefited from repeated measurements of thecoordinates of epoch and permanent GPS stations of the Central European GPS Reference Network(CEGRN), starting in 1994. Here we report on the results of the systematic processing of availabledata up to 2005. The analysis has yielded velocities for some 60 sites, covering a variety of CentralEuropean tectonic provinces, from the Adria indenter to the Tauern window, the Dinarides, thePannonian Basin, the Vrancea seismic zone and the Carpathian Mountains. The estimated velocitiesdefine kinematical patterns which outline, with varying spatial resolution depending on the stationdensity and history, the present day surface kinematics in Central Europe. Horizontal velocities areanalyzed after removal from the ITRF2000 estimated velocities of a rigid rotation accounting forthe mean motion of Europe: a ~2.3 mm/yr north-south oriented convergence rate between Adria andthe Southern Alps that can be considered to be the present day velocity of the Adria indenterrelative to the European foreland. An eastward extrusion zone initiates at the Tauern Window. Thelateral eastward flow towards the Pannonian Basin exhibits a gentle gradient from 1-1.5 mm/yrimmediately east of the Tauern Window to zero in the Pannonian Basin. This kinematic continuityimplies that the Pannonian plate fragment recently suggested by seismic data does not require aspecific Eulerian pole. On the southeastern boundary of the Adria microplate, we report a velocitydrop from 4-4.5 mm/yr motion near Matera to ~1 mm/yr north of the Dinarides, in the southwesternpart of the Pannonian Basin. A positive velocity gradient as one moves south from West Ukraineacross Rumania and Bulgaria is estimated to be 2 mm/yr on a scale of 600-800 km, as if the crustwere dragged by the counterclockwise rotation along the North Anatolian Fault Zone. This regimeapparently does not interfere with the Vrancea seismic zone: earthquakes there are sufficiently deep(> 100 km) that the brittle deformation at depth can be considered as decoupled from the creep atthe surface. We conclude that models of the Quaternary tectonics of Central and Eastern Europeshould not neglect the long wavelength, nearly aseismic deformation affecting the upper crust in theRomanian and Bulgarian regions

A Self-supervised Classification Algorithm for Sensor Fault Identification for Robust Structural Health Monitoring

A self-supervised classification algorithm is proposed for detecting and isolating sensor faults of health monitoring devices. This is achieved by automatically extracting information from failure investigations. This approach uses (i) failure reports for extracting comprehensive failure labels; (ii) recorded data of a faulty monitoring device and the information of the failure type for selecting fault-sensitive features. The features-label pairs are then used to train a classification algorithm, so that when a new set of measurements becomes available, the algorithm is capable of identifying with a high accuracy one of the possible failure types included in the training data set. The proposed approach is successfully applied to the failure investigations conducted on a low-cost wearable device, displaying similar challenges encountered in SHM.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Engineering StructuresMechanics and Physics of Structure

Sensor fault label identification for robust structural health monitoring

Health Monitoring strategies rely on tracking the health status of critical engineering structures (Structural Health Monitoring) and of people (monitoring of medical conditions) to detect anomalies in the measurements and make inferences on the health condition for supporting decisions on preventive actions to be implemented to restore normal conditions. In these applications, the health monitoring devices are subjected daily to various events that can damage internal electrical components and sensors. As a result, the quality of the data collected can be compromised and therefore lead to a wrong health assessment. Therefore, robust health monitoring strategies need to be capable of automatically detecting sensors failures. Having the sensors' data is often not enough to gain insights into a monitoring system failure since the data variation can be related to changes in operating and environmental conditions. Alternatively, a supervised machine learning approach can be used. However, this requires an engineer to label the data in real-time, which rarely happens. Nonetheless, the common practice when a system fails is to write failure reports from which information about the failure can be extracted. Manually extracting comprehensive labels from the failure reports can be time-consuming. A strategy for automatically extracting failure labels from a set of failure reports written to describe failures of different types of sensors of a monitoring device is presented. This strategy consists in transforming the reports in their word vector form, processing each failure report to reduce the list of important words and identifying clusters of reports. The feasibility of the proposed approach is shown through its application to the failure reports compiled to describe seven types of failure of a low-cost wearable device based on an Arduino programmable board. Comparisons between manually extracted labels, and labels extracted with the proposed strategy when considering semi-supervised and unsupervised clustering strategies are presented. It is shown that the proposed strategy is capable of identify the failure label of a cluster of reports with a good accuracy. Therefore, enabling the development of a self-supervised classification algorithm for sensor fault identification for robust Structural Health Monitoring.Mechanics and Physics of Structure