Robust Modeling of Epistemic Mental States

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913.Comment: Accepted for Publication in Multimedia Tools and Application, Special Issue: Socio-Affective Technologie

LOCL: Learning Object-Attribute Composition using Localization

This paper describes LOCL (Learning Object Attribute Composition using Localization) that generalizes composition zero shot learning to objects in cluttered and more realistic settings. The problem of unseen Object Attribute (OA) associations has been well studied in the field, however, the performance of existing methods is limited in challenging scenes. In this context, our key contribution is a modular approach to localizing objects and attributes of interest in a weakly supervised context that generalizes robustly to unseen configurations. Localization coupled with a composition classifier significantly outperforms state of the art (SOTA) methods, with an improvement of about 12% on currently available challenging datasets. Further, the modularity enables the use of localized feature extractor to be used with existing OA compositional learning methods to improve their overall performance.Comment: 20 pages, 7 figures, 11 tables, Accepted in British Machine Vision Conference 202

Causes and incidence of community-acquired serious infections among young children in south Asia (ANISA): an observational cohort study.

BACKGROUND: More than 500 000 neonatal deaths per year result from possible serious bacterial infections (pSBIs), but the causes are largely unknown. We investigated the incidence of community-acquired infections caused by specific organisms among neonates in south Asia. METHODS: From 2011 to 2014, we identified babies through population-based pregnancy surveillance at five sites in Bangladesh, India, and Pakistan. Babies were visited at home by community health workers up to ten times from age 0 to 59 days. Illness meeting the WHO definition of pSBI and randomly selected healthy babies were referred to study physicians. The primary objective was to estimate proportions of specific infectious causes by blood culture and Custom TaqMan Array Cards molecular assay (Thermo Fisher, Bartlesville, OK, USA) of blood and respiratory samples. FINDINGS: 6022 pSBI episodes were identified among 63 114 babies (95·4 per 1000 livebirths). Causes were attributed in 28% of episodes (16% bacterial and 12% viral). Mean incidence of bacterial infections was 13·2 (95% credible interval [CrI] 11·2-15·6) per 1000 livebirths and of viral infections was 10·1 (9·4-11·6) per 1000 livebirths. The leading pathogen was respiratory syncytial virus (5·4, 95% CrI 4·8-6·3 episodes per 1000 livebirths), followed by Ureaplasma spp (2·4, 1·6-3·2 episodes per 1000 livebirths). Among babies who died, causes were attributed to 46% of pSBI episodes, among which 92% were bacterial. 85 (83%) of 102 blood culture isolates were susceptible to penicillin, ampicillin, gentamicin, or a combination of these drugs. INTERPRETATION: Non-attribution of a cause in a high proportion of patients suggests that a substantial proportion of pSBI episodes might not have been due to infection. The predominance of bacterial causes among babies who died, however, indicates that appropriate prevention measures and management could substantially affect neonatal mortality. Susceptibility of bacterial isolates to first-line antibiotics emphasises the need for prudent and limited use of newer-generation antibiotics. Furthermore, the predominance of atypical bacteria we found and high incidence of respiratory syncytial virus indicated that changes in management strategies for treatment and prevention are needed. Given the burden of disease, prevention of respiratory syncytial virus would have a notable effect on the overall health system and achievement of Sustainable Development Goal. FUNDING: Bill & Melinda Gates Foundation

Safety and efficacy of simplified antibiotic regimens for outpatient treatment of serious infection in neonates and young infants 0-59 days of age in Bangladesh: design of a randomized controlled trial.

BACKGROUND: Because access to care is limited in settings with high mortality, exclusive reliance on the current recommendation of 7-10 days of parenteral antibiotic treatment is a barrier to provision of adequate treatment of newborn infections. METHODS: We are conducting a trial to determine if simplified antibiotic regimens with fewer injections are as efficacious as the standard course of parenteral antibiotics for empiric treatment of young infants with clinical signs suggestive of severe infection in 4 urban hospitals and in a rural surveillance site in Bangladesh. The reference regimen of intramuscular procaine-benzyl penicillin and gentamicin given once daily for 7 days is being compared with (1) intramuscular gentamicin once daily and oral amoxicillin twice daily for 7 days and (2) intramuscular penicillin and gentamicin once daily for 2 days followed by oral amoxicillin twice daily for additional 5 days. All regimens are provided in the infant's home. The primary outcome is treatment failure (death or lack of clinical improvement) within 7 days of enrolment. The sample size is 750 evaluable infants enrolled per treatment group, and results will be reported at the end of 2013. DISCUSSION: The trial builds upon previous studies of community case management of clinical severe infections in young infants conducted by our research team in Bangladesh. The approach although effective was not widely accepted in part because of feasibility concerns about the large number of injections. The proposed research that includes fewer doses of parenteral antibiotics if shown efficacious will address this concern

IMAPS: A smart phone based real-time framework for prediction of affect in natural dyadic conversation

The lack of ability to perceive emotions and affective states is a setback for people who are blind or visually impaired in professional and social communications. Towards developing assistive technology solution in facilitating natural dyadic conversations for people with such disability, this paper describes the development of a smart phone based system called interactive mobile affect perception system (iMAPS) for prediction of affective dimensions (valence-arousal-dominance). The proposed solution utilizes an Android platform in conjunction with a wireless network to build a fully integrated iMAPS. Empirical analyses were conducted to measure the efficacy and utility of the proposed solution. It was found that the proposed framework can predict affect dimensions with good accuracy (Maximum Correlation Coefficient for valence: 0.68, arousal: 0.71, and dominance: 0.67) in natural dyadic conversation. The overall minimum and maximum response times are (181 milliseconds) and (500 milliseconds), respectively. © 2012 IEEE

Expression

Limited access to non-verbal cues hinders the dyadic conversation or social interaction of people who are blind or visually impaired. This paper presents Expression-an integrated assistive solution using Google Glass. The key function of the system is to enable the user to perceive social signals during a natural face-to-face conversation. Empirical evaluation of the system is presented with qualitative (Likert score: 4.383/5) and quantitative results (overall F-measure of the nonverbal expression recognition: 0.768). Copyright 2014 978-1-4503-3047-3/14/09

E m o A s s i s t: emotion enabled assistive tool to enhance dyadic conversation for the blind

This paper presents the design and implementation of EmoAssist: a smart-phone based system to assist in dyadic conversations. The main goal of the system is to provide access to more non-verbal communication options to people who are blind or visually impaired. The key functionalities of the system are to predict behavioral expressions (such a yawn, a closed lip smile, a open lip smile, looking away, sleepy, etc.) and 3-D affective dimensions (valence, arousal, and dominance) from visual cues in order to provide the correct auditory feedback or response. A number of challenges related to the data communication protocols, efficient tracking of the face, modeling of behavioral expressions/affective dimensions, feedback mechanism and system integration were addressed to build an effective and functional system. In addition, orientation-sensor information from the smart-phone was used to correct image alignment to improve the robustness for real world application. Empirical studies show that the EmoAssist can predict affective dimensions with acceptable accuracy (Maximum Correlation-Coefficient for valence: 0.76, arousal: 0.78, and dominance: 0.76) in natural dyadic conversation. The overall minimum and maximum response-times are (64.61 milliseconds) and (128.22 milliseconds), respectively. The integration of sensor information for correcting the orientation improved (16 % in average) the accuracy in recognizing behavioralexpressions. A usability study with ten blind people in social interaction shows that the EmoAssist is highly acceptable with an Average acceptability rating using of 6.0 in Likert scale (where 1 and 7 are the lowest and highest possible ratings, respectively)

Robust modeling of epistemic mental states

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913