Combined Reinforcement Learning via Abstract Representations

In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages. In this paper we propose a new way of explicitly bridging both approaches via a shared low-dimensional learned encoding of the environment, meant to capture summarizing abstractions. We show that the modularity brought by this approach leads to good generalization while being computationally efficient, with planning happening in a smaller latent state space. In addition, this approach recovers a sufficient low-dimensional representation of the environment, which opens up new strategies for interpretable AI, exploration and transfer learning.Comment: Accepted to the Thirty-Third AAAI Conference On Artificial Intelligence, 201

Knee extension strength in obese and nonobese male adolescents

The aim of the present study was to compare “absolute” and “relative” knee extension strength between obese and nonobese adolescents. Ten nonobese and 12 severely obese adolescent boys of similar chronological age, maturity status, and height were compared. Total body and regional soft tissue composition were determined using dual-energy X-ray absorptiometry (DXA). Knee extensors maximum voluntary contraction (MVC) torque was measured using an isometric dynamometer at a knee angle of 60° (0° is full extension). Absolute MVC torque was significantly higher in obese adolescents than in controls. However, although MVC torque expressed per unit of body mass was found to be significantly lower in obese adolescent boys, no significant difference in MVC torque was found between groups when normalized to fat-free mass. Conversely, when correcting for thigh lean mass and estimated thigh muscle mass, MVC torque was significantly higher in the obese group (17.9% and 22.2%, respectively; P <0.05). To conclude, our sample of obese adolescent boys had higher absolute and relative knee extension strength than our nonobese controls. However, further studies are required to ascertain whether or not relative strength, measured with more accurate in vivo methods such as magnetic resonance imaging, is higher in obese adolescents than in nonobese controls

Impact of an obesogenic diet program on bone densitometry, micro architecture and metabolism in male rat.

International audienceABSTRACT: Background The relationships between fat mass and bone tissue are complex and not fully elucidated. A high-fat/high-sucrose diet has been shown to induce harmful effects on bone micro architecture and bone biomechanics of rat. When such diet leads to obesity, it may induce an improvement of biomechanical bone parameters in rodent. Here, we examined the impact of a high-fat/high-sucrose diet on the body composition and its resulting effects on bone density and structure in male rats. Forty three Wistar rats aged 7 months were split into 3 groups: 1 sacrificed before diet (BD, n=14); 1 subjected to 16 weeks of high-fat/high-sucrose diet (HF/HS, n=14); 1 subjected to standard diet (Control, n=15). Abdominal circumference and insulin sensitivity were measured and visceral fat mass was weighed. The bone mineral density (BMD) was analyzed at the whole body and tibia by densitometry. Microcomputed tomography and histomorphometric analysis were performed at L2 vertebrae and tibiae to study the trabecular and cortical bone structures and the bone cell activities. Osteocalcin and CTX levels were performed to assess the relative balance of the bone formation and resorption. Differences between groups have been tested with an ANOVA with subsequent Scheffe post-hoc test. An ANCOVA with global mass and global fat as covariates was used to determine the potential implication of the resulting mechanical loading on bone. RESULTS: The HF/HS group had higher body mass, fat masses and abdominal circumference and developed an impaired glucose tolerance compared to Control group (p<0.001). Whole body bone mass (p<0.001) and BMD (p<0.05) were higher in HF/HS group vs. Control group. The trabecular thickness at vertebrae and the cortical porosity of tibia were improved (p<0.05) in HF/HS group. Bone formation was predominant in HF/HS group while an unbalance bone favoring bone resorption was observed in the controls. The HF/HS and Control groups had higher total and abdominal fat masses and altered bone parameters vs. BD group. Conclusions The HF/HS diet had induced obesity and impaired glucose tolerance. These changes resulted in an improvement of quantitative, qualitative and metabolic bone parameters. The fat mass increase partly explained these observations

A Meta-Reinforcement Learning Algorithm for Causal Discovery

Uncovering the underlying causal structure of a phenomenon, domain or environment is of great scientific interest, not least because of the inferences that can be derived from such structures. Unfortunately though, given an environment, identifying its causal structure poses significant challenges. Amongst those are the need for costly interventions and the size of the space of possible structures that has to be searched. In this work, we propose a meta-reinforcement learning setup that addresses these challenges by learning a causal discovery algorithm, called Meta-Causal Discovery, or MCD. We model this algorithm as a policy that is trained on a set of environments with known causal structures to perform budgeted interventions. Simultaneously, the policy learns to maintain an estimate of the environment’s causal structure. The learned policy can then be used as a causal discovery algorithm to estimate the structure of environments in a matter of milliseconds. At test time, our algorithm performs well even in environments that induce previously unseen causal structures. We empirically show that MCD estimates good graphs compared to SOTA approaches on toy environments and thus constitutes a proof-of-concept of learning causal discovery algorithms