Improving Real-Time Bidding Using a Constrained Markov Decision Process

Online advertising is increasingly switching to real-time bidding on advertisement inventory, in which the ad slots are sold through real-time auctions upon users visiting websites or using mobile apps. To compete with unknown bidders in such a highly stochastic environment, each bidder is required to estimate the value of each impression and to set a competitive bid price. Previous bidding algorithms have done so without considering the constraint of budget limits, which we address in this paper. We model the bidding process as a Constrained Markov Decision Process based reinforcement learning framework. Our model uses the predicted click-through-rate as the state, bid price as the action, and ad clicks as the reward. We propose a bidding function, which outperforms the state-of-the-art bidding functions in terms of the number of clicks when the budget limit is low. We further simulate different bidding functions competing in the same environment and report the performances of the bidding strategies when required to adapt to a dynamic environment.</p

QCD and strongly coupled gauge theories : challenges and perspectives

We highlight the progress, current status, and open challenges of QCD-driven physics, in theory and in experiment. We discuss how the strong interaction is intimately connected to a broad sweep of physical problems, in settings ranging from astrophysics and cosmology to strongly coupled, complex systems in particle and condensed-matter physics, as well as to searches for physics beyond the Standard Model. We also discuss how success in describing the strong interaction impacts other fields, and, in turn, how such subjects can impact studies of the strong interaction. In the course of the work we offer a perspective on the many research streams which flow into and out of QCD, as well as a vision for future developments.Peer reviewe

In Re: Experimental Education

This chapter appeared in a volume that summarized the results of the three-year "Patterns of Intuition" project that Gerhard Nierhaus and others directed: the project investigated the creative act of composing by means of algorithmic composition. Central to the investigation were the compositional strategies of twelve composers, which were documented through a dialogic and cyclic process of modelling and evaluating musical materials. In the second part of the book contributions by various authors considered the project from different perspectives, offered independent approaches, or provided more general reflections from their respective research fields. The present chapter situates the project in the context of experimental practice in music, especially in the tradition of pragmatist aesthetics and education best represented in the writings of John Dewey

Clonal haematopoiesis and risk of chronic liver disease

Chronic liver disease is a major public health burden worldwide. Although different aetiologies and mechanisms of liver injury exist, progression of chronic liver disease follows a common pathway of liver inflammation, injury and fibrosis. Here we examined the association between clonal haematopoiesis of indeterminate potential (CHIP) and chronic liver disease in 214,563 individuals from 4 independent cohorts with whole-exome sequencing data (Framingham Heart Study, Atherosclerosis Risk in Communities Study, UK Biobank and Mass General Brigham Biobank). CHIP was associated with an increased risk of prevalent and incident chronic liver disease (odds ratio = 2.01, 95% confidence interval (95% CI) [1.46, 2.79]; P < 0.001). Individuals with CHIP were more likely to demonstrate liver inflammation and fibrosis detectable by magnetic resonance imaging compared to those without CHIP (odds ratio = 1.74, 95% CI [1.16, 2.60]; P = 0.007). To assess potential causality, Mendelian randomization analyses showed that genetic predisposition to CHIP was associated with a greater risk of chronic liver disease (odds ratio = 2.37, 95% CI [1.57, 3.6]; P < 0.001). In a dietary model of non-alcoholic steatohepatitis, mice transplanted with Tet2-deficient haematopoietic cells demonstrated more severe liver inflammation and fibrosis. These effects were mediated by the NLRP3 inflammasome and increased levels of expression of downstream inflammatory cytokines in Tet2-deficient macrophages. In summary, clonal haematopoiesis is associated with an elevated risk of liver inflammation and chronic liver disease progression through an aberrant inflammatory response