Visualization of leukocyte transendothelial and interstitial migration using reflected light oblique transillumination in intravital video microscopy

Dynamic visualization of the intravascular events leading to the extravasation of leukocytes into tissues by intravital microscopy has significantly expanded our understanding of the underlying molecular processes. In contrast, the detailed observation of leukocyte transendothelial and interstitial migration in vivo has been hampered by the poor image contrast of cells within turbid media that is obtainable by conventional brightfield microscopy. Here we present a microscopic method, termed reflected light oblique transillumination microscopy, that makes use of the optical interference phenomena generated by oblique transillumination to visualize subtle gradients of refractive indices within tissues for enhanced image contrast. Using the mouse cremaster muscle, we demonstrate that this technique makes possible the reliable quantification of extravasated leukocytes as well as the characterization of morphological phenomena of leukocyte transendothelial and interstitial migration

Optimistic Agents are Asymptotically Optimal

We use optimism to introduce generic asymptotically optimal reinforcement learning agents. They achieve, with an arbitrary finite or compact class of environments, asymptotically optimal behavior. Furthermore, in the finite deterministic case we provide finite error bounds.Comment: 13 LaTeX page

Extreme State Aggregation Beyond MDPs

We consider a Reinforcement Learning setup where an agent interacts with an environment in observation-reward-action cycles without any (esp.\ MDP) assumptions on the environment. State aggregation and more generally feature reinforcement learning is concerned with mapping histories/raw-states to reduced/aggregated states. The idea behind both is that the resulting reduced process (approximately) forms a small stationary finite-state MDP, which can then be efficiently solved or learnt. We considerably generalize existing aggregation results by showing that even if the reduced process is not an MDP, the (q-)value functions and (optimal) policies of an associated MDP with same state-space size solve the original problem, as long as the solution can approximately be represented as a function of the reduced states. This implies an upper bound on the required state space size that holds uniformly for all RL problems. It may also explain why RL algorithms designed for MDPs sometimes perform well beyond MDPs.Comment: 28 LaTeX pages. 8 Theorem

Mass Density Fluctuations in Quantum and Classical descriptions of Liquid Water

First principles molecular dynamics simulation protocol is established using revised functional of Perdew-Burke-Ernzerhof (revPBE) in conjunction with Grimme's third generation of dispersion (D3) correction to describe properties of water at ambient conditions. This study also demonstrates the consistency of the structure of water across both isobaric (NpT) and isothermal (NVT) ensembles. Going beyond the standard structural benchmarks for liquid water, we compute properties that are connected to both local structure and mass density uctuations that are related to concepts of solvation and hydrophobicity. We directly compare our revPBE results to the Becke-Lee-Yang-Parr (BLYP) plus Grimme dispersion corrections (D2) and both the empirical fixed charged model (SPC/E) and many body interaction potential model (MB-pol) to further our understanding of how the computed properties herein depend on the form of the interaction potential

On the Computability of Solomonoff Induction and Knowledge-Seeking

Solomonoff induction is held as a gold standard for learning, but it is known to be incomputable. We quantify its incomputability by placing various flavors of Solomonoff's prior M in the arithmetical hierarchy. We also derive computability bounds for knowledge-seeking agents, and give a limit-computable weakly asymptotically optimal reinforcement learning agent.Comment: ALT 201

Bayesian reinforcement learning with exploration

We consider a general reinforcement learning problem and show that carefully combining the Bayesian optimal policy and an exploring policy leads to minimax sample-complexity bounds in a very general class of (history-based) environments. We also prove lower bounds and show that the new algorithm displays adaptive behaviour when the environment is easier than worst-case

Probabilities on Sentences in an Expressive Logic

Automated reasoning about uncertain knowledge has many applications. One difficulty when developing such systems is the lack of a completely satisfactory integration of logic and probability. We address this problem directly. Expressive languages like higher-order logic are ideally suited for representing and reasoning about structured knowledge. Uncertain knowledge can be modeled by using graded probabilities rather than binary truth-values. The main technical problem studied in this paper is the following: Given a set of sentences, each having some probability of being true, what probability should be ascribed to other (query) sentences? A natural wish-list, among others, is that the probability distribution (i) is consistent with the knowledge base, (ii) allows for a consistent inference procedure and in particular (iii) reduces to deductive logic in the limit of probabilities being 0 and 1, (iv) allows (Bayesian) inductive reasoning and (v) learning in the limit and in particular (vi) allows confirmation of universally quantified hypotheses/sentences. We translate this wish-list into technical requirements for a prior probability and show that probabilities satisfying all our criteria exist. We also give explicit constructions and several general characterizations of probabilities that satisfy some or all of the criteria and various (counter) examples. We also derive necessary and sufficient conditions for extending beliefs about finitely many sentences to suitable probabilities over all sentences, and in particular least dogmatic or least biased ones. We conclude with a brief outlook on how the developed theory might be used and approximated in autonomous reasoning agents. Our theory is a step towards a globally consistent and empirically satisfactory unification of probability and logic.Comment: 52 LaTeX pages, 64 definiton/theorems/etc, presented at conference Progic 2011 in New Yor