Rewriting a Deep Generative Model

A deep generative model such as a GAN learns to model a rich set of semantic and physical rules about the target distribution, but up to now, it has been obscure how such rules are encoded in the network, or how a rule could be changed. In this paper, we introduce a new problem setting: manipulation of specific rules encoded by a deep generative model. To address the problem, we propose a formulation in which the desired rule is changed by manipulating a layer of a deep network as a linear associative memory. We derive an algorithm for modifying one entry of the associative memory, and we demonstrate that several interesting structural rules can be located and modified within the layers of state-of-the-art generative models. We present a user interface to enable users to interactively change the rules of a generative model to achieve desired effects, and we show several proof-of-concept applications. Finally, results on multiple datasets demonstrate the advantage of our method against standard fine-tuning methods and edit transfer algorithms.Comment: ECCV 2020 (oral). Code at https://github.com/davidbau/rewriting. For videos and demos see https://rewriting.csail.mit.edu

Speed/Accuracy Trade-Off between the Habitual and the Goal-Directed Processes

Instrumental responses are hypothesized to be of two kinds: habitual and goal-directed, mediated by the sensorimotor and the associative cortico-basal ganglia circuits, respectively. The existence of the two heterogeneous associative learning mechanisms can be hypothesized to arise from the comparative advantages that they have at different stages of learning. In this paper, we assume that the goal-directed system is behaviourally flexible, but slow in choice selection. The habitual system, in contrast, is fast in responding, but inflexible in adapting its behavioural strategy to new conditions. Based on these assumptions and using the computational theory of reinforcement learning, we propose a normative model for arbitration between the two processes that makes an approximately optimal balance between search-time and accuracy in decision making. Behaviourally, the model can explain experimental evidence on behavioural sensitivity to outcome at the early stages of learning, but insensitivity at the later stages. It also explains that when two choices with equal incentive values are available concurrently, the behaviour remains outcome-sensitive, even after extensive training. Moreover, the model can explain choice reaction time variations during the course of learning, as well as the experimental observation that as the number of choices increases, the reaction time also increases. Neurobiologically, by assuming that phasic and tonic activities of midbrain dopamine neurons carry the reward prediction error and the average reward signals used by the model, respectively, the model predicts that whereas phasic dopamine indirectly affects behaviour through reinforcing stimulus-response associations, tonic dopamine can directly affect behaviour through manipulating the competition between the habitual and the goal-directed systems and thus, affect reaction time

Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits

Background Over the last several years, it has become apparent that there are critical problems with the hypothesis that brain dopamine (DA) systems, particularly in the nucleus accumbens, directly mediate the rewarding or primary motivational characteristics of natural stimuli such as food. Hypotheses related to DA function are undergoing a substantial restructuring, such that the classic emphasis on hedonia and primary reward is giving way to diverse lines of research that focus on aspects of instrumental learning, reward prediction, incentive motivation, and behavioral activation. Objective The present review discusses dopaminergic involvement in behavioral activation and, in particular, emphasizes the effort-related functions of nucleus accumbens DA and associated forebrain circuitry. Results The effects of accumbens DA depletions on food-seeking behavior are critically dependent upon the work requirements of the task. Lever pressing schedules that have minimal work requirements are largely unaffected by accumbens DA depletions, whereas reinforcement schedules that have high work (e.g., ratio) requirements are substantially impaired by accumbens DA depletions. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related decision making. Rats with accumbens DA depletions reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and instead, these rats select a less-effortful type of food-seeking behavior. Conclusions Along with prefrontal cortex and the amygdala, nucleus accumbens is a component of the brain circuitry regulating effort-related functions. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as energy-related disorders such as psychomotor slowing, fatigue, or anergia in depression

Depletion of nucleus accumbens dopamine leads to impaired reward and aversion processing in mice: Relevance to motivation pathologies

Dopamine (DA) neurotransmission, particularly the ventral tegmental area-nucleus accumbens (VTA-NAcc) projection, underlies reward and aversion processing, and deficient DA function could underlie motivational impairments in psychiatric disorders. 6-hydroxydopamine (6-OHDA) injection is an established method for chronic DA depletion, principally applied in rat to study NAcc DA regulation of reward motivation. Given the increasing focus on studying environmental and genetic regulation of DA function in mouse models, it is important to establish the effects of 6-OHDA DA depletion in mice, in terms of reward and aversion processing. This mouse study investigated effects of 6-OHDA-induced NAcc DA depletion using the operant behavioural test battery of progressive ratio schedule (PRS), learned non-reward (LNR), learned helplessness (LH), treadmill, and in addition Pavlovian fear conditioning. 6-OHDA NAcc DA depletion, confirmed by ex vivo HPLC-ED, reduced operant responding: for gustatory reward under effortful conditions in the PRS test; to a stimulus recently associated with gustatory non-reward in the LNR test; to escape footshock recently experienced as uncontrollable in the LH test; and to avoid footshock by physical effort in the treadmill test. Evidence for specificity of effects to NAcc DA was provided by lack of effect of medial prefrontal cortex DA depletion in the LNR and LH tests. These findings add significantly to the evidence that NAcc DA is a major regulator of behavioural responding, particularly at the motivational level, to both reward and aversion. They demonstrate the suitability of mouse models for translational study of causation and reversal of pathophysiological DA function underlying motivation psychopathologies

Unusual Histomorphometric Changes in the Iliac Crest in Ovariectomized and Sham-operated Ewes

SummaryA large animal model to study cancellous bone loss and the effect of various therapeutic agents following oestrogen deficiency-related bone loss is needed. Following double fluorochrome labeling at the time of surgery, six and 12 months later, static and dynamic histomorphometry was performed on undecalcified sections of the iliac crests of 16 mature (4 to 5-yearold) ewes following ovariectomy (OVX; n = 8) or sham-operation (Sham; n = 8). We found a slight decrease in bone mass associated with oestrogen deficiency as indicated by a statistically significant increase in trabecular separation (Tb.Sp: +14%, p &lt;0.05) and decrease in wall thickness (W.Th: -10%, p &lt;0.05) in the OVX group. However, at six months, we saw evidence of complete cessation of bone formation in both OVX and Sham animals. At that time there was a statistically significant (p &lt;0.05) decline in the following parameters in both the Sham and OVX animals respectively: osteoid surface (OS: -79%, -77%), osteoid thickness (O.Th: -40%, -30%), flat osteoblasts (Fl.Ob.S: -87%, -94%), cuboidal osteoblasts (Cu.Ob.S: -88%, -65%), osteoid surface as a percentage of the total cancellous bone perimeter (OS/BS: -82%, -80%), osteoblast surface as a percentage of total cancellous bone surface (Ob.S/BS: -94%, -77%), mineralizing surface as a percentage of double labels plus half single labels (MS/OS: -44%, -42%), osteoid volume as a percentage of total area of mineralized bone plus osteoid (OV/TV: -88%, -88%), osteoid as a percentage of mineralized bone (OV/BV: -87%, -86%), osteoid maturation time (Omt: -27%, -35%), bone formation rate, surface referent (BFR/BS: -90%, -87%), bone formation rate, volume referent (BFR/BV: -89%, -86%), and bone formation rate, tissue referent (BFR/TV: -91%, -88%). The data was compared to static and dynamic histomorphometry of three intact ewes (similar age, breed, source) whose biopsies were taken two months after the second biopsies in the present study. The data indicate that there is a slight loss of trabecular bone following OVX but we are unable to explain the dramatic depression of bone turnover with little change in bone resorption. The data resembles the response seen when sheep are given daily doses of methylprednisolone. We speculate that this phenomenon may be a response to endogenous corticosteroid release in a response to stresses of transport, and surgery. A seasonal effect may be another explanation fro these changes, whereas dietary alteration (e. g. poisonous plants), hormonal changes or response to changes in physical activity are unlikely causes.A large animal model to study cancellous bone loss and effect of various therapeutic agents following oestrogen deficiency – related bone loss is needed. Following double fluorochrome labeling at the time of surgery, six and 12 months later, static and dynamic histomorphome-try was performed on undecal-cified sections of the iliac crests of 16 mature (4 to 5-year-old) ewes following ovariectomy (OVX; n = 8) or sham-operation (Sham; n = 8). A slight decrease in bone mass associated with oestrogen deficiency was seen but there was also evidence of complete cessation of bone formation in both OVX and Sham animals. The data resembles the responses seen when sheep are given daily doses of methylpednisolone and may be a response to endogenous corticosteroid release. A seasonal effect may be another explanation for these changes, whereas dietary alteration (e. g. poisonous plants), hormonal changes or response to changes in physical activity are unlikely causes.</jats:p