Particular flows and attracting sets: A comment on "How particular is the physics of the Free Energy Principle?" by Aguilera, Millidge, Tschantz and Buckley

In this commentary, I expand on the analysis of the recent article "How particular is the physics of the Free Energy Principle?" by Aguilera et al. by studying the flow fields of linear diffusions, and particularly the rotation of their attracting sets in the presence of different types of solenoidal coupling. This analysis sheds new light on previous claims made in the FEP literature (and contested in the target article) that the internal dynamics of stochastic systems can be cast performing a gradient flow on variational free energy, and thus endowed with an inferential interpretation, i.e., as if internal states are performing inference about states external to the system. I express general agreement with the target article's statement that the marginal flow of internal states does not point along variational free energy gradients evaluated at the most likely internal state (i.e., the conditional mode). However, in this commentary I focus on the flow of particular states (internal and blanket states) and their variational free energy gradients, and show that for a wide but restricted class of solenoidal couplings, the average flow of these systems point along variational free energy gradients. This licenses a different but perhaps stronger re-description of the flow of particular states as performing inference, which importantly holds at arbitrary points in state space, not just at the conditional modes

An analytical model of active inference in the Iterated Prisoner's Dilemma

This paper addresses a mathematically tractable model of the Prisoner's Dilemma using the framework of active inference. In this work, we design pairs of Bayesian agents that are tracking the joint game state of their and their opponent's choices in an Iterated Prisoner's Dilemma game. The specification of the agents' belief architecture in the form of a partially-observed Markov decision process allows careful and rigourous investigation into the dynamics of two-player gameplay, including the derivation of optimal conditions for phase transitions that are required to achieve certain game-theoretic steady states. We show that the critical time points governing the phase transition are linearly related to each other as a function of learning rate and the reward function. We then investigate the patterns that emerge when varying the agents' learning rates, as well as the relationship between the stochastic and deterministic solutions to the two-agent system

Spin glass systems as collective active inference

An open question in the study of emergent behaviour in multi-agent Bayesian systems is the relationship, if any, between individual and collective inference. In this paper we explore the correspondence between generative models that exist at two distinct scales, using spin glass models as a sandbox system to investigate this question. We show that the collective dynamics of a specific type of active inference agent is equivalent to sampling from the stationary distribution of a spin glass system. A collective of specifically-designed active inference agents can thus be described as implementing a form of sampling-based inference (namely, from a Boltzmann machine) at the higher level. However, this equivalence is very fragile, breaking upon simple modifications to the generative models of the individual agents or the nature of their interactions. We discuss the implications of this correspondence and its fragility for the study of multiscale systems composed of Bayesian agents.Comment: Accepted for publication: 3rd International Workshop on Active Inferenc

The free energy principle made simpler but not too simple

This paper provides a concise description of the free energy principle, starting from a formulation of random dynamical systems in terms of a Langevin equation and ending with a Bayesian mechanics that can be read as a physics of sentience. It rehearses the key steps using standard results from statistical physics. These steps entail (i) establishing a particular partition of states based upon conditional independencies that inherit from sparsely coupled dynamics, (ii) unpacking the implications of this partition in terms of Bayesian inference and (iii) describing the paths of particular states with a variational principle of least action. Teleologically, the free energy principle offers a normative account of self-organisation in terms of optimal Bayesian design and decision-making, in the sense of maximising marginal likelihood or Bayesian model evidence. In summary, starting from a description of the world in terms of random dynamical systems, we end up with a description of self-organisation as sentient behaviour that can be interpreted as self-evidencing; namely, self-assembly, autopoiesis or active inference.Comment: 44 pages, 9 Figures; 64 pages including graphical abstract, highlights and reference

Path integrals, particular kinds, and strange things

This paper describes a path integral formulation of the free energy principle. The ensuing account expresses the paths or trajectories that a particle takes as it evolves over time. The main results are a method or principle of least action that can be used to emulate the behaviour of particles in open exchange with their external milieu. Particles are defined by a particular partition, in which internal states are individuated from external states by active and sensory blanket states. The variational principle at hand allows one to interpret internal dynamics - of certain kinds of particles - as inferring external states that are hidden behind blanket states. We consider different kinds of particles, and to what extent they can be imbued with an elementary form of inference or sentience. Specifically, we consider the distinction between dissipative and conservative particles, inert and active particles and, finally, ordinary and strange particles. Strange particles (look as if they) infer their own actions, endowing them with apparent autonomy or agency. In short - of the kinds of particles afforded by a particular partition - strange kinds may be apt for describing sentient behaviour.Comment: 31 pages (excluding references), 6 figure

A Theory of Generalized Coordinates for Stochastic Differential Equations

Stochastic differential equations are ubiquitous modeling tools in applied mathematics and the sciences. In most modeling scenarios, random fluctuations driving dynamics or motion have some nontrivial temporal correlation structure, which renders the SDE non‐Markovian; a phenomenon commonly known as ‘colored’’ noise. Thus, an important objective is to develop effective tools for mathematically and numerically studying (possibly non‐Markovian) SDEs. In this paper, we formalize a mathematical theory for analyzing and numerically studying SDEs based on so‐called “generalized coordinates of motion.” Like the theory of rough paths, we analyze SDEs pathwise for any given realization of the noise, not solely probabilistically. Like the established theory of Markovian realization, we realize non‐Markovian SDEs as a Markov process in an extended space. Unlike the established theory of Markovian realization however, the Markovian realizations here are accurate on short timescales and may be exact globally in time, when flows and fluctuations are analytic. This theory is exact for SDEs with analytic flows and fluctuations, and is approximate when flows and fluctuations are differentiable. It provides useful analysis tools, which we employ to solve linear SDEs with analytic fluctuations. It may also be useful for studying rougher SDEs, as these may be identified as the limit of smoother ones. This theory supplies effective, computationally straightforward methods for simulation, filtering and control of SDEs; among others, we rederive generalized Bayesian filtering, a state‐of‐the‐art method for time‐series analysis. Looking forward, this paper suggests that generalized coordinates have far‐reaching applications throughout stochastic differential equations

From pixels to planning: scale-free active inference

This paper describes a discrete state-space model and accompanying methods for generative modeling. This model generalizes partially observed Markov decision processes to include paths as latent variables, rendering it suitable for active inference and learning in a dynamic setting. Specifically, we consider deep or hierarchical forms using the renormalization group. The ensuing renormalizing generative models (RGM) can be regarded as discrete homologs of deep convolutional neural networks or continuous state-space models in generalized coordinates of motion. By construction, these scale-invariant models can be used to learn compositionality over space and time, furnishing models of paths or orbits: that is, events of increasing temporal depth and itinerancy. This technical note illustrates the automatic discovery, learning, and deployment of RGMs using a series of applications. We start with image classification and then consider the compression and generation of movies and music. Finally, we apply the same variational principles to the learning of Atari-like games

From pixels to planning: scale-free active inference

This paper describes a discrete state-space model -- and accompanying methods -- for generative modelling. This model generalises partially observed Markov decision processes to include paths as latent variables, rendering it suitable for active inference and learning in a dynamic setting. Specifically, we consider deep or hierarchical forms using the renormalisation group. The ensuing renormalising generative models (RGM) can be regarded as discrete homologues of deep convolutional neural networks or continuous state-space models in generalised coordinates of motion. By construction, these scale-invariant models can be used to learn compositionality over space and time, furnishing models of paths or orbits; i.e., events of increasing temporal depth and itinerancy. This technical note illustrates the automatic discovery, learning and deployment of RGMs using a series of applications. We start with image classification and then consider the compression and generation of movies and music. Finally, we apply the same variational principles to the learning of Atari-like games.Comment: 64 pages, 28 figure

Federated inference and belief sharing

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves. But, how is this possible? Here, we show how all the necessary components arise from minimising free energy. We use numerical studies to simulate the generation, acquisition and emergence of language in synthetic agents. Specifically, we consider inference, learning and selection as minimising the variational free energy of posterior (i.e., Bayesian) beliefs about the states, parameters and structure of generative models, respectively. The common theme-that attends these optimisation processes-is the selection of actions that minimise expected free energy, leading to active inference, learning and model selection (a.k.a., structure learning). We first illustrate the role of communication in resolving uncertainty about the latent states of a partially observed world, on which agents have complementary perspectives. We then consider the acquisition of the requisite language-entailed by a likelihood mapping from an agent's beliefs to their overt expression (e.g., speech)-showing that language can be transmitted across generations by active learning. Finally, we show that language is an emergent property of free energy minimisation, when agents operate within the same econiche. We conclude with a discussion of various perspectives on these phenomena; ranging from cultural niche construction, through federated learning, to the emergence of complexity in ensembles of self-organising systems

Supervised structure learning

This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move - in the ensuing schemes - is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states - and their characteristic paths or dynamics