In the early 1990s, British neuroscientist Karl Friston was poring over brain scans. The scans produced terabytes of digital output, and Friston had to find new techniques to sort and classify the massive flows of data.
Along the way he had a revelation. The techniques he was using might be similar to what the brain itself was doing when it processed visual data.
Could it be he had stumbled upon a solution to a data engineering problem that nature had discovered long ago? Friston’s eureka moment led to a “theory of everything”, which claims to explain the behaviour of the brain, the mind, and life itself.
As we discovered when we put together a collection of papers, the theory – known as the “free energy principle” – is controversial among scientists and philosophers.
Re-engineering nature
Friston’s initial idea was appealing because the problem facing the brain is similar to that facing an experimental scientist. Both must use the data they have to draw conclusions about events they cannot observe directly.
The neuroscientist uses scan data to infer facts about brain processes. The brain uses sensory input to infer facts about the external world.
The algorithm Friston used to draw conclusions from his data – a mathematical operation called “minimising free energy” – was based on existing techniques in statistical analysis.
Friston (and others such as computer scientist Geoff Hinton) realised artificial neural networks could easily carry out this operation. And if artificial neural networks could do it, perhaps biological neural networks could too.
But Friston didn’t stop there. He reasoned that the problem of drawing conclusions from limited information is a problem faced by all living things.
This led him to the “free energy principle”: that every living thing, everywhere, minimises free energy.
The free energy principle
The Free Energy Principle and Active Inference
This entails that reducing free energy additionally reduces the agent’s model surprise, restricting its existence to a limited set of craved beliefs. The free energy principle originated from the work of von Helmholtz on ‘unconscious inference’ [57], postulating that humans inevitably perform inference in order to perform perception.
This implies that the human perceptual system continuously adjusts beliefs about the hidden states of the world in an unconscious way. The variational formulation of the free energy [1,58], along with the introduction of actions as part of the inference process, expanded the original free energy principle leading to the development of active inference.
In Figure 2, we illustrate the interplay between the main factors that determine the embodied perception process as described in active inference. At any time, the environment is in a certain state η, which is external to the agent and not directly observable. The agent interacts with the environment in two ways: either through (passive) sensorial perception, which is characterized by the observation of sensorial states o, or by actions, which can be cast as a set of active states a that the agent imposes on the environment.
Crucially, these internal states do not need to be isomorphic to the external ones, as their purpose is explaining sensorial states in accordance with active states, rather than replicating the exact dynamics of the environment Isomorphism, in this context, refers to considering a structure-preserving mapping of the state space.
According to active inference, internal and environment states are not necessarily equal and the way the internal states are organized may even differ from agent to agent, despite having to deal with similar concepts/observations/sensory states. From a biological perspective, this finds evidence in the fact that different living systems have developed different organs/tissues along their evolutionary process [16].
The role of the internal state representation is, in fact, to provide the sufficient statistics that allow a ‘best guess’ about the causes of the agent’s observations and the selection of adaptive action policies [59].
The external environment states η are the hidden causes of sensorial states o (observations). The environment attempts to represents such hidden causes through its internal model states s. Crucially, internal states may or may not correspond to external states, which means that hidden causes in the brain do not need to be represented in the same way as in the environment. Active states a (actions), which are developed according to internal states, allow the agent to condition the environment states.
As a consequence of minimizing free energy, the agent possesses beliefs about the process generating outcomes, but also about action policies that lead to generating those outcomes [60]. This corresponds to a probabilistic model of how sensations are caused and how states should be actively sampled to drive the environment’s generation of sensory data.
Because of these assumptions, the concept of ‘reward’ in active inference is very different from rewards in RL, as rewards are not signals used to attract trajectories, but rather sensory states that the agents aims to frequently visit in order to minimize its free energy [61].
From an engineering perceptive, this difference is reflected in the fact that rewards in RL are part of the environment and, thus, each environment should provide its unique reward signal, while in active inference ‘rewards’ are intrinsic to the agent, which would pursue its preferences in any environment, developing a set of most frequently visited states.
In the remainder of this section, we discuss how the agent’s probabilistic model for perception and action is learned by minimizing free energy, providing a mathematical synthesis. We consider the environment as a partially observable Markov decision process (POMDP), represented in Figure 3.
Using subscripts to specify the discrete time steps, we indicate the observation or outcome at time t with ot. To indicate sequences that span over an undefined number of time steps, we use the superscript ∼, i.e., for outcomes o˜={o1,o2,…,ot}. The succession of states s˜={s1,s2,…,st} is influenced by sequences of actions, or policies, that we indicate with π=a˜={a1,a2,…,at}.
Parameterization of the state-outcome likelihood mapping is indicated with θ. A precision parameter ζ influences action selection working as an inverse temperature over policies.
The diagram illustrates the interplay between the different factors that compose the graphical model. (1) Policy precision; (2) beliefs about policies; (3) transition probabilities, also known as dynamics; (4) parameters of the likelihood mapping; (5) likelihood model.
The section is divided in two parts: the first explains how the internal model is learned with respect to past experience, minimizing a variational free energy functional that explains the dynamics of the environment’s outcomes, given a sequence of actions. In the second part, we discuss the minimization of expected free energy with respect to the future, when actions are selected to reduce surprise with respect to the agent’s preferred outcomes. Importantly, our treatment refers to a discrete-time instantiation of active inference. For a discussion on continuous time, the reader may refer to [34].
reference link :https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8871280/#:~:text=The%20free%20energy%20principle%20originated,world%20in%20an%20unconscious%20way.
No surprises?
Minimising free energy means choosing to believe in the unobserved situation that makes your observations least surprising.
Here’s an example: imagine you are picnicking in the park, watching two friends kick a football to and fro. Your view is occluded by a tree, so you don’t see the full trajectory of the kicked ball.
Now, it is possible that there is a third person behind the tree, who catches the ball each time it passes them and then throws on a spare ball they have handy.
However, there is no evidence for the existence of this third person, so their existence would be very surprising. So you can minimise your surprise by believing there is no secret third person behind the tree.
Minimizing free energy can help guide our actions, too. According to the free energy principle, you should do things that will change the world in such a way that your experiences are less likely to be surprising!
Seen from this perspective, we eat to avoid the surprise of extreme hunger, and we seek shelter to avoid the surprise of being cold.
How much does a ‘theory of everything’ actually explain?
So the free energy principle is a “theory of everything” spanning neuroscience, psychology and biology! But not everyone is convinced it’s a useful idea.
Some of the skepticism concerns whether or not the theory is true. An even bigger concern is that, even if it is true, it may not be very useful.
But why would people think this?
The American population biologist Richard Levins famously outlined a dilemma facing scientists who study biological systems.
These systems contain a huge amount of potentially important detail, and when we model them we cannot hope to capture all of it. So how much detail should we attempt to capture, and how much should we leave out?
Levins concluded there is a trade-off between the level of detail in a model and the number of systems it applies to. A model that captures a lot of detail about a specific system will be less informative about other, similar systems.
For instance, we can model the technique of an Olympic swimmer in order to improve their performance. But that model will not faithfully represent a different swimmer.
On the other hand, a model that covers more systems will be less informative about any particular system. By modelling how humans swim in general, we can design swimming lessons for children, but individual differences between children will be ignored.
The moral is that our models should fit our aims. If you want to explain the workings of a particular system, produce a highly specific model. If you want to say things about a lot of different systems, produce a general model.
Too general to be useful?
The free energy principle is a highly general model. It might even be the most general model in the life sciences today.
But how useful are such models in the day-to-day practice of biology or psychology? Critics argue Friston’s theory is so general that it is hard to see how it might be put to practical use.
Proponents claim successes for the free energy principle, but will it turn out to be an enormous breakthrough? Or do theories that try to explain everything end up explaining nothing?
Author: Ross Pain, Michael David Kirchhoff, and Stephen Francis Mann
Source: The Conversation
