Title: free_energy_jared
Tags: free_energy

Q: What is the Friston definition of an organism's 'phenotype'?
A: The set of physiological and sensory states in which an organism can survive

Q: The set of [...] and [...] states which an organism can survive in is its [...]
A: sensory; physiological; phenotype

The free energy is an information theory measure that is an upper bound on the surprise an organism experiences when sampling some data, given a generative model

Q: What quantity is represented by this formula: \(-\ln P(x)\)
A: Surprisal

Q: Free energy is an upper bound on [...]
A: surprisal

Q: How would you write the total probability of a state as the sum over all environmental states T that could cause it:
A: \(P(S) = \sum_{i}{P(S|T_i)P(T_i)}\)

Q: Why do we want to minimise the KL-divergence between the R-density and true posterior belief:
A: Doing so makes our R-density \(q(T)\) approximate the true posterior belief \(P(T|S)\)

Q: What does \(P(T|S)\) represent?
A: The probability that the true environmental temperature is \(x\) given that we're experiencing sensation \(S\)

Q: What does the Kullback-Liebler Divergence quantify?
A: The difference between two probability distributions

Q: Why does this integral equal 1?
\(\int \text{dT}\ q \left( T \right) = 1\)
A: Probabilities must sum to 1

Q: What does this equation read as in english?
\(F\  \geq \  - \ln P(S)\)
A: Free energy is an upper bound on surprise

Q: State the key idea in this equation in plain english: \(F = \int \text{dT}\ q\left( T \right) \mathbf{E} (T,S) + \int \text{dT}\ q( T ) \ln q( T )\)
A: The free energy can be written as the average of an energy, minus an entropy term
(remember the negative of a negative gives the positive above)

Q: What is this the formula for?
\(\int \text{dT}\ q( T ) \ln \frac{ q ( T )}{ P ( T | S )}\)
A: The KL-divergence

Q: What is the R-density, \(q(T)\)?
A: The organism's current best guess at the true posterior belief \(P(T|S)\)

Q: What is the G-density, P(T,S)?
A: A model encoding how environmental states correlate with sensory data

Q: Why can't the organism just minimise the KL-divergence given by \(D_{\text{KL}} (q(T)||P(T|S))  = \int \text{dT} q( T ) \ln \frac{q( T }{P( T | S )}\)
A: Because this would require the organism to already know \(P(T|S)\), which is the unknown quantity we're trying to compute!