Carlo Rovelli won the second prize in the 2013 FQXi community’s essay competition with an essay entitled, “Relative Information at the foundation of physics”. This is how FQXi competition topic is introduced:
It From Bit or Bit From It?
The past century in fundamental physics has shown a steady progression away from thinking about physics, at its deepest level, as a description of material objects and their interactions, and towards physics as a description of the evolution of information about and in the physical world. Moreover, recent years have shown an explosion of interest at the nexus of physics and information, driven by the “information age” in which we live, and more importantly by developments in quantum information theory and computer science.
We must ask the question, though, is information truly fundamental or not? Can we realize John Wheeler’s dream, or is it unattainable? We ask: “It From Bit or Bit From It?”
Possible topics or sub-questions include, but are not limited to:
What IS information? What is its relation to “Reality”?
How does nature (the universe and the things therein) “store” and “process” information?
How does understanding information help us understand physics, and vice-versa?
In the analysis that follows, I reproduce Rovelli’s essay verbatim, but intersperse the text with my own commentary which is highlighted as red text. Incidentally, there are some spelling and grammar errors in the original text that have not been altered. Commentary on the essay can be found here on the FQXi website
anthropomorphic: anthropo – word-forming element meaning “pertaining to man or human beings,”
infopomorphic: infopo – pertaining to information. Infopomorphic, is the misappropriation of information to objects, states, or processes, rather than about objects, states, or processes.
‘Relative Information at the foundation of physics’ by Carlo Rovelli
Shannon’s notion of relative information between two physical systems can function as foundation for statistical mechanics and quantum mechanics, without referring to subjectivism or idealism. It can also represent a key missing element in the foundation of the naturalistic picture of the world, providing the conceptual tool for dealing with its apparent limitations. I comment on the relation between these ideas and Democritus.
I. IS THERE A SUBJECTIVE ELEMENT IN STATISTICAL MECHANICS?
Thermodynamical quantities such as entropy and temperature depend on the macroscopical variables chosen to describe systems with many degrees of freedom. They depend on coarse-graining. For instance, entropy can be defined (in the microcanonical) in terms of the the number of microstates compatible with what we know about the system. With this definition, it changes if we know more. This appears to insert a puzzling subjective element in physics. There is a tension with the fact that termodynamical laws seem to hold quite independently on any choice by us. Is the Sun “hot” just because we “choose” a certain coarse graining for describing it? Does entropy increases because of our choices?
The way out of the puzzle is simple. Entropy is neither something inherent to the microstate of a system, nor something depending on our subjective “knowledge” about it. Rather, it [entropy] is a property [color emphasis added] of certain (macroscopic) variables.
The follow point may appear pedantic, but it is important nonetheless. Entropy is not a “property”. Rather variables display macroscopic characteristics so one might say, entropy is a description for, or function of macroscopic variables. (see below)
For instance, the full state of a gas in a box is described by the position and velocity of its molecules. No entropy so far. But volume, total energy and (time averaged) pressure on the box boundaries are well defined functions of this state, and Entropy is a function of these [color emphasis added]. This is the first step.
Now consider a situation where the gas interacts with a second system coupled only to volume, total energy and pressure of the gas (for instance, it interacts with the gas by a thermometer and a spring holding a piston).
Although not explicitly stated as such, “it” re-enforces the notion that ‘the body’ of gas is a single naturally occurring entity; a single body with “properties”… but of course it is not. It becomes a body only by virtue of the artificial constrains imposed upon it, be those constraints actual or hypothetical.
Then the physical interactions between the gas and this system are objectively described by thermodynamics.
In other words, it is not an arbitrary or subjective choice of a coarse-graining that makes thermodynamics physically relevant: it is the concrete way another physical system is coupled to the gas. If the coupling is such that it depends only on certain gas macroscopic variables, then the physical interactions of the gas and this system are objectively well governed by thermodynamics.
But bear in mind that ‘the’ gas and ‘the’ system are subjects of artificial constraint.
This key observation clarifies the role that information plays in physics. Entropy, indeed, is information [color emphasis added]: in the micro-canonical language entropy is determined by the number of microstates compatible with a given macrostate. The number of states in which something can be, is precisely the definition of “information” (more precisely, “lack of information”) given by Shannon in his celebrated 1948 work that started the development of information theory .
The statement, “Entropy, indeed, is information” is relying on a significant and profound presumption:
the Shannonian presumption is that information, in reference to artificially constrained constructs, is indistinguishable or identical to information in reference to naturally self-constrained constructs.
As an alternative, consider the following:
for a construct to be self-constrained is for it to be a true ‘information construct’; but
for a construct to be artificially constrained is only for that means of constraint to be informative.
Infopomorphism (see glossary above), is a term I use to describe the tendency, particularly of physicists and philosophers, to see these two aspects of information as equivalent or indistinguishable. The importance of this will become apparent later.
Boundary conditions – be they physical or hypothetical – determine the manner in which microstates are composites of a unified whole, but said boundary conditions may be artificially or naturally occurring.
But “information”, that is, the number of alternatives compatible with what we know, is not significative in physics insofar as it depends on idealistic subjective knowledge: it is relevant in physics when it refers to the interaction between two systems where the effects of the interaction on the second depend only on few variables of the first, and are independent on the rest of the variables. Under these circumstances, the number of states of the first system which are not distinguished by these variables is the number of Shannon “alternatives”, relevant for the definition of thermodynamical entropy. Here “information”, counts the number of states of a system which behave equally in the interaction with a second system.
Therefore the information relevant in physics is always the relative information between two systems. There is no subjective element in it: it is fully determined by the state and the interaction Hamiltonian which dictates which variables are the relevant ones in the interaction.
Pictorially: it is not the microstate of the Sun which is hot, it is the manner the Sun affects the Earth which is objectively hot.
II. RELATIVE IRREVERSIBILITY
Reconsider the quintessential irreversible phenomenon: a cup falls to the floor and breaks, in the light of the observation above. On one account this is obviously an irreversible phenomenon, but is it so on any possible account? The event is one among the many possible dynamical evolutions of a bunch of molecules. What makes the starting configuration more “special” that the final one?
The first state is macroscopically artificial. It is an example of an artificially constrained ‘bunch of molecules’.
Something does so, but it is not in the microstate of the molecules: it is the manner we describe, or better, at the light of the previous section, we interact with it. It is because of our macroscopic account of the cup, dictated by the variables we interact with, that the initial state is special and therefore entropy increases.
To illustrate this, consider a box full of balls, characterized by two properties, say color and electrical charge. Say they have two possible colors: white and black; and two possible value of the charge: neutral and charged. Consider a microstate Col where white balls are on the left of the box and black balls on the right, while charge is randomly distributed. And consider a different microstate Ch, where charged balls are on the left and neutral balls on the right, while color is randomly dis- tributed. To normal eyes, Col looks as a low-entropy state and Ch as a high entropy state. But to a person who is color blind but has an electrometer it us Ch that looks low-entropy and Col that appears to have high entropy. Who is right? Both, of course. Entropy is relational: it pertains to the relation between two interacting systems, not to a single system.
These balls do not illustrate variations of entropy interpretation because neither the colors nor the charges constitute “interacting systems”. In like fashion two pebbles in a bowl may be informative to an observer – in a manner dependent on the nature of the observer’s mode of observation – but in itself, the bowl and pebbles, are not an information construct.
Could the breaking cup be observed by somebody else, coupling differently to it, as a process where entropy decreases? Yes of course. Imagine each fragment of the cup moving to a picture of itself on the ground, pictured in color to which you are color blind.
If these considerations are correct, then the irreversibility of the worlds is to be understood as a property of the couplings between systems, rather than a property of isolated systems.
This assumes no distinction between systems with artificially induced boundaries and those with naturally occurring boundaries. For example, is this interpretation consistent with the view that systems with artificial boundaries – boundaries generated by either subjective reasoning or material constraint – are coherent interpretations of physical reality or could artificial constraint create a distorted view through false inference and presumption? Smolin (2013 – Time Reborn) concludes:
“Neither mathematical beauty nor agreement with experiment can guarantee that the ideas a theory is based on bear the slightest relation to reality.” – See Edwin Eugene Klingman’s essay entry entitled ‘Gravity and the Nature of Information‘ for more on this.
III. THE LIMITS OF MICROPHYSCS WITHOUT INFORMATION
The idea that the world can be described as a vast see of interacting atoms and nothing else, can be traced back to the ancient atomism of Democritus. The naturalistic and materialistic world view of Democritus was soon criticized by Plato and Aristotle on the ground that it fails to account for the forms, or the objects, that we see in the world. What makes a certain ensemble of atoms into a given object we recognize? Plato and Aristotle (in different manners) wanted to add “forms” to the naturalistic view of Democritus. For Plato, a horse is not just an aggregate of matter: it is an imprecise realization of the abstract form (“idea”) of a horse. For Aristotle the same horse is the union of its substance and its form. But if the form is something above the substance, what is it?
What is it that makes a random disposition of molecules into a cup?
Of course, the point I have been making thus far, is that a horse is an example of an evolved, naturally constrained construct derived from natural interactions, whereas a cup is an artificially constrained construct.
Which the properties of the Democritean atoms generate collective variables? And how?
In fact, Democritus’s idea was more subtle than everything being just atoms. Democritus says that three features are relevant about the atoms: the shape of each individual atom, the order in which they are disposed, and their orientation in the structure. And Democritus uses then a powerful metaphor: like twenty letters of an alphabet can be combined in innumerable manners to give rise to comedies or tragedies, similarly the atoms can be combined in innumerable manners to give rise to the innumerable phenomena of the world.
But what is the relevance of the way in which atoms combine, in a world in which there is nothing else than atoms? If they are like letters of an alphabet, whom do they tell stories to?
I think that the key of the answer brings us back to the observation in the first section: physical systems interact and affect one another. In the course of these interactions, the way one happens to be, leaves traces on the way another is: correlations are established.
This is exactly what Hierarchical Construct Theory states but, it does make the necessary distinction between artificial (closed) and naturally occurring (open) systems constructs. Understanding the importance of this distinction may become apparent in the following passage.
Following Shannon, we can say that a system s has information about a system S if there is a physical constraint such that the number of total states of the two systems is smaller than the product of the number of states of each. For instance: if the system s can be in the states a and b and the system S can be in the states A and B, but there is a physical constraint (say do to the way the two have interacted) that forbids the combinations (a, B) and (b, A), thus allowing only the two states (a, A) and (b, B), then we say that s has (one bit of) information about S. In words, if we see the state of s, we also know the state of S. Physical interactions determine constraints among systems: if a tree happen to fall on my head, then I cannot be standing smiling anymore: I have some information about the tree.
Thus, systems have necessarily information about one another, in the sense of Shannon.
Note a “system” can only be said to have information, in and of itself, if it is not an artificially constrained systems-construct. Otherwise, one can only say of it, as per Shannon, that it is informative; that it can provide information about its construction as pertaining to those artificial constraints.
The lack of information that a system has about another is precisely the entropy of the second with respect to the first. It is relevant for the interactions with the first. It is the conventional thermodynamical entropy.
Before pursuing this line of thinking, let me bring quantum theory into the picture.
IV. QUANTUM THEORY
The discovery of quantum theory has sharpened the role of information in our understanding of the world. If we measure the state of a system with a certain precision, the resulting information specifies a region R of the phase space of the system. The unit of phase space volume is action (length2 x mass x time?1) per degree of freedom. In classical mechanics we can in principle arbitrarily refine measurements, therefore there is always a continuous (infinite) amount of missing information about a system, whatever the precision of the measurement.
No longer so after the discovery of quantum theory. If we measure the energy of a harmonic oscillator and we obtain the result that this is between E1 and E2, then there is only a finite number of possible values that the energy can have. This is given by the area of the region of phase space included between the two surfaces E1 and E2, divided by the Planck constant.
This is a general result: for all quantum systems, there is only a finite number of the orthogonal states per each finite region of phase space. The Planck constant determines the minimal phase space volume. Phase space volume measures the (missing) information we have about a system. It follows that quantum mechanics affirms that information is no longer continuous as in classical physics. It is discrete, and the Planck constant is the minimal unit of information.
This leads to a first principle at the basis of quantum theory: The information contained in any finite region of the phase space of any system is finite.
To say “the information contained in…”, is an infopomorphism. That is, it is to assume that the reasoning from which we derive an informed understanding, grants the object of consideration those very same “properties”. We are informed by our reasoning, but from this reasoning should not be infopomorthic and hastily assume that said artificially constructed terms correctly describe that object physically.
This principle does not exhaust quantum theory, because it holds for any discrete classical system as well. What further characterises quantum theory is that information can become “irrelevant”, and be renewed. By this I mean the following. If we have measured a system, the information we have about it allows us to predict its future. In quantum theory, we can always add new information to the state of a system, even after we have reached maximal information about it. By doing so, part of the old information becomes irrelevant. That is, is has no effect on future predictions. The typical case is a sequence of measurements of spins along different axes, in a two-state system. Each measurement brings novel information and makes the previous one irrelevant.
This leads to the second principle at the basis of quantum theory: It is always possible to acquire new information about a system.
The combination of these two principles generates the entire mathematical structure of quantum theory, up to some technical aspects, as was shown in . Thus, relative information that systems have about one another is a key language for grounding quantum theory.
Let us now remember that spacetime geometry is dynamical. Then any physical system must include the its spacetime region. This implies that there is natural identification between systems and spacetime regions. The interactions between spacetime regions are quantum interactions between systems. Therefore there is an exchange of informations across spacial regions.
These are quantized and discrete, because information is discrete.
The quantum discreetness, combined to the fact that geometry is dynamical and therefore quantized, leads to the discretization of space, idea that can be traced back to the thirties [3, 4] and has been concretized more recently. Discreetness translates into the discreetness of the area of two-dimensional surfaces [5, 6]. The discreteness of the area is an immediate reflex of the discreetness of the quantum information that can be transmitted across these surfaces.
V. REALITY AND INFORMATION
It seems to me that this ensemble of considerations conspire towards a picture where the fog begins a bit to dissipate over the intriguing role of information at the foundation of physics.
Information that physical systems have about one another, in the sense of Shannon, is ubiquitous in the universe.
This statement is an infopomorphism attributing information to physical systems, i.e., it is to define a system by virtue of one’s ability to become informed about it. Thus, if one identifies a whole made of complex parts about which one becomes informed, one is being infopomorphic to conclude that the whole must be a ‘system of information’. One is then liable to infer, there is no distinction between entities that are self-constrained and those to which artificial constraint has been applied; see below.
It has the consequence that on top of the microstate of a system we have also the informational state that a second system O has about any system S.
The universe is not just the position of all its Democritean atoms. It is also the net of information that all systems have about one another. Objects are not just aggregate of atoms. They are configurations of atoms…
And this is the point of this analytic exercise. Artificially constrained atoms (those that might form the macroscopic object we identify as ‘a cup’) are merely informative aggregates whereas, naturally constrained atoms (those that might form the macroscopic object we identify as ‘a horse’) truly are information configurations. They are not comparable when relating to information.
…singled out because of the manner a given other system interacts with them. An object is only such with respect to an observer interacting with it.
Among all systems, living ones are those that selection has led to persist and reproduce by, in particular, making use of the information they have about the exterior world.
Yes, this is true but only because they are an example of a construct that is open and, therefore, not artificially constrained.
This is why we can understand them in terms of finality and intentionality. They are those that have persisted thanks to the finality in their structure. Thus, it is not finality that drives structure, but selected structures define finality. Since the interaction with the world is described by information, it is by dealing with information that these systems most effectively persist. This is why we have DNA code, immune systems, sensory organs, neural systems, memory, complex brains, language, books, MAC’s and the ArXives. To maximize the management of information.
This illustrates the dangers of infopomorphism, of which the Integrated Information Theory of mind is an example – IIT being a theoretical incarnation of Dennett’s Intentional Stance; committing his same error. An object is an information construct only when it is a self-constrained emergent consequence of interactive exchanges in an environment. When one becomes informed by processes of artificial constraint one must not infer that those constrained objects are information constructs in their own right.
The statue that Aristotle wants made of more than atoms, is made by more than atoms: it is something that pertains to the interaction between the stone and brain of Aristotle, or ours. It is something that pertains to the stone, the goddess represented, Phidias, a woman he met, our education, and else. The atoms of that statue talk to us precisely in the same manner in which a white ball in my hand “says” that the ball in your hand is also white, if the two are correlated. By carrying information.
This is why, I think, from the basis of genetics, to the foundation of quantum mechanics and thermodynamics, all the way to sociology and quantum gravity, the notion of information has a pervasive and unifying role. The world is not just a blind wind of atoms, or general covariant quantum fields. It is also the infinite game of mirrors reflecting one another formed the correlations among the structures made by the elementary objects. To go back to Democritus metaphor: atoms are like an alphabet, but an immense alphabet so rich to be capable of reading itself and thinking itself. In Democritus words:
“The Universe is change, life is opinion that adapt itself”.
 C. E. Shannon, The Bell System Technical Journal, XXVII, 379 (1948).
 C. Rovelli, Int. J. Theor. Phys., 35, 1637 (1996), arXiv:9609002 [quant-ph].
 M. P. Bronstein, Zh. Eksp. Tear. Fiz., 6, 195 (1936).
 M. P. Bronstein, Phys. Z. Sowjetunion, 9, 140 (1936).