4. To illustrate the limitations of entropy
we created a very simple graph:
1. Let 1 → 2 be a starting graph G connecting a
node with label 1 to a node with label 2. If a node
with label n has degree n, we call it a core node;
otherwise, we call it a supportive node.
2. Iteratively add a node n + 1 to G such that the
number of core nodes in G is maximised.
5.
6. Algorithm:
The degree sequence d of the labeled nodes
d = 1,2, 3, 4, . . . , n
is the Champernowne constant in base 10, a transcendental real
whose decimal expansion is Borel normal.
7. Algorithm:
The sequence of number of edges is a function of core and supportive nodes, defined by
[1/r] + [2/r] +···+ [n/r],
where r = (1 + √5)/2 is the golden ratio and [· ] the floor
function (sequence A183136 in the OEIS), whose values are
1, 2, 4, 7, 10, 14, 18, 23, 29, 35, 42, 50, 58, 67, 76, 86, 97, 108,
120, 132, 145,....
12. Maximum Entropy (MaxEnt)
Model Principle
The principle of maximum entropy states that the
probability distribution which best represents the
current state of knowledge is the one with largest
entropy…
E.T. Jaynes
15. K(s) = min{ |p| | U(p) = s}
Computability, Algorithmic
Complexity & Causality
(Un)Computability mediates in the challenge of causality
by way of Algorithmic Information Theory:
16. Computability, Algorithmic
Complexity & Causality
(Un)Computability mediates in the challenge of causality
by way of Algorithmic Information Theory:
Generating
mechanism
Observation
K(s) = min{ |p| | U(p) = s}
17. Generating
mechanism
Observation
K(s) = min{ |p| | U(p) = s}
Most likely
mechanism
according
to Ockham’s
Computability, Algorithmic
Complexity & Causality
(Un)Computability mediates in the challenge of causality
by way of Algorithmic Information Theory:
18. Constructor
Generating
mechanism
Observation
K(s) = min{ |p| | U(p) = s}
Most likely
mechanism
according
to Ockham’s
Computability, Algorithmic
Complexity & Causality
(Un)Computability mediates in the challenge of causality
by way of Algorithmic Information Theory:
21. Algorithmic Data Analytics, Small Data Matters and Correlation
versus Causation. In Computability of the World? Philosophy and
Science in the Age of Big Data), Springer Verlag, 2017
suggestive
methods
Coding Theorem Method
(CTM)
22. Algorithmic Data Analytics, Small Data Matters and Correlation
versus Causation. In Computability of the World? Philosophy and
Science in the Age of Big Data), Springer Verlag, 2017
Coding Theorem Method
(CTM)
suggestive
methods
39. Divide and Conquer
sample
sample
sample
sample
sample
If any part of the whole system (samples) is of high m(x) and low K(x), then that part can
be generated by mechanistic/algorithmic means and thus is causal. The lower BDM the
more causal.
Rube
Goldberg
machine
40. Block Decomposition Method
(BDM)
sample
sample
sample
sample
If any part of the whole system (samples) is of high m(x) and low K(x), then that part can
be generated by mechanistic/algorithmic means and thus is causal. The lower BDM the
more causal.
Rube
Goldberg
machine
41. Formally:
Block Decomposition Method
(BDM)
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
42. Block Decomposition Method
(BDM)
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
43. Block Decomposition Method
(BDM)
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
44. Block Decomposition Method
(BDM)
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
45. Block Decomposition Method
(BDM)
For example:
If 2 strings are 10111 and 10111, a tighter upper bound of their
algorithmic complexity K is not:
|K(U(p)=10111)| + |K(U(p)=10111)| but
|K(U(p)=10111 2 times)|
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
46. Hybrid Measure:
Shannon Entropy +
Local Algorithmic Complexity
Because clearly:
|K(U(p)=10111)| + |K(U(p)=10111)| >|K(U(p)=10111 2 times)|
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
47. Hybrid Measure:
Shannon Entropy +
Local Algorithmic Complexity
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
48. e.g. These 2 strings are of high Entropy (and high Entropy rate)
but low K:
010101110100, 011011010010
Gaining Causation Power
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
49. Examples
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
50. CTM and BDM conform with the
theoretical expectation
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
51. Example of a generating
mechanism/model
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0096223
52. Algorithmic Sequence Model
Sequential aggregation of small models produce a model able to explain larger
data. TMs: 2205 <> 1351 <> 1447 <> 599063 can produce:
S = 00000101111111111111111111110
+ +
+
54. Version history and future
of the OACC
We keep expanding the calculator to wider
horizons of methodological and numerical capabilities:
• Version 1: Estimations of K for short binary strings
• Version 2: Expanded CTM and BDM capabilities for non-binary
strings and arrays
• Version 2.5: Estimations of Bennett's logical depth based on
CTM
• Version 3 (current version): Algorithmic Information dynamics
of strings, arrays and networks
• Version 4: Algorithmic complexity of models, algorithmic feature
selection, algorithmic dimensionality reduction and model
generator. (BETA version ready!)
complexitycalculator.com
55. We have seen how important AIT and
the role of Computability theory is in
the challenge of causality in science in
the 4th revolution of causal discover
towards model-driven approaches
https://www.youtube.com/watch?v=BEaXyDS_1Cw&t=30s
63. Boundary Conditions
H Zenil, F Soler-Toscano, N.A. Kiani, S. Hernández-Orozco, A. Rueda-Toicen
A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic
Complexity, Entropy 20(8), 605, 2018.
87. The 2D CA Game of Life (GoL)
Density diagram showing the
persistence of structures:
The Game of Life
88. The Game of Life Space-time
evolution
H. Zenil et al., Algorithmic-information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of Life.
In A. Adamatzky (ed), From Parallel to Emergent Computing (book),
Taylor & Francis / CRC Press, 2018
89. The Game of Life Space-time
evolution
H. Zenil et al., Algorithmic-information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of Life.
In A. Adamatzky (ed), From Parallel to Emergent Computing (book),
Taylor & Francis / CRC Press, 2018
90. GoL’s glider
(a ‘moving’ particle)
A piece of information moving through the grid from one side to another
92. Characterization of evolving
patterns
H. Zenil et al., Algorithmic-information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of Life.
In A. Adamatzky (ed), From Parallel to Emergent Computing (book),
Taylor & Francis / CRC Press, 2018
95. Dynamics of a 4-particle
collision
New particles
Fixed point
H. Zenil et al., Algorithmic-information Properties of Dynamic Persistent Patterns and Colliding Particles in the Game
of Life. In A. Adamatzky (ed), From Parallel to Emergent Computin, Taylor & Francis / CRC, 2018
96. Algorithmic dynamics of a stable
‘near miss’
Zenil et al., Algorithmic-information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of Life.
In A. Adamatzky (ed), From Parallel to Emergent Computing (book),
Taylor & Francis / CRC Press, 2018
97. Sensitivity of dynamics to initial conditions
Periodic fixed point
Zenil et al., Algorithmic-information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of Life.
In A. Adamatzky (ed), From Parallel to Emergent Computing (book),
Taylor & Francis / CRC Press, 2018
98. Particle annihilation
H. Zenil, N.A. Kiani and J. Tegnér, Algorithmic-
information Properties of Dynamic Persistent
Patterns and Colliding Particles in the Game of
Life, In A. Adamatzky (ed), From Parallel to
Emergent Computing (book), Taylor & Francis /
CRC Press, 2018
100. All cases can be reduced to 4 density
plots (+ annihilation)
Algorithmicdynamics
ofglidercollisions
0 50 100 150 200
50
100
500
1000
BDM all cases
102. A Causal Calculus Based on
Algorithmic Information Dynamics
Let S be a non-random binary file containing a string in binary:
S= 10101010101010101010101010101010101010101010101010101010101010
Clearly, S is algorithmically compressible, with pS = “Print(01) 31 times” a short
computer program generating S. Let S’ be equal to S except for a bitwise operation
(bitwise NOT, or complement) in, say, position 23:
S’ = 10101010101010101010100010101010101010101010101010101010101010
We have that the relationship of their algorithmic complexity denoted by C is:
C(S) < C(S’)
for any single-bit mutation of S, where C(S) = |pS| and C(S’) = |pS’| are the lengths
of the shortest computer programs generating S and S’.
103. The algorithmic information dynamics of a string (step 0) pushed towards and away
from randomness by digit removal using BDM.
The initial string consists of a simple segment of ten 1s followed by a random-looking
segment of 10 digits. The resulting strings mostly extract each of the respective
segments.
Moving a string
104. Neutral Deletion
Neutral digit deletion maximises the preservation of the elements contributing to
the algorithmic information content of the original string thus the most important
(computable) features (of which statistical regularities are a subset). Here applied
to a string (step 0) after
removal of 10 digits.
105. Random data is of
different nature
Now consider S to be a binary file of the same size but consisting of, say,
random data:
01101100100011001101010001110110001100011010011100010000110
So pS = Print(S) is the shortest possible generating program, then, in a
random object:
All perturbations are neutral!
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming
Systems, bioaRXiv DOI: https://doi.org/10.1101/185637
106. Identifying random objects by
the effect of perturbations
01101100100011001101010001110110001100011010011100010000110
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S.
Elias, A. Schmidt, G. Ball, J. Tegnér, An
Algorithmic Information Calculus for Causal
Discovery and Reprogramming
Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
107. The Principle of Algorithmic
Information Dynamics
C(rule 30)
C(rule 30 after 1 step) = C(rule 30 after 2 steps) + log(2) = …
C(rule 30 after 100 steps) + log(100) … = C(rule 30 after 1000 steps) + log(1000)
In a deterministic isolated dynamical system algorithmic complexity never
changes if it did, then it is neither deterministic or isolated and we can
identify those elements
108. Networks as programs
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
109. Information Ranking and
Information Spectra
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
110. Information Signatures
σ(G)
H Zenil, N.A. Kiani, F. Marabita, Y.
Deng, S. Elias, A. Schmidt, G. Ball,
J. Tegnér,
An Algorithmic Information Calculus
for Causal Discovery and
Reprogramming Systems, bioaRXiv
DOI: https://doi.org/10.1101/185637
111. H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
112. Moving Networks
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
114. AID Application to
Dynamical Systems
Original
candidate
model set M
with individual
models mi in M
After negative
perturbation:
candidate
models M’
grow in length (in
average):
Σi=1
|M’||m’i|>Σi=1
|M||mi|
Positive
perturbation:
candidate
models
decrease in length
Neutral
perturbation
|M| is the
cardinality of M
and |mi| the size
of model m in M.
115. Reconstructing space-time
dynamics of ECAs
original
reconstructed
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
116. Reconstructing time in space-time
dynamics
We are generalizing these tools to continuous dynamical systems such as strange attractors
original
reconstructed
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
117. Reconstructing space-phase dynamics of
ECA (more observations)
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
118. Sensitivity to perturbations at
difference time steps
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér,
An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems, bioaRXiv DOI:
https://doi.org/10.1101/185637
121. Evidence of pervasive
Reprogrammability
Reprogrammability of ECA and
CA
J. Riedel, H. Zenil
Cross-boundary Behavioural Reprogrammability Reveals Evidence of Pervasive Universality
International Journal of Unconventional Computing, vol 13:14-15 pp. 309-357
122. J. Riedel, H. Zenil
Cross-boundary Behavioural Reprogrammability Reveals Evidence of Pervasive Universality
International Journal of Unconventional Computing, vol 13:14-15 pp. 309-357
Reprogrammability Networks
Evidence of pervasive
Turing universality
123. J. Riedel and H. Zenil
Rule Primality, Minimal Generating Sets and Turing-Universality in the Causal Decomposition of Elementary Cellular
Automata
Journal of Cellular Automata, vol. 13, pp. 479–497, 2018
New Universal
Cellular Automata
125. An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems
H Zenil, N.A. Kiani, F. Marabita, Y. Deng, S. Elias, A. Schmidt, G. Ball, J. Tegnér
doi: https://doi.org/10.1101/185637
Algorithmic space Dynamical phase space
Algorithmic/Dynamic
Landscape Relationship