Layer 24
Information, Measurement & Computation
Quantify uncertainty, noise, error correction, inference and the thermodynamic cost of irreversible information processing.
interactivemodel assumptions visibleestablished science separated from analogy
Story step: Signals and measurements are useful only when uncertainty can be represented, transmitted, corrected and acted on. Information makes that quantitative.
1 · Shannon entropy
A binary source is most uncertain when both outcomes are equally likely.
H₂(p) = −p log₂p − (1−p) log₂(1−p)
—bits / symbol
—surprisal of outcome 1
—surprisal of outcome 0
2 · Noise & channel capacity
For a binary symmetric channel with bit-flip probability ε, the maximum reliable information rate is reduced.
C = 1 − H₂(ε) bits/use
—capacity
—raw correctness
—channel entropy
3 · Redundancy & error correction
An odd-length repetition code can reduce independent bit errors by majority vote, at the cost of extra channel uses.
—majority-decoding failure
—code rate
—error reduction factor
4 · Measurement as Bayesian updating
A measurement changes what you should believe when you know the prior and the sensor's true- and false-positive rates.
P(H|+) = P(+|H)P(H) / P(+)
—posterior after +
—likelihood ratio
—overall + probability
5 · Landauer limit
Erasing one bit in a logically irreversible operation has a minimum thermodynamic cost at temperature T.
Emin = kBT ln 2
—minimum energy
—per bit
not a device forecastreal computers dissipate much more
Scientific boundary: Shannon entropy is an information measure, not automatically thermodynamic entropy in every context. Bayesian updating is a general inference rule, not the same as quantum measurement. Landauer’s principle applies to logically irreversible information erasure.