Novelty flux density: the quantity the whole thing is about

Strip the framework on this site down to one number and you get this: the rate at which a system produces genuinely new, compressible structure per unit of resource. I call it novelty flux density. Every claim in the programme is ultimately a claim about this rate. The plateau argument is a claim that it decays. Varney’s law is a claim about its integral. The case for building a multiverse is a claim that you can raise it by buying physics from somewhere else. And the original definition of it, I have to report, was not good.

What it has to measure

The word novelty is doing heavy work, so let me be exact. An observation is novel if it is not compressible by the structure you already have. A data stream full of regularities you have already absorbed carries no novelty, no matter how complicated it looks. A stream with a new regularity carries exactly as much novelty as the description length you save by absorbing the regularity. Novelty is a saving, and savings are measured against a current model, which makes novelty relative to the discoverer. That relativity is not a defect. It is the reason a plateau can exist at all.

Where the original definition went wrong

The first version of the quantity, in the early framework papers, was defined through agents and the complexity of rule-sets: how intricate a set of laws the agents in a simulated environment had to model. It looked quantitative and it was not, for three reasons.

First, it was uncomputable. Rule-set complexity as defined was a Kolmogorov-style quantity, and those cannot be measured, only bounded, and bounding them requires exactly the kind of discovery the quantity was supposed to quantify.

Second, it was observer-relative in a bad way. The same environment had different novelty depending on who was counting, with no way to say which count was the one the theory was about.

Third, and worst, it was circular. The plateau, the central empirical claim of the programme, was partially baked into the definition. Anything that made discovery look hard was classified as complexity, so of course discovery looked hard. A definition should not vote on the outcome of its own experiment.

The replacement

The fix, worked through in Paper 1 and the posts in the computational series of this archive, is to measure novelty as a description length under a two-part code: the cost of your current model, plus the cost of the data’s residuals under that model. A discovery is an event that lowers the total. The novelty of the event is the size of the drop.

This version is computable in practice, because description lengths are what compressors estimate. It is objective given a model class, and the model class is stated rather than hidden. And it is not circular, because the plateau becomes a theorem about the measure instead of a property of the definition. If discovery decays, it decays in spite of the definition, not because of it.

There is a price, and I would rather state it than bury it. Description length is relative to a model class, which means the plateau inherits a dependence on the class. When the truth sits inside your class, discovery decays to zero. When it sits outside, the rate still dies but floors at a level your class sets. That floor is a real result and a real limitation, and it gets its own post. The honest summary is that the replacement made the quantity measurable and shifted its arbitrariness from the definition to the model class, which is where arbitrariness can be seen and argued about.

Everything from here depends on this one rate. The walkthroughs that follow build the machinery it runs on, starting with the number at the bottom of the whole framework.