This post is to address the relation between the uncertainty on the state of a system with the information we have of it. Said in those terms is kind of obvious, the more information the less uncertainty. More information can take the form of knowing some other aspect of the system or being more precise information on previously acknowledge aspects.
In mathematics the way to deal with this kind of problem is probability and the way probabilities change when additional data is taken into account is the law of Bayes, hence Bayesian probabilities.
To illustrate how this works I’m going to go through an example. Lets imagine we have a tank of worm water, like a bath tub, and we want to know what temperature the water in the tank has. We could stick a thermometer in the tank and measure its temperature, assuming the temperature of the water in the tank is well mixed then that would yield the temperature of the tank with the uncertainty characteristic of our measuring device.
If we are using a mercury thermometer the uncertainty would be around . Measuring devices’ uncertainties are usually assumed to be normally distributed unless specific evidence is available. Usually the uncertainty level used is 95% or (two standard deviations). Meaning that when I say that the water in the tank is at , it means I’m 95% certain that the temperature is between and . The probability distribution of Fig. (1) shows the exact meaning, we are more confident the closer we get to the mean value.
To see how our knowledge of the system varies when further information is taken into account we are going to consider that this tank is not alone in the universe but it interacts with it.
Lets make the tank be in thermal equilibrium by adding a hot water inlet and a outlet, configured in such a way that the level of water is constant. Lets assume the tank is well insulated on all of its lateral walls and floor but open to the room temperature air on its top surface. For this system to be in equilibrium the mass flows of the inlet and outlet must coincide and the incoming heat through the inlet must be equal to the heat losses to the ambient.
This system is easily described with a simple equation relating the variables in play:
- is the mass flow.
- is the specific heat of the water, which we are going to take as a constant known with absolute certainty to be .
- is the temperature of the water comming into the tank.
- is the temperature of the tank.
- is the area of the surface water of the tank, which we also will assume is a perfectly known quantity
- is the convection film coeficient, which we’ll assume is .
- is the ambient temperature of the air in contact with the water surface.
Lets assume we are measuring the inlet temperature, the ambient temperature and the mass flow, as follows:
Having these measurements gives us a pretty good idea of the temperature of the tank even before explicitly measuring it. Fig. (2) shows this probability, it turns out we already know the temperature of the tank is . It is less precise that the direct measurement but the monitoring of the interactions with the outside does provide an estimate of the temperature of the system.
Now lets consider what happens when we measure the temperature and the measurement device shows like before . Since our prior knowledge of the state of the system is different, in the previous case we didn’t know anything of the system before hand, now we believe the temperature is . How this influences the ultimate state of our knowledge, after the measurement?
Bayes’ theorem provides the method that allows us to to update beliefs when new evidence arrives (more on that on wikipedia). Applying the theorem to the example at hand we find that the prior beliefs modify the end result bringing the mean a little towards the mean of our prior beliefs and reduces a little the uncertainty of the measurement. Figure (3) shows how our prior probabilities (blue) bring the measurement (green) slightly towards the right to, transforming our prior believes (blue) to to our later, more precise, believes (red). The final state of knowledge of the system is , the uncertainty has gone down from 0.2 to 0.187 because of our prior knowledge, moving the mean from 36.9 to 36.946.