Uncertainty and Bayesian probability

bayesian_processThis post is to address the relation between the uncertainty on the state of a system with the information we have of it. Said in those terms is kind of obvious, the more information the less uncertainty. More information can take the form of knowing some other aspect of the system or being more precise information on previously acknowledge aspects.

In mathematics the way to deal with this kind of problem is probability and the way probabilities change when additional data is taken into account is the law of Bayes, hence Bayesian probabilities.

To illustrate how this works I’m going to go through an example. Lets imagine we have a tank of worm water, like a bath tub, and we want to know what temperature the water in the tank has. We could stick a thermometer in the tank and measure its temperature, assuming the temperature of the water in the tank is well mixed then that would yield the temperature of the tank with the uncertainty characteristic of our measuring device.

Fig. (1) The uncertainty distribution of a simple measurement of the water in the tank

If we are using a mercury thermometer the uncertainty would be around \pm0.2\textrm{ }\textdegree\textrm{C}. Measuring devices’ uncertainties are usually assumed to be normally distributed unless specific evidence is available. Usually the uncertainty level used is 95% or 2\sigma (two standard deviations). Meaning that when I say that the water in the tank is at 36.9\pm0.2\textrm{ }\textdegree\textrm{C}, it means I’m 95% certain that the temperature is between 36.7 and 37.1. The probability distribution of Fig. (1) shows the exact meaning, we are more confident the closer we get to the mean value.

To see how our knowledge of the system varies when further information is taken into account we are going to consider that this tank is not alone in the universe but it interacts with it.

Lets make the tank be in thermal equilibrium by adding a hot water inlet and a outlet, configured in such a way that the level of water is constant. Lets assume the tank is well insulated on all of its lateral walls and floor but open to the room temperature air on its top surface. For this system to be in equilibrium the mass flows of the inlet and outlet must coincide and the incoming heat through the inlet must be equal to the heat losses to the ambient.

This system is easily described with a simple equation relating the variables in play:

mc_p(T_{in}-T)=Ah(T-T_{amb})                                   (1)


  • m is the mass flow.
  • c_p is the specific heat of the water, which we are going to take as a constant known with absolute certainty to be c_p=4.187\quad\frac{\textrm{kJ}}{\textrm{kg}\textdegree\textrm{C}}   .
  • T_{in} is the temperature of the water comming into the tank.
  • T is the temperature of the tank.
  • A is the area of the surface water of the tank, which we also will assume is a perfectly known quantity T_{in}=0.64\quad\textrm{m}^2
  • h is the convection film coeficient, which we’ll assume is h=10 \quad\frac{\textrm{W}}{\textrm{m}^2\textdegree\textrm{C}}.
  • T_{amb} is the ambient temperature of the air in contact with the water surface.

Lets assume we are measuring the inlet temperature, the ambient temperature and the mass flow, as follows:

  1. T_{in}= 58 \pm 0.2\quad \textdegree\textrm{C}.
  2. T_{amb}= 17 \pm 0.2\quad \textdegree\textrm{C}
  3. m= 0.0015 \pm 3.7E-5\quad \frac{\textrm{kg}}{\textrm{s}}

Having these measurements gives us a pretty good idea of the temperature of the tank even before explicitly measuring it. Fig. (2) shows this probability, it turns out we already know the temperature of the tank is 37.257 \pm 0.52\textrm{ }\textdegree\textrm{C}. It is less precise that the direct measurement but the monitoring of the interactions with the outside does provide an estimate of the temperature of the system.

Fig. (2) Prior probability of the state of the tank, before measuring it explicitly.

Now lets consider what happens when we measure the temperature and the measurement device shows like before 36.9\pm0.2\textrm{ }\textdegree\textrm{C}. Since our prior knowledge of the state of the system is different, in the previous case we didn’t know anything of the system before hand, now we believe the temperature is 37.257 \pm 0.52\textrm{ }\textdegree\textrm{C}. How this influences the ultimate state of our knowledge, after the measurement?

Bayes’ theorem provides the method that allows us to to update beliefs when new evidence arrives (more on that on wikipedia). Applying the theorem to the example at hand we find that the prior beliefs modify the end result bringing the mean a little towards the mean of our prior beliefs and reduces a little the uncertainty of the measurement. Figure (3) shows how our prior probabilities (blue) bring the measurement (green) slightly towards the right to, transforming our prior believes (blue) to to our later, more precise, believes (red). The final state of knowledge of the system is 36.946 \pm 0.187\textrm{ }\textdegree\textrm{C}, the uncertainty has gone down from 0.2 to 0.187 because of our prior knowledge, moving the mean from 36.9 to 36.946.

Fig. (3) The Bayesian process. Our prior believes (blue) influence our final state of knowledge (red) after measuring the system (green). It doesn’t only modify the most likely value but also reduces the uncertainty of the measurement.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s