Shulou(Shulou.com)06/03 Report--

This article is about how python implements Bayesian inference. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

1. Preface

An example of python implementation of Bayesian inference, and shows the difference between the implementation of scalar operation and the implementation of matrix operation based on numpy.

two。 Problem description

The example of this question is taken from Ref1-Chapter1.

Problem description: suppose there is a machine that makes light bulbs. You want to know whether the machine is working properly or there is something wrong with it. To get the answer, you can test every light bulb, but there are so many light bulbs that each test may be unbearable in the actual production process. Using Bayesian inference, you can estimate whether the machine is working properly (in probabilistic way) based on a small number of samples (such as sampling results).

When building Bayesian inference, you first need two elements:

(1) Prior distribution

(2) likelihood rate

A priori distribution is our initial belief about the working state of machines. First of all, we determine the first random variable that describes the working state of the machine, which is marked as M. This random variable has two working states: {working, broken}, abbreviated as {w, br} (abbreviated to br to distinguish it from the following Bad abbreviation b). As an initial belief, we believe that the machine is good and can work properly. A priori distribution is defined as follows:

P (M=working) = 0.99

P (M=broken) = 0.01

This shows that our belief in the normal operation of the machine is very high, and there is a 99% chance that it will work properly.

The second random variable is L, which represents the working state of the light bulb produced by the machine. The light bulb can be good or bad, with two states: {good, bad}. The following is abbreviated as {gjingo b}. Note the difference between br and b.

We need to give the prior distribution of L based on the working state of the machine, that is, the conditional probability P (L | M), which represents the likelihood probability (likelihood) in the Bayesian formula.

Define this likelihood probability distribution (since M and L each have two states, they contain a total of four conditional probabilities) as follows:

P (L=Good | maw) = 0.99

P (L=Bad | MW) = 0.01,

P (L=Good | M=br) = 0.6

P (L=Bad | M=br) = 0.4

The above likelihood probability shows that when the machine is normal, we believe that there will be only one bad light bulb for every 100 light bulbs, and not all the light bulbs will be broken when the machine is abnormal, but 40% will be broken. For ease of implementation, it can be written in the following matrix form:

Now that we have a complete description of the Bayesian model, we can use it to do some magical estimation and prediction.

Our input is the sampling results of some light bulbs. Suppose we sampled ten light bulbs and the results are as follows:

{bad, good, good}

Let's take a look at how our belief (a posteriori probability) about the working state of the machine based on Bayesian inference changes.

3. Bayesian rule

Bayesian inference rules are expressed in the form of Bayesian formulas as follows:

The specific mapping to this question can be expressed as follows:

Shellfish

The priority of Bayesian inference is that it can be carried out online (online), that is, the observation data can come one by one, and each time a new observation data is received, a posterior probability based on Bayesian formula is updated, and the updated a posteriori probability is used as the prior probability of the next Bayesian inference. So the basic processing flow of online Bayesian inference is as follows:

4. Bayes engine: scalar implementation

First of all, we write a function in the way of scalar operation for bayes inference processing.

Prior stores a priori probability distribution in the form of vectors, prior [0] denotes P (M=working) and prior [1] denotes P (M=broken).

Likelihood stores likelihood probability distribution in the form of matrix. The first line represents P (L/M=working) and the second line represents P (L/M=broken).

In this case, when you enter

The formula for evidence (note that evidence depends on the input observation data) is:

Notice that when I write P (w), it actually means P (M), but it actually means that the rest, and so on. Depending on the context, these should not lead to confusion.

The first function has the following code:

Def bayes_scalar (prior, likelihood, data): "" Bayesian inference function example. Parameters-prior: float, 1mi D vector prior information, P (X). Likelihood: float 2murd matrix likelihood function, P (Y | X). Data: List of strings. Value: 'Good','Bad' Observed data samples sequence Returns-posterior: float P (XQuery Y), posterior sequence. " Posterior = np.zeros ((len (data) + 1Magne2)) posterior [0good': L_value:] = prior # Not used in computation, just for the later plotting for kmai L in enumerate (data): if L = = 'good': L_value = 0 else: L_value = 1 # print (L, L_value, likelihood [:, L_value]) evidence = likelihood [0 L_value] * prior [0] + likelihood [1Magnum Lemma value] * prior [1] LL0_prior_prod= likelihood [0MagneVal value] * prior [0] posterior [KLV 1] = LL0_prior_prod / evidence LL1_prior_prod= likelihood [1LV value] * prior [1] posterior [KLV 1] = LL1_prior_prod / evidence prior = posterior [KLV 1] :] # Using the calculated posterior at this step as the prior for the next step return posterior5. Bayes engine: vectorization

We note that the calculation of evidence can be expressed as the dot product of two vectors, as shown below.

This makes it very convenient to implement with numpy. In this example, there are only two values for each random variable. in complex cases, when each random variable has many values, the effective use of vector or matrix operation is an indispensable element of concise operation. The dot product of the above two vectors can be realized by numpy.dot ().

In addition, the product of likelihood and prior is calculated separately for the two states of M (note that we need to calculate the posterior for the two different states of M respectively), not using the dot product of vectors, but a kind of element-wise multiplication, which can be calculated with numpy.multiply (). So in the vectorization version, the Bayesian update processing is reduced to two statements, which is very elegant and concise compared to the above scalar version. (well, it may not show that obvious advantage in this simple example, but as the complexity of the problem increases, the advantage will become more and more obvious. )

As a result, we get the following functions for vectorization processing:

Def bayes_vector (prior, likelihood, data): "" Bayesian inference function example. Parameters-prior: float, 1mi D vector prior information, P (X). Likelihood: float 2murd matrix likelihood function, P (Y | X). Data: List of strings. Value: 'Good','Bad' Observed data samples sequence Returns-posterior: float P (XQuery Y), posterior sequence. " Posterior = np.zeros ((len (data) + 1Magne2)) posterior [0Magi:] = prior # Not used in computation, just for the later plotting for kmai L in enumerate (data): if L = 'good': L_value = 0 else: L_value = 1 # print (L, L_value, likelihood [:, L_value]) evidence = np.dot (likelihood [:, L_value] Prior [:]) posterior [Karmu1Magi:] = np.multiply (likelihood [:, L_value], prior) / evidence prior = posterior [Kang1Magi:] # Using the calculated posterior at this step as the prior for the next step return posterior6. test

Let's see how the posterior probability will change when we use the above functions to process our observed data.

Import numpy as npimport matplotlib.pyplot as plt prior = np.array, likelihood = np.array, data = ['bad','good','good','good','good','good','good','good'] posterior1 = bayes_scalar (prior,likelihood,data) posterior2 = bayes_vector (prior,likelihood,data) if np.allclose (posterior1) Posterior2): print ('posterior1 and posterior2 are idlers') Fig, ax = plt.subplots () ax.plot (posterior1 [:, 0]) ax.plot (posterior1 [: 1]) ax.grid () # fig.suptitle ('Poeterior curve vs observed data') ax.set_title (' Posterior curve vs observed data') plt.show ()

Run the above code to get a posteriori probability change as shown in the following figure (note that the first point is prior):

Of course, the above code also verifies that the two versions of the bayes function are completely equivalent.

Thank you for reading! This is the end of this article on "how to achieve Bayesian inference in python". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.