# Naive Bayes Classification Program in Python from Scratch

Naive Bayes classifiers are a family of “probabilistic classifiers” based on Bayes’ theorem with strong independence between the features. They are among the simplest Bayesian network models and are capable of achieving high accuracy levels.

Bayes theorem states mathematically as:
P(A|B) = ( P(B|A) * P(A) )/ P(B)
where A and B are events and P(B) != 0.
P(A|B) is a conditional probability: the probability of event A occurring given that B is true.
P(B|A) is also a conditional probability: the probability of event B occurring given that A is true.
P(A) and P(B) are the probabilities of observing A and B respectively without any given conditions.
A and B must be different events.

In this post, we will write the program for Naive Bayes Classification. We will use python to write this program and we will not use any libraries.

### Input

We have the dataset given below. It consists of weather information of the past few days and also whether golf can be played on that day or not.

### Converting to list

Now, using the above dataset, we have to predict whether golf can be played on the given weather conditions or not.

For this we will first create convert our dataset into list.

```Assigning numbers to the data

outlook

rainy = 0, overcast = 1, sunny = 2

temperature

hot = 0, mild = 1, cool = 2

humidity

normal = 0, high = 1

windy

false = 0, true = 1

play golf

no = 0, yes = 1```
```#dataset converted to integers using above notations
dataset = [
[0,0,1,0,0],
[0,0,1,1,0],
[1,0,1,0,1],
[2,1,1,0,1],
[2,2,0,0,1],
[2,2,0,1,0],
[1,2,0,1,1],
[0,1,1,0,0],
[0,2,0,0,1],
[2,1,0,0,1],
[0,1,0,1,1],
[1,1,1,1,1],
[1,0,0,0,1],
[2,1,1,1,0]
]```

### Separating the data

Now we have to separate the data according to the Play Golf column (which is ‘y’ since this is what we have to predict). For this, we will create a dictionary and append the list into it.

```mp = dict()
for i in range(len(dataset)):
row = dataset[i]
y = row[-1]
if (y not in mp):
mp[y] = list()
mp[y].append(row)
for label in mp:
print(label)
for row in mp[label]:
print(row)```
```0
[0, 0, 1, 0, 0]
[0, 0, 1, 1, 0]
[2, 2, 0, 1, 0]
[0, 1, 1, 0, 0]
[2, 1, 1, 1, 0]
1
[1, 0, 1, 0, 1]
[2, 1, 1, 0, 1]
[2, 2, 0, 0, 1]
[1, 2, 0, 1, 1]
[0, 2, 0, 0, 1]
[2, 1, 0, 0, 1]
[0, 1, 0, 1, 1]
[1, 1, 1, 1, 1]
[1, 0, 0, 0, 1]```

### Algorithm

Let’s first define the test input:

`test = [2,1,0,1]`

After this, we will apply the naive bayes algorithm:

```probYes = 1

count = 0
total = 0
for row in dataset:
if(row[-1] == 1):
count+=1
total+=1
print("Total yes: "+str(count)+" / "+str(total))
probYes *= count/total
for i in range(len(test)):
count = 0
total = 0
for row in mp:
if(test[i] == row[i]):
count += 1
total += 1
print('for feature '+str(i+1))
print(str(count)+" / "+str(total))
probYes *= count/total

probNo = 1
count = 0
total = 0
for row in dataset:
if(row[-1] == 0):
count+=1
total+=1
probNo *= count/total
print("Total no: "+str(count)+" / "+str(total))
for i in range(len(test)):
count = 0
total = 0
for row in mp:
if(test[i] == row[i]):
count += 1
total += 1
print('for feature '+str(i+1))
print(str(count)+" / "+str(total))
probNo *= count/total```
```Total yes: 9 / 14
for feature 1
3 / 9
for feature 2
4 / 9
for feature 3
6 / 9
for feature 4
3 / 9
Total no: 5 / 14
for feature 1
2 / 5
for feature 2
2 / 5
for feature 3
1 / 5
for feature 4
3 / 5```

Let’s print the probability of playing golf and not playing golf:

```print(probYes)
print(probNo)```
```0.021164021164021163
0.006857142857142859```

### Calculating the probability of playing golf

To calculate the probability of playing golf, we will use the following code:

```prob = probYes/(probYes+probNo)
print("Probability of playing golf: "+str(prob*100)+"%")```
`Probability of playing golf: 75.5287009063444%`

That’s it. The probability of playing golf will be calculated according to the previous data using naive bayes algorithm.

### Complete code

```dataset = [
[0,0,1,0,0],
[0,0,1,1,0],
[1,0,1,0,1],
[2,1,1,0,1],
[2,2,0,0,1],
[2,2,0,1,0],
[1,2,0,1,1],
[0,1,1,0,0],
[0,2,0,0,1],
[2,1,0,0,1],
[0,1,0,1,1],
[1,1,1,1,1],
[1,0,0,0,1],
[2,1,1,1,0]
]
mp = dict()
for i in range(len(dataset)):
row = dataset[i]
y = row[-1]
if (y not in mp):
mp[y] = list()
mp[y].append(row)

for label in mp:
print(label)
for row in mp[label]:
print(row)

test = [2,1,0,1]

probYes = 1

count = 0
total = 0
for row in dataset:
if(row[-1] == 1):
count+=1
total+=1
print("Total yes: "+str(count)+" / "+str(total))
probYes *= count/total
for i in range(len(test)):
count = 0
total = 0
for row in mp:
if(test[i] == row[i]):
count += 1
total += 1
print('for feature '+str(i+1))
print(str(count)+" / "+str(total))
probYes *= count/total

probNo = 1
count = 0
total = 0
for row in dataset:
if(row[-1] == 0):
count+=1
total+=1
probNo *= count/total
print("Total no: "+str(count)+" / "+str(total))
for i in range(len(test)):
count = 0
total = 0
for row in mp:
if(test[i] == row[i]):
count += 1
total += 1
print('for feature '+str(i+1))
print(str(count)+" / "+str(total))
probNo *= count/total

print(probYes)
print(probNo)

prob = probYes/(probYes+probNo)
print("Probability of playing golf: "+str(prob*100)+"%")```

### Other Machine Learning algorithms:

Let us know in the comments if you are having any questions regarding this machine learning algorithm.