Linear regression is a machine learning algorithm in which a best scalar response is established according to the variables. This scalar has the least error and can be used for prediction tasks.
Linear regression works by finding the best values for the variables for equation y = w0 + w1*x, where w0, and w1 are variables. This can be extended to any number of features.
For example: y = w0 + w1*x1 + w2*x2 + … + wn*xn. The best values are found using an algorithm known as gradient descend. But in this post, we will covariance, variance, and mean to find the values for the variables.
We will write the program for the linear regression implementation. We will use python to write this program and we will not use any libraries.
Input
We have the dataset given below. It consists of 2D points.
data = [
[0,1],
[1,2],
[2,4],
[3,6],
[4,7],
[5,8],
[6,9],
[7,9],
[8,10],
[9,12]
]
x = [i[0] for i in data]
y = [i[1] for i in data]
import matplotlib.pyplot as plt
plt.scatter(x,y)
plt.show()
Mean, covariance, and variance function
Now, we will define functions to calculate mean, covariance and variance.
import math def mean(n): return sum(n) / float(len(n)) def covariance(x, mx, y, my): covar = 0.0 for i in range(len(x)): covar += (x[i] - mx) * (y[i] - my) return covar def variance(values, mean): return sum([(x-mean)**2 for x in values])
Algorithm
Now, we will find the variables using the following code:
x_mean, y_mean = mean(x), mean(y) w1 = covariance(x, x_mean, y, y_mean) / variance(x, x_mean) w0 = y_mean - w1 * x_mean print(w0) print(w1)
1.6181818181818173 1.1515151515151516
Plotting the line
After applying the algorithm, we will plot the predicted line.
import numpy as np x = np.array(x) plt.scatter(x,y) plt.plot(x, w0 + w1*x, linestyle='solid') plt.show()
Calculating error
We will now find the RMSE error for the predicted line.
def rmse(actual, predicted): sum_error = 0.0 for i in range(len(actual)): prediction_error = predicted[i] - actual[i] sum_error += (prediction_error ** 2) mean_error = sum_error / float(len(actual)) return (mean_error)**0.5 yPred = [w0 + q*w1 for q in x] print(rmse(y,yPred))
0.6485414871895709
Predicting the input
Now, we can predict the y value for any input x.
print("Enter the x to classify")
test = [int(i) for i in input().split()]
yNew = w0 + w1 * test[0]
print(yNew)
Enter the x to classify 5 7.375757575757575
That’s it. This is the complete implementation of linear regression.
Complete code
data = [
[0,1],
[1,2],
[2,4],
[3,6],
[4,7],
[5,8],
[6,9],
[7,9],
[8,10],
[9,12]
]
x = [i[0] for i in data]
y = [i[1] for i in data]
import matplotlib.pyplot as plt
plt.scatter(x,y)
plt.show()
import math
def mean(n):
return sum(n) / float(len(n))
def covariance(x, mx, y, my):
covar = 0.0
for i in range(len(x)):
covar += (x[i] - mx) * (y[i] - my)
return covar
def variance(values, mean):
return sum([(x-mean)**2 for x in values])
x_mean, y_mean = mean(x), mean(y)
w1 = covariance(x, x_mean, y, y_mean) / variance(x, x_mean)
w0 = y_mean - w1 * x_mean
print(w0)
print(w1)
import numpy as np
x = np.array(x)
plt.scatter(x,y)
plt.plot(x, w0 + w1*x, linestyle='solid')
plt.show()
def rmse(actual, predicted):
sum_error = 0.0
for i in range(len(actual)):
prediction_error = predicted[i] - actual[i]
sum_error += (prediction_error ** 2)
mean_error = sum_error / float(len(actual))
return (mean_error)**0.5
yPred = [w0 + q*w1 for q in x]
print(rmse(y,yPred))
print("Enter the x to classify")
test = [int(i) for i in input().split()]
yNew = w0 + w1 * test[0]
print(yNew)
Other Machine Learning algorithms:
- Naive Bayes Classification
- K Nearest Neighbors
- Linear Regression
- K Means Clustering
- Apriori Algorithm
- Principal Component Analysis (PCA)
Let us know in the comments if you are having any questions regarding this machine learning algorithm.
And if you found this post helpful, then please help us by sharing this post with your friends. Thank You
