This article will explain brief summary of linear regression and how to implement it using TensorFlow 2. If you are beginner, I would recommend to read following posts first:

- Setup Deep Learning environment: Tensorflow, Jupyter Notebook and VSCode

- Tensorflow 2: Build Your First Machine Learning Model with tf.keras

Regression

It is a process where a model learns to predict a continuous value output for a given input data. For example, we are given some data points of x and corresponding y and we need to learn the relationship between them that is called a hypothesis.

In case of Linear regression, the hypothesis is a straight line, i.e, the response value(y) as accurately as possible as a function of the feature or independent variable(x).

y = W*x + b

Where W is a vector called Weights and b is a scalar called Bias. The Weights and Bias are called the parameters of the model.

Simple Example

Let us consider a dataset where we have a value of response y for every feature x:

We have to predict value for x=10

Let's build Tensorflow 2 model for this:

import tensorflow as tf

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
y = [5, 8, 11, 14, 17, 20, 23, 26, 29, 32]

# Define layer
layer0 = tf.keras.layers.Dense(units=1, input_shape=[1])

model = tf.keras.Sequential([layer0])

# Compile model
model.compile(loss='mean_squared_error',
              optimizer=tf.keras.optimizers.Adam(1))

# Train the model
history = model.fit(x, y, epochs=100, verbose=False)

# Prediction
print('Prediction: {}'.format(model.predict([10])))

# Get weight and bias
weights = layer0.get_weights()
print('weight: {} bias: {}'.format(weights[0], weights[1]))

In above code, we defined x and y with provided data. then define keras Dense layer with single Neuron and compiled the model with mean squared error and Adam optimizer. After training, we define to get prediction for x=10 and display to check weight and bias.

Here is the output

Prediction: [[35.]]
weight: [[3.]] bias: [5.]

Which is correct. As it is very simple example for understanding purpose.

y = W*x + b
y = 3x + 5

for x= 0, y=5

x=1, y = 8

...

x=10, y= 3*10+5 = 35

Another Example

Let's prepare a linear dataset

import numpy as np 
import tensorflow as tf 
import matplotlib.pyplot as plt 

x_train = np.linspace(0, 50, 51) 
y_train = np.linspace(5, 155, 51)  

It is similar to previous one ((x,y)(0,5)(1,8)(2,11)...) except total number of data points. Let's add some noise:

y_train = y_train + np.random.normal(0,5,51)

Let's visualize the training data

plt.xlabel('x_train') 
plt.ylabel('y_train') 
plt.scatter(x_train, y_train)
plt.show()

Build the model:

layer0 = tf.keras.layers.Dense(units=1, input_shape=[1])

model = tf.keras.Sequential([layer0])

model.compile(loss='mean_squared_error',
              optimizer=tf.keras.optimizers.Adam(0.1))

history = model.fit(x_train,y_train, epochs=100, verbose=False)

The model is similar to previous except Adam arg value.

Check the training loss:

plt.xlabel('Epoch Number')
plt.ylabel("Loss Magnitude")
plt.plot(history.history['loss'])
plt.show()

As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near "perfect" towards the end.

Let's get prediction for x=100

print('Prediction: {}'.format(model.predict([100])))

Output:

Prediction: [[304.8801]]

Which is ~305. Let's plot the result

weights = layer0.get_weights()
weight = weights[0][0]
bias = weights[1]
print('weight: {} bias: {}'.format(weight, bias))
y_learned = x_train * weight + bias 
plt.scatter(x_train, y_train, label='Training Data')
plt.plot(x_train, y_learned, color='orangered', label='Fit Line')
plt.legend() 
plt.show()

Conclusion

In this article, we saw what is linear regression with a very simple example and learned how to build a linear regression model, evaluate it, and use it to predict new data values using TensorFlow 2.0 Keras API.

Enjoy TensorFlow!!