A Transfer Learning based approach for detecting COVID-19 with Chest Radiography Images

 A Transfer Learning-based approach for detecting COVID-19 with Chest Radiography Images

Environment setup/Project Requirements :

1. Python
2. Tensorflow, Matplotlib, Scikit-Learn libraries need to be installed.
3. Jupyter Notebook Environment 

Note: You can implement this project in Google-Colaboratory. I don't recommend Atom/Pycharm environment because it would be hard to visualize the data.

Workflow:

1. Importing Data (training images)
2. Dividing the data set paths
3. Append the images and labels into two empty lists
4. Convert images into RGB format
5. Apply LabelBinarizer
6. Dividing training and testing data
7. Loading the transfer learning model(VGG-16)
8. Visualizing the sample data  
9. Compiling and training the model.
10. Visualizing the model predictions 
11. Confusion matrix
12. Testing accuracy.

Step 1: Importing Data  

Depending on the workflow of the project, we need to import the required libraries and the data required for the project.

dataset = "...........input/Data"

Step 2: Dividing the data set paths 
The data set contains two folders along with the label name s for each folder, So we need to divide the path for each folder in order to loop through each folder.

for iPath in iPaths:
    label = iPath.split(os.path.sep)[-2]   
    image = cv2.imread(iPath)

Step 3: Append the images 

Since we had a common label for the entire folder, but we need a label for each and every image. So we need to loop through the images and append them into the empty list and labels into another empty list. 

data = []
labels = []
for iPath in iPaths:
    label = iPath.split(os.path.sep)[-2]
    image = cv2.imread(iPath)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) 
    image = cv2.resize(image, (224, 224))
    data.append(image)
    labels.append(label)

Step 4: Convert images into RGB format

The transfer learning model is a pre-trained model that was trained on millions of RGB images. So transfer learning model can be used only for RGB images. So we need to convert each and every image into RGB format(Red, Green, Blue pixels)

    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

Step 5: Apply Label Binarizer

Since Machine Learning/Deep Learning models cannot understand string formats, we need to label them with numbers. Usually, one-Hot encoding is used to label with numbers, since this is just a binary classification problem we can use LabelBinarizer.

LB = LabelBinarizer() 
labels = LB.fit_transform(labels)
labels = to_categorical(labels);

print(labels)

Step 6: Dividing the training and testing data.

In order to prevent overfitting and to test the model, we shouldn't use the data that is used for training. So we will split the data into training and testing data.

(X_train, X_test, Y_train, Y_test) = train_test_split(data, labels, test_size=0.20, 

                                                   stratify=labels, random_state=42)

Step 7: Loading transfer learning model

Usually, we need to build our own architecture of neural network since we had a limited amount of data, we are using a transfer learning model which basically means the transferring of weights.

Since it was trained to classify millions of classes but we need to classify only two classes we need to modify the output layer. 

bModel = VGG16(weights="imagenet", include_top=False,input_tensor=Input(shape=(224, 224, 3))) 
hModel = bModel.output
hModel = AveragePooling2D(pool_size=(4, 4))(hModel)
hModel = Flatten(name="flatten")(hModel)
hModel = Dense(64, activation="relu")(hModel)
hModel = Dropout(0.5)(hModel)
hModel = Dense(2, activation="softmax")(hModel)

Step 8: Visualizing the sample data 

This is not an important step but in order to visualize the encodings that are applied to the image, we visualize the data with its encodings.

W_grid = 4
L_grid = 4
fig, axes = plt.subplots(L_grid, W_grid, figsize = (25, 25))
axes = axes.ravel()
n_training = len(X_train)
for i in np.arange(0, L_grid * W_grid):
    index = np.random.randint(0, n_training)
    axes[i].imshow(X_train[index])
    axes[i].set_title(Y_train[index])
    axes[i].axis('off')

Step 9: Compiling and training the model

After making sure that data was encoded correctly, we start compiling the model and training the neural network. 

model.compile(loss="binary_crossentropy", optimizer=opt,metrics=["accuracy"])
R = model.fit_generator(
    trainAug.flow(X_train, Y_train, batch_size=BS),
    steps_per_epoch=len(X_train) // BS,
    validation_data=(X_test, Y_test),
    validation_steps=len(X_test) // BS,
    epochs=EPOCHS)

Step 10: Visualizing the model predictions 

After training the model, in order to test the model training, we test the model on the test data set and visualize its predictions compared to original labels.

y_pred = model.predict(X_test, batch_size=BS)
for i in np.arange(0,L*W):
    axes[i].imshow(X_test[i])
    axes[i].set_title('Prediction = {}\n True = {}'.format(y_pred.argmax(axis=1)[i], Y_test.argmax(axis=1)[i]))
    axes[i].axis('off')

Step 11:Confusion matrix

In order to see how the model performed on the entire data set, we use the confusion matrix.

cm = confusion_matrix(Y_test.argmax(axis=1), y_pred

Step 12: Testing accuracy

After completing the training and testing of the model, we will save the weights of the model in .h5 format. When it comes to real-time circumstances, It is hard to run the model every time so we will load the previously saved weights and try to predict the new data(images)

model = tf.keras.models.load_model('Covid_model.h5')
img = image.load_img('......../Data/Covid/01E392EE-69F9-4E33-BFCE-E5C968654078.jpeg', target_size=(224, 224))
imgplot = plt.imshow(img)
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
img_data = preprocess_input(x)
classes = model.predict(img_data)
New_pred = np.argmax(classes, axis=1)
if New_pred==[1]:
    print('Prediction: Normal')
else:
    print('Prediction: Corona')

You can access the complete code in my GitHub by clicking here

Note: Thanks to each and every opensource contributor

Expected Viva:

1)What is transfer learning?

A) Transfer learning is basically a method of transferring weights of the model which was trained with some other data like Image-Net data.

2)Which transfer learning model is used in the above project?

A) VGG-16 which was proposed in 2015 was used. basically, we can use some other pre-trained/transfer learning models. I prefer VGG_16 because the main aim of VGG-16 is "Keep deep and Keep Simple"

3)What is the accuracy you got?

A)Around 99%.

4)Why did you use the transfer learning model?

A)Neural networks need a lot of data in order to train and give better accuracy. In this case, we have a very little amount of open-sourced data. So in order to achieve better accuracy with fewer amount of data, the network should have optimized weights. so I choose the transfer learning method.

5)Transfer learning models are made by which neural network (or) Which type of neural network you used in the project?

A)Convolutional neural networks.

6)What are the measures you took to prevent overfitting?

A)I have used "Dropout" techniques which drops some of the neurons in a layer and I have used the data augmentation technique(Step 6).

made with 💓by Sai Dheeraj