MNIST
author: Juma Shafara date: "2024-08-12" title: One Hidden Layer keywords: [Training Two Parameter, Mini-Batch Gradient Decent, Training Two Parameter Mini-Batch Gradient Decent] description: In this lab, you will see how many neurons it takes to classify noisy XOR data with one hidden layer neural network.
Neural Networks with One Hidden Layer
Objective
- How to classify handwritten digits using Neural Network.
Table of Contents
In this lab, you will use a single layer neural network to classify handwritten digits from the MNIST database.
- Neural Network Module and Training Function
- Make Some Data
- Define the Neural Network, Optimizer, and Train the Model
- Analyze Results
Estimated Time Needed: 25 min
Preparation
We'll need the following libraries
# Import the libraries we need for this lab
# Using the following line code to install the torchvision library
# !mamba install -y torchvision
import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dsets
import torch.nn.functional as F
import matplotlib.pylab as plt
import numpy as np
Use the following helper functions for plotting the loss:
# Define a function to plot accuracy and loss
def plot_accuracy_loss(training_results):
plt.subplot(2, 1, 1)
plt.plot(training_results['training_loss'], 'r')
plt.ylabel('loss')
plt.title('training loss iterations')
plt.subplot(2, 1, 2)
plt.plot(training_results['validation_accuracy'])
plt.ylabel('accuracy')
plt.xlabel('epochs')
plt.show()
Use the following function for printing the model parameters:
# Define a function to plot model parameters
def print_model_parameters(model):
count = 0
for ele in model.state_dict():
count += 1
if count % 2 != 0:
print ("The following are the parameters for the layer ", count // 2 + 1)
if ele.find("bias") != -1:
print("The size of bias: ", model.state_dict()[ele].size())
else:
print("The size of weights: ", model.state_dict()[ele].size())
Define the neural network module or class:
Neural Network Module and Training Function
Define the neural network module or class:
Define a function to train the model. In this case, the function returns a Python dictionary to store the training loss and accuracy on the validation data.
# Define a training function to train the model
def train(model, criterion, train_loader, validation_loader, optimizer, epochs=100):
i = 0
useful_stuff = {'training_loss': [],'validation_accuracy': []}
for epoch in range(epochs):
for i, (x, y) in enumerate(train_loader):
optimizer.zero_grad()
z = model(x.view(-1, 28 * 28))
loss = criterion(z, y)
loss.backward()
optimizer.step()
#loss for every iteration
useful_stuff['training_loss'].append(loss.data.item())
correct = 0
for x, y in validation_loader:
#validation
z = model(x.view(-1, 28 * 28))
_, label = torch.max(z, 1)
correct += (label == y).sum().item()
accuracy = 100 * (correct / len(validation_dataset))
useful_stuff['validation_accuracy'].append(accuracy)
return useful_stuff
Make Some Data
Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.
Load the testing dataset and convert it to a tensor by placing a transform object in the argument transform:
Create the criterion function:
Create the training-data loader and the validation-data loader objects:
Define the Neural Network, Optimizer, and Train the Model
Create the model with 100 neurons:
Print the model parameters:
Define the optimizer object with a learning rate of 0.01:
Train the model by using 100 epochs (this process takes time):
Analyze Results
Plot the training total loss or cost for every iteration and plot the training accuracy for every epoch:
Plot the first five misclassified samples:
Practice
Use nn.Sequential to build exactly the same model as you just built. Use the function plot_accuracy_loss to see the metrics. Also, try different epoch numbers.
Double-click here for the solution.