UDACITY SDCE Nanodegree: Term 1- Project 3: Behavioral Cloning!
The goals / steps of this project were to use a simulator to collect data of good driving behavior; and then build, a convolution neural network in Keras that predicts steering angles from images. The objective was to mimic the human driving behavior so that the the model successfully drives around a track without leaving the road.
Simulator
For this project we used a simulator provided by UDACITY that provided two modes of operation and two different tracks to test the car driving behavior. Training Mode and Autonomous Mode. Training mode was used to drive the car like it’s a video game and record the camera images and respective steering angles with which to train the neural network. To test the trained neural network we can use the autonomous mode with the steering angles being controlled by the network.
Training Data Generation
UDACITY provided us with training data that we could use to train the network. Upon inspecting the data it could be observe that the majority (around 4000 images out of 8000) of angles correspond to 0 degrees.
I started out using the UDACITY dataset provided in the project resources. For all my experiments, I used only the center images provided within the csv file. I used the RGB information from the images, and scaled them between -1 and 1. Also I resize the input images to 80x40 to reduce training time and to cope with the limited memory on the GPU.
I followed different approaches with regard to generating data, however, the approach that worked for me is as follows. By observing the distribution of the data in the image shown below one can observe that almost half the images correspond to zero steering angles. This was also pointed out by my fellow peers in the slack discussions. The approach that I followed was to balance the data a bit was to increase the amount of non-zero images. I did this by flipping those images and changing the sign of their angles. Also, I discarded 2000 images corresponding to zero-angle images. The total number of training samples is 9711. Dropping less than this amount caused the car to understeer in tight corners. In addition, I generated the data for each epoch using the Keras image generator that performs random shifts and rotations on the images at each batch. I chose 64 images per batch due to compensate for GPU memory but still achieve fast training. I finally randomly shuffled the data set before training. I used this training data for training the model. The ideal number of epochs for me was 10 as when trying more epochs the loss did not decrease. I used an adam optimizer so that manually training the learning rate wasn’t necessary.
Convolutional Neural Network Architecture
The overall strategy for deriving a model architecture was to read the NVIDIA paper [1] to understand the thought process of generating such a network. It turns out that an architecture of convolutional layers followed by fully connected was used. The NVIDIA model has 5 convolutional layers and 2 fully connected (excluding the final neuron). I followed a similar architecture, however, I decided to start off with a simpler mode. of two convolutional and two fully connected since we have a smaller dataset and the test case is simpler. I tried this model with the UDACITY data as is without any augmentations. The first attempt was not successful since the car went straight on the first corner.
Then I focused on augmenting the data (random transformations) and decreased the model capacity to 3 convolutional layers and increase the neurons of the fully connected layers. At the end of the process, the vehicle is able to drive autonomously around the track without leaving the road. I also tried it on track 2 but the car crashed after a few turns. I believe that more training data are necessary to succeed in that track. The final model architecture consisted of 3 convolutional neural network and 2 fully connected layers as shown in the summary bellow and a visualization of the architecture:
Interesting through the process of training a neural network I realized that I resized all the images wrongly by swapping the columns and rows in the openCV resize function in both the model.py and drive.py files. However, to my surprise, the car still navigated the track flawlessly.
Successful Autonomous Driving
The convolutional neural network with the dataset described above was able to produce a model that successfully navigated the track! See the video below of the car doing multiple laps! This was by far the most fun project up to this point in the course. I look forward for more!
References
[1] End to End Learning for Self-Driving Cars, NVIDIA
