UDACITY SDCE Nanodegree: Term 1- Project 1 — Finding road Lanes!

Finding lanes is perhaps the important task for any self-driving car. Hence, not surprisingly the first project in the nanodegree revolved around just that. The goal was to write some code to implement a series of steps (pipeline) using the tools that we learned about during the lesson, and build upon them to identify and draw the lane lines on a few test images and some videos. Even though I had experience working with openCV before in various applications, this project allowed me to learn new things specifically regarding the Hough transform and how to fit a lines to polynomials. Below I will share some of the main concepts of my solution.

  1. Defining the ROI

The first step is to define a region-of-interest (ROI) wherein we would search for potential line markings. I selected a pyramid shaped ROI which I parametrized with respect to the image height and width in order to make it adaptable to different images and video where the camera would be placed at a different location. The trapezoid shaped ROI is defined by four vertices (v1,v2,v3,v4) which are calculated based on the percentage of image height and width that we need defined by variables (V_top,V_side_top,V_side_bottom,V_bottom).

Definition of ROI: A trapezoidal shape that is adjusted according to a percentage of the image width and height.

2. Isolating lane markings

The next step is to isolate the lane markings so that we can find the lines using Hough transform in step 3. An obvious choice is to use edge detection to identify the markings, however, in cases where additional edges are present due to shadows or reflections on the road then this approach will yield many false lines. Hence, in order to overcome this approach I fuse the edge detection results with color processing. The color processing is performed in two different ways. First, any yellow lanes are detected in the HSV space using the inRange functionin opencv, and then any white white lanes are detected in RGB color space. I found that this dual approach was more robust (less outliers) than trying to detected both yellow and white lines in the same colorspace.

3. Line Finding

This is the gist of the whole project, how to detect the lines from the lanes so that an autonomous vehicle can follow them! After tuning the Hough transform parameters the challenging part was discriminating between lines corresponding to the right and left lanes and removing any outliers.

I used the slope of each line to determine whether or not it should be considered. If the slope was pointing to a different direction from the one suitable for each lane side then that line was discarded. In addition any lines with zero slope where also discarded. This eliminated some lines and avoided the problem shown in the figure below which was caused by the ourliers. An additional step that I used is to save the lines from the previous frame and reuse the for the next so that the marked lanes are more stable.

The impact of averaging and discarding the line outliers

4. Extrapolating Lines

This can perhaps be considered the easy part after we have clearly detected the lane markings and removed all the outliers. The next step is to extend the lines to cover the road lanes. This can be achieved by using the OpenCV function fitline which given the line points can provide us with the line parameters after which we can draw the whole line to the desired extend. The resulting image looks something like the one below.

Final outcome of the pipeline

5. The Real Fun

As rewarding as it was to detect lanes in images the real fun is when we do this on a real video! Doing so can “break” the algorithm in the sense that when using a single image can make us overfit the parameters for that specific image. I had to play around with the parameters but eventually I think that my processing pipeline works well on the most “challenging” of the three videos we had to test for the assignment. I also decided to test it on a very different video taken from a police car vehicle with infrared camera. To my surprise it worked well without having to tune the parameters! Of course, the overall framework can be further improved as very curvy lanes can cause problems and also there is no strategy for when the car changes lanes, but as you can see in the second video the fact that previous lane lines are preserved allows for a smoother transition between lanes. I look forward to the next project as I have been waiting to get my hands dirty with some deep learning applications!

PhD in Computer Engineering, Self-Driving Car Engineering Nanodegree, Computer Vision, Visual Perception and Computing

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store