A single board computer such as the Raspberry Pi or RBPi can work wonders on a remote controlled car. Running Python on the RBPi allows it to handle three tasks a remote controlled car needs most – self-driving on a track, detection of sign and traffic lights and avoiding front collisions. The RC car has three subsystems – input units consisting of a camera and ultrasonic sensors, a processing unit and a control unit.
The processing unit on the RC car communicates with the RBPi to handle several tasks. These include receiving data from the RBPi, training, and predicting the neural network, detecting objects, measuring distances, and sending instructions to the Arduino through the USB connection.
The computer also runs a multithread TCP server program for receiving streamed image frames and ultrasonic data from the RBPi. The computer converts the image frames into gray scale and decodes them into numpy arrays.
To make object recognition and steering simple and fast, the RC car uses a neural network. The advantage is once the network is trained, it can work with only the trained parameters, making predictions very fast. The output layer of the network has four nodes corresponding to the steering control instructions – forward, reverse, left, and right. The input layer has over 38,000 nodes and uses only the lower half of the input images for training and prediction.
Although the project uses the shape-based approach for object detection, it only focuses on detecting the stop sign and traffic lights. Detection and training was both using OpenCV using both positive and negative samples. Positive samples are images that contain the desired object while negative samples are random images without the desired object.
The controller on the RC car needs four low-going signals corresponding to the forward, reverse, left, and right actions. Four pins on the Arduino provide these signals simulating button-press actions that drive the RC car.
The ultrasonic sensor measures the distance of an obstacle in front of the RC car. This includes measuring proper sensing angle and other surface conditions. Other measurements from the Pi camera allow the RC car to stop at the correct distance from the object.
The monocular vision approach of the RC car makes it difficult to get accurate distance measurements. In turn, other factors also influence the distance measurement, which includes errors in the actual measurement, variations in detecting the bounding box of the object, and nonlinear relationship between distance and camera coordinates. The error increases when camera distances are great and the camera coordinates are changing rapidly.
The traffic light recognition process uses image processing for detecting red and green lights. First part of the training involves detecting the traffic light by decoding its bounding box. Next, Gaussian blur reduces the image noise to find the brightest point within the bounding box. Finally, red or green state determination within the brightest spot detects the actual state of the traffic light.
The project uses an RBPi Model B+, a Pi camera and an ultrasonic sensor, HC-SR04. The RBPi streams ultrasonic sensor and color video data via its local Wi-Fi connection. It scales the video down to QVGA resolution to achieve low latency.
