Video Streaming Car Robot using Raspberry Pi & Camera

Overview

In this guide, you will learn how to build your own Raspberry Pi Video Streaming Robot Car and create a First Person View (FPV) driving experience. For this, we will use a Robotic Car Kit called Picar-X Robot from Sunfounder.

The first step is to configure video streaming on the Raspberry Pi using the HAT & Raspberry Pi camera. The Robot is capable of Video Streaming with a Raspberry Pi Camera using OpenCV. The Robot can also control camera servos as Pan and Tilt functions as its head is movable. Apart from this you can connect everything together & drive the car in any direction.

The PiCar-X is an AI self-driving robot car that uses Raspberry Pi as its control center. The Robotic Kit is designed by SunFounder. The Kit comes with a Robot HAT for Raspberry Pi. The HAT integrates the motor driving, servo driving, and preset ADC, PWM, and digital pins to extend the functionality. The PiCar-X’s has a 2-axis camera module, ultrasonic module, and line tracking modules that can provide the functions of color/face/traffic signs detection, automatic obstacle avoidance, automatic line tracking, etc.

This project can be advanced to AI-based autonomous-driving cars and fully autonomous cars. The remote control enables valuable operational data to be obtained, thus laying the groundwork for gradually increasing autonomous driving performance in the future. Moreover, human assistance through remote driving can offer more flexibility and intelligence than a single artificial intelligence.

Bill of Materials

We need the following components along with the SunFounder Robot Car Kit to make a Video Streaming Car Robot using Raspberry Pi Camera. You can purchase all these components from Amazon links:

S.N.	Components	Quantity	Purchase Links
1	Raspberry Pi Ai Car Kit PiCar-X	1	Amazon \| SunFounder
2	Raspberry Pi 4	1	Amazon \| SunFounder
3	Samsung 18650 Battery	2	AliExpress
4	SD Card 16/32 GB	1	Amazon \| SunFounder
5	SD Card Adapter	1	Amazon \| AliExpress
6	18650 Battery Charger	1	Amazon \| AliExpress

Robot HAT cannot charge the battery, so you need to buy a battery charger at the same time.

Robot Assembly

The Sunfounder Picar-X Robot Kit comes with all the necessary components required for the Robot which include the following components. You need to buy a Raspberry Pi 4 Board and a pair of Samsung 18650 Batteries separately.

The most important part of this Kit is Robot HAT. With the Robot HAT board, the PiCar-X integrates left/right driving motors, servo motors for steering, and the camera’s pan/tilt functions, and pre-sets the Robot HAT’s ADC, PWM, and Digital I2C pins to allow for extensions to the standard functionality of the Raspberry Pi.

Both a speaker and a Bluetooth chip have been engineered into the Robot HAT for remote control of Text-to-Speech, sound effects, or even background music functionality.

To assemble the components together and make a perfect Robotic Car you can follow the Assembly Guide. The PDF Document has all the images and a pictorial view for easy help with assembly.

The Robotic Kit Part details, Assembly guide, HAT parts & functionality, calibration process, Robot Testing and Setting Up Rasbian OS has been explained in PiCar-X Getting Started guide.

Read the tutorial thoroughly to understand the Hardware & Software Setup Part.

Installing Python Modules & Libraries

The OS installation part and setup part can be followed on Raspberry Pi OS Setup. But for this robotic project part, it is recommended to install Raspberry Pi OS(Legacy).

You can either use the HDMI Screen or enable SSH and connect the Raspberry Pi to VNC Viewer using the local IP Address, username, and password. It is best to connect the Robot Wirelessly via a Remote Desktop as we need a remote control for it.

The Picar-X Robot should be turned ON and Raspberry Pi should be connected to the network. Now let’s install all the Python Modules and libraries.

Let’s first update the system using the following commands.

1 2	sudo apt update sudo apt upgrade

Then install the Python3-related packages.

1	sudo apt install git python3-pip python3-setuptools python3-smbus

Download & install the libraries for Robot-Hat.

cd /home/pi/

git clone https://github.com/sunfounder/robot-hat.git

cd robot-hat

sudo python3 setup.py install

Running ‘setup.py‘ will download some necessary components. Your download may have failed due to network issues. You may need to download it again at this point. See the following interface, type ‘Y‘, and press Enter.

Now, download and install the ;vilib‘ module.

cd /home/pi/

git clone https://github.com/sunfounder/vilib.git

cd vilib

sudo python3 install.py

Similarly download and install the ‘picar-x‘ module.

cd /home/pi/

git clone -b v2.0 https://github.com/sunfounder/picar-x.git

cd picar-x

sudo python3 setup.py install

This will download a large number of files, therefore it will take time.

Once the download is complete, run the ‘i2samp.sh‘ script to install the components required by the i2s amplifier, otherwise the picar-x will have no sound.

1 2	cd /home/pi/picar-x sudo bash i2samp.sh

Type ‘y‘ and press enter to continue running the script.

Type ‘y‘ and press enter to run ‘/dev/zero‘ in the background.

Type y and press enter to restart the Picar-X.

After restarting, the Raspberry Pi will connect to the Network and will play a sound.

If there is no sound after restarting, you may need to run the ‘i2samp.sh‘ script several times.

All the installation and setup part is complete here.

Python Code for Video Streaming Car Robot

Here is a complete Python Code for Video Streaming Car Robot using Raspberry Pi & Camera. But before that we need to fulfill some requirements for the fully functionality of this Robot:

This project requires access to the Raspberry Pi desktop to view the footage taken by the camera module.
You can connect a screen to the PiCar-X or refer to the tutorial Remote Desktop to access it with VNC or XRDP.

Once inside the Raspberry Pi desktop, open Terminal and type the following command to run it, or just open and run it with a Python editor.

1 2	cd /home/pi/picar-x/example sudo python3 video_car.py

The code is also located in the source code path ‘picar-x/example/video_car.py‘. You can directly open the code from this location. You can Modify/Reset/Copy/Run/Stop the code below.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

# #!/usr/bin/env python3

print('Please run under desktop environment (eg: vnc) to display the image window')

from utils import reset_mcu

reset_mcu()

from picarx import Picarx

from vilib import Vilib

from time import sleep, time, strftime, localtime

import readchar

manual = '''

Press key to call the function(non-case sensitive)：

O: speed up

P: speed down

W: forward

S: backward

A: turn left

D：turn right

F: stop

T: take photo

ESC / Ctrl+C: quit

'''

px = Picarx()

def take_photo():

_time = strftime('%Y-%m-%d-%H-%M-%S',localtime(time()))

name = 'photo_%s'%_time

path = "/home/pi/Pictures/picar-x/"

Vilib.take_photo(name, path)

print('\nphoto save as %s%s.jpg'%(path,name))

def move(operate:str, speed):

if operate == 'stop':

px.stop()

else:

if operate == 'forward':

px.set_dir_servo_angle(0)

px.forward(speed)

elif operate == 'backward':

px.set_dir_servo_angle(0)

px.backward(speed)

elif operate == 'turn left':

px.set_dir_servo_angle(-30)

px.forward(speed)

elif operate == 'turn right':

px.set_dir_servo_angle(30)

px.forward(speed)

def main():

speed = 0

status = 'stop'

Vilib.camera_start(vflip=False,hflip=False)

Vilib.display(local=True,web=True)

sleep(2) # wait for startup

print(manual)

while True:

print("\rstatus: %s , speed: %s "%(status, speed), end='', flush=True)

# readkey

key = readchar.readkey().lower()

# operation

if key in ('wsadfop'):

# throttle

if key == 'o':

if speed <=90:

speed += 10

elif key == 'p':

if speed >=10:

speed -= 10

if speed == 0:

status = 'stop'

# direction

elif key in ('wsad'):

if speed == 0:

speed = 10

if key == 'w':

# Speed limit when reversing,avoid instantaneous current too large

if status != 'forward' and speed > 60:

speed = 60

status = 'forward'

elif key == 'a':

status = 'turn left'

elif key == 's':

if status != 'backward' and speed > 60: # Speed limit when reversing

speed = 60

status = 'backward'

elif key == 'd':

status = 'turn right'

# stop

elif key == 'f':

status = 'stop'

# move

move(status, speed)

# take photo

elif key == 't':

take_photo()

# quit

elif key == readchar.key.CTRL_C or key in readchar.key.ESCAPE_SEQUENCES:

print('\nquit ...')

px.stop()

Vilib.camera_close()

break

sleep(0.1)

if __name__ == "__main__":

try:

main()

except Exception as e:

print("error:%s"%e)

finally:

px.stop()

Vilib.camera_close()

Testing Video Streaming Car Robot using Raspberry Pi & Camera

This program will provide a First Person View from the PiCar-X! Use the keyboards ‘WSAD‘ keys to control the direction of movement, and the ‘O‘ and ‘P‘ to adjust the speed.

Once the code is running, you can see what PiCar-X is shooting and control it by pressing the following keys.