Hand Gesture Recognition using PAJ7620 Sensor & Arduino

Overview

In this project, we will make Hand Gesture Recognition System using PAJ7620 Sensor & Arduino Board. This PAJ7620 gesture sensor can recognize nine gestures, including moving Up, Down, Left, Right, Forward, Backward, Clockwise, anti-clockwise, and wave. We can use the gesture recognition system in medical applications or control lights, robots, HMI, games, using IR LED and optical CMOS array.

Gesture Recognition is a technology that uses sensors to read and interpret hand movements as commands. Gestures originate from any bodily motion or state but commonly originate from the face or hand. A gesture is an action that has to be seen by someone else and has to convey some piece of information. Gesture Recognition can be considered a way for a computer to understand human body language.

Earlier we learned about APDS9960 Gesture Recognition & RGB Color Sensor. You can check the article to get started with Gesture Sensors.

Bill of Materials

S.N.	Components Name	Quantity	Purchase Links
1	Arduino Nano Board	1	Amazon \| AliExpress
2	PAJ7620 Gesture Recognition Sensor	1	Amazon \| AliExpress
3	16x2 I2C LCD Display	1	Amazon \| AliExpress
4	Jumper Wires	5	Amazon \| AliExpress
5	Breadboard	1	Amazon \| AliExpress

PAJ7620 Gesture Recognition Sensor

This product is based on the PAJ7620U2 gesture recognition sensor, which recognizes gestures in 9 different directions, including up, down, left, right, front, back, clockwise, counterclockwise, and rocking.

The gesture recognition module uses the I2C interface and can be programmed and controlled by using the corresponding PAJ7620 Arduino library function. The signal returned by the gesture recognition module can be used as a control signal by the robot, thereby realizing the control of the robot.

The built-in recognition algorithm is quite intelligent, freeing both hands from blunt keys. Gesture recognition sensors can be used for non-contact control scenarios such as contactless mice, smart homes, car click device controls, robot interactions, and more.

PAJ7620U2 also offers built-in proximity detection for the purpose of sensing objects approaching or departing. The PAJ7620U2 is designed with great flexibility in a power-saving mechanism. The PAJ7620U2 is designed to operate from 2.8V to 3.3V over -40°C to +85°C and the pull-up voltage for the I2C bus and interrupt line is from 1.8V to 3.3V.

Refer to the datasheet for more information: PAJ7620 Datasheet

Features of PAJ7620

Nine gesture recognition (Up / Down / Left / Right / Push / Pull / CW / CCW / Wave)
Gesture speed is 60°/s to 600°/s in Normal Mode and 60°/s to 1200°/s in Gaming Mode
The supply voltage is 2.8V- 3.3V and I/O voltage is 1.8V~3.3V
Working Current: 3mA-10mA
Detection distance: 50-100mm
Ambient light immunity: < 100k Lux
Built-in proximity detection
Flexible power-saving scheme
I2C interface up to 400 kbit/s, Pull-up voltage from 1.8V to 3.3V
Ambient light noise cancellation

PAJ7620 Pinout

PIN	SYMBOL	Description
1	VCC	3.3V/5V
2	GND	Ground
3	SDA	I2C data pin
4	SCL	I2C clock pin
5	INT	External interrupt pin

PAJ7620 Applications

Smart Home
Offices and teaching
Human-robot interaction
Gesture toys
Domatosensory game equipment
PAD Phone
Tablet Personal Computer
Automobile Application

Interfacing PAJ7620 Gesture Recognition Sensor with Arduino

Let us interface the PAJ7620 Gesture Recognition Sensor with Arduino using the simple I2C interface.

The connection is fairly simple. Connect the VCC, GND, SDA & SCL pin of PAJ7620 to Arduino 3.3V, GND, A4 & A5 respectively.

In my case, I am using an Arduino Nano board. You can use any other Arduino-compatible microcontroller board.

Source Code/Program

There are multiple libraries for interfacing the PAJ7620 Sensor with Arduino. One of the libraries is written by Seeed Studio. Download the library from the following link.

Download: PAJ7620 Library

The following code is taken from the library example. Copy the following code and paste it to the Arduino IDE. Then upload the code.

100

101

102

103

104

105

106

107

108

109

#include <Wire.h>

#include "paj7620.h"

Notice: When you want to recognize the Forward/Backward gestures, your gestures' reaction time must less than GES_ENTRY_TIME(0.8s).

You also can adjust the reaction time according to the actual circumstance.

#define GES_REACTION_TIME 500 // You can adjust the reaction time according to the actual circumstance.

#define GES_ENTRY_TIME 800 // When you want to recognize the Forward/Backward gestures, your gestures' reaction time must less than GES_ENTRY_TIME(0.8s).

#define GES_QUIT_TIME 1000

void setup() {

uint8_t error = 0;

Serial.begin(9600);

Serial.println("\nPAJ7620U2 TEST DEMO: Recognize 9 gestures.");

error = paj7620Init(); // initialize Paj7620 registers

if (error) {

Serial.print("INIT ERROR,CODE:");

Serial.println(error);

} else {

Serial.println("INIT OK");

}

Serial.println("Please input your gestures:\n");

}

void loop() {

uint8_t data = 0, data1 = 0, error;

error = paj7620ReadReg(0x43, 1, &data); // Read Bank_0_Reg_0x43/0x44 for gesture result.

if (!error) {

switch (data) { // When different gestures be detected, the variable 'data' will be set to different values by paj7620ReadReg(0x43, 1, &data).

case GES_RIGHT_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG) {

Serial.println("Forward");

delay(GES_QUIT_TIME);

} else if (data == GES_BACKWARD_FLAG) {

Serial.println("Backward");

delay(GES_QUIT_TIME);

} else {

Serial.println("Right");

}

break;

case GES_LEFT_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG) {

Serial.println("Forward");

delay(GES_QUIT_TIME);

} else if (data == GES_BACKWARD_FLAG) {

Serial.println("Backward");

delay(GES_QUIT_TIME);

} else {

Serial.println("Left");

}

break;

case GES_UP_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG) {

Serial.println("Forward");

delay(GES_QUIT_TIME);

} else if (data == GES_BACKWARD_FLAG) {

Serial.println("Backward");

delay(GES_QUIT_TIME);

} else {

Serial.println("Up");

}

break;

case GES_DOWN_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG) {

Serial.println("Forward");

delay(GES_QUIT_TIME);

} else if (data == GES_BACKWARD_FLAG) {

Serial.println("Backward");

delay(GES_QUIT_TIME);

} else {

Serial.println("Down");

}

break;

case GES_FORWARD_FLAG:

Serial.println("Forward");

delay(GES_QUIT_TIME);

break;

case GES_BACKWARD_FLAG:

Serial.println("Backward");

delay(GES_QUIT_TIME);

break;

case GES_CLOCKWISE_FLAG:

Serial.println("Clockwise");

break;

case GES_COUNT_CLOCKWISE_FLAG:

Serial.println("anti-clockwise");

break;

default:

paj7620ReadReg(0x44, 1, &data1);

if (data1 == GES_WAVE_FLAG) {

Serial.println("wave");

}

break;

}

delay(100);

}

After uploading the code, open the Serial Monitor. Then swipe your hand in different directions and make different gestures. The gesture indication will be shown in the Serial Monitor. The tests are straightforward, just plug in and start moving your hands.

There are two ways of detecting:

Near, which is on 5-15 cm distance with a 60° angle
Far, on a 15-30 cm distance with a 30° angle

You can modify the detection speed for all and for the forward and backward alone. Also, keep the module on a flat surface, otherwise, the clockwise and anti-clockwise gestures won’t be detected.

#define GES_REACTION_TIME 500

#define GES_ENTRY_TIME 800

#define GES_QUIT_TIME 1000

Displaying Gestures on 16×2 LCD Display

In this part, I’ll use an LCD i²c screen to show the gesture detected. Using this method you can adapt it to whatever project you’re working on.

Connect the I2C LCD Display to the I2C Pin and power the LCD Display with 5V Pin of Arduino.

Source Code/Program

Copy the following code and upload it to the Arduino Board.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

#include <Wire.h>

#include "paj7620.h"

#include <LiquidCrystal_I2C.h>

LiquidCrystal_I2C lcd(0x27, 16, 2);

#define GES_REACTION_TIME 300 // You can adjust the reaction time according to the actual circumstance.

#define GES_ENTRY_TIME 500 // When you want to recognize the Forward/Backward gestures, your gestures' reaction time must less than GES_ENTRY_TIME(0.8s).

#define GES_QUIT_TIME 1000

void setup()

{

lcd.init();

lcd.backlight();

Serial.begin(9600);

Serial.println("\nPAJ7620U2 TEST DEMO: Recognize 9 gestures.");

lcd.setCursor(0, 0);

lcd.print(" Gesture ");

lcd.setCursor(0, 1);

lcd.print(" Recognition ");

delay(4000);

uint8_t error = 0;

error = paj7620Init(); // initialize Paj7620 registers

if (error)

{

Serial.print("INIT ERROR,CODE:");

Serial.println(error);

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("INIT ERROR,CODE:");

lcd.setCursor(0, 1);

lcd.print(error);

delay(3000);

}

else

{

Serial.println("INIT OK");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("INIT OK");

delay(3000);

}

Serial.println("Please input your gestures:\n");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Input Your");

lcd.setCursor(0, 1);

lcd.print("Gestures");

delay(3000);

}

void loop()

{

uint8_t data = 0, data1 = 0, error;

error = paj7620ReadReg(0x43, 1, &data); // Read Bank_0_Reg_0x43/0x44 for gesture result.

if (!error)

{

switch (data)

{ // When different gestures be detected, the variable 'data' will be set to different values by paj7620ReadReg(0x43, 1, &data).

case GES_RIGHT_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG)

{

Serial.println("Forward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Forward");

delay(GES_QUIT_TIME);

}

else if (data == GES_BACKWARD_FLAG)

{

Serial.println("Backward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Backward");

delay(GES_QUIT_TIME);

}

else

{

Serial.println("Right");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Right");

}

break;

case GES_LEFT_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG)

{

Serial.println("Forward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Forward");

delay(GES_QUIT_TIME);

}

else if (data == GES_BACKWARD_FLAG)

{

Serial.println("Backward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Backward");

delay(GES_QUIT_TIME);

}

else

{

Serial.println("Left");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Left");

}

break;

case GES_UP_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG)

{

Serial.println("Forward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Forward");

delay(GES_QUIT_TIME);

}

else if (data == GES_BACKWARD_FLAG)

{

Serial.println("Backward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Backward");

delay(GES_QUIT_TIME);

}

else

{

Serial.println("Up");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Up");

}

break;

case GES_DOWN_FLAG:

delay(GES_ENTRY_TIME);

paj7620ReadReg(0x43, 1, &data);

if (data == GES_FORWARD_FLAG)

{

Serial.println("Forward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Forward");

delay(GES_QUIT_TIME);

}

else if (data == GES_BACKWARD_FLAG)

{

Serial.println("Backward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Backward");

delay(GES_QUIT_TIME);

}

else

{

Serial.println("Down");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Down");

}

break;

case GES_FORWARD_FLAG:

Serial.println("Forward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Forward");

delay(GES_QUIT_TIME);

break;

case GES_BACKWARD_FLAG:

Serial.println("Backward");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Backward");

delay(GES_QUIT_TIME);

break;

case GES_CLOCKWISE_FLAG:

Serial.println("Clockwise");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Clockwise");

break;

case GES_COUNT_CLOCKWISE_FLAG:

Serial.println("anti-clockwise");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Anti-Clockwise");

break;

default:

paj7620ReadReg(0x44, 1, &data1);

if (data1 == GES_WAVE_FLAG)

{

Serial.println("wave");

lcd.clear();

lcd.setCursor(0, 0);

lcd.print("Wave");

}

break;

}

delay(100);

}

Testing Gesture Movements

After uploading the code the LCD will display the following message.

Then the LCD will ask you to input the Gesture.

Now you can swipe the hand in a different direction to show the Gesture movement on the LCD screen.

Similarly, move your hand in forward and backward directions for a forward-backward gesture.

Move your hand in the left and right direction for a left-right gesture.

Finally move your hand in an upward and downward direction for an up-down gesture.

You can rotate your hand in clockwise or counterclockwise for rotational movement detection.

Apart from the basic 9 Gesture Moment, you can add right-left, left-right, up-down, down-up, forward-backward, and backward-forward for other gesture detection.