Object Detection & Identification with AMB82-Mini IoT AI Camera

---

Overview

In this tutorial, we will use Realtek AMB82-Mini IoT AI Camera board for Object Detection & Identification task. In this example, we will be using AMB82-Mini board to detect 80 different kinds of objects such as person, bicycle, car, laptop etc using on-board camera sensor (JXF37). Earlier we used ESP32-CAM & OpenCV for Object Detection & Identification, which you can check to learn more.

Check Here: ESP32 CAM Based Object Detection & Identification with OpenCVObject detection and identification using a camera is a key aspect of computer vision, a subfield of artificial intelligence (AI). The goal is to enable machines to interpret and understand the visual world as humans do, automatically detecting and identifying objects within an image or video stream. This process generally involves two steps: object detection and object identification (or object classification).

The Realtek AMB82-Mini IoT AI Camera Board uses the algorithm like YOLOv3, YOLOv4 and YOLOv7. The applications of object detection and identification are widespread and include object recognition, autonomous vehicles, video surveillance, and many more. Therefore AMB82-Mini board can be used for numerous applications and building various AI-based projects such as Embedded AI Vision Camera.

---

Bill of Materials

We need the following components for this tutorial. You can purchase the components from given links.

S.N.	Components	Quantity	Purchase Links
1	Realtek AMB82-Mini IoT AI Camera Board	1	Seeed Studio
2	Micro-USB Cable	1	Amazon | AliExpress

---

Realtek AMB82-Mini IoT AI Camera Board

The Realtek AMB82-Mini IoT AI Camera Board is an advanced development tool for AI network camera applications.

This compact and robust board is equipped with the integrated Realtek RTL8735BDM SoC, a powerful system-on-a-chip that merges an Arm® v8M MCU, DualBand Wi-Fi, Bluetooth BLE5, audio codec, an ISP, H264/H265 encoder, DDR2 128MB memory, and a neural network intelligent engine. This combination of features allows for efficient coordination of various applications and controls, making it a versatile tool for developers.

The AMB82-Mini has been optimized for battery-powered appliances, displaying a quick boot-up time and ultra-low power consumption, making it perfect for IoT applications. It also incorporates robust security measures, including TrustZone/security mechanism. The board is compatible with multiple programming platforms such as RTOS, IAR, GCC, and Arduino IDE, giving developers the flexibility to use their preferred development environment.

The AMB82-Mini offers more than just wireless network camera solutions. The on-board Neural Network engine accelerates the AI Model processing, that supports variety of AI Models. This includes object detection, audio recognition, and facial recognition technologies, expanding the potential use cases for this board in IoT and AI applications. With the AMB82-Mini, developers are given a versatile platform to create innovative, intelligent products.

One of the major components of the AMB82-Mini is the JXF37 5 megapixels camera module. This Full HD CMOS image sensor boasts a resolution of 1920×1080 pixels, providing crystal clear images with fine details. The wide field of view FOV (D,H,V) of 130 degrees, coupled with the high-quality optical lens, makes this camera ideal for wide area surveillance, photography, and videography.

To learn more about the AMB82-Mini IoT AI Camera and to learn how to get started with it, you may refer to our previous article: Getting Started with Realtek AMB82-Mini IoT AI Camera Board

---

Algorithm Used for Object Detection & Identification

For Object Detection & Identification, the Realtek AMB82-Mini development board supports object detection through neural network tasks such as YOLO (You Only Look Once) versions 3, 4, and 7 TINY.

Here’s a brief explanation of the algorithm: YOLO (You Only Look Once): YOLO is a state-of-the-art, real-time object detection system that looks at the whole image at test time, so its predictions are informed by the global context in the image.

It frames object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. It comes in various versions, with each new version improving on the previous one. YOLOv3, YOLOv4, and YOLOv7 TINY are mentioned in your context.

---

Object Detection & Identification with AMB82-Mini AI Camera

Now let us move to the project part and learn about the Object Detection & Identification with AMB82-Mini AI Camera. In this example we will utilize the JXF37 camera sensor on AMB82-Mini AI. This advanced board is capable of identifying a broad range of 80 distinct objects, spanning from people and bicycles to cars and laptops, among others.

---

Arduino IDE Setup & Example Code

To being with this example, you need to install Ameba Boards to the Arduino IDE. If you haven’t installed and configured the Arduino IDE, you may follow the Getting Started Tutorial to setup the Arduino IDE completely.

The following examples show how Object Detection and On-Screen Display are used. The main difference between the two examples is where the Object Detection results are processed. “ObjectDetectionCallback” uses a callback function, while “ObjectDetectionLoop” deals with the results in the loop function.

1. ObjectDetectionLoop
2. ObjectDetectionCallback

Open one of the Object Detection examples in “File” -> “Example” -> “AmebaNN” -> “ObjectDetectionLoop”.

Here is the code part of this project:

---

ObjectDetectionLoop.ino

---

ObjectClassList.h

We need to make some modifications to the code before uploading the code. From the following lines make changes to the WiFi SSID and Password.

Select Neural Network (NN) tasks and models using modelSelect() function.

This function takes 4 arguments: Neural Network task, Object Detection model, Face Detection model, and Face Recognition model.

Replace with “NA_MODEL” if they are not necessary for your selected Neural Network task. Note that it is mandatory to call modelSelect() function before calling the begin() function.

Valid Neural Network task: OBJECT_DETECTION, FACE_DETECTION, FACE_RECOGNITION

Valid Object Detection model:
YOLOv3 model: DEFAULT_YOLOV3TINY, CUSTOMIZED_YOLOV3TINY
YOLOv4 model: DEFAULT_YOLOV4TINY, CUSTOMIZED_YOLOV4TINY
YOLOv7 model: DEFAULT_YOLOV7TINY, CUSTOMIZED_YOLOV7TINY

Now Compile the code after compilation you can upload the code to the AMB82-Mini board. To do that press and hold the boot button and then Press and release the Reset button and finally release the boot button.

Once code get uploaded the Arduino IDE will show the following messages.

---

Object Detection Testing & Demo

Once code uploads, we can start testing Object Detection & Identification project using AMB82-Mini AI Camera.

Press the Reset button and wait for the AMB82-Mini board to connect to the WiFi network. The Serial Monitor will show the board’s IP address and network port number for RTSP.

The result of detected objects can be validated using VLC. You may download VLC media player from the link (here).
Upon the completion of the software installation, open VLC media player, and go to “Media” -> “Open Network Stream”.

Make sure your PC is connected to the same network as the AMB82-Mini board for streaming. Since RTSP is used as the streaming protocol, key in “rtsp://{IPaddress}:{port}” as the Network URL in VLC media player.

Replace {IPaddress} with the IP address of your AMB82-Mini board board, and {port} with the RTSP port shown in Serial Monitor (e.g., “rtsp://192.168.1.154:554”). The default RTSP port number is 554.

Following the setup, proceed by clicking the “Play” button to initiate RTSP streaming and view the outcome. The live video feed from the camera will be displayed in the VLC media player.

Upon successful object detection, a rectangular frame, known as a bounding box, will appear around the identified objects within the RTSP video stream. This box will be accompanied by the label of the detected object and a confidence score, reflecting the system’s certainty in its identification. This information assists in providing a clear visualization of the recognized objects.

Here are some samples of of images and object recognized inside the room.

Here are some samples of of images and object recognized outdoor.

This example is only able to identify the object that are trained (i.e. 80 objects).

The pre-trained model can recognize 80 different types of objects in total. The objects can be found in ObjectClassList.h. The index number for each object is fixed and should not be changed. To deactivate the detection of certain objects, set the filter value to 0. For example, set the filter value to 0 to exclude the detection of bicycles.

 

---

Conclusion

In conclusion, this tutorial has demonstrated the potent capabilities of the Realtek AMB82-Mini IoT AI Camera board in performing Object Detection and Identification tasks. With its ability to detect a diverse range of 80 different objects using the integrated JXF37 camera sensor, the AMB82-Mini board proves to be a powerful tool in the realm of computer vision, a prominent subfield of AI.

By utilizing advanced algorithms like YOLOv3, YOLOv4, YOLOv7 Tiny, the board ensures precise and reliable object identification. The widespread applications of object detection and identification, such as person recognition, autonomous vehicles, and video surveillance, make the AMB82-Mini board a versatile asset for developing an array of AI-centric projects. Thus, the Realtek AMB82-Mini IoT AI Camera board stands as a comprehensive and efficient solution for a myriad of AI and IoT applications.

---

Video Tutorial & Guide