ESP32 CAM is an ESP32 development board with a built-in camera. It is powerful, compact, and suitable for Internet of Things (IoT) projects. It can meet the market's demands for device connectivity, data transmission, and security. The ESP32 CAM offers the following advantages:
Powerful processing capabilities and rich features: ESP32 CAM integrates a high-performance ESP32 chip, providing powerful processing capabilities and various functional modules such as Wi-Fi, Bluetooth, and a camera. This enables easy device connectivity and data processing, meeting the requirements of IoT projects.
Stable wireless connection: ESP32 CAM has a built-in robust Wi-Fi module, offering a stable and reliable wireless connection. This facilitates reliable communication between devices and supports remote monitoring and control functionality.
Flexible device integration: ESP32 CAM has ample GPIO pins and communication interfaces, allowing for flexible integration with other sensors, actuators, and external devices. This simplifies the expansion and customization process of IoT projects.
Security assurance: ESP32 CAM supports secure data transmission and communication protocols such as SSL/TLS. Additionally, being open-source and having an active developer community ensures timely fixes and updates for security vulnerabilities, providing reliable security assurance.
Due to the advantages of the ESP32 CAM, it has a wide range of applications in IoT projects. It can be used in areas such as smart homes, industrial automation, smart agriculture, and smart cities. For example, leveraging the high-performance camera and image processing capabilities of the ESP32 CAM, intelligent video surveillance and facial recognition systems can be implemented.
In the following, we'll use the M5Stack TimerCam as an example and provide a guide to help you set up your own video surveillance project.
We will provide tutorials for four different uses: Web camera, scheduled wake-up capture, push and pull RTSP streaming. However, before getting started, it is necessary to set up the Arduino working environment.
To set up the Arduino environment, we can complete the basic software installation by following the first step in the Arduino IDE environment setup tutorial. And then install the "M5Stack board management" and the "TimerCam-Arduino library" in the Library Manager, or manually clone it from the GitHub link and install it into the "C:\Users\PC\Documents\Arduino\libraries" directory.
1. Set up your webcam
1). Using the code below, we will explain how to initialize the camera, capture a frame of image data, and enable the Webserver Demo for image preview.
// Include the necessary header files
#include "battery.h"
#include "esp_camera.h"
#include <WiFi.h>
#include "soc/soc.h"
#include "soc/rtc_cntl_reg.h"
#include "camera_pins.h"
//Configure the Wi-Fi information for the camera to connect to
const char *ssid = "******";
const char *password = "******";
void startCameraServer( );
void setup( ) {
Serial.begin(115200);
WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); // disable detector
bat_init();
// After starting the TimerCAM, set the power pin to a high level to maintain power supply.
bat_hold_output();
Serial.setDebugOutput(true);
Serial.println();
pinMode(2, OUTPUT);
digitalWrite(2, HIGH);
// Configuration of the camera includes settings for pins, image size, and compression format.
//Setting a large image size or high image quality may result in insufficient memory allocation, leading to initialization failure.
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.pixel_format = PIXFORMAT_JPEG;
config.frame_size = FRAMESIZE_UXGA;
config.jpeg_quality = 10;
config.fb_count = 2;
// Initialize the camera
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf("Camera init failed with error 0x%x", err);
return;
}
sensor_t *s = esp_camera_sensor_get();
// Initially flip the sensor vertically and adjust brightness and saturatio
s->set_vflip(s, 1); // flip it back
s->set_brightness(s, 1); // up the blightness just a bit
s->set_saturation(s, -2); // lower the saturation
// Set Frame Size
s->set_framesize(s, FRAMESIZE_QVGA);
Serial.printf("Connect to %s, %s\r\n", ssid, password);
// Connect WiFi
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
// If you don't have access to a Wi-Fi network to connect to, you can switch to using the following code to enable the camera to create its own Access Point (AP)
// WiFi.softAP(ssid, password);
// IPAddress IP = WiFi.softAPIP();
// Serial.print("AP IP address: ");
// Serial.println(IP);
// enable the Web server
startCameraServer();
}
void loop() {
// After startup, you can view the IP address assigned to the camera by serial communication. Here's the code that will continuously print the IP address for easy viewing
Serial.print("Camera Ready! Use 'http://");
Serial.print(WiFi.localIP());
Serial.println("' to connect");
delay(1000);
digitalWrite(2, HIGH);
delay(1000);
digitalWrite(2, LOW);
}
2). Modify the code with the WiFi information section, and compile and burn the program to the device. Then, you can use the serial monitor to view the IP address assigned to the camera.
WiFi connected
Camera Ready! Use 'http://192.168.31.11' to connect
For devices on the same network, you can access 192.168.31.11 through a web browser to access the control page, where can obtain real-time images and adjust image properties.
(Note: You need replace the IP address with the actual IP address printed on the serial monitor).
2. Scheduled wake-up capture
To achieve scheduled wake-up and capture using the RTC (Real-Time Clock) functionality for extended battery life, you can refer the code below. It captures an image during the period between camera startup and entering sleep mode. You can then transmit the image data to another location using a method such as HTTP POST for scheduled capture.
#include "battery.h"
#include "esp_camera.h"
#include <WiFi.h>
#include "soc/soc.h"
#include "soc/rtc_cntl_reg.h"
#include "led.h"
#include "camera_pins.h"
#include "bmm8563.h"
void setup() {
Serial.begin(115200);
WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); // disable detector
bat_init();
led_init(CAMERA_LED_GPIO);
bmm8563_init();
// Configuration of the camera includes settings for pins, image size, and compression format.
// Setting a large image size or high image quality may result in insufficient memory allocation, leading to initialization failure.
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sscb_sda = SIOD_GPIO_NUM;
config.pin_sscb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.pixel_format = PIXFORMAT_JPEG;
config.frame_size = FRAMESIZE_UXGA;
config.jpeg_quality = 10;
config.fb_count = 2;
// Initialize the camera
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf("Camera init failed with error 0x%x", err);
return;
}
sensor_t *s = esp_camera_sensor_get();
// Initially flip the sensor vertically and adjust brightness and saturatio
s->set_vflip(s, 1); // flip it back
s->set_brightness(s, 1); // up the blightness just a bit
s->set_saturation(s, -2); // lower the saturation
// Set Frame Size
s->set_framesize(s, FRAMESIZE_QVGA);
camera_fb_t *fb = NULL;
esp_err_t res = ESP_OK;
// Get a frame of image data
fb = esp_camera_fb_get();
if (!fb) {
Serial.println("Camera capture failed");
}
Serial.println("Camera capture OK");
// Acquired image data
uint8_t *data_buffer = fb->buf;
size_t data_len = fb->len;
//...................
// You can perform data processing in this section and send it to the server using the HTTP method.
//...................
// Set a 5-second wake-up interval.
bmm8563_setTimerIRQ(5);
delay(1000);
}
void loop( ) {
// Power off.
// The device enters sleep mode and wakes up after 5 seconds to start executing the program from the beginning. (This only applies when powered by a battery.)
bat_disable_output();
}
3. Push and pull RTSP streaming
RTSP, short for Real-Time Streaming Protocol, is a network transmission protocol used to control the real-time transmission of data in multimedia applications. It is commonly used in conjunction with streaming media servers and clients to achieve real-time streaming and playback of audio and video content. Here is a brief description of the applications of RTSP and the meanings related to push and pull:
RTSP Applications:
Streaming Media Transmission: RTSP is widely used in streaming media applications such as online video streaming, video-on-demand, and video conferencing. It enables clients to work collaboratively with streaming media servers by negotiating session parameters, controlling playback status, and data transmission methods, thus achieving real-time media data transmission and playback.
Surveillance Systems: RTSP is commonly used in surveillance cameras and camera systems, allowing users to remotely monitor video streams over the network. This is particularly useful for home security, enterprise security, and public safety.
Video Conferencing: RTSP is used in video conferencing applications, allowing multiple participants to share audio and video data in real-time, enabling remote collaboration.
Audio Streaming: RTSP not only supports video streaming but also facilitates the transmission of audio streams. This is important for applications such as audio broadcasting and audio conferences.
Push and Pull
Push: Push refers to the process of sending multimedia data (typically audio and video) from a source to the network, allowing other devices or users to access and play it in real-time. Typically, the media source generates data and then pushes it to a streaming media server using a streaming media transfer protocol (such as RTSP, RTMP, HLS, etc.). The process from the source to the server is called "push".
Pull: Pull refers to the process of retrieving multimedia data from a streaming media server and decoding and playing it at the receiving end. Users or client devices request specific media content from the streaming media server using the same streaming media transfer protocol. This process is called "pull".
In summary, RTSP is an essential protocol for real-time streaming transmission widely used in the multimedia domain, enabling users to transmit and play audio and video data in real-time over the internet. Push involves sending media data to the network, while pull involves retrieving and playing media data from the network. Together, they form the core of streaming media services.
Using the Camera as an RTSP Server
By running the examples -> rtsp_stream code, you can configure the device as an RTSP server. Devices on the same network can access the real-time image by using a media player (such as VLC Player) and accessing rtsp://192.168.xxx.xxx:8554/mjpeg/1 (Note: Please replace the IP address with the actual IP address obtained from the serial monitor).
« ..WiFi connected
RTSP URL: rtsp://192.168.2.175:8554/mjpeg/1
LinkedListElement (0x3ffdb9ac)->(0x3ffdb9ac)->(0x3ffdb9ac)
Creating TSP streamer
Created streamer width=640, height=480
Relevant Links
rtsp_stream relies on the Micro-RTSP library, which can be installed by following the dependency prompts when installing the TimerCam-arduino library. Alternatively, you can manually clone and install it using the GitHub link provided. VLC Player is a powerful media player that we will use for pulling/streaming tests in the subsequent testing phase.
VLC Player Pull Streaming
Configuration of Streaming Relay Server
Distributing HTTP Streaming Sources
In the previous demo, we used the camera as an RTSP server, but there are limitations in terms of performance, especially when multiple clients want to pull the stream, it can be challenging. In some cases, you may want to distribute media resources to external servers or live streaming platforms, or achieve stronger multi-device access capabilities. In the following, we will use an open-source RTSP server project combined with the ffmpeg tool to implement streaming (forwarding TimerCAM data to the server) and pulling (clients pulling video resources from the server).
Here we need to burn the camera with the first web_cam demo program mentioned at the beginning of the article, and we can directly access the image stream through http://IP:81/stream.
Visit EasyDarwin - Github releases, download the release version to your local machine, and extract it. Run it as a service and start the server. (Note: Please use the sudo command to run it, otherwise, you may not have permission to open the port.)
cd EasyDarwin
sudo ./start.sh
#sudo ./stop.sh
Then please use ffmpeg to configure forwarding, and use the IP and port used in the previous demo for TimerCAM's running port and IP. The latter is the IP of the RTSP server (EasyDarwin) (default port 554, followed by a customizable ID number).
Note: Before executing this command, make sure to start the camera and ensure that the image can be previewed correctly on the webpage.
ffmpeg -re -i http://192.168.2.175:81/stream -rtsp_transport tcp -vcodec h264 -f rtsp rtsp://192.168.2.89:554/666
After completing the previous configuration, open a browser and enter http://IP:10008 to access the EasyDarwin control page. The default username and password are admin/admin. (Replace IP with the IP address of the Linux host).
By refreshing the page, we can see a new streaming address added to the control page. We can use general media player software such as VLC to fill in the corresponding streaming address to pull the video and locally cache the recording. Additionally, you can use multiple clients simultaneously for streaming playback.
Distributing Pull Sources
If the camera is being used as an RTSP server, you can also directly configure the pull forwarding in the EasyDarwin backend. For more details, please refer to the EasyDarwin GitHub repository.
