Shulou(Shulou.com)06/02 Report--

This article analyzes "how to read and process video streams in OpenCV2". The content is detailed and easy to understand. Friends who are interested in "how to read and process video streams in OpenCV2" can follow the editor's train of thought to read it slowly and deeply. I hope it will be helpful to you after reading. Let's follow the editor to learn more about how to read and process video streams in OpenCV2.

Preface

Due to the needs of the project, we plan to achieve nine-way video splicing, so we must be familiar with the processing of video sequences by OpenCV. Video signal processing is an extension of image processing. The so-called video sequence is composed of images emitted in a certain order, namely Frame. Here, we mainly record how to use Qt+OpenCV to read every frame in the video, and then apply some image processing algorithms to each frame (such as using Canny operator to detect the edge of the video).

one。 Read video sequence

OpenCV provides an easy-to-use framework to extract image frames from video files and USB cameras. If you just want to read a video, you just need to create an cv::VideoCapture instance and extract each frame in a loop. Create a new Qt console project and add it directly to the main function:

# include # include int main (int argc, char * argv []) {QCoreApplication a (argc, argv); / / read video stream cv::VideoCapture capture ("e:/BrokeGirls.mkv"); / / check whether the video has been read successfully if (! capture.isOpened ()) {qDebug () = 0) stop= true;} return a.exec ();}

(note: to open the video file correctly, the corresponding decoder must be installed on the computer, otherwise cv::VideoCapture cannot understand the video format! After running, a window appears to play the selected video (you need to specify the file name of the video when you create the cv::VideoCapture object).

two。 Processing video Fram

In order to process each frame of the video, we create our own class VideoProcessor, which encapsulates the video capture framework of OpenCV, which allows us to specify the handler function called for each frame.

First, we want to specify a callback handler that will be called in every frame. This function accepts a cv::Mat object and outputs the processed cv::Mat object with the following function signature:

Void processFrame (cv::Mat& img, cv::Mat& out)

As an example of such a processing function, the following Canny function calculates the edge of the image and adds it directly to the mian file when you use it:

/ / A pair of video frames to do Canny operator edge detection void canny (cv::Mat& img, cv::Mat& out) {/ / first convert each frame image to a grayscale image cv::cvtColor (img,out,CV_BGR2GRAY); / / call the Canny function cv::Canny (out,out,100200); / / a pair of pixels to flip cv::threshold (out,out,128,255,cv::THRESH_BINARY_INV);}

Now we need to create a VideoProcessor class to deploy the video processing module. Before you do that, you need to create another class, the frame handling class used internally by VideoProcessor. This is because in an object-oriented context, it is more appropriate to use a frame processing class than a frame handling function, and using classes can give programmers more flexibility when it comes to algorithms (as described in the book). Name this internal frame handling class FrameProcessor, which is defined as follows:

# ifndef FRAMEPROCESSOR_H#define FRAMEPROCESSOR_H#include # include class FrameProcessor {public: virtual void process (cv:: Mat & input, cv:: Mat & output) = 0;}; # endif / / FRAMEPROCESSOR_H

Now you are ready to define the VideoProcessor class, as shown in videoprocessor.h:

# ifndef VIDEOPROCESSOR_H#define VIDEOPROCESSOR_H#include # include "frameprocessor.h" class VideoProcessor {private: / / create a video capture object cv::VideoCapture capture; / / callback function void (* process) (cv::Mat&, cv::Mat&) called per frame; / / FrameProcessor interface FrameProcessor * frameProcessor; / / bool signal bool callIt that determines whether to call the callback function / / name of input window std::string windowNameInput; / / name of output window std::string windowNameOutput; / / delay int delay; / / number of frames processed long fnumber; / / stop long frameToStop; at this frame / / whether to stop processing bool stop / / when the input image sequence is stored in different files, you can use the following settings / / use the array of image file names as the std::vector::const_iterator itImg of the input std::vector images; / / image vector / / get the next frame / / possibly from: video file or camera bool readNextFrame (cv::Mat & frame) {if (images.size () = = 0) return capture.read (frame); else {if (itImg! = images.end ()) {frame= cv::imread (* itImg) ItImg++; return frame.data! = 0 } public: / / default settings digits (0), frameToStop (- 1), VideoProcessor (): callIt (false), delay (- 1), fnumber (0), stop (false), process (0), frameProcessor (0) {} / / create input window void displayInput (std::string wt) / / create the output window void displayOutput (std::string wn); / / No longer display the processed frame void dontDisplay (); / / set the input image vector bool setInput (std::string filename) by the following three functions; / / if the input is a camera, set ID bool setInput (int id) / / if the input is a set of image sequences, apply this function bool setInput (const std::vector& imgs); / / set the delay between frames / / 0 means waiting for the keystroke response at each frame / / negative number means no delay void setDelay (int d); / / return the frame rate of the image double getFrameRate () / need to call the callback function void callProcess (); / / do not need to call the callback function void dontCallProcess (); / / set the FrameProcessor instance void setFrameProcessor (FrameProcessor* frameProcessorPtr); / / set the callback function void setFrameProcessor (void (* frameProcessingCallback) (cv::Mat&, cv::Mat&)); / / stop running void stopIt () / / determine if bool isStopped () has been stopped; / / has the capture device started? Bool isOpened (); / / returns the number of frames in the next frame long getFrameNumber (); / / this function acquires and processes video frames void run ();}; # endif / / VIDEOPROCESSOR_H

Then, define the functions of each function in videoprocessor.cpp:

# include "videoprocessor.h" / / create input window void VideoProcessor::displayInput (std::string wt) {windowNameInput= wt; cv::namedWindow (windowNameInput);} / create output window void VideoProcessor::displayOutput (std::string wn) {windowNameOutput= wn; cv::namedWindow (windowNameOutput);} / / No longer display processed frame void VideoProcessor::dontDisplay () {cv::destroyWindow (windowNameInput); cv::destroyWindow (windowNameOutput); windowNameInput.clear () WindowNameOutput.clear ();} / / sets the input image vector bool VideoProcessor::setInput (std::string filename) {fnumber= 0; / / releases the previously opened video resource capture.release (); images.clear (); / / opens the video return capture.open (filename);} / / if the input is a camera, set IDbool VideoProcessor::setInput (int id) {fnumber= 0 / / release the previously opened video resource capture.release (); images.clear (); / / Open the video file return capture.open (id);} / / if the input is a set of image sequences, apply the function bool VideoProcessor::setInput (const std::vector& imgs) {fnumber= 0; / / release the previously opened video resource capture.release (); / / input the vector images= imgs that will be the image ItImg= images.begin (); return true;} / / set delay between frames / / 0 means waiting for keystroke response at every frame / / negative number means no delay void VideoProcessor::setDelay (int d) {delay= d;} / / frame rate double VideoProcessor::getFrameRate () {if (images.size ()! = 0) return 0 of the returned image; double r = capture.get (CV_CAP_PROP_FPS); return r } / / need to call callback function void VideoProcessor::callProcess () {callIt= true;} / / there is no need to call callback function void VideoProcessor::dontCallProcess () {callIt= false;} / / set FrameProcessor instance void VideoProcessor::setFrameProcessor (FrameProcessor* frameProcessorPtr) {/ / invalidate callback function process= 0; / / reset FrameProcessor instance frameProcessor= frameProcessorPtr; callProcess () } / / set the callback function void VideoProcessor::setFrameProcessor (void (* frameProcessingCallback) (cv::Mat&, cv::Mat&)) {/ / invalidate the FrameProcessor instance frameProcessor= 0; / / reset the callback function process= frameProcessingCallback; callProcess ();} / / the following function indicates the reading status of the video / / stops running void VideoProcessor::stopIt () {stop= true } / / determine if bool VideoProcessor::isStopped () {return stop;} / / has started capturing the device? Bool VideoProcessor::isOpened () {return capture.isOpened () | |! images.empty ();} / return the number of frames in the next frame long VideoProcessor::getFrameNumber () {if (images.size () = = 0) {/ / get the information of the capture device long f = static_cast (capture.get (CV_CAP_PROP_POS_FRAMES)); return f } else / / what happens when input comes from a set of image sequences {return static_cast (itImg-images.begin ());}} / / this function acquires and processes video frame void VideoProcessor::run () {/ / current frame cv::Mat frame; / / output frame cv::Mat output / / if (! isOpened ()) {qDebug () process (frame,output); / / increase the number of frames fnumber++;} else {output= frame;} / / display the output frame if (windowNameOutput.length ()! = 0) cv::imshow (windowNameOutput,output) / / introduce delay if (delay > = 0 & & cv::waitKey (delay) > = 0) stopIt (); / / check whether you need to stop running if (frameToStop > = 0 & & getFrameNumber () = = frameToStop) stopIt ();}}

Define the video processing class, which will be associated with a callback function. Using this class, you can create an instance, specify the input video file, bind the callback function, and then start processing each frame. To call this video processing class, simply add to the main function:

/ / define a video processing class to process video frames / / first create an instance VideoProcessor processor; / / Open the video file processor.setInput ("e:/BrokeGirls.mkv"); / / declare the display window / / as input and output video processor.displayInput ("Input Video"); processor.displayOutput ("Output Video") / / play video processor.setDelay (1000./processor.getFrameRate ()) at the original frame rate; / / set callback function processor.setFrameProcessor (canny); / / start frame processing processor.run (); cv::waitKey ()

Effect:

OpenCV: open the camera to get the video stream # include#includeusing namespace cv;using namespace std;int main () {/ / [1] read the video from the camera VideoCapture capture (1); if (! capture.isOpened ()) {cout frame; / / read the current frame key = waitKey (20); if (key = = 27) / / the ESC key exits break If (key = = 32) / / the space bar saves the image {sprintf (filename, "Picture_%d.png", + + count); imwrite (filename, frame); / / namedWindow ("[frame]", WINDOW_NORMAL); imshow ("[frame]", frame);} imshow ("image", frame); / / display the current frame} return 0 } about how to read and process the video stream in OpenCV2, so much for sharing here. I hope the above content can improve everyone. If you want to learn more knowledge, please pay more attention to the editor's updates. Thank you for following the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.