3. Basic Video Manipulation

Video Handling Basics

Understanding how to manipulate images was critical to learning video manipulation. This is because a video is just a loop of images! So toying with videos was a piece a pie after getting comfortable with image handling. Since I learned how to load images and display them, the only thing I needed to play videos was a loop. The next program should look very similar to the first two…

# include "stdio.h"
# include "highgui.h"

int main( int argc, char** argv ) {
	IplImage* frame = 0;

	if( argc < 2 ) {
		printf( "Usage: Accepts one video as argument\n" );
		exit( 0 );
	}

	cvNamedWindow( "Example2", CV_WINDOW_AUTOSIZE );
	cvMoveWindow( "Example2", 100, 100);
	CvCapture* capture = cvCreateFileCapture( argv[1] );

	while(1) {
	    frame = cvQueryFrame( capture );
	    if( !frame) break;
	    cvShowImage( "Example2", frame );
	    char c = cvWaitKey(33);
	    if( c == 27 ) break;
	}

	cvReleaseCapture( &capture );
	cvDestroyWindow( "Example2" );
}

Get the code from my Git Repository
If you need help with Git, follow this quick introduction – Getting Started With Git

First I created another window “Example2″ with my usual approach, no problems here.

Dealing with videos seemed simple enough in theory, but I was slightly taken aback at it’s complexity once I delved into the source files on my computer. CvCapture is another data structure, which has a complicated definition. All of it’s members are declared virtually and has different capture objects it uses for different situations. (of which, I don’t know yet…)

The way I loaded a video’s data into the CvCapture data structure is through the CreateFileCapture function. After looking through the source, this function calls the other functions with various usages, they were of the form

CvCapture* cvCreateFileCapture_XXX( const char* filename );

filename - Filename of the video you want to pass in.

where the XXX was different names. I just took note to remember this function takes a pointer to the filename of the video I’m trying to load.

Now remembering that a video is just a loop of images, I could be sure I was going to need an IplImage structure to store each one in. After creating the image variable, there was really nothing left but to jump into the loop.

while(1) {
   frame = cvQueryFrame( capture );
   if( !frame) break;
   cvShowImage( "Example2", frame );
   char c = cvWaitKey(33);
   if( c == 27 ) break;
}

Now inside the loop, my thinking process is this. I want to take each frame from the CvCapture data structure and display them one at a time, quickly. The first step was extracting each frame. I accomplished this using the QueryFrame function.

 IplImage* cvQueryFrame( CvCapture* capture );

capture - Here you pass the video structure.

It takes a pointer to a CvCapture structure, parses through the data and returns a pointer to each frame which is an IplImage structure. This function is just a combination of cvGrabFrame and cvRetrieveFrame in one call. Pretty handy, so I just stored it in frame, my IplImage pointer.

I then displayed the image in my “Example2″ window using my old friend cvShowImage. My question was “Okay, how long do I let this image sit here?”. After a little reading a good time to assume without knowledge of the captures frame rate it 30ms. the other 3 milliseconds are waiting for input (the ecs key == 27 in ASCII). There is a way to extract the correct frame rate from the CvCapture structure, but I’ve yet to glean this procedure…

char c = cvWaitKey(33);
if( c == 27 ) break;

Then if the esc key is pressed I could exit the loop. The loop only has two exit methods.

if( !frame ) break;
if( c == 27 ) break;

Then the usual clean ups were done, minus one. Notice I didn’t kill any of the IplImage structures? Apparently the CvCapture structure took care of that for me, how kind. Anyway, trying to release information that isn’t there is clearly not a good idea!

I saved the file, edited my CMakeLists.txt to accommodate the extra program and built it. I tested it on the sample video tree.avi and some other vids on my hard drive. You may need to finagle with the frame rate a bit to get a good flow for now.

It would be wise to put more exit procedures and provide more error checking. However, this was boring… I wanted to get to the fun stuff.

The Fun Stuff

As with the images, once I got my video basics down I wanted to try some more simple manipulations. My thought, play a video I normally enjoy with the previous procedure, then play another one next to it with each frame being blurred.

Alright so here it is.

# include "stdio.h"
# include "cv.h"
# include "highgui.h"

int main( int argc, char** argv ) {
	IplImage* frame = 0;
	IplImage* augframe = 0;

	if( argc < 2 ) {
		printf( "Usage: Accepts one video as argument\n" );
		exit( 0 );
	}

	cvNamedWindow( "Example2", CV_WINDOW_AUTOSIZE );
	cvMoveWindow( "Example2", 100, 100);
	cvNamedWindow( "output", CV_WINDOW_AUTOSIZE );
	cvMoveWindow( "output", 500, 100);
	CvCapture* capture = cvCreateFileCapture( argv[1] );

	while(1) {
	    frame = cvQueryFrame( capture );
	    if( !frame) break;
	    augframe = cvCreateImage( cvGetSize( frame ),
                                 IPL_DEPTH_8U, 3 );
		cvSmooth( frame, augframe, CV_BLUR, 3, 3 );
	    cvShowImage( "Example2", frame );
	    cvShowImage( "output", augframe );
	    char c = cvWaitKey(73);
	    if( c == 27 ) break;
	}

	cvReleaseCapture( &capture );
	cvDestroyWindow( "Example2" );
}

Get the code from my Git Repository
If you need help with Git, follow my quick introduction – Getting Started With Git

Everything highlighted I’ve used before, so there is nothing new to talk about. I used the same cvSmooth function from the previous post, I just chose to use another one of the smoothing procedures that was listed in the table. CV_BLUR was my choice, I don’t see much of a difference between it and the CV_GAUSSIAN. I suppose it’s just a different algorithm.

I did want to note one thing though. Recall that the CvCapture structure is taking care of releasing the frames for me now, this means that as soon as the call cvShowImage terminated, the image stored in frame was released… This meant that in order to create the augframe from the original using the cvCreateImage function it had to be done before frame was displayed!

I say try this out with some videos on your computer. Then perhaps take a look at some of the other filters, and what they do.

References

I’m studying from Gary Bradski’s Learning OpenCV