Reading every nth frame from VideoCapture in OpenCV

Question 1

I'm afraid there's not much you can do and it's not just a shortcoming of OpenCV. You see, modern video codecs, are, generally, complex beasts. To get a higher compression rate the encoding of a frame is often dependent on previous and sometimes even successive frames.

So, most of the time you have to decode frames before the desired one even if you don't need them.

There are rather non-trivial tricks to specifically encode a video file, so that it would be cheap to get every Nth frame, but it's not feasible in general case.

That said, you can try the seeking functionality OpenCV provides (see OpenCV Seek Function/Rewind). It may (as well as may not) work faster depending on the circumstances. However, personally, I wouldn't bet on it.

Question 2

I've had success in Python 3 using a simple counter and setting the capture to that counter's frame, as follows:

import cv2

cap = cv2.VideoCapture('XYZ.avi')
# For streams:
#   cap = cv2.VideoCapture('rtsp://url.to.stream/media.amqp')
# Or e.g. most common ID for webcams:
#   cap = cv2.VideoCapture(0)
count = 0

while cap.isOpened():
    ret, frame = cap.read()

    if ret:
        cv2.imwrite('frame{:d}.jpg'.format(count), frame)
        count += 30 # i.e. at 30 fps, this advances one second
        cap.set(cv2.CAP_PROP_POS_FRAMES, count)
    else:
        cap.release()
        break

I've tried to find a way to make this a little more pythonic using a with statement but I don't believe the CV2 library has been updated for it.

Question 3

I got it to work in Python... See below for two sample use cases and some caveats.

# First, import some packages

import cv2
import math
import numpy as np

# Make sure that the print function works on Python 2 and 3
from future import print_function

# Capture every n seconds (here, n = 5) 

#################### Setting up the file ################
videoFile = "Jumanji.mp4"
vidcap = cv2.VideoCapture(videoFile)
success, image = vidcap.read()

#################### Setting up parameters ################

seconds = 5
fps = vidcap.get(cv2.CAP_PROP_FPS) # Gets the frames per second
multiplier = fps * seconds

#################### Initiate Process ################

while success:
    frameId = int(round(vidcap.get(1))) #current frame number, rounded b/c sometimes you get frame intervals which aren't integers...this adds a little imprecision but is likely good enough
    success, image = vidcap.read()

    if frameId % multiplier == 0:
        cv2.imwrite("FolderSeconds/frame%d.jpg" % frameId, image)
        
vidcap.release()
print("Complete")

# Alternatively, capture every n frames (here, n = 10)


#################### Setting up the file ################
videoFile = "Jumanji.mp4"
vidcap = cv2.VideoCapture(videoFile)
success, image = vidcap.read()

#################### Setting up parameters ################

#OpenCV is notorious for not being able to good to 
# predict how many frames are in a video. The point here is just to 
# populate the "desired_frames" list for all the individual frames
# you'd like to capture. 

fps = vidcap.get(cv2.CAP_PROP_FPS)
est_video_length_minutes = 3         # Round up if not sure.
est_tot_frames = est_video_length_minutes * 60 * fps  # Sets an upper bound # of frames in video clip

n = 5                             # Desired interval of frames to include
desired_frames = n * np.arange(est_tot_frames) 


#################### Initiate Process ################

for i in desired_frames:
    vidcap.set(1, i-1)                      
    success, image = vidcap.read(1)         # image is an array of array of [R,G,B] values
    frameId = vidcap.get(1)                # The 0th frame is often a throw-away
    cv2.imwrite("FolderFrames/frame%d.jpg" % frameId, image)

vidcap.release()
print("Complete")

That's pretty much it.

Some unfortunate caveats... depending on your version of `opencv` (this is built for `opencv` V3), you may need to set the fps variable differently. See [here][1] for details. To find out your version, you can do the following:

major_ver, minor_ver, subminor_ver = cv2.__version__.split('.')
print(major_ver)

Question 4

I encountered the same problem. All i did was this:

import cv2

vs = cv2.VideoCapture("<path of your video>.mp4")

print("Showing frames...")
c=1
while True:

    grabbed, frame = vs.read()
    if c%5==0:
        cv2.imshow('Frame',frame)
        cv2.waitKey(1)
    c+=1

vs.release()

Question 5

You should use the grab function to move to next frame. And only use retrieve to decode the frame you need.

bool bSuccess
int FramesSkipped = 5;
for (int a = 0;  < FramesSkipped; a++)
      bSuccess = cap.grab();
bSuccess = cap.retrieve(NextFrameBGR);

Question 6

Here is what I suggest:

     CvCapture* capture = cvCaptureFromFile("input_video_path");
     int loop = 0;
     IplImage* frame = NULL;
     Mat matframe;                                                                   
     char fname[20];                                                                 
     do {
        frame = cvQueryFrame(capture);  
        matframe = cv::cvarrToMat(frame);
        cvNamedWindow("video_frame", CV_WINDOW_AUTOSIZE);                                 
        cvShowImage("video_frame", frame);
        sprintf(fname, "frame%d.jpg", loop);
        cv::imwrite(fname, matframe);//save each frame locally
        loop++;
        cvWaitKey(100);
     } while( frame != NULL );

Now that you have saved all the frames locally you can quickly read the nth frame that you want.
CATUION:A sample video of 12 secs I had was composed of >200 images. This will eat up lot of space.

A simple yet effective optimization will be to read the nth frame using the approach that you are using or the one suggested by @sergie. After this you can save the image with its index so that later query at same index will return the saved image rather than having to skip frames like you are. This way you will save the space that you would have wasted in saving frames that you wouldn't have queried and time taken to read & save those unwanted frames aswell.

Question 7

When I had the same goal with OpenCV, I just tuned around the number of "keyframes" I wanted per second of video, regardless of the frame rate or total number of frames. So, this gets me the N-th key given my target KPS.

# python3.6 code using OpenCV 3.4.2

import cv2
KPS = 5 # Target Keyframes Per Second
VIDEO_PATH = "path/to/video/folder" # Change this
IMAGE_PATH = "path/to/image/folder" # ...and this 
EXTENSION = ".png"

cap = cv2.VideoCapture(VIDEO_PATH)
    # Set frames-per-second for capture
    fps = round(cap.get(cv2.CAP_PROP_FPS))
    hop = round(fps / KPS)
    curr_frame = 0
    while(True):
        ret, frame = cap.read()
        if not ret: break
        if curr_frame % hop == 0:
            print('Creating... {0}'.format(name,))
            name = IMAGE_PATH + "_" + str(curr_frame) + EXTENSION
            cv2.imwrite(name, frame)
        curr_frame += 1
    cap.release()

Note that I'm rolling through all the frames, but only writing the N-th frame using hop as N.

Question 8

I use this repo!

Main idea is:

main.py

from camera import VideoCam
SKIPFRAME = 8
url = 0
v1 = VideoCam(url)
v1.check_camera(v1.cap)
ct = 0
while True:
    ct += 1
    try:
        ret = v1.cap.grab()
        if ct % SKIPFRAME == 0:  # skip some frames
            ret, frame = v1.get_frame()
            if not ret:
                v1.restart_capture(v1.cap)
                v1.check_camera(v1.cap)
                continue
            # frame HERE
            v1.show_frame(frame, 'frame')
    except KeyboardInterrupt:
        v1.close_cam()
        exit(0)

camera.py

import cv2
import logging


class VideoCam():
    def __init__(self, url=0):
        self.url = url
        self.cap = cv2.VideoCapture(self.url)
        self.get_frame()
        self.get_frame_read()
        logging.basicConfig(format='%(asctime)s %(message)s', level=logging.INFO)

    def check_camera(self, cap):
        logging.info('Camera {} status: {}'.format(self.url, cap.isOpened()))

    def show_frame(self, frame, name_fr='NAME'):
        cv2.imshow(name_fr, frame)
        # cv2.imshow(name_fr, cv2.resize(frame, (0, 0), fx=0.4, fy=0.4))
        cv2.waitKey(1)


    def get_frame(self):
        return self.cap.retrieve()

    def get_frame_read(self):
        return self.cap.read()

    def close_cam(self):
        self.cap.release()
        cv2.destroyAllWindows()

    def restart_capture(self, cap):
        cap.release()
        self.cap = cv2.VideoCapture(self.url)

Question 9

if anyone will need to capture every 5th frame and save it as jpg, based on Ishan Shah's code:

import cv2

vid = cv2.VideoCapture('C:/python_works/creatives_gardenscapes/0c52b83ed1dec617092aaf83278f12ad.mp4')

if not os.path.exists('images'):
    os.makedirs('images')

index = 0
while(True):
    ret, frame = vid.read()
    if not ret: 
        break
    name = 'C:/python_works/creatives_gardenscapes/frames/0c52b83ed1dec617092aaf83278f12ad' + str(index) + '.jpg'
    if index%50==0:
        cv2.imwrite(name, frame)
    index += 1

Question 10

code:

import cv2
vidcap = cv2.VideoCapture('bali.mp4')
success,image = vidcap.read()

fps = vidcap.get(cv2.CAP_PROP_FPS)
print('video fps :', fps)
length = int(vidcap.get(cv2.CAP_PROP_FRAME_COUNT))
print( 'total frames :', length )

count = 0
skip = 10 

success = True
while success:
  success,image = vidcap.read()
  if count%skip==0:
    print(count)
    cv2.imwrite("frames/frame%d.jpg" % count, image)     # save frame as JPEG file
  if cv2.waitKey(10) == 27:                     # exit if Escape is hit
      break
  count += 1

output:

Question 11

You can effectively "skip" a frame by reading it then discarding it. Granted, this does exactly not what you are asking, but might still suffice as a work-around. Obviously this has implications on performance but may be adequate for your purposes.

vs = cv2.videoCapture('/my/clip.mp4')

for i in (thing):
    # Read a frame
    _, frame = vs.read()
    # Skip the frame you just read by reading another
    _, frame = vs.read()
    # Run a process on said frame
    cv2.imshow("Window", frame)  
    

vs.release()

Question 12

This is what i did on skip frame

skip_frame = cap.get(cv2.CAP_PROP_POS_FRAMES) #get current frames
sk = skip_frame + 10 # How many frames you want to skip
cap.set(cv2.CAP_PROP_POS_FRAMES, sk) # set the Current frame to the sk

Question 13

It is not possible to extract random frames as the encoding scheme is generally extremely complex. For example in MPEG-4, Only the information containing the difference between two frames is stored, Hence clearly the previous frames are required.

Question 14

I've refactored @benJephunneh's answer to be more "Pythonic" and wrapped the code in an iterator.

def every_n_frames(path, n):
    video = cv2.VideoCapture(path)
    # start at frame 0
    count = 0

    while video.isOpened():
        success, frame = video.read()

        if not success:
            break

        try:
            yield frame
        except Exception as e:
            video.release()
            raise e

        count += n  # advance by n frames
        video.set(cv2.CAP_PROP_POS_FRAMES, count)

    video.release()


# Usage:
framerate = 30
for i, frame in enumerate(every_n_frames("XYZ.avi", framerate)):
    cv2.imwrite("frame{:d}.jpg".format(i * framerate), frame)

Note: If you're using lower values of n (e.g. every 2, 3 frames), setting cv2.CAP_PROP_POS_FRAMES will be much slower than just reading every frame then discarding the ones you don't need.

Question 15

Skipping frames in cv2 python

cap = cv2.VideoCapture(0)
x = 0
   while True:
      ret, frame = cap.read()

      gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
      blurred = cv2.GaussianBlur(gray,(7, 7), 0)
      (T, thresh) = cv2.threshold(blurred, 135, 255, cv2.THRESH_BINARY)
      contours, hierarchy=cv2.findContours(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
      contours = sorted(contours, key=cv2.contourArea, reverse=True)[:10]

      for contour in contours:
         area = cv2.contourArea(contour)
         if(x != 0):
               x -= 1
               break
         if(area >= 517000 and area <= 522000):

               holdFrame = frame
               results = yourAlgorithm(holdFrame) # Your Algorithm Here
               x = 3
      if cv2.waitKey(1) & 0xFF == ord('q'):
         break


   cv2.waitKey(0)
   cv2.destroyAllWindows()

In the Code Above, I have done 3 things totally, that solved my problem

Capturing Frames and initializing the variable x = 0
Then if my required area of contour is detected I run my algorithm.
After running the algorithm I set my x = 3
Then next time my function goes to trigger my algorithm, it checks if x != 0, while x = 3 so it breaks out of the loop twice while decrementing the value of x by 1 each time. So, when the value of x becomes 0, approximately 3,4 frames have been skipped.