Saving Numpy Array as Video Relative to a Time Array

Asked 2 months ago
Viewed 6 times

Let a numpy array video of shape (T,w,h,3) be given. Here T is a positive integer representing number of frames, w is a positive integer representing the width, h is a positive integer representing the height. Every entry of video is an integer from 0 to 255. In other words, video is a numpy array represents a video in the sense that video[t] is an RGB image for every non-negative integer t < T. After video is given, another array of floats time of shape (T) is given. This array time satisfy time[0]=0 and time[t] < time[t+1] for every non-negative integer t < T. An example of the above situation is given here:

import numpy as np

shape = (200, 500, 1000, 3)
random = np.random.randint(0, 255, shape, dtype= np.uint16)
time = np.zeros((shape[0]), dtype = np.float16)
time[0] = 0
for i in range(1, shape[0]):
    x = np.random.random_sample()
    time[i] = time[i-1] + x

My goal is to save video and time a playable video file such that:

  1. The video file is in format of either avi or mp4 (so that we can just double click it and play it).
  2. Each frame of the video respects the time array in the following sense: for every non-negative integer t < T, the viewer is seeing the picture video[t] during the time period from time[t] to time[t+1]. The moment time[T-1] is the end of the video.
  3. If possible, keep the original size (in the given example the size is (500,1000)).

How can I achieve this? I tried using the opencv's video writer and it seems I have to enter some fps information which I do not have because the time array can be very non-uniform in terms of when each picture is displayed.

asked 2 months ago

Correct Answer

The solution to your problem seems to be very simple. As a video needs a constant frame rate you have to decide first at which granularity you want your video.

After you have decided the FPS value you go and add as numpy arrays all of the missing frames, so you can use the export to video method with a constant frame rate.

The added numpy arrays will be always the same as the last one displayed until the time of change to another one. If the chosen frame rate does not 'hit' all your time values exactly you have the choice of increasing the frame rate or adjust a bit the time spans for displaying the same frame.

answered 2 months ago