Av rescale q
lastmod 2022-01-27created 2022-01-271 edits
Time Stamp? PTS? DTS?
Whether you're saving data with ffmpeg or sending it somewhere, one of the most annoying things is setting the PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp) values.
First, if you don't know much about it, you don't even know what a PTS is, so you probably thought "it's a timestamp value, so I can just put in a tick value, right?" and wasted time throwing in something like GetTickCount() (on a Windows environment), or putting in values incremented by 1 sequentially. Or maybe you didn't put anything in at all?
Then in some cases it plays fine (here ffmpeg generates the values on its own), and in other cases it doesn't.
If you've studied it to some extent, you'll know what PTS/DTS and the like are.
To take encoded data and put it on the screen, DTS is what tells you "it must be decoded by this time at the latest", and PTS is what tells you "it must be shown on screen by this time at the latest".
To explain again, a timestamp is literally a "time stamp". We perceive video as moving when still pictures change with a certain time gap. Each picture would carry time information about when it was taken, and we call that a timestamp. Just think of PTS and DTS as timestamps with a few more messy purposes attached.
Why does a proper Time Stamp need to go in?
Back to the main topic..
So how do you put in the right PTS and DTS?
First, "right" can be explained as "conforming to predefined rules". Then what does this right time depend on? It depends on the system(?) of the codec you use. The concept that comes up here is PCR (Program Clock Reference), but I don't know it precisely myself, so I won't write an explanation.
Anyway, for example, what is the right timestamp period for H.264? It's 90000. 90 khz. (Where do you get this information? If you search around, it's in the specs in documents mostly written in English (rfc, ietf).)
This means that, assuming you shot 1 second of video, the difference in timestamp units between the first still image taken and the last still image taken is 90000. (For audio, this corresponds to the sample rate.) In other words, if I save or send 30 images over 1 second and the timestamp of the first image is 1, the timestamp of the last, 30th image becomes 90001.
So for PTS/DTS, you can simply multiply the Tick value (in ms) by 90 and send it!
But if this Tick value were perfectly accurate there'd be nothing to talk about; even a slight error often causes the file not to play properly, or a stream you sent to jump around strangely on the receiving side.
av_rescale_q for accurate Time Stamp calculation
The thing that exists to solve this is av_rescale_q. Ah, that was way too much background just to explain this one thing. First, let's look at the av_rescale_q declaration.
It looks roughly like this. ffmpeg's prototype declarations are a bit unfriendly.
To explain, the return value spits out the calculated timestamp value.
The first argument, a, is the increment value. If I managed the video's time values in ms and it's 30fps, the video will have time values of (1, 34, 67, 100 ....), so you put in values like (1, 34, 67... ).
The second argument is which time base system the first argument runs on. Since it's stamped in ms, pass (AVRational){1,1000}.
The third argument specifies the time base system of the timestamp unit to convert to. For h.264, it would be (AVRational){1, 90000}.