Libswscale
lastmod 2022-01-26created 2014-11-1115 edits
sws_getCachedContext VS sws_getContext
There are two ways to use libswscale in FFmpeg.
-
sws_getCachedContextcallssws_freeContextinternally within the function itself, - whereas with
sws_getContextyou must callsws_freeContextexplicitly.
If you don't call sws_freeContext, a Memory Overflow occurs.
sws_scale
convert RGB(A) to YUV
It turns out you can convert RGB or RGBA data into YUV using FFmpeg itself (SwScale), which then is compatible with output to a file. The basics are just a few lines: first, create an SwsContext that specifies the image size, and the source and destination data formats:
AVCodec *codec = avcodec_find_encoder(AV_CODEC_ID_MPEG2VIDEO);
AVCodecContext *c = avcodec_alloc_context3(codec);
// ...set up c's params
AVFrame *frame = av_frame_alloc();
// ...set up frame's params and allocate image buffer
SwsContext * ctx = sws_getContext(c->width, c->height,
AV_PIX_FMT_RGBA,
c->width, c->height,
AV_PIX_FMT_YUV420P,
0, 0, 0, 0);
And then apply the conversion to each RGBA frame (the rgba32Data pointer) as it’s generated:
uint8_t *inData[4] = { rgba32Data, 0, 0, 0 }; // sws_scale 안에서 4개의 배열로 접근한다. 4개의 배열로 만들고 남은 공간은 공백으로 채우는게 좋다.
int inLinesize[4] = { 4 * c->width, 0, 0, 0 }; // sws_scale 안에서 4개의 배열로 접근한다. 4개의 배열로 만들고 남은 공간은 공백으로 채우는게 좋다.
sws_scale(ctx, inData, inLinesize, 0, c->height,
frame->data, frame->linesize);
Other input formats
The third argument to sws_getContext describes the format/packing of your data. There are a huge number of formats defined in FFmpeg (see pixfmt.h), so if your raw data is not RGBA you shouldn’t have to change how your image is generated. Be sure to compute the correct line width ( inLinesize in the code snippets) when you change the input format specification. I don’t know which input formats are supported by sws_scale (all, most, just a few?), so it would be wise to do a little experimentation.
For example, if your data is packed 24-bit RGB, and not 32-bit RGBA, then the code would look like this:
SwsContext * ctx = sws_getContext(c->width, c->height,
AV_PIX_FMT_RGB24,
c->width, c->height,
AV_PIX_FMT_YUV420P,
0, 0, 0, 0);
uint8_t *inData[4] = { rgb24Data, 0, 0, 0 };
int inLinesize[4] = { 3 * c->width, 0, 0, 0 };
sws_scale(ctx, inData, inLinesize, 0, c->height,
frame->data, frame->linesize);
Input frame
The sws_scale function uses aligned memory for H/W acceleration (SIMD, etc.). So it is best to allocate memory with a function like av_frame_get_buffer and use that as the buffer.
static AVFrame *alloc_picture(enum AVPixelFormat pix_fmt, int width, int height)
{
AVFrame *picture;
int ret;
picture = av_frame_alloc();
if (!picture)
return NULL;
picture->format = pix_fmt;
picture->width = width;
picture->height = height;
/* allocate the buffers for the frame data */
ret = av_frame_get_buffer(picture, 32);
if (ret < 0) {
fprintf(stderr, "Could not allocate frame data.\n");
exit(1);
}
return picture;
}
// AVFrame -> input_frame
// source data -> frame
int const frame_size = frame_height * frame_width * frame_channels;
memcpy(input_frame->data[0], frame->data, frame_size);
sws_scale(context, input_frame->data, input_frame->linesize, 0, height, scaled_frame.getDatas(), scaled_frame.getLineSizes());
YUV420P to RGB24
When FFMPEG decodes a video frame, it decodes to the YUV420p format by default, so it has to be converted to RGB24 to be displayed in a Win32 environment.
yuv420p_to_rgb24 usually uses img_convert(), but if FFMPEG was compiled with the --enable-swscale option, you must use the sws_scale() function.
# img_convert()
# img_convert ( (AVPicture *)frameRGB, PIX_FMT_RGB24, (AVPicture*)frame,
is->video_st->codec->pix_fmt, is->video_st->codec->width, is->video_st->codec->height);
# sws_getContext(), sws_scale()
# static struct SwsContext *img_convert_ctx;
# img_convert_ctx = sws_getContext (is->video_st->codec->width,
is->video_st->codec->height, is->video_st->codec->pix_fmt,
is->video_st->codec->width, is->video_st->codec->height,
PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);
# sws_scale (img_convert_ctx, frame->data, frame->linesize, 0, is->video_st->codec->height, frameRGB->data, frameRGB->linesize);
Troubleshooting
deprecated pixel format used, make sure you did set range correctly
This message is defined in the sws_init_context function in the libswscale/utils.c file, as follows.
av_cold int sws_init_context(SwsContext *c, SwsFilter *srcFilter,
SwsFilter *dstFilter)
{
// ...
enum AVPixelFormat srcFormat = c->srcFormat;
enum AVPixelFormat dstFormat = c->dstFormat;
// ...
if(srcFormat!=c->srcFormat || dstFormat!=c->dstFormat)
av_log(c, AV_LOG_WARNING, "deprecated pixel format used, make sure you did set range correctly\n");
// ...
}
It seems you're trying to read AV_PIX_FMT_YUVJXXXP frames which are deprecated (see the libav doc). You can use this workaround to manage it :
AVPixelFormat pixFormat;
switch (_videoStream->codec->pix_fmt) {
case AV_PIX_FMT_YUVJ420P :
pixFormat = AV_PIX_FMT_YUV420P;
break;
case AV_PIX_FMT_YUVJ422P :
pixFormat = AV_PIX_FMT_YUV422P;
break;
case AV_PIX_FMT_YUVJ444P :
pixFormat = AV_PIX_FMT_YUV444P;
break;
case AV_PIX_FMT_YUVJ440P :
pixFormat = AV_PIX_FMT_YUV440P;
break;
default:
pixFormat = _videoStream->codec->codec->pix_fmts;
break;
}