2

我正在设计一个管道，将来自 opencv 应用程序（从网络摄像头获取）的视频帧编码为 video/x-h264 格式，通过网络发送并在另一个不同类型的设备（可能是树莓派）上解码到适合我的项目的 RGB 流。

为此，我应该使用硬件加速的编码器和解码器。由于整个场景非常庞大，目前的开发是在 Intel 机器上使用 gstreamer VAAPI 插件（vaapiencode_h264 & vaapidecode）进行的。此外，我们不需要使用任何网络插件，如 TCPServer 或 UDPServer

为此，我将以下管道用于我的目的：在编码器端：

appsrc name=applicationSource ! videoconvert ! video/x-raw, format=I420, width=640, height=480,framerate=30/1, pixel-aspect-ratio=1/1,interlace-mode=progressive ! vaapiencode_h264 bitrate=600 tune=high-compression ! h264parse config-interval=1 ! appsink name=applicationSink sync=false

Appsrc 部分运行良好，而 appsink 部分存在一些问题。

此管道的应用程序部分已设置为以下上限：

"video/x-h264, format=(string){avc,avc3,byte-stream },alignment=(string){au,nal};video/mpeg, mpegversion=(int)2, profile=(string)simple "

我的appsink的数据提取代码是

    bool HWEncoder::grabData()
{

    // initial checks..

    if (!cameraPipeline)
    {

        GST_ERROR("ERROR AS TO NO PIPE FOUND ... Stopping FRAME GRAB HERE !! ");
        return false;
    }


    if (gst_app_sink_is_eos (GST_APP_SINK(applicationSink)))
    {

        GST_WARNING("APP SINK GAVE US AN EOS! BAILING OUT ");
        return false;
    }

    if (sample)
    {
        cout << "sample available ... unrefing it ! "<< endl;
        gst_sample_unref(sample);
    }


    sample = gst_app_sink_pull_sample (GST_APP_SINK(applicationSink));

    if (!sample)
    {
        GST_WARNING("No valid sample");
        return false; // no valid sample pulled !
    }

    sink_buffer = gst_sample_get_buffer(sample);

    if (!sink_buffer)
    {
        GST_ERROR("No Valid Buffer ");return false;
    }

    return true;
}

在启动管道并检查我的应用程序接收器中填充的缓冲区后，我无限期地陷入了下面所说的代码行：

sample = gst_app_sink_pull_sample (GST_APP_SINK(applicationSink));

我有以下问题：1）我的appsink Caps 是否正确？如果不是，我如何确定他们的上限？2）我上面的管道有问题吗？

如何使用 Appsink 解决此问题？

任何形式的帮助都会很有用！

谢谢 ！！

4

1 回答 1

1

Just a guess (I had similar problems) the problem having appsink and appsrc in same pipeline may be that when you fill/empty one of them it will block the other(more on that below).

appsink and appsrc would block when they are full/empty - this is normal desired behaviour. There is option drop for appsink or for appsrc there is option block - but using these it may be just workaround and you will get glitches in your stream. Proper solution is to handle the synchronisation between appsrc and appsink in a better way.

You can react on appsrc signals enough-data and need-data - this is our way. Also we fiddled with properties of appsrc: is-live, do-timestamp and buffer size (this may or may not help you):

g_object_set(src->appsrc,
    "stream-type", GST_APP_STREAM_TYPE_STREAM,
    "format", GST_FORMAT_TIME,
    "do-timestamp", TRUE,
    "is-live", TRUE,
    "block", TRUE,
    NULL);

Why do they block each other? Because (I guess) you process appsink and at the same time appsrc in main application thread. When one of the appsink/appsrc block the thread there is no one that would handle the processing for the other one. So when appsink is blocked because it does not have any data there is noone that can feed appsrc with new data - thus endless deadlock.

We also implemented noblock version of appsink *pull_sample method but it was just a workaround and resulted in more problems than solutions.

If you want to debug what is happening you can add GST_DEBUG entry for appsrc/appsink (I do not remember what they were), you can add callback on mentioned enough-data and need-data signals or you may add queues and enable GST_DEBUG=queue_dataflow:5 to see which queue is filled first etc.. this is always helpful when debugging the "data-deadlock".

于 2015-11-27T08:22:05.870 回答