5

我想用 AVSampleBufferDisplayLayer 显示一些 CMSampleBuffer，但在显示第一个样本后它会冻结。

我从 AVCaptureVideoDataOutputSampleBuffer 委托中获取样本缓冲区：

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CFRetain(sampleBuffer);
    [self imageToBuffer:sampleBuffer];
    CFRelease(sampleBuffer);
}

将它们放入向量中

-(void) imageToBuffer: (CMSampleBufferRef )source{
//buffers is defined as: std::vector<CMSampleBufferRef> buffers;
        CMSampleBufferRef newRef;
        CMSampleBufferCreateCopy(kCFAllocatorDefault, source, &newRef);
        buffers.push_back(newRef);
}

然后尝试通过 AVSampleBufferDisplayLayer 显示它们（在另一个 ViewController 中）

AVSampleBufferDisplayLayer * displayLayer = [[AVSampleBufferDisplayLayer alloc] init];

    displayLayer.bounds = self.view.bounds;
    displayLayer.position = CGPointMake(CGRectGetMidX(self.displayOnMe.bounds), CGRectGetMidY(self.displayOnMe.bounds));
    displayLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
    displayLayer.backgroundColor = [[UIColor greenColor] CGColor];

    [self.view.layer addSublayer:displayLayer];
    self.view.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;

    dispatch_queue_t queue = dispatch_queue_create("My queue", DISPATCH_QUEUE_SERIAL);
    [displayLayer setNeedsDisplay];
    [displayLayer requestMediaDataWhenReadyOnQueue:queue
                                        usingBlock:^{
                                            while ([displayLayer isReadyForMoreMediaData]) {

                                                if (samplesKey < buffers.size()) {
                                                    CMSampleBufferRef buf = buffers[samplesKey];
                                                    [displayLayer enqueueSampleBuffer:buffers[samplesKey]];
                                                    samplesKey++;

                                                }else
                                                {
                                                    [displayLayer stopRequestingMediaData];
                                                    break;
                                                }
                                            }

                                        }];

但它显示第一个样本然后冻结，并且什么都不做。

而我的视频数据输出设置如下：

//set up our output
self.videoDataOutput = [[AVCaptureVideoDataOutput alloc] init];
dispatch_queue_t queue = dispatch_queue_create("VideoQueue", DISPATCH_QUEUE_SERIAL);
[_videoDataOutput setSampleBufferDelegate:self queue:queue];
[_videoDataOutput setVideoSettings:[NSDictionary dictionaryWithObjectsAndKeys:
                                                [NSNumber numberWithInt:kCVPixelFormatType_32BGRA],(id)kCVPixelBufferPixelFormatTypeKey,
                                                nil]];
4

1 回答 1

5

我在相同的上下文中遇到了这个问题，试图从 AVCaptureVideoDataOutput 获取输出并将其显示在 AVSampleDisplay 层中。

如果您的帧按显示顺序显示，那么修复非常简单，只需在 CMSampleBufferRef 上设置立即显示标志。

获取委托返回的样本缓冲区，然后...

CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);
CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);

CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);

如果您的帧以编码器顺序（而不是显示顺序）出现，则 CMSampleBuffer 上的时间戳需要零偏差并重新加盖，以使第一帧时间戳等于时间 0。

 double pts = CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(sampleBuffer));

 // ptsStart is equal to the first frames presentationTimeStamp so playback starts from time 0.
 CMTime presentationTimeStamp = CMTimeMake((pts-ptsStart)*1000000,1000000);

 CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer, presentationTimeStamp);

更新：

当我使用零偏差方法并进一步调查时，我遇到了一些视频仍然无法流畅播放的情况。正确的答案似乎是从您打算播放的第一帧开始使用 PTS。

我的答案在这里，但我也会在这里发布。

设置 AVSampleBufferDisplayLayer 渲染样本缓冲区的速率

时基需要设置为您打算解码的第一帧的演示时间戳 (pts)。通过从所有后续点中减去初始点并将时基设置为 0，我将第一帧的点索引为 0。无论出于何种原因，这不适用于某些视频。

你想要这样的东西（在调用解码之前调用）：

CMTimebaseRef controlTimebase;
CMTimebaseCreateWithMasterClock( CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &controlTimebase );

displayLayer.controlTimebase = controlTimebase;

// Set the timebase to the initial pts here
CMTimebaseSetTime(displayLayer.controlTimebase, CMTimeMake(ptsInitial, 1));
CMTimebaseSetRate(displayLayer.controlTimebase, 1.0);

为 CMSampleBuffer 设置 PTS...

CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer, presentationTimeStamp);

也许确保立即显示没有设置....

CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanFalse);

这在 WWDC 2014 Session 513 中有非常简短的介绍。

于 2015-11-09T19:53:15.117 回答