3

我得到了一个解码的AVFrame，其格式显示为 160/ Videotoolbox_vld。在搜索了一些文章（此处）并查看了FFmpeg源代码（此处和此处）后，CVBuffer句柄应位于AVFrame.data[3]. 但CVBuffer我得到的似乎无效，任何CVPixelBufferGetXXX()函数都返回 0 或 nil。

如果我使用av_hwframe_transfer_data()ffmpeg 的example/hw_decode.c所做的那样，则可以将样本从 HW 下载到 SW 缓冲区。它AVFrame.format会的nv12。通过 转换sws_scale为后bgra，样本可以以正确的内容显示在视图中。

我认为 VideoToolbox 解码帧是可以的。我转换的方式AVFrame.data[3]可能CVBuffer是错误的。刚刚学会了快速访问 c 指针，但我不确定如何正确读取指针中的资源句柄（CVBuffer）。

以下是我尝试从 AVFrame 中提取 CVBuffer 的方法

var pFrameOpt: UnsafeMutablePointer<AVFrame>? = av_frame_alloc()
avcodec_receive_frame(..., pFrameOpt)

let data3: UnsafeMutablePointer<UInt8>? = pFrameOpt?.pointee.data.3
data3?.withMemoryRebound(to: CVBuffer.self, capacity: 1) { pCvBuf in
    let fW = pFrameOpt!.pointee.width // print 3840
    let fH = pFrameOpt!.pointee.height // print 2160
    let fFmt = pFrameOpt!.pointee.format // print 160

    let cvBuf: CVBuffer = pCvBuf.pointee

    let a1 = CVPixelBufferGetDataSize(cvBuf)               // print 0
    let a2 = CVPixelBufferGetPixelFormatType(cvBuf)        // print 0
    let a3 = CVPixelBufferGetWidth(cvBuf)                  // print 0
    let a4 = CVPixelBufferGetHeight(cvBuf)                 // print 0
    let a5 = CVPixelBufferGetBytesPerRow(cvBuf)            // print 0
    let a6 = CVPixelBufferGetBytesPerRowOfPlane(cvBuf, 0)  // print 0
    let a7 = CVPixelBufferGetWidthOfPlane(cvBuf, 0)        // print 0
    let a8 = CVPixelBufferGetHeightOfPlane(cvBuf, 0)       // print 0
    let a9 = CVPixelBufferGetPlaneCount(cvBuf)             // print 0
    let a10 = CVPixelBufferIsPlanar(cvBuf)                 // print false
    let a11 = CVPixelBufferGetIOSurface(cvBuf)             // print nil
    let a12 = CVPixelBufferGetBaseAddress(cvBuf)           // print nil
    let a13 = CVPixelBufferGetBaseAddressOfPlane(cvBuf, 0) // print nil

    let b1 = CVImageBufferGetCleanRect(cvBuf)   // print 0, 0, 0, 0
    let b2 = CVImageBufferGetColorSpace(cvBuf)  // print nil 
    let b3 = CVImageBufferGetDisplaySize(cvBuf) // print 0, 0, 0, 0
    let b4 = CVImageBufferGetEncodedSize(cvBuf) // print 0, 0, 0, 0
    let b5 = CVImageBufferIsFlipped(cvBuf)      // print false

    // bad exec
    var cvTextureOut: CVMetalTexture?
    CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, ..., cvBuf, nil, .bgra8Unorm, 3840, 2160, 0, ...) 


}
4

4 回答 4

1

CVBuffer不是固定大小，因此重新绑定内存不会以这种方式工作。你需要这样做：

Unmanaged<CVBuffer>.fromOpaque(data!).takeRetainedValue()

但是，底线是 FFmpeg 的 VideoToolbox 后端没有创建一个CVPixelBufferwith kCVPixelBufferMetalCompatibilityKeyset to true。CVMetalTextureCacheCreateTextureFromImage(...)无论如何，您将无法成功调用。

您可以考虑使用CVPixelBufferPool适当的设置（包括kCVPixelBufferMetalCompatibilityKey设置为true），然后使用VTPixelTransferSession快速将FFmpeg的像素缓冲区复制到您自己的。

于 2019-11-15T23:41:13.880 回答
0 
/**
    @function   CVPixelBufferGetBaseAddressOfPlane
    @abstract   Returns the base address of the plane at planeIndex in the PixelBuffer.
    @discussion Retrieving the base address for a PixelBuffer requires that the buffer base address be locked
                via a successful call to CVPixelBufferLockBaseAddress. On OSX 10.10 and earlier, or iOS 8 and
                earlier, calling this function with a non-planar buffer will have undefined behavior.
    @param      pixelBuffer Target PixelBuffer.
    @param      planeIndex  Identifying the plane.
    @result     Base address of the plane, or NULL for non-planar CVPixelBufferRefs.
*/
@available(iOS 4.0, *)
public func CVPixelBufferGetBaseAddressOfPlane(_ pixelBuffer: CVPixelBuffer, _ planeIndex: Int) -> UnsafeMutableRawPointer?

也许您可以在使用 CVPixelBufferGetBaseAddressOfPlane 之前尝试使用 CVPixelBufferLockBaseAddress

于 2021-03-08T07:35:29.073 回答
0

尝试这个

let cvBuf: CVBuffer = Array(UnsafeMutableBufferPointer(start: data3, count: 3))
.withUnsafeBufferPointer {
   $0.baseAddress!.withMemoryRebound(to: CVBuffer.self, capacity: 1) { $0 }
}.pointee

甚至可能

let cvBuf: CVBuffer = unsafeBitcast(UnsafeMutableBufferPointer(start: data3, count: 3), to: CVBuffer.self)
于 2019-08-23T18:36:16.213 回答
0

似乎我错误地投射void*到CVPixelBuffer*而不是void*直接投射到CVPixelBuffer. 我找不到一种快速的方法来进行从指针到数值的这种 c 风格转换。（使用as! CVPixelBuffer导致崩溃）。

所以我在 C 代码中为void*to创建了一个函数CVPixelBufferRef来完成这样的转换工作。

// util.h
#include <CoreVideo/CVPixelBuffer.h>
CVPixelBufferRef CastToCVPixelBuffer(void* p);

// util.c
CVPixelBufferRef CastToCVPixelBuffer(void* p)
{
    return (CVPixelBufferRef)p;
}

// BridgeHeader.h
#include "util.h"

然后传入UnsafeMutablePointer<UInt8>，得到CVPixelBuffer句柄。

let pFrameOpt: UnsafeMutablePointer<AVFrame>? = ...
let data3: UnsafeMutablePointer<UInt8>? = pFrameOpt?.pointee.data.3

let cvBuf: CVBuffer = CastToCVPixelBuffer(data3).takeUnretainedValue()

let width = CVPixelBufferGetWidth(cvBuf)   // print 3840
let height = CVPixelBufferGetHeight(cvBuf) // print 2160
于 2019-08-13T04:51:18.847 回答