26

我正在使用 AVFoundation 并从中获取示例缓冲区AVCaptureVideoDataOutput，我可以使用以下方法将其直接写入 videoWriter：

- (void)writeBufferFrame:(CMSampleBufferRef)sampleBuffer {
    CMTime lastSampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);    
    if(self.videoWriter.status != AVAssetWriterStatusWriting)
    {
        [self.videoWriter startWriting];
        [self.videoWriter startSessionAtSourceTime:lastSampleTime];
    }

    [self.videoWriterInput appendSampleBuffer:sampleBuffer];

}

我现在要做的是在 CMSampleBufferRef 内裁剪和缩放图像，而不将其转换为 UIImage 或 CGImageRef，因为这会降低性能。

4

5 回答 5

35

如果您使用 vimage，您可以直接处理缓冲区数据，而无需将其转换为任何图像格式。

outImg包含裁剪和缩放的图像数据。outWidth和之间的关系cropWidth设置缩放。 图像裁剪

int cropX0, cropY0, cropHeight, cropWidth, outWidth, outHeight;

CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);                   
CVPixelBufferLockBaseAddress(imageBuffer,0);
void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
                                    
vImage_Buffer inBuff;                       
inBuff.height = cropHeight;
inBuff.width = cropWidth;
inBuff.rowBytes = bytesPerRow;

int startpos = cropY0*bytesPerRow+4*cropX0;
inBuff.data = baseAddress+startpos;

unsigned char *outImg= (unsigned char*)malloc(4*outWidth*outHeight);
vImage_Buffer outBuff = {outImg, outHeight, outWidth, 4*outWidth};

vImage_Error err = vImageScale_ARGB8888(&inBuff, &outBuff, NULL, 0);
if (err != kvImageNoError) NSLog(@" error %ld", err);

因此，将cropX0 = 0and cropY0 = 0and cropWidthand设置cropHeight为原始大小意味着不裁剪（使用整个原始图像）。设置outWidth = cropWidth并outHeight = cropHeight导致无缩放。请注意，inBuff.rowBytes应该始终是完整源缓冲区的长度，而不是裁剪后的长度。

于 2012-04-08T13:02:35.763 回答
10

注意：我没有注意到原始问题也要求缩放。但无论如何，对于那些只需要裁剪 CMSampleBuffer 的人来说，这就是解决方案。

缓冲区只是一个像素数组，因此您实际上可以直接处理缓冲区而无需使用 vImage。代码是用 Swift 编写的，但我认为很容易找到对应的 Objective-C。

首先，确保您的 CMSampleBuffer 是 BGRA 格式。如果不是，您使用的预设可能是 YUV，并且会破坏以后将使用的每行字节数。

dataOutput = AVCaptureVideoDataOutput()
dataOutput.videoSettings = [
    String(kCVPixelBufferPixelFormatTypeKey): 
    NSNumber(value: kCVPixelFormatType_32BGRA)
]

然后，当您获得样本缓冲区时：

let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!

CVPixelBufferLockBaseAddress(imageBuffer, .readOnly)

let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
let cropWidth = 640
let cropHeight = 640
let colorSpace = CGColorSpaceCreateDeviceRGB()

let context = CGContext(data: baseAddress, width: cropWidth, height: cropHeight, bitsPerComponent: 8, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)
// now the cropped image is inside the context. 
// you can convert it back to CVPixelBuffer 
// using CVPixelBufferCreateWithBytes if you want.

CVPixelBufferUnlockBaseAddress(imageBuffer, .readOnly)

// create image
let cgImage: CGImage = context!.makeImage()!
let image = UIImage(cgImage: cgImage)

如果要从某个特定位置裁剪，请添加以下代码：

// calculate start position
let bytesPerPixel = 4
let startPoint = [ "x": 10, "y": 10 ]
let startAddress = baseAddress + startPoint["y"]! * bytesPerRow + startPoint["x"]! * bytesPerPixel

并将baseAddressin更改CGContext()为startAddress. 确保不要超过原始图像的宽度和高度。

于 2015-07-28T17:45:01.530 回答
9

您可以考虑使用 CoreImage (5.0+)。

CIImage *ciImage = [CIImage imageWithCVPixelBuffer:CMSampleBufferGetImageBuffer(sampleBuffer)
                                           options:[NSDictionary dictionaryWithObjectsAndKeys:[NSNull null], kCIImageColorSpace, nil]];
ciImage = [[ciImage imageByApplyingTransform:myScaleTransform] imageByCroppingToRect:myRect];
于 2011-12-13T18:32:27.970 回答
1

对于扩展，您可以让 AVFoundation 为您执行此操作。在这里查看我最近的帖子。如果图像的尺寸不同，则设置 AVVideoWidth/AVVideoHeight 键的值将缩放图像。看看这里的属性。至于裁剪，我不确定您是否可以让 AVFoundation 为您执行此操作。您可能不得不求助于使用 OpenGL 或 CoreImage。这个SO question的顶部帖子中有几个很好的链接。

于 2011-12-16T03:53:54.210 回答
1

在 Swift3 上试试这个

func resize(_ destSize: CGSize)-> CVPixelBuffer? {
        guard let imageBuffer = CMSampleBufferGetImageBuffer(self) else { return nil }
        // Lock the image buffer
        CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
        // Get information about the image
        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
        let bytesPerRow = CGFloat(CVPixelBufferGetBytesPerRow(imageBuffer))
        let height = CGFloat(CVPixelBufferGetHeight(imageBuffer))
        let width = CGFloat(CVPixelBufferGetWidth(imageBuffer))
        var pixelBuffer: CVPixelBuffer?
        let options = [kCVPixelBufferCGImageCompatibilityKey:true,
                       kCVPixelBufferCGBitmapContextCompatibilityKey:true]
        let topMargin = (height - destSize.height) / CGFloat(2)
        let leftMargin = (width - destSize.width) * CGFloat(2)
        let baseAddressStart = Int(bytesPerRow * topMargin + leftMargin)
        let addressPoint = baseAddress!.assumingMemoryBound(to: UInt8.self)
        let status = CVPixelBufferCreateWithBytes(kCFAllocatorDefault, Int(destSize.width), Int(destSize.height), kCVPixelFormatType_32BGRA, &addressPoint[baseAddressStart], Int(bytesPerRow), nil, nil, options as CFDictionary, &pixelBuffer)
        if (status != 0) {
            print(status)
            return nil;
        }
        CVPixelBufferUnlockBaseAddress(imageBuffer,CVPixelBufferLockFlags(rawValue: 0))
        return pixelBuffer;
    }
于 2017-06-26T15:30:12.107 回答