4

我有一个有趣的问题，我需要研究与非常低级的视频流相关的问题。

有没有人有将原始字节流（分隔为每个像素信息，但不是标准的视频格式）转换为低分辨率视频流的经验？我相信我可以将数据映射为每像素字节的 RGB 值，因为与原始数据中的值对应的颜色值将由我们确定。我不确定从那里去哪里，或者每个像素需要什么 RGB 格式。

我看过 FFMPeg，但它的文档非常庞大，我不知道从哪里开始。

我的具体问题包括，是否可以使用该像素数据创建 CVPixelBuffer？如果我要这样做，我需要将每像素数据转换为哪种格式？

另外，我是否应该更深入地研究 OpenGL，如果是的话，在哪里可以找到有关该主题的信息的最佳位置？

那么 CGBitmapContextCreate 呢？例如，如果我使用类似这样的东西，典型的像素字节需要是什么样的？这是否足够快以将帧速率保持在 20fps 以上？

编辑：

我认为在你们两个的出色帮助下，以及我自己的一些研究，我已经制定了一个计划，如何构建原始 RGBA 数据，然后从该数据构建一个 CGImage，然后从该 CGImage 创建一个 CVPixelBuffer从这里CVPixelBuffer 来自 CGImage。

但是，要在数据输入时实时播放，我不确定我会看到什么样的 FPS。我是否将它们绘制到 CALayer，或者是否有一些与 AVAssetWriter 类似的类，我可以在附加 CVPixelBuffers 时使用它来播放它。我的经验是使用 AVAssetWriter 将构建的 CoreAnimation 层次结构导出到视频，因此视频总是在开始播放之前构建，而不是显示为实时视频。

4

2 回答 2

5

I've done this before, and I know that you found my GPUImage project a little while ago. As I replied on the issues there, the GPUImageRawDataInput is what you want for this, because it does a fast upload of RGBA, BGRA, or RGB data directly into an OpenGL ES texture. From there, the frame data can be filtered, displayed to the screen, or recorded into a movie file.

Your proposed path of going through a CGImage to a CVPixelBuffer is not going to yield very good performance, based on my personal experience. There's too much overhead when passing through Core Graphics for realtime video. You want to go directly to OpenGL ES for the fastest display speed here.

I might even be able to improve my code to make it faster than it is right now. I currently use glTexImage2D() to update texture data from local bytes, but it would probably be even faster to use the texture caches introduced in iOS 5.0 to speed up refreshing data within a texture that maintains its size. There's some overhead in setting up the caches that makes them a little slower for one-off uploads, but rapidly updating data should be faster with them.

于 2013-04-06T16:32:24.600 回答
1

我的 2 美分：

I made an opengl game which lets the user record a 3d scene. Playback was done via replaying the scene (instead of playing a video because realtime encoding did not yield a comfortable FPS.

There is a technique which could help out, unfortunately I didn't have time to implement it: http://allmybrain.com/2011/12/08/rendering-to-a-texture-with-ios-5-texture-cache-api/

This technique should cut down time on getting pixels back from openGL. You might get an acceptable video encoding rate.

于 2013-04-04T04:55:41.020 回答