9

On Android, I'm trying to perform some OpenGL processing on camera frames, show those frames in the camera preview and then encode the frames in a video file. I'm trying to do this with OpenGL, using the GLSurfaceView and GLSurfaceView.Renderer and with FFMPEG for video encoding.

I've successfully processed the image frames using a shader. Now I need to encode the processed frames to video. The GLSurfaceView.Renderer provides the onDrawFrame(GL10 ..) method. It's in this method that I'm attempting to read the image frames using just glReadPixels() and then place the frames on a queue for encoding to video. On it's own, glReadPixels() is much too slow - my frame rate is in the single digits. I'm attempting to speed this up using Pixel Buffer Objects. This is not working. After plugging in the pbo, the frame rate is unchanged. This is my first time using OpenGL and I do not know where to begin looking for the problem. Am I doing this right? Can anyone give me some direction? Thanks in advance.

public class MainRenderer implements GLSurfaceView.Renderer, SurfaceTexture.OnFrameAvailableListener {

.
.

    public void onDrawFrame ( GL10 gl10 ) {

        //Create a buffer to hold the image frame     
        ByteBuffer byte_buffer = ByteBuffer.allocateDirect(this.width * this.height * 4);
        byte_buffer.order(ByteOrder.nativeOrder());

        //Generate a pointer to the frame buffers
        IntBuffer image_buffers = IntBuffer.allocate(1); 
        GLES20.glGenBuffers(1, image_buffers);

        //Create the buffer
        GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, image_buffers.get(0));
        GLES20.glBufferData(GLES20.GL_ARRAY_BUFFER, byte_buffer.limit(), byte_buffer, GLES20.GL_STATIC_DRAW);
        GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, image_buffers.get(0));

        //Read the pixel data into the buffer
        gl10.glReadPixels(0, 0, this.width, this.height, GL10.GL_RGBA, GL10.GL_UNSIGNED_BYTE, byte_buffer);

        //encode the frame to video
        enQueueForEncoding(byte_buffer);

        //unbind the buffer
        GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);
    }

.
.

}
4

2 回答 2

1

我以前从未尝试过这样的事情（opengl+video enconding），但我可以告诉你从设备内存读取速度很慢。尝试双缓冲，这可能会有所帮助，因为 GPU 可以在 DMA 控制器读回内容时继续渲染到第二个缓冲区。

加载分析器（检查您设备的 GPU 供应商），这可能会给您一些想法。另一件可能有帮助的事情是将内部 pbuffer 格式设置为其他格式，尝试使用较小的数字并删除通道 (alpha)。

编辑：如果您有这种感觉，您可以在 GPU 上对视频进行编码，这将提高您的应用程序的内存和处理能力。

于 2013-05-06T19:12:50.947 回答
1

我记得glBufferData()不是将您的内部缓冲区映射到 GPU 内存，它只是将数据从您的内存复制到缓冲区（初始化）。

要访问由 分配的内存，glBufferData()您应该使用glMapBufferRange(). 该函数返回一个您可以读取的 Java Buffer 对象。

于 2014-07-09T09:50:51.180 回答