33

考虑到以下逻辑(据我了解),我编写了从 YUV_420_888 到 Bitmap 的转换:

在此处输入图像描述

总结该方法:内核的坐标 x 和 y 与 Y 平面(2d 分配)的非填充部分的 x 和 y 以及输出位图的 x 和 y 一致。然而,U 平面和 V 平面具有与 Y 平面不同的结构,因为它们使用 1 个字节来覆盖 4 个像素,此外,它们的 PixelStride 可能大于 1,此外它们可能也有一个可以与 Y 平面不同的填充。因此,为了让内核有效地访问 U 和 V,我将它们放入一维分配中并创建了一个索引“uvIndex”,它给出了对应的 U 和 V 在该一维分配中的位置,对于给定的 ( x,y) 在(未填充的)Y 平面(以及输出位图)中的坐标。

为了保持 rs-Kernel 精简,我通过 LaunchOptions 限制 x 范围,排除了 yPlane 中的填充区域(这反映了 y 平面的 RowStride,因此可以在内核中忽略)。所以我们只需要考虑 uvIndex 中的 uvPixelStride 和 uvRowStride,即用于访问 u 值和 v 值的索引。

这是我的代码:

渲染脚本内核,命名为 yuv420888.rs

  #pragma version(1)
  #pragma rs java_package_name(com.xxxyyy.testcamera2);
  #pragma rs_fp_relaxed

  int32_t width;
  int32_t height;

  uint picWidth, uvPixelStride, uvRowStride ;
  rs_allocation ypsIn,uIn,vIn;

 // The LaunchOptions ensure that the Kernel does not enter the padding  zone of Y, so yRowStride can be ignored WITHIN the Kernel.
 uchar4 __attribute__((kernel)) doConvert(uint32_t x, uint32_t y) {

 // index for accessing the uIn's and vIn's
uint uvIndex=  uvPixelStride * (x/2) + uvRowStride*(y/2);

// get the y,u,v values
uchar yps= rsGetElementAt_uchar(ypsIn, x, y);
uchar u= rsGetElementAt_uchar(uIn, uvIndex);
uchar v= rsGetElementAt_uchar(vIn, uvIndex);

// calc argb
int4 argb;
    argb.r = yps + v * 1436 / 1024 - 179;
    argb.g =  yps -u * 46549 / 131072 + 44 -v * 93604 / 131072 + 91;
    argb.b = yps +u * 1814 / 1024 - 227;
    argb.a = 255;

uchar4 out = convert_uchar4(clamp(argb, 0, 255));
return out;
}

Java端:

    private Bitmap YUV_420_888_toRGB(Image image, int width, int height){
    // Get the three image planes
    Image.Plane[] planes = image.getPlanes();
    ByteBuffer buffer = planes[0].getBuffer();
    byte[] y = new byte[buffer.remaining()];
    buffer.get(y);

    buffer = planes[1].getBuffer();
    byte[] u = new byte[buffer.remaining()];
    buffer.get(u);

    buffer = planes[2].getBuffer();
    byte[] v = new byte[buffer.remaining()];
    buffer.get(v);

    // get the relevant RowStrides and PixelStrides
    // (we know from documentation that PixelStride is 1 for y)
    int yRowStride= planes[0].getRowStride();
    int uvRowStride= planes[1].getRowStride();  // we know from   documentation that RowStride is the same for u and v.
    int uvPixelStride= planes[1].getPixelStride();  // we know from   documentation that PixelStride is the same for u and v.


    // rs creation just for demo. Create rs just once in onCreate and use it again.
    RenderScript rs = RenderScript.create(this);
    //RenderScript rs = MainActivity.rs;
    ScriptC_yuv420888 mYuv420=new ScriptC_yuv420888 (rs);

    // Y,U,V are defined as global allocations, the out-Allocation is the Bitmap.
    // Note also that uAlloc and vAlloc are 1-dimensional while yAlloc is 2-dimensional.
    Type.Builder typeUcharY = new Type.Builder(rs, Element.U8(rs));

    //using safe height
    typeUcharY.setX(yRowStride).setY(y.length / yRowStride);

    Allocation yAlloc = Allocation.createTyped(rs, typeUcharY.create());
    yAlloc.copyFrom(y);
    mYuv420.set_ypsIn(yAlloc);

    Type.Builder typeUcharUV = new Type.Builder(rs, Element.U8(rs));
    // note that the size of the u's and v's are as follows:
    //      (  (width/2)*PixelStride + padding  ) * (height/2)
    // =    (RowStride                          ) * (height/2)
    // but I noted that on the S7 it is 1 less...
    typeUcharUV.setX(u.length);
    Allocation uAlloc = Allocation.createTyped(rs, typeUcharUV.create());
    uAlloc.copyFrom(u);
    mYuv420.set_uIn(uAlloc);

    Allocation vAlloc = Allocation.createTyped(rs, typeUcharUV.create());
    vAlloc.copyFrom(v);
    mYuv420.set_vIn(vAlloc);

    // handover parameters
    mYuv420.set_picWidth(width);
    mYuv420.set_uvRowStride (uvRowStride);
    mYuv420.set_uvPixelStride (uvPixelStride);

    Bitmap outBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    Allocation outAlloc = Allocation.createFromBitmap(rs, outBitmap, Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT);

    Script.LaunchOptions lo = new Script.LaunchOptions();
    lo.setX(0, width);  // by this we ignore the y’s padding zone, i.e. the right side of x between width and yRowStride
    //using safe height
    lo.setY(0, y.length / yRowStride);

    mYuv420.forEach_doConvert(outAlloc,lo);
    outAlloc.copyTo(outBitmap);

    return outBitmap;
}

在 Nexus 7 (API 22) 上进行测试会返回漂亮的彩色位图。然而,该设备具有微不足道的像素步幅 (=1) 并且没有填充(即行步幅=宽度)。在全新的三星 S7 (API 23) 上进行测试,我得到的图片颜色不正确——除了绿色。但图片并没有显示出对绿色的普遍偏见,只是似乎没有正确再现非绿色。请注意,S7 应用 u/v 像素步幅为 2,并且没有填充。

由于最关键的代码行在 rs 代码中,因此我认为 u/v 平面的访问 uint uvIndex= (...) 可能存在问题,可能是此处对像素步幅的考虑不正确。有没有人看到解决方案?谢谢。

更新:我检查了一切,我很确定有关 y,u,v 访问的代码是正确的。所以问题一定出在 u 和 v 值本身。非绿色有紫色的倾斜,从 u,v 值来看,它们似乎在大约 110-150 的相当窄的范围内。我们真的有可能需要处理特定于设备的 YUV -> RBG 转换......?!我错过了什么吗?

更新 2:已更正代码,现在可以使用,感谢 Eddy 的反馈。

4

8 回答 8

9

看着

floor((float) uvPixelStride*(x)/2)

它从 Y x 坐标计算您的 U、V 行偏移量 (uv_row_offset)。

如果 uvPixelStride = 2,那么随着 x 的增加:

x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 1
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 3

这是不正确的。在 uv_row_offset = 1 或 3 处没有有效的 U/V 像素值,因为 uvPixelStride = 2。

你要

uvPixelStride * floor(x/2)

(假设你不相信自己记住整数除法的关键舍入行为,如果你这样做的话):

uvPixelStride * (x/2)

应该够了

这样,您的映射将变为:

x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 0
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 2

看看这是否解决了颜色错误。实际上,此处不正确的寻址意味着所有其他颜色样本都来自错误的颜色平面,因为底层 YUV 数据很可能是半平面的(因此 U 平面从 V 平面 + 1 字节开始,两个平面交错)

于 2016-03-25T19:25:43.570 回答
5

对于遇到错误的人

android.support.v8.renderscript.RSIllegalArgumentException: Array too small for allocation type

使用buffer.capacity()而不是buffer.remaining()

如果您已经对图像进行了一些操作,则需要rewind()在缓冲区上调用方法。

于 2016-12-21T11:09:03.283 回答
4

此外,对于其他任何人来说

android.support.v8.renderscript.RSIllegalArgumentException:数组对于分配类型来说太小了

我通过更改yAlloc.copyFrom(y);yAlloc.copy1DRangeFrom(0, y.length, y);

于 2017-02-02T09:27:28.333 回答
2

发布转换 YUV->BGR 的完整解决方案(也可用于其他格式)并使用渲染脚本将图像旋转为直立。分配用作输入,字节数组用作输出。它也在包括三星设备在内的 Android 8+ 上进行了测试。

爪哇

/**
 * Renderscript-based process to convert YUV_420_888 to BGR_888 and rotation to upright.
 */
public class ImageProcessor {

    protected final String TAG = this.getClass().getSimpleName();

    private Allocation mInputAllocation;
    private Allocation mOutAllocLand;
    private Allocation mOutAllocPort;

    private Handler mProcessingHandler;
    private ScriptC_yuv_bgr mConvertScript;
    private byte[] frameBGR;

    public ProcessingTask mTask;
    private ImageListener listener;
    private Supplier<Integer> rotation;

    public ImageProcessor(RenderScript rs, Size dimensions, ImageListener listener, Supplier<Integer> rotation) {

        this.listener = listener;
        this.rotation = rotation;
        int w = dimensions.getWidth();
        int h = dimensions.getHeight();

        Type.Builder yuvTypeBuilder = new Type.Builder(rs, Element.YUV(rs));
        yuvTypeBuilder.setX(w);
        yuvTypeBuilder.setY(h);
        yuvTypeBuilder.setYuvFormat(ImageFormat.YUV_420_888);
        mInputAllocation = Allocation.createTyped(rs, yuvTypeBuilder.create(),
                Allocation.USAGE_IO_INPUT | Allocation.USAGE_SCRIPT);

        //keep 2 allocations to handle different image rotations
        mOutAllocLand = createOutBGRAlloc(rs, w, h);
        mOutAllocPort = createOutBGRAlloc(rs, h, w);

        frameBGR = new byte[w*h*3];

        HandlerThread processingThread = new HandlerThread(this.getClass().getSimpleName());
        processingThread.start();
        mProcessingHandler = new Handler(processingThread.getLooper());

        mConvertScript = new ScriptC_yuv_bgr(rs);
        mConvertScript.set_inWidth(w);
        mConvertScript.set_inHeight(h);

        mTask = new ProcessingTask(mInputAllocation);
    }

    private Allocation createOutBGRAlloc(RenderScript rs, int width, int height) {
        //Stored as Vec4, it's impossible to store as Vec3, buffer size will be for Vec4 anyway
        //using RGB_888 as alternative for BGR_888, can be just U8_3 type
        Type.Builder rgbTypeBuilderPort = new Type.Builder(rs, Element.RGB_888(rs));
        rgbTypeBuilderPort.setX(width);
        rgbTypeBuilderPort.setY(height);
        Allocation allocation = Allocation.createTyped(
            rs, rgbTypeBuilderPort.create(), Allocation.USAGE_SCRIPT
        );
        //Use auto-padding to be able to copy to x*h*3 bytes array
        allocation.setAutoPadding(true);

        return allocation;
    }

    public Surface getInputSurface() {
        return mInputAllocation.getSurface();
    }

    /**
     * Simple class to keep track of incoming frame count,
     * and to process the newest one in the processing thread
     */
    class ProcessingTask implements Runnable, Allocation.OnBufferAvailableListener {

        private int mPendingFrames = 0;

        private Allocation mInputAllocation;

        public ProcessingTask(Allocation input) {
            mInputAllocation = input;
            mInputAllocation.setOnBufferAvailableListener(this);
        }

        @Override
        public void onBufferAvailable(Allocation a) {
            synchronized(this) {
                mPendingFrames++;
                mProcessingHandler.post(this);
            }
        }

        @Override
        public void run() {
            // Find out how many frames have arrived
            int pendingFrames;
            synchronized(this) {
                pendingFrames = mPendingFrames;
                mPendingFrames = 0;

                // Discard extra messages in case processing is slower than frame rate
                mProcessingHandler.removeCallbacks(this);
            }

            // Get to newest input
            for (int i = 0; i < pendingFrames; i++) {
                mInputAllocation.ioReceive();
            }

            int rot = rotation.get();

            mConvertScript.set_currentYUVFrame(mInputAllocation);
            mConvertScript.set_rotation(rot);

            Allocation allocOut = rot==90 || rot== 270 ? mOutAllocPort : mOutAllocLand;

            // Run processing
            // ain allocation isn't really used, global frame param is used to get data from
            mConvertScript.forEach_yuv_bgr(allocOut);

            //Save to byte array, BGR 24bit
            allocOut.copyTo(frameBGR);

            int w = allocOut.getType().getX();
            int h = allocOut.getType().getY();

            if (listener != null) {
                listener.onImageAvailable(frameBGR, w, h);
            }
        }
    }

    public interface ImageListener {

        /**
         * Called when there is available image, image is in upright position.
         *
         * @param bgr BGR 24bit bytes
         * @param width image width
         * @param height image height
         */
        void onImageAvailable(byte[] bgr, int width, int height);
    }
}

RS

#pragma version(1)
#pragma rs java_package_name(com.affectiva.camera)
#pragma rs_fp_relaxed

//Script convers YUV to BGR(uchar3)

//current YUV frame to read pixels from
rs_allocation currentYUVFrame;

//input image rotation: 0,90,180,270 clockwise
uint32_t rotation;
uint32_t inWidth;
uint32_t inHeight;

//method returns uchar3  BGR which will be set to x,y in output allocation
uchar3 __attribute__((kernel)) yuv_bgr(uint32_t x, uint32_t y) {

    // Read in pixel values from latest frame - YUV color space

    uchar3 inPixel;
    uint32_t xRot = x;
    uint32_t yRot = y;

    //Do not rotate if 0
    if (rotation==90) {
      //rotate 270 clockwise
      xRot = y;
      yRot = inHeight - 1 - x;
    } else if (rotation==180) {
      xRot = inWidth - 1 - x;
      yRot = inHeight - 1 - y;
    } else if (rotation==270) {
      //rotate 90 clockwise
      xRot = inWidth - 1 - y;
      yRot = x;
    }

    inPixel.r = rsGetElementAtYuv_uchar_Y(currentYUVFrame, xRot, yRot);
    inPixel.g = rsGetElementAtYuv_uchar_U(currentYUVFrame, xRot, yRot);
    inPixel.b = rsGetElementAtYuv_uchar_V(currentYUVFrame, xRot, yRot);

    // Convert YUV to RGB, JFIF transform with fixed-point math
    // R = Y + 1.402 * (V - 128)
    // G = Y - 0.34414 * (U - 128) - 0.71414 * (V - 128)
    // B = Y + 1.772 * (U - 128)

    int3 bgr;
    //get red pixel and assing to b
    bgr.b = inPixel.r +
            inPixel.b * 1436 / 1024 - 179;
    bgr.g = inPixel.r -
            inPixel.g * 46549 / 131072 + 44 -
            inPixel.b * 93604 / 131072 + 91;
    //get blue pixel and assign to red
    bgr.r = inPixel.r +
            inPixel.g * 1814 / 1024 - 227;

    // Write out
    return convert_uchar3(clamp(bgr, 0, 255));
}
于 2019-12-19T06:19:55.013 回答
1

在三星 Galaxy Tab 5(平板电脑)、android 版本 5.1.1 (22) 上,据称 YUV_420_888 格式,以下渲染脚本数学运行良好并产生正确的颜色:

uchar yValue    = rsGetElementAt_uchar(gCurrentFrame, x + y * yRowStride);
uchar vValue    = rsGetElementAt_uchar(gCurrentFrame, ( (x/2) + (y/4) * yRowStride ) + (xSize * ySize) );
uchar uValue    = rsGetElementAt_uchar(gCurrentFrame, ( (x/2) + (y/4) * yRowStride ) + (xSize * ySize) + (xSize * ySize) / 4);

我不明白为什么水平值(即y)按四倍而不是二倍缩放,但效果很好。我还需要避免使用 rsGetElementAtYuv_uchar_Y|U|V。我相信相关的分配步幅值设置为零而不是适当的值。使用 rsGetElementAt_uchar() 是一种合理的解决方法。

在三星 Galaxy S5(智能手机),android 版本 5.0 (21) 上,据称 YUV_420_888 格式,我无法恢复 u 和 v 值,它们全为零。这会产生一个绿色的图像。Luminous 可以,但图像垂直翻转。

于 2016-04-04T20:05:02.600 回答
1

此代码需要使用 RenderScript 兼容性库 (android.support.v8.renderscript.*)。

为了使兼容性库与 Android API 23 一起使用,我根据王淼在如何在 Android Studio 上创建 Renderscript 脚本并使其运行?

如果您按照他的回答并收到错误“需要 Gradle 版本 2.10”,请不要更改

classpath 'com.android.tools.build:gradle:2.1.0'

相反,将 Project\gradle\wrapper\gradle-wrapper.properties 文件的 distributionUrl 字段更新为

distributionUrl=https\://services.gradle.org/distributions/gradle-2.10-all.zip

并更改File > Settings > Builds,Execution,Deployment > Build Tools > Gradle > Gradle使用默认的 gradle 包装器,按照“需要 Gradle 版本 2.10”。错误

于 2016-05-11T18:08:31.060 回答
0

回复:RSIllegalArgumentException

在我的情况下,buffer.remaining() 不是步幅的倍数:最后一行的长度小于步幅(即仅达到实际数据所在的位置。)

于 2019-07-23T06:22:17.213 回答
0

仅供参考,以防其他人在尝试代码时收到“android.support.v8.renderscript.RSIllegalArgumentException: Array too small for allocation type”。就我而言,事实证明,在为 Y 分配缓冲区时,我不得不倒带缓冲区,因为它被留在了错误的一端并且没有复制数据。通过执行 buffer.rewind(); 在分配新字节数组之前,它现在可以正常工作了。

于 2020-08-27T17:18:42.113 回答