0

如何通过 TangoService_connectOnFrameAvailable() 保存帧并在我的计算机上正确显示?正如此参考页面所述,像素以 HAL_PIXEL_FORMAT_YV12 格式存储。在我的 TangoService_connectOnFrameAvailable 回调函数中,我像这样保存框架:

static void onColorFrameAvailable(void* context, TangoCameraId id, const TangoImageBuffer* buffer) 
{
  ...
  std::ofstream fp;
  fp.open(imagefile, std::ios::out | std::ios::binary );
  int offset = 0;
  for(int i = 0; i < buffer->height*2 + 1; i++) {
    fp.write((char*)(buffer->data + offset), buffer->width);
    offset += buffer->stride;
  }
  fp.close();
}

然后摆脱第一行中的元数据并显示我运行的图像:

$ dd if="input.raw" of="new.raw" bs=1 skip=1280
$ vooya new.raw

我小心翼翼地确保在 vooya 中频道顺序是 yvu。结果输出是: yv12 测试图片

我在保存图像并显示它时做错了什么?

根据 Mark Mullin 的回复更新:

int offset = buffer->stride; // header offset
// copy Y channel
for(int i = 0; i < buffer->height; i++) {
  fp.write((char*)(buffer->data + offset), buffer->width);
  offset += buffer->stride;
}
// copy V channel
for(int i = 0; i < buffer->height / 2; i++) {
  fp.write((char*)(buffer->data + offset), buffer->width / 2);
  offset += buffer->stride / 2;
}
// copy U channel
for(int i = 0; i < buffer->height / 2; i++) {
  fp.write((char*)(buffer->data + offset), buffer->width / 2);
  offset += buffer->stride / 2;
}

这现在显示了下图,但仍有一些伪影;我想知道这是来自 Tango 平板相机还是我对原始数据的处理……有什么想法吗?

yv12 测试 2

4

2 回答 2

1

不能确切地说出你做错了什么,探戈图像中经常有伪影——你的是新的,但我经常看到淡蓝色作为一种颜色,眩光似乎让更深层次的系统感到恼火,并且随着它开始与在加载深度系统时,您经常会看到看起来像闪亮的网格(我认为它的 IR 模式) - 最后,任何用 openCV 等处理图像的合理尝试都失败了,所以我亲手编写了解码器SO线程的一些帮助here

也就是说,给定的图像缓冲区包含一个指向来自 Tango 的原始数据的指针,并且从回调中接收到的数据中填充了各种其他变量,如高度和步幅,然后这个逻辑将创建一个 RGBA 映射 - 是的,我优化了数学它,所以它有点难看 - 它速度较慢但功能等效的双胞胎排在第二位。我自己的经验表明,尝试在回调中直接进行解码是一个可怕的想法(我相信 Tango 完全出于恶意的原因能够失去与闪光灯的同步),所以我在渲染阶段运行。

快速地

uchar* pData = TangoData::cameraImageBuffer;
uchar* iData = TangoData::cameraImageBufferRGBA;
int size = (int)(TangoData::imageBufferStride * TangoData::imageBufferHeight);
float invByte = 0.0039215686274509803921568627451;  // ( 1 / 255)

int halfi, uvOffset, halfj, uvOffsetHalfj;
float y_scaled, v_scaled, u_scaled;
int uOffset = size / 4 + size;
int halfstride = TangoData::imageBufferStride / 2;
for (int i = 0; i < TangoData::imageBufferHeight; ++i)
{
    halfi = i / 2;
    uvOffset = halfi * halfstride;
    for (int j = 0; j < TangoData::imageBufferWidth; ++j)
    {
        halfj = j / 2;
        uvOffsetHalfj = uvOffset + halfj;
        y_scaled = pData[i * TangoData::imageBufferStride + j] * invByte;
        v_scaled = 2 * (pData[uvOffsetHalfj + size] * invByte - 0.5f) * Vmax;
        u_scaled = 2 * (pData[uvOffsetHalfj + uOffset] * invByte - 0.5f) * Umax;
        *iData++ = (uchar)((y_scaled + 1.13983f * v_scaled) * 255.0);;
        *iData++ = (uchar)((y_scaled - 0.39465f * u_scaled - 0.58060f * v_scaled) * 255.0);
        *iData++ = (uchar)((y_scaled + 2.03211f * u_scaled) * 255.0);
        *iData++ = 255;
    }
}

可以理解

for (int i = 0; i < TangoData::imageBufferHeight; ++i)
{
    for (int j = 0; j < TangoData::imageBufferWidth; ++j)
    {
        uchar y = pData[i * image->stride + j];
        uchar v = pData[(i / 2) * (TangoData::imageBufferStride / 2) + (j / 2) + size];
        uchar u = pData[(i / 2) * (TangoData::imageBufferStride / 2) + (j / 2) + size + (size / 4)];
        YUV2RGB(y, u, v);
        *iData++ = y;
        *iData++ = u;
        *iData++ = v;
        *iData++ = 255;
    }
}
于 2015-01-26T20:55:57.313 回答
0

我认为如果可以离线进行,还有更好的方法。保存图像的最佳方法应该是这样的(不要忘记创建文件夹图片,否则您将不会保存任何内容)

void onFrameAvailableRouter(void* context, TangoCameraId id, const TangoImageBuffer* buffer) {
  //To write the image in a txt file.
  std::stringstream name_stream;
  name_stream.setf(std::ios_base::fixed, std::ios_base::floatfield);
  name_stream.precision(3);
  name_stream << "/storage/emulated/0/Pictures/"
                <<cur_frame_timstamp_
                <<".txt";

  std::fstream f(name_stream.str().c_str(), std::ios::out | std::ios::binary);
  // size = 1280*720*1.5 to save YUV or 1280*720 to save grayscale
  int size = stride_ * height_ * 1.5;
  f.write((const char *) buffer->data,size * sizeof(uint8_t));
  f.close();
}

然后将 .txt 文件转换为 png 你可以这样做

inputFolder = "input"
outputFolderRGB = "output/rgb"
outputFolderGray = "output/gray"

input_filename  = "timestamp.txt"
output_filename = "rgb.png"
allFile = listdir(inputFolder)
numberOfFile = len(allFile)

if "input" in glob.glob("*"):
    if  "output/rgb" in glob.glob("output/*"):
        print ""
    else:
        makedirs("output/rgb")
        if "output/gray" in glob.glob("output/*"):
            print ""
        else:
            makedirs("output/gray")

    #The output reportories are ready
    for file in allFile:
        count+=1
        print "current file : ",count,"/",numberOfFile
        input_filename = file
        output_filename = input_filename[0:(len(input_filename)-3)]+"png"

        # load file into buffer
        data = np.fromfile(inputFolder+"/"+input_filename, dtype=np.uint8)  

        #To get RGB image  
        # create yuv image
        yuv = np.ndarray((height + height / 2, width), dtype=np.uint8, buffer=data)    
        # create a height x width x channels matrix with the datatype uint8 for rgb image
        img = np.zeros((height, width, channels), dtype=np.uint8);    
        # convert yuv image to rgb image
        cv2.cvtColor(yuv, cv2.COLOR_YUV2BGRA_NV21, img, channels)
        cv2.imwrite(outputFolderRGB+"/"+output_filename, img)

        #If u saved the image in graysacale use this part instead
        #yuvReal = np.ndarray((height, width), dtype=np.uint8, buffer=data)
        #cv2.imwrite(outputFolderGray+"/"+output_filename, yuvReal)
else:
    print "not any input"

您只需将 .txt 放入文件夹输入中。这是一个 python 脚本,但如果您更喜欢 c++ 版本,它非常接近。

于 2016-08-10T08:32:01.767 回答