1

我正在开发一个使用 Kinect 的 XNA 游戏。屏幕上看到的玩家是在 Kinect 传感器前玩耍的人的真实形象。为了消除背景并仅获取玩家的图像，我正在执行以下操作kinect.AllFramesReady：

using (ColorImageFrame colorVideoFrame = imageFrames.OpenColorImageFrame())
{
    if (colorVideoFrame != null)
    {
        //Getting the image of the colorVideoFrame to a Texture2D named colorVideo
    }
    //And setting its information on a Color array named colors with GetData
    colorVideo.GetData(colors); 
}

using (DepthImageFrame depthVideoFrame = imageFrames.OpenDepthImageFrame())
{
    if (depthVideoFrame != null){
        //Copying the the image to a DepthImagePixel array
        //Using only the pixels with PlayerIndex > 0 to create a Color array
        //And then setting the colors of this array from the 'colors' array by using MapDepthPointToColorPoint method, provided by Kinect SDK
        //Finally I use SetData function in order to set the colors to a Texture2D I created before
    }
}

但不出所料，性能非常低。因为我必须在每帧GetData中使用长度为 640*480 = 307200 的颜色数组（因为ColorImageFormat）和SetData另一个长度为 320*480 = 76800 的颜色数组（因为 ） ！DepthImageFormat

我想知道这个问题是否有任何其他解决方案，SetData以及GetData可能的任何替代方案。因为我知道这些函数在 GPU 和 CPU 之间移动数据，这对于大数据来说是一项昂贵的操作。谢谢你的帮助。

4

1 回答 1

1

Kinect for Windows Toolbox 附带一个“GreenScreen-WPF”示例，它应该提供一些关于处理信息的见解。因为您在 XNA 中工作，所以可能存在一些差异，但总体概念应该适用于两个示例。

该示例通过提取多个玩家来工作。这里是处理函数的业务端：

private void SensorAllFramesReady(object sender, AllFramesReadyEventArgs e)
{
    // in the middle of shutting down, so nothing to do
    if (null == this.sensor)
    {
        return;
    }

    bool depthReceived = false;
    bool colorReceived = false;

    using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
    {
        if (null != depthFrame)
        {
            // Copy the pixel data from the image to a temporary array
            depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);

            depthReceived = true;
        }
    }

    using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
    {
        if (null != colorFrame)
        {
            // Copy the pixel data from the image to a temporary array
            colorFrame.CopyPixelDataTo(this.colorPixels);

            colorReceived = true;
        }
    }

    // do our processing outside of the using block
    // so that we return resources to the kinect as soon as possible
    if (true == depthReceived)
    {
        this.sensor.CoordinateMapper.MapDepthFrameToColorFrame(
            DepthFormat,
            this.depthPixels,
            ColorFormat,
            this.colorCoordinates);

        Array.Clear(this.greenScreenPixelData, 0, this.greenScreenPixelData.Length);

        // loop over each row and column of the depth
        for (int y = 0; y < this.depthHeight; ++y)
        {
            for (int x = 0; x < this.depthWidth; ++x)
            {
                // calculate index into depth array
                int depthIndex = x + (y * this.depthWidth);

                DepthImagePixel depthPixel = this.depthPixels[depthIndex];

                int player = depthPixel.PlayerIndex;

                // if we're tracking a player for the current pixel, do green screen
                if (player > 0)
                {
                    // retrieve the depth to color mapping for the current depth pixel
                    ColorImagePoint colorImagePoint = this.colorCoordinates[depthIndex];

                    // scale color coordinates to depth resolution
                    int colorInDepthX = colorImagePoint.X / this.colorToDepthDivisor;
                    int colorInDepthY = colorImagePoint.Y / this.colorToDepthDivisor;

                    // make sure the depth pixel maps to a valid point in color space
                    // check y > 0 and y < depthHeight to make sure we don't write outside of the array
                    // check x > 0 instead of >= 0 since to fill gaps we set opaque current pixel plus the one to the left
                    // because of how the sensor works it is more correct to do it this way than to set to the right
                    if (colorInDepthX > 0 && colorInDepthX < this.depthWidth && colorInDepthY >= 0 && colorInDepthY < this.depthHeight)
                    {
                        // calculate index into the green screen pixel array
                        int greenScreenIndex = colorInDepthX + (colorInDepthY * this.depthWidth);

                        // set opaque
                        this.greenScreenPixelData[greenScreenIndex] = opaquePixelValue;

                        // compensate for depth/color not corresponding exactly by setting the pixel 
                        // to the left to opaque as well
                        this.greenScreenPixelData[greenScreenIndex - 1] = opaquePixelValue;
                    }
                }
            }
        }
    }

    // do our processing outside of the using block
    // so that we return resources to the kinect as soon as possible
    if (true == colorReceived)
    {
        // Write the pixel data into our bitmap
        this.colorBitmap.WritePixels(
            new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
            this.colorPixels,
            this.colorBitmap.PixelWidth * sizeof(int),
            0);

        if (this.playerOpacityMaskImage == null)
        {
            this.playerOpacityMaskImage = new WriteableBitmap(
                this.depthWidth,
                this.depthHeight,
                96,
                96,
                PixelFormats.Bgra32,
                null);

            MaskedColor.OpacityMask = new ImageBrush { ImageSource = this.playerOpacityMaskImage };
        }

        this.playerOpacityMaskImage.WritePixels(
            new Int32Rect(0, 0, this.depthWidth, this.depthHeight),
            this.greenScreenPixelData,
            this.depthWidth * ((this.playerOpacityMaskImage.Format.BitsPerPixel + 7) / 8),
            0);
    }
}

如果您只对单个玩家感兴趣，您可以考虑使用玩家掩码来更快地提取适当的像素集。你会

using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
    if (skeletonFrame != null && skeletonFrame.SkeletonArrayLength > 0)
    {
        if (_skeletons == null || _skeletons.Length != skeletonFrame.SkeletonArrayLength)
        {
            _skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
        }

        skeletonFrame.CopySkeletonDataTo(_skeletons);

        // grab the tracked skeleton and set the playerIndex for use pulling
        // the depth data out for the silhouette.
        this.playerIndex = -1;
        for (int i = 0; i < _skeletons.Length; i++)
        {
            if (_skeletons[i].TrackingState != SkeletonTrackingState.NotTracked)
            {
                this.playerIndex = i+1;
            }
        }
    }
}

然后，您可以逐步浏览深度数据以提取适当的位：

depthFrame.CopyPixelDataTo(this.pixelData);

for (int i16 = 0, i32 = 0; i16 < pixelData.Length && i32 < depthFrame32.Length; i16++, i32 += 4)
{
    int player = pixelData[i16] & DepthImageFrame.PlayerIndexBitmask;
    if (player == this.playerIndex)
    {
        // the player we are tracking
    }
    else if (player > 0)
    {
        // a player, but not the one we want.
    }
    else
    {
        // background or something else we don't care about
    }
}

我从用于生成轮廓的控件中提取此代码，因此它不处理颜色流。但是，在适当的时候调用MapDepthFrameToColorFrame应该允许您处理颜色流数据并将相应的像素提取到播放器的掩码中。

于 2013-02-15T20:36:13.873 回答