Kinect

By | 2011-01-30

Yesterday I bought a Kinect and hooked it up to my PC. After a couple of hours of compiling drivers and finally realizing I had to switch USB ports for the camera to work I could start writing code.

I used the CLNUIDeviceTest as a base since it already had the lines of code required to pull the image from Kinect. My initial problem was how to access the raw bytestream of image data. I overcame this and made a small copy operation that “makes it appear as if you are walking through a wall”. At least that was the point, but the Kinect obviously needs some calibrating and I’m copying source image wrong. Both easy to fix.

Anyhow I just wanted to share the code used for raw image access. Speed is great

public class ImageProcessor
{
    private byte[] myImageBytes;
    public ImageProcessor()
    {
        JpegBitmapDecoder myImage = new JpegBitmapDecoder(new Uri("brick_wall.jpg", UriKind.Relative), BitmapCreateOptions.DelayCreation, BitmapCacheOption.OnLoad);
        myImageBytes = new byte[myImage.Frames[0].PixelWidth * 4 * myImage.Frames[0].PixelHeight];
        myImage.Frames[0].CopyPixels(myImageBytes, myImage.Frames[0].PixelWidth * 4, 0);
    }
 
    public void Process(NUIImage colorImage, NUIImage depthImage, NUIImage processedImage)
    {
        unsafe
        {
            byte* colorImgPtr = (byte*)colorImage.ImageData;
            byte* srcImgPtr = (byte*)depthImage.ImageData;
            byte* dstImgPtr = (byte*)processedImage.ImageData;
            int height = (int)depthImage.BitmapSource.Height;
            int width = (int)depthImage.BitmapSource.Width;
            int stride = width * 4;
 
            for (int x = 4; x < height; x++)
            {
                for (int y = 0; y < width - 4; y++)
                {
                    int p = x * (width * 4) + (y * 4);
 
                    //dstImgPtr[p + 0] = (byte)srcImgPtr[p + 0];
                    //dstImgPtr[p + 1] = (byte)srcImgPtr[p + 1];
                    //dstImgPtr[p + 2] = (byte)srcImgPtr[p + 2];
                    //dstImgPtr[p + 3] = (byte)srcImgPtr[p + 3];
                    byte b = srcImgPtr[p + 0];
                    dstImgPtr[p + 0] = b > (byte)128 ? colorImgPtr[p + 0] : myImageBytes[p + 0];
                    dstImgPtr[p + 1] = b > (byte)128 ? colorImgPtr[p + 1] : myImageBytes[p + 1];
                    dstImgPtr[p + 2] = b > (byte)128 ? colorImgPtr[p + 2] : myImageBytes[p + 2];
                    dstImgPtr[p + 3] = b > (byte)128 ? colorImgPtr[p + 3] : myImageBytes[p + 3];
 
                    //int avg = (srcImgPtr[p + 0] + srcImgPtr[p + 1] + srcImgPtr[p + 2] + srcImgPtr[p + 3]) / 4;
                    //dstImgPtr[p + 0] = (byte)avg;
                    //dstImgPtr[p + 1] = (byte)avg;
                    //dstImgPtr[p + 2] = (byte)avg;
                    //dstImgPtr[p + 3] = (byte)avg;
                }
            }
        }
    }
}

 

And a sample picture…

image

As you can see the black region on the right represents the picture while the blue region represents the height. I’m guessing I’m sitting a bit too close, maybe I should be processing the green channel instead.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.