Kinect

Yesterday I bought a Kinect and hooked it up to my PC. After a couple of hours of compiling drivers and finally realizing I had to switch USB ports for the camera to work I could start writing code.

I used the CLNUIDeviceTest as a base since it already had the lines of code required to pull the image from Kinect. My initial problem was how to access the raw bytestream of image data. I overcame this and made a small copy operation that “makes it appear as if you are walking through a wall”. At least that was the point, but the Kinect obviously needs some calibrating and I’m copying source image wrong. Both easy to fix.

Anyhow I just wanted to share the code used for raw image access. Speed is great

public class ImageProcessor
{
 private byte[] myImageBytes;
 public ImageProcessor()
 {
 JpegBitmapDecoder myImage = new JpegBitmapDecoder(new Uri("brick_wall.jpg", UriKind.Relative), BitmapCreateOptions.DelayCreation, BitmapCacheOption.OnLoad);
 myImageBytes = new byte[myImage.Frames[0].PixelWidth * 4 * myImage.Frames[0].PixelHeight];
 myImage.Frames[0].CopyPixels(myImageBytes, myImage.Frames[0].PixelWidth * 4, 0);
 }
 
 public void Process(NUIImage colorImage, NUIImage depthImage, NUIImage processedImage)
 {
 unsafe
 {
 byte* colorImgPtr = (byte*)colorImage.ImageData;
 byte* srcImgPtr = (byte*)depthImage.ImageData;
 byte* dstImgPtr = (byte*)processedImage.ImageData;
 int height = (int)depthImage.BitmapSource.Height;
 int width = (int)depthImage.BitmapSource.Width;
 int stride = width * 4;
 
 for (int x = 4; x < height; x++)
 {
 for (int y = 0; y < width - 4; y++)
 {
 int p = x * (width * 4) + (y * 4);
 
 //dstImgPtr[p + 0] = (byte)srcImgPtr[p + 0];
 //dstImgPtr[p + 1] = (byte)srcImgPtr[p + 1];
 //dstImgPtr[p + 2] = (byte)srcImgPtr[p + 2];
 //dstImgPtr[p + 3] = (byte)srcImgPtr[p + 3];
 byte b = srcImgPtr[p + 0];
 dstImgPtr[p + 0] = b > (byte)128 ? colorImgPtr[p + 0] : myImageBytes[p + 0];
 dstImgPtr[p + 1] = b > (byte)128 ? colorImgPtr[p + 1] : myImageBytes[p + 1];
 dstImgPtr[p + 2] = b > (byte)128 ? colorImgPtr[p + 2] : myImageBytes[p + 2];
 dstImgPtr[p + 3] = b > (byte)128 ? colorImgPtr[p + 3] : myImageBytes[p + 3];
 
 //int avg = (srcImgPtr[p + 0] + srcImgPtr[p + 1] + srcImgPtr[p + 2] + srcImgPtr[p + 3]) / 4;
 //dstImgPtr[p + 0] = (byte)avg;
 //dstImgPtr[p + 1] = (byte)avg;
 //dstImgPtr[p + 2] = (byte)avg;
 //dstImgPtr[p + 3] = (byte)avg;
 }
 }
 }
 }
} 

And a sample picture…

As you can see the black region on the right represents the picture while the blue region represents the height. I’m guessing I’m sitting a bit too close, maybe I should be processing the green channel instead.