Utilizing Kinect V2 Depth File?

[dantep]

Hello,
My sandbox is in its final stages of being built (painted and projector/mount being built) and I have been trying to familiarize myself with the software over the last few weeks. I am doing a project for my master’s research and my hope is to use the current software packages (Kinect 3.9, SARndbox 2.7 and Vrui 5.2) to be able to calculate the volume of sand/material that is removed with various mining tools. However, I have been having a hard time getting my head around how the depth image is interpreted. I was wondering if someone has experience utilizing the output from the depth image for more than just coloring and if they could point me in the direction of files that I should be looking at to understand how this all connects. The most similar to what I think I need that I have found so far is DepthImageRenderer.cpp and KinectV2DepthStreamReader.cpp. Any advice is greatly appreciated. Thanks!

[dantep]

I have found multiple places where the depth file is passes as an argument but I cannot find where the file is stored/updated at. If anyone knows the name or location of the depth file or how I can save a depth frame to a depth file, I would greatly appreciate the insight.

[Sean Robinson]

I don’t know about a “depth file”. But you can get DepthPixel values in Sandbox::frame().

Almost a year ago, as an exploration, I added a rough feature to dump a single frame of depth values to a text file for import into LibreOffice Calc. Using LO’s Color Range conditional cell formatting, I could get a quick,
visual verification that the 300k depth values represented the scene I was capturing with the Kinect.

I hope the following will give you an idea for one way to proceed.

@@ -1143,10 +1150,21 @@ void Sandbox::frame(void)
    /* Check if the filtered frame has been updated: */
    if(filteredFrames.lockNewValue())
        {
        /* Update the depth image renderer's depth image: */
        depthImageRenderer->setDepthImage(filteredFrames.getLockedValue());
+
+	      typedef GLfloat DepthPixel;
+
+         Kinect::FrameBuffer fb = filteredFrames.getLockedValue();
+
+         const DepthPixel *dfPtr = fb.getData<DepthPixel>();
+         const int x = fb.getSize()[0];
+         const int y = fb.getSize()[1];
+         for (unsigned int i = 0; i < x * y; ++i, ++dfPtr) {
+             std::cout << "(" << (i % x) << ", " << floor(i / x) << ") = "<< *dfPtr << std::endl;
+         }
        }

    if(handExtractor!=0)
        {
        /* Lock the most recent extracted hand list: */

[dantep]

Hi Sean,

I have been working with the code you provided above and I have it working and I am on the right track. However, I am building a tool where the depth frame is saved so I am working through trying to figure out how to use the Framebuffer from a different class. Before I go through all the work of doing that, I want to be able to convert the depth value to some real world value (meters or centimeters) so I can do calculations with the data. I have found various formulas online for doing this but my depth frame values range from 1400-1600 and the formulas give me ~63cm to 110cm when I know the base of my sand is ~152cm away or 60 inches. I was wondering if you had any experience with this or did anything similar with your data?

[Sean Robinson]

dantep,

I’m glad to read this helped you. Unfortunately, I did not do more with the code I posted. I verified the depth data represented my real-world target and moved on.

I suspect you’ll need to translate values to another basis. BoxLayout.txt has several physical sandbox values in centimeters, so this conversion to another 3D space is possible. But I do not know enough about Vrui and SARndbox to say how.

[Author]

Posts