I’m trying to do some marker recognition with the Sandbox and managed to get most of it working but I am struggling with mapping the color and depth images together. I know that there is an offset between the two and the images don’t quite match up so ideally I’d want to have them both in the same coordinate system. I was wondering if anybody has tried something like this before or would know roughly how to go about something like this.
I have access to both the color projection matrix and the depth projection matrix but they give me very different results unfortunately. Would I have to create an orthogonal transform for the color data as well?