Kinect Camera & Projector Alignment

The projector should be aligned physically such that it projects orthogonally or on-axis onto the “ideal” level sand surface, and that its projected image exactly fits the sandbox. This usually means that the projector has to be mounted vertically above the back long edge of the sandbox, due to short-throw projectors’ usual above-axis projection. The Kinect camera should be aligned such that its field-of-view exactly matches the sandbox as well. This is most easily achieved by mounting the Kinect vertically above the center of the sandbox, orienting it straight down, and then fine-tuning its position and orientation while observing its depth image in the RawKinectViewer application.


Kinect Camera & Projector Calibration

Prior to mounting the Kinect perform  Step 1: Kinect Camera Intrinsic Calibration

Next mount the Kinect above the sandbox and perform  Step 2: Kinect Camera Extrinsic Calibration

Finally, perform  Step 3: Kinect Camera and Projector Calibration


1. Kinect Camera Intrinsic Calibration

(Perform prior to mounting Kinect above sandbox)

Ideally, the Kinect camera should be calibrated intrinsically to capture a proper 1:1 representation of the sand surface. All Kinect cameras are pre-calibrated at the factory, but while that calibration is servicable, it is not very good. Most importantly, Kinect cameras benefit from per-pixel depth correction; without that, a Kinect will capture a completely flat surface as a bowl-like shape, which noticeably affects elevation contour lines and water flow.

There is no video for the per-pixel depth correction calibration step, which needs to be done first, but it is fairly simple and explained in the README file. For simplicity, intrinsic calibration should be performed before the Kinect is mounted above the sandbox.

Intrinsic calibration is described in detail in the Kinect package’s README file, and the following two videos:


2. Kinect Camera Extrinsic Calibration

(Perform with camera mounted above sandbox. Recalibrate whenever physical changes are made to the sandbox layout)

Extrinsic calibration establishes the position and orientation of the Kinect with respect to the sandbox, and the mathematical plane equation of the “ideal” level sand surface and its position and size in 3D size.

Unlike intrinsic calibration, extrinsic calibration has to be performed with the Kinect mounted above the sandbox, and needs to be repeated any time changes are made to the physical sandbox layout ( i.e. changes to the position or orientation of the projector, camera, or sandbox itself). Also, depending on the sandbox assembly’s mechanical stability, it might have to be redone on a regular basis (every month or so). Fortunately, the process only takes a few minutes.

The process is explained in the following two videos:


3. Kinect Camera and Projector Calibration

(Perform with camera mounted above sandbox. Recalibrate whenever physical changes are made to the sandbox layout)

The final calibration step is to measure the precise alignment of the Kinect camera’s field of view and the projector’s image. Like extrinsic calibration, this procedure has to be performed any time the AR Sandbox’ physical layout changes in any way.This is done using a calibration prop and a dedicated calibration utility, explained in detail in the following video:


Running the Augmented Reality Sandbox

See README file included with the AR Sandbox software package.