DEM Help

[ndscs.lscetParticipant]

I am hoping someone has instructions on how to input a DEM into the sandbox which will allow you to recreate the DEM in the sand. I have been looking everywhere and cant seem to find any information about this. Any help or guidance would be greatly appreciated. Thanks!

[Oliver KreylosModerator]

The first step is to prepare a digital elevation model (DEM) in the (very simple) file format read by the AR Sandbox. I might add support for standard file formats at some later point, but there’s no funding right now.

The format is binary, with little-endian (Intel) byte order. The first two entries are the number of columns and rows in the DEM, stored as 4-byte signed integers.

The next four entries are the spatial extents of the DEM represented as a rectangle using some arbitrary units and coordinate system. More precisely, they are the left, bottom, right, and top edges of the DEM, stored as 4-byte IEEE 754 standard floating-point numbers.

After that follow the DEM’s elevation postings, as numRows x numColumns 4-byte IEEE 754 floating-point numbers in row-major order. The AR Sandbox assumes that the elevation values are in the same unit of measurement as the DEM’s spatial extents.

To load a DEM in this format into the AR Sandbox, press any key while it’s running, and select “Show DEM” from the tool selection menu. In the file selection box that appears, select a DEM file. If the file could be read correctly, pressing the same key afterwards will toggle the AR Sandbox into DEM mode, where blue indicates an excess of sand, and red indicates a lack of sand. Move sand from the blue to the red areas until everything is white (ideally). Pressing the bound key again will toggle back to regular mode.

The DEM mapping tool can be configured to adjust the DEM’s vertical level (to make the DEM match the total amount of sand in the box), its vertical exaggeration, and the tolerance level when the sand is colored white. To adjust these settings, it is easiest to assign a DEM tool as above, and then save the current tool setup by selecting “Save Input Graph” from the “Devices” sub-menu of the Vrui system menu. This creates a text file with the settings for the DEM tool inside. After adjusting the settings, the complete tool setup can be loaded back from inside the AR Sandbox by selecting “Load Input Graph,” or at startup by passing -loadInputGraph <your input graph file name> on SARndbox’s command line.

[ndscs.lscetParticipant]

Thanks, for the help Oliver! I will give it a try.

[ryanwildevansParticipant]

I have been trying to get the “Show Dem” to work without an error message showing up.
I have had success saving my own DEM’s with the Sandbox. I can upload them up into QGIS no problem.
Then when I go to upload the DEM using “ShowDEM” I just created it doesn’t work.
Has anyone had success trying to load DEMs? Could you help shed light on getting a DEM into the very simple format for the sandbox program to read?

This is probably the most exciting feature of the ARSandbox for me, really excited to get it to work. Any help is greatly appreciated! Thanks!

[sharnedParticipant]

Ryan,
I’ve posted information about what I’ve been able to do with DEM data in this thread.
Perhaps it will help you too.

S. Harned

[sikiParticipant]

I’ve started to write a small python utility program which converts the saved output USGS DEM file to input binary grid format. The saved bathymetry file contains 639 rows and 480 columns. I thing there should be 640 rows in the output. Converting the DEM file to grid, SARnbox can’t read it I get a ‘Short read by 2854419504 bytes’ error message. That number is larger then the file size.

[sikiParticipant]

The “Short read by” error was my mistake (using 2 byte integers instead of 4 bytes). I’ve just finished my small utility which convert dem file to grid file. The code is beta, no extensive tests were made. It is written in Python (tested in Python 2.7 but is should run in python 3 too).
Here it is:

“””
Convert USGS ASCII DEM file to SARnbox grid format
(c) Zoltan Siki siki.zoltan@epito.bme.hu

Usage:
python dem2grid.py input.dem [output.grid]
or
dem2grid.py input.dem [output.dgrid]
“””

import sys
import os
import osgeo.gdal
from gdalconst import *
import struct

# check command line parameters
if len(sys.argv) <= 1:
print(“Usage: dem2grid.py input.dem output.grid”)
sys.exit(1)
ifilename = sys.argv[1]
# generate output file name if not given on the command line
if len(sys.argv) > 2:
ofilename = sys.argv[2]
else:
ofilename = os.path.splitext(ifilename)[0] + “.grid”
# use gdal to read USGS DEM file
idataset = osgeo.gdal.Open(ifilename, GA_ReadOnly)
if idataset is None:
print(“Cant read input file”);
sys.exit(2)
# get size of dem
cols = idataset.RasterXSize
rows = idataset.RasterYSize
# get and calculate coordinate limits
tr = idataset.GetGeoTransform()
xul = tr[0]
yul = tr[3]
xlr = xul + (cols – 1) * tr[1]
ylr = yul + (rows – 1) * tr[5]
# write data to binary output
of = open(ofilename, “wb”)
of.write(struct.pack(“2i”, cols, rows))
of.write(struct.pack(“4f”, xul, ylr, xlr, yul))
band = idataset.GetRasterBand(1)
of.write(band.ReadRaster(0, 0, cols, rows, cols, rows, band.DataType))
of.close()

[sikiParticipant]

I’ve finished the grid->dem conversion too. Both tools (dem2grid.py, grid2dem.py) are available on github.com
I’ve tested using Python 2.7.
See https://github.com/zsiki/SARndbox_util

Testers are highly welcome

[jKrienertParticipant]

I have had success with producing a *.grid file capable of being opened in the SARndbox viewer with this Python utility.
However, I am having difficulty in establishing the appropriate DEM offset to make it usable with the current SARndbox plane.
Has anyone resolved a simple means of addressing this other than using a GIS calculator to make several .grid surfaces with a gradually adjusted coefficient to find the proper range?
Thanks!

[Oliver KreylosModerator]

See my earlier reply. DEM tools have to applicable settings: demVerticalShift and demVerticalScale.

demVerticalShift moves the DEM up or down to match the total amount of sand in the sandbox. demVerticalScale applies vertical exaggeration to the DEM. If the DEM appears to be flat or just noise, it is probably using a different unit of measurement for the vertical than for the horizontal.

[jKrienertParticipant]

Thanks for reiterating those commands Dr. Kreylos.

I have experimented with [demVerticalScale, demVerticalShift] further, but the flat plane problem remains.
I am wondering if anyone out there would consider reviewing the input files in question.
The data is accessible via the link below, including original DEM, translated *.asc data, and conclusive *.grid input.

DEM-Testing

For vertical exaggeration of the original z-axis elevation offset of ~10 meters, neither pre-processing of the raster image in GIS software, or actively adjusting the demVerticalScale within the SARndbox program seems to help.
Consistently, the red/blue areal indications suggest the construction of a flat plane. This is so even when using exaggerations greater than 10 (the provided inputs are at 20x) and upwards of 1000.

The input elevation data (z-axis) is in meters, and the horizontal (x,y-axes) are in UTM (meters).
It seems as though I might be missing a simple step in this process and remain hopeful to resolve.

Again, thanks.

• This reply was modified 7 months ago by  jKrienert. Reason: Clarification of attached documents

[jKrienertParticipant]

Update;
One possible source of complication was the (column x row) pixel resolution of the input dataset shared previously.
The files have been updated with 480 x 640 (to respectively reflect the Kinects’ capabilities as previously mentioned by Dr. Kreylos), and are available for troubleshooting at the following link:

https://drive.google.com/open?id=0B4hmYjvNucnIeE5YbjFRZm50ZE0
(includes OG elevations and per-processed vertical exaggerations)

Even with these changes, the sandbox continues to suggest a flat DEM surface construction.
Are the *.grid elevation values related to the surface plane measurements for the Kinect calibration?

Also, the (appreciated) LakeTahoe.grid file distributed by Dr. Kreylos exhibits the same problem.
This might indicate that the shared datasets are sufficient, yet this particular sandboxes configuration and/or exaggeration is improper?

• This reply was modified 7 months ago by  jKrienert. Reason: further confirmation of troubles

[Oliver KreylosModerator]

I don’t know. I looked at your original DEM, which is a bit flat for the sandbox and for which I would recommend a vertical exaggeration around 5x, and at your scaled DEM which appears to have a 20x factor baked in.

I use the Lake Tahoe DEM I made available on my web server at 4x vertical exaggeration, and it works fine then.

Just for clarification: what do you actually see when you run your scaled DEM, or the Lake Tahoe DEM at 4x?

[jKrienertParticipant]

Dr. Kreylos,
The following screenshots were both captured from the SARndbox in full-screen mode.
In the left image, the LakeTahoe.grid file was loaded with demVerticalScale: 4.0, and demVerticalShift: -20.
In the right image, the most recently submitted OG-Elev_CedarLakeDam was loaded with demVerticalScale: 4.0, and demVerticalShift: -12.

DEM-Comparison

It appears both images suggest the construction of a planer sand surface. Also, the two projections do not exhibit any indications of varied topography, which is contrary to their true representation when opening the associated *.dem data in GIS software.

To aid diagnostics, the following pasted text was extracted from this SARndboxes BoxLayout.txt file.

(-0.00618249, 0.027504, 0.999603), -110.904
( -49.2924, -37.6834, -106.106)
( 50.3001, -38.25, -105.131)
( -49.7734, 38.0511, -107.141)
( 51.0307, 37.0223, -104.757)

For general context, an image of the entire apparatus is also included, taken at the time of the above tests.

SARndbox-Apparatus_Whole

Glad to provide any other data which might help solve this issue, and enable this great SARndbox feature!

• This reply was modified 7 months ago by  jKrienert. Reason: Repaired hyperlinks
• This reply was modified 7 months ago by  jKrienert.
• This reply was modified 7 months ago by  jKrienert.
• This reply was modified 7 months ago by  jKrienert.
• This reply was modified 7 months ago by  jKrienert.

[Oliver KreylosModerator]

Hard to tell what’s going on. Can you mold the sand surface until the color map is white-ish everywhere, and then see what it looks like? Also, just in case, demVerticalShift and demVerticalScale are case-sensitive. Here is a relevant tool section from my configuration file:

section DEMTool2
    toolClass DEMTool
    bindings ((Mouse, 4))
    demFileName /home/okreylos/Projects/Sandbox/LakeTahoe.grid
    demVerticalScale 4.0
    demVerticalShift -1.5 # Positive values make it blue-er, negative values red-er
endsection

[Author]

Posts