DEM Help

Home Forums AR Sandbox Forum DEM Help

This topic contains 23 replies, has 7 voices, and was last updated by  ABCottrell 1 month, 1 week ago.

Viewing 15 posts - 1 through 15 (of 24 total)
  • Author
    Posts
  • #102407

    ndscs.lscet
    Participant

    I am hoping someone has instructions on how to input a DEM into the sandbox which will allow you to recreate the DEM in the sand. I have been looking everywhere and cant seem to find any information about this. Any help or guidance would be greatly appreciated. Thanks!

    #102788

    Oliver Kreylos
    Moderator

    The first step is to prepare a digital elevation model (DEM) in the (very simple) file format read by the AR Sandbox. I might add support for standard file formats at some later point, but there’s no funding right now.

    The format is binary, with little-endian (Intel) byte order. The first two entries are the number of columns and rows in the DEM, stored as 4-byte signed integers.

    The next four entries are the spatial extents of the DEM represented as a rectangle using some arbitrary units and coordinate system. More precisely, they are the left, bottom, right, and top edges of the DEM, stored as 4-byte IEEE 754 standard floating-point numbers.

    After that follow the DEM’s elevation postings, as numRows x numColumns 4-byte IEEE 754 floating-point numbers in row-major order. The AR Sandbox assumes that the elevation values are in the same unit of measurement as the DEM’s spatial extents.

    To load a DEM in this format into the AR Sandbox, press any key while it’s running, and select “Show DEM” from the tool selection menu. In the file selection box that appears, select a DEM file. If the file could be read correctly, pressing the same key afterwards will toggle the AR Sandbox into DEM mode, where blue indicates an excess of sand, and red indicates a lack of sand. Move sand from the blue to the red areas until everything is white (ideally). Pressing the bound key again will toggle back to regular mode.

    The DEM mapping tool can be configured to adjust the DEM’s vertical level (to make the DEM match the total amount of sand in the box), its vertical exaggeration, and the tolerance level when the sand is colored white. To adjust these settings, it is easiest to assign a DEM tool as above, and then save the current tool setup by selecting “Save Input Graph” from the “Devices” sub-menu of the Vrui system menu. This creates a text file with the settings for the DEM tool inside. After adjusting the settings, the complete tool setup can be loaded back from inside the AR Sandbox by selecting “Load Input Graph,” or at startup by passing -loadInputGraph <your input graph file name> on SARndbox’s command line.

    #102794

    ndscs.lscet
    Participant

    Thanks, for the help Oliver! I will give it a try.

    #102810

    ryanwildevans
    Participant

    I have been trying to get the “Show Dem” to work without an error message showing up.
    I have had success saving my own DEM’s with the Sandbox. I can upload them up into QGIS no problem.
    Then when I go to upload the DEM using “ShowDEM” I just created it doesn’t work.
    Has anyone had success trying to load DEMs? Could you help shed light on getting a DEM into the very simple format for the sandbox program to read?

    This is probably the most exciting feature of the ARSandbox for me, really excited to get it to work. Any help is greatly appreciated! Thanks!

    #102868

    sharned
    Participant

    Ryan,
    I’ve posted information about what I’ve been able to do with DEM data in this thread.
    Perhaps it will help you too.

    S. Harned

    #102888

    siki
    Participant

    I’ve started to write a small python utility program which converts the saved output USGS DEM file to input binary grid format. The saved bathymetry file contains 639 rows and 480 columns. I thing there should be 640 rows in the output. Converting the DEM file to grid, SARnbox can’t read it I get a ‘Short read by 2854419504 bytes’ error message. That number is larger then the file size.

    #102889

    siki
    Participant

    The “Short read by” error was my mistake (using 2 byte integers instead of 4 bytes). I’ve just finished my small utility which convert dem file to grid file. The code is beta, no extensive tests were made. It is written in Python (tested in Python 2.7 but is should run in python 3 too).
    Here it is:

    “””
    Convert USGS ASCII DEM file to SARnbox grid format
    (c) Zoltan Siki siki.zoltan@epito.bme.hu

    Usage:
    python dem2grid.py input.dem [output.grid]
    or
    dem2grid.py input.dem [output.dgrid]
    “””

    import sys
    import os
    import osgeo.gdal
    from gdalconst import *
    import struct

    # check command line parameters
    if len(sys.argv) <= 1:
    print(“Usage: dem2grid.py input.dem output.grid”)
    sys.exit(1)
    ifilename = sys.argv[1]
    # generate output file name if not given on the command line
    if len(sys.argv) > 2:
    ofilename = sys.argv[2]
    else:
    ofilename = os.path.splitext(ifilename)[0] + “.grid”
    # use gdal to read USGS DEM file
    idataset = osgeo.gdal.Open(ifilename, GA_ReadOnly)
    if idataset is None:
    print(“Cant read input file”);
    sys.exit(2)
    # get size of dem
    cols = idataset.RasterXSize
    rows = idataset.RasterYSize
    # get and calculate coordinate limits
    tr = idataset.GetGeoTransform()
    xul = tr[0]
    yul = tr[3]
    xlr = xul + (cols – 1) * tr[1]
    ylr = yul + (rows – 1) * tr[5]
    # write data to binary output
    of = open(ofilename, “wb”)
    of.write(struct.pack(“2i”, cols, rows))
    of.write(struct.pack(“4f”, xul, ylr, xlr, yul))
    band = idataset.GetRasterBand(1)
    of.write(band.ReadRaster(0, 0, cols, rows, cols, rows, band.DataType))
    of.close()

    #102893

    siki
    Participant

    I’ve finished the grid->dem conversion too. Both tools (dem2grid.py, grid2dem.py) are available on github.com
    I’ve tested using Python 2.7.
    See https://github.com/zsiki/SARndbox_util

    Testers are highly welcome

    #102921

    jKrienert
    Participant

    I have had success with producing a *.grid file capable of being opened in the SARndbox viewer with this Python utility.
    However, I am having difficulty in establishing the appropriate DEM offset to make it usable with the current SARndbox plane.
    Has anyone resolved a simple means of addressing this other than using a GIS calculator to make several .grid surfaces with a gradually adjusted coefficient to find the proper range?
    Thanks!

    #102923

    Oliver Kreylos
    Moderator

    See my earlier reply. DEM tools have to applicable settings: demVerticalShift and demVerticalScale.

    demVerticalShift moves the DEM up or down to match the total amount of sand in the sandbox. demVerticalScale applies vertical exaggeration to the DEM. If the DEM appears to be flat or just noise, it is probably using a different unit of measurement for the vertical than for the horizontal.

    #102924

    jKrienert
    Participant

    Thanks for reiterating those commands Dr. Kreylos.

    I have experimented with [demVerticalScale, demVerticalShift] further, but the flat plane problem remains.
    I am wondering if anyone out there would consider reviewing the input files in question.
    The data is accessible via the link below, including original DEM, translated *.asc data, and conclusive *.grid input.

    DEM-Testing

    For vertical exaggeration of the original z-axis elevation offset of ~10 meters, neither pre-processing of the raster image in GIS software, or actively adjusting the demVerticalScale within the SARndbox program seems to help.
    Consistently, the red/blue areal indications suggest the construction of a flat plane. This is so even when using exaggerations greater than 10 (the provided inputs are at 20x) and upwards of 1000.

    The input elevation data (z-axis) is in meters, and the horizontal (x,y-axes) are in UTM (meters).
    It seems as though I might be missing a simple step in this process and remain hopeful to resolve.

    Again, thanks.

    • This reply was modified 2 months, 1 week ago by  jKrienert. Reason: Clarification of attached documents
    #102928

    jKrienert
    Participant

    Update;
    One possible source of complication was the (column x row) pixel resolution of the input dataset shared previously.
    The files have been updated with 480 x 640 (to respectively reflect the Kinects’ capabilities as previously mentioned by Dr. Kreylos), and are available for troubleshooting at the following link:

    https://drive.google.com/open?id=0B4hmYjvNucnIeE5YbjFRZm50ZE0
    (includes OG elevations and per-processed vertical exaggerations)

    Even with these changes, the sandbox continues to suggest a flat DEM surface construction.
    Are the *.grid elevation values related to the surface plane measurements for the Kinect calibration?

    Also, the (appreciated) LakeTahoe.grid file distributed by Dr. Kreylos exhibits the same problem.
    This might indicate that the shared datasets are sufficient, yet this particular sandboxes configuration and/or exaggeration is improper?

    • This reply was modified 2 months ago by  jKrienert. Reason: further confirmation of troubles
    #102931

    Oliver Kreylos
    Moderator

    I don’t know. I looked at your original DEM, which is a bit flat for the sandbox and for which I would recommend a vertical exaggeration around 5x, and at your scaled DEM which appears to have a 20x factor baked in.

    I use the Lake Tahoe DEM I made available on my web server at 4x vertical exaggeration, and it works fine then.

    Just for clarification: what do you actually see when you run your scaled DEM, or the Lake Tahoe DEM at 4x?

    #102932

    jKrienert
    Participant

    Dr. Kreylos,
    The following screenshots were both captured from the SARndbox in full-screen mode.
    In the left image, the LakeTahoe.grid file was loaded with demVerticalScale: 4.0, and demVerticalShift: -20.
    In the right image, the most recently submitted OG-Elev_CedarLakeDam was loaded with demVerticalScale: 4.0, and demVerticalShift: -12.

    DEM-Comparison

    It appears both images suggest the construction of a planer sand surface. Also, the two projections do not exhibit any indications of varied topography, which is contrary to their true representation when opening the associated *.dem data in GIS software.

    To aid diagnostics, the following pasted text was extracted from this SARndboxes BoxLayout.txt file.

    (-0.00618249, 0.027504, 0.999603), -110.904
    ( -49.2924, -37.6834, -106.106)
    ( 50.3001, -38.25, -105.131)
    ( -49.7734, 38.0511, -107.141)
    ( 51.0307, 37.0223, -104.757)

    For general context, an image of the entire apparatus is also included, taken at the time of the above tests.

    SARndbox-Apparatus_Whole

    Glad to provide any other data which might help solve this issue, and enable this great SARndbox feature!

    • This reply was modified 2 months ago by  jKrienert. Reason: Repaired hyperlinks
    • This reply was modified 2 months ago by  jKrienert.
    • This reply was modified 2 months ago by  jKrienert.
    • This reply was modified 2 months ago by  jKrienert.
    • This reply was modified 2 months ago by  jKrienert.
    #102938

    Oliver Kreylos
    Moderator

    Hard to tell what’s going on. Can you mold the sand surface until the color map is white-ish everywhere, and then see what it looks like? Also, just in case, demVerticalShift and demVerticalScale are case-sensitive. Here is a relevant tool section from my configuration file:

    section DEMTool2
        toolClass DEMTool
        bindings ((Mouse, 4))
        demFileName /home/okreylos/Projects/Sandbox/LakeTahoe.grid
        demVerticalScale 4.0
        demVerticalShift -1.5 # Positive values make it blue-er, negative values red-er
    endsection
Viewing 15 posts - 1 through 15 (of 24 total)

You must be logged in to reply to this topic.

Comments are closed.