Model Export

Home Forums AR Sandbox Forum Model Export

This topic contains 11 replies, has 4 voices, and was last updated by  sotoglzz 3 years, 7 months ago.

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #101175

    wsgrah
    Participant

    I’ve just started working with SARndbox and one of the things we’re trying to do is generate 3D models from the sandbox. Unless I missed something, SARndbox doesn’t currently do this, and before I started down the road to write something, I wanted to check if there was another solution (like another application) that could generate something (mesh, stl, whatever) that could be manipulated/converted in to something that could be printed.

    Thanks!

    Wayne

    #101178

    Oliver Kreylos
    Keymaster

    There is no export functionality built into the AR Sandbox itself, but the Kinect package contains an (unsupported) utility to convert a previously saved 3D video stream to a sequence of 3D mesh files in Lightwave Object (.lwo) format, which in turn can be read by most 3D modeling software. This is not the same as exporting from inside the AR Sandbox, as that contains special filter algorithms to ensure a watertight surface.

    Unfortunately, I have not maintained that utility, and it fell prey to some API changes. I am currently packaging an updated version of the Kinect package which will fix those issues.

    With the new package, here is the sequence to build the utility, called LWOWriter:

    1. Change into the Kinect package’s source directory.
    2. Build the LWOWriter utility:
      $ make PACKAGES=MYKINECT LWOWriter
    3. Run KinectViewer and save a stretch of 3D video data via the main menu’s “Save Streams…” entry.
    4. Exit KinectViewer, locate the saved video stream (a pair of files with .color and .depth extension), and run it through LWOWriter:

      $ ./bin/LWOWriter <video stream base name> <first exported frame> <last exported frame>

      This will create one Lightwave Object file for each video frame between the given start and end indices. Base name is just the name of the video stream files without the .depth/.color extension.

    If you don’t want to wait for Kinect-2.8-002, you can fix LWOWriter.cpp yourself: Change line 304 from
    projector.setFilterDepthFrames(true);
    to
    projector.setFilterDepthFrames(false,true);
    and replace line 325,
    const Kinect::MeshBuffer& mesh=projector.processDepthFrame(depth);
    with

    Kinect::MeshBuffer mesh;
    projector.processDepthFrame(depth,mesh);
    
    #101179

    wsgrah
    Participant

    Thanks so much for this! Sounds exactly like what I’m looking for.

    I pulled the code off of github and patched LWOWriter. It compiles and appears to function, but I ran in to an issue reading the stream:

    
    Processing frame 293terminate called after throwing an instance of 'IO::File::ReadError' 
        what(): IO::File::ReadError: Short read by 2610 bytes
    Aborted (core dumped)
    

    I just made a quick stream and quit KinectViewer; did I corrupt the end frames or something? Is there a “proper” way to quit saving the video stream?

    #101180

    Oliver Kreylos
    Keymaster

    293 frames were probably all you had. I might have forgotten to check for end-of-file in LWOWriter.

    I recommend passing the min and max frame indices on the command line, as in

    $ ./bin/LWOWriter FooFile 60 60

    That should only export a single LWO file from video frame 60 (two seconds in to get around early start problems).

    #101187

    wsgrah
    Participant

    That did the trick, works like a champ!

    #101202

    matthias
    Participant

    Thankfully, this thread was opened before I could even ask for the same thing.

    I have now also been using LWOwriter to export 3D data. As I found, the arguments should rather read:

    $ ./bin/LWOWriter <video stream base name> <output file name.lwo> <first exported frame> <last exported frame>

    When entering two differing numbers, i.e. more than frame, the LWOWriter still only generates one single file, though. Would I have to specify more output file names, or does it average multiple frames into one file then (which doesn’t seem to be the case), or can it simply not generate more than one LWO per run?
    No matter what frame numbers were given as argument, it will always prompt that it is processing all the frames, i.e. it starts at 1 and runs all the way through to the end of the stream.

    The resulting mesh is very noisy, I guess because no averaging took place. I got a savvy friend interested in trying to use this code to run it over the saved streams before sending them to the LWOWriter. Any thoughts on that?

    Wsgrah, what do your meshes look like? Could you post a pic of some results? Here is an example of a single frame LWOWriter export, imported in Blender and exported as .stl for import in Meshlab:

    Look how bumpy and noisy the sandbox walls are. In reality, they are just flat boards 😉

    I am currently experimenting on how to achieve the best possible results in gaining 3D data. Here’s what I am trying:
    – LWOWriter, single frame export > smoothing in Meshlab
    – LWOWriter, multiple frames export > merging/averaging in Meshlab
    – (possibly, if my friend succeeds) Smoothing saved stream > LWOWriter
    – external 3D scanning software (Skanect & ReconstructMe, running under Windows, requires moving the Kinect for best results)

    Obviously, using 3D scanning software and moving the sensor will yield the best results. It kinda disrupts the workflow to switch OS, and remove the Kinect from its mount, though. That’s why I am looking for a viable solution within Linux, and without dismounting the Kinect.

    What are the chances of having data export implemented in the Sandbox software itself, with watertightness and some averaging included?

    #101203

    wsgrah
    Participant

    Hi matthias,

    The result is kind of noisy, but a place to start.

    screenshot

    The big bump in the middle is a bucket, but it is pretty noisy. I had just planned to bring it in to Meshlab to smooth out.

    Would love to hear the results of your experiments! I had a thought of using VirtualBox to run Skanect, but I’m not sure it’ll work very well 🙂

    #101204

    wsgrah
    Participant

    I was just playing around with some of the 3D Video settings, and I think some of this noise may be coming from some low triangle depth range settings. Haven’t had a chance to try this out much, but under “Show Streamer Dialog” you can increase the depth range a bit to smooth some of that out…

    #101612

    sotoglzz
    Participant

    Hi there!

    I have a VSARndbox up and running using Linux Mint using a Kinect for Xbox 1473. It works great. I have been trying to capture *.lwo files following the LWOwriter of this topic nevertheless I get the following error message:
    I ran ~/Vrui-3.1/bin/KinectViewer -c 0
    and created SavedStreams0001 files

    /home/trinidad/src/Kinect-2.8-002/bin/LWOWriter SavedStreams0001 SavedStreams0001.lwo 60 60

    according to the code from August 31, 2015 at 3:02 pm post
    I get an error the following error message:

    Processing frame 0terminate called after throwing an instance of ‘std::runtime_error’
    what(): IFFChunkWriter::writeChunk: Chunk is too large to write
    Abortado

    I really need to get the LWO files in order to build 3D mesh.

    Thanks in advance Alberto GT

    #101628

    Oliver Kreylos
    Keymaster

    Right you are, the correct invocation is:

    $ ./bin/LWOWriter <depth/color file name prefix> <output file name> <first frame> <last frame>

    <output file name> has to include a printf-style number template to generate one file name per frame, as in OutputFile%06d.lwo, otherwise subsequent frames will overwrite previous ones.

    LWOWriter has to iterate through all frames as depth files are compressed, and have to be processed sequentially. But just skipping a frame is much faster than generating an LWO file.

    Yes, the quality is what you should expect. That’s raw Kinect v1 data for you. This is the reason why the sandbox uses a long baseline low-pass filter to smooth raw data, which causes the 30 frame (1s) delay in topography updates.

    I’m going to add model output as a feature to the next SARndbox version.

    #101629

    Oliver Kreylos
    Keymaster

    That shouldn’t happen, and I don’t know what could be causing it. Try capturing a longer video sequence, and experiment with different frame indices, and see if it makes a difference. Use a frame index range, say 60 100, to check if you get at least a few frames, and look at my comment above regarding the output file name format (it needs to include a number conversion).

    #101662

    sotoglzz
    Participant

    Thank you Oliver, I solved the problem.

Viewing 12 posts - 1 through 12 (of 12 total)

You must be logged in to reply to this topic.

Comments are closed.