Ross Scott
Staff member
These are some leads I've had on 2D image / video & model generation. This is not the dream software, but it is an interesting halfway point I'm seeing how far I can take. Some stuff worth checking out:
Best 2D to 3D depth map generator I've found (Midas v3):
https://github.com/isl-org/MiDaS
Instructions + code for a way to fix the flickering from Midas, however, I believe it is only for v2 of Midas + the instructions are only for Linux, but the code is apparently cross-platform:
https://colab.research.google.com/drive/1YOLXsb4JUOD1wt5TXB2ln78koocm-bu8?usp=sharing#scrollTo=zUYrfiNRfTUv
https://github.com/facebookresearch/robust_cvd
Method to generate model from a depth map + contains links to open source code on how this was made:
https://depthplayer.ugocapeto.com/
Sample content I made containing footage + different versions of depth maps to experiment with (about 330MB). I generated it with Midas v3, and I used 2 different temporal blurring scripts on the depth maps in order to balance out the flickering:
https://mega.nz/file/H6BERRqZ#QWuZlHimMmq83zy9AZpMPnVsui7WrRCNZiXGNBGKG0M
There's also the program StereoPhoto Maker + Depthgate, but I found that while they do an okay job, the depth can't be pushed without skewing the convergence too much, which can hurt your eyes. I think converting to actual models that can be manipulated will have much better results.
STUFF TO SOLVE:
1. Figure out how to get the anti-flicker code working (preferably for Windows 10, but Linux would be good also).
2. Figure out how to have a dynamically changing model based on the individual footage frames + depth maps. Maybe modify the code from the model generator?
3. Figure out how to export above models to VR + and / or batch convert it to frames for stereoscopic video.
4. Integrate the Midas v3 depth map generation code with the anti-flicker code (this one is a longshot and may not be doable).
Feel free to reply here or email me ([email protected]). I'll more reliably notice the emails. Work on this could go towards a future video!
Best 2D to 3D depth map generator I've found (Midas v3):
https://github.com/isl-org/MiDaS
Instructions + code for a way to fix the flickering from Midas, however, I believe it is only for v2 of Midas + the instructions are only for Linux, but the code is apparently cross-platform:
https://colab.research.google.com/drive/1YOLXsb4JUOD1wt5TXB2ln78koocm-bu8?usp=sharing#scrollTo=zUYrfiNRfTUv
https://github.com/facebookresearch/robust_cvd
Method to generate model from a depth map + contains links to open source code on how this was made:
https://depthplayer.ugocapeto.com/
Sample content I made containing footage + different versions of depth maps to experiment with (about 330MB). I generated it with Midas v3, and I used 2 different temporal blurring scripts on the depth maps in order to balance out the flickering:
https://mega.nz/file/H6BERRqZ#QWuZlHimMmq83zy9AZpMPnVsui7WrRCNZiXGNBGKG0M
There's also the program StereoPhoto Maker + Depthgate, but I found that while they do an okay job, the depth can't be pushed without skewing the convergence too much, which can hurt your eyes. I think converting to actual models that can be manipulated will have much better results.
STUFF TO SOLVE:
1. Figure out how to get the anti-flicker code working (preferably for Windows 10, but Linux would be good also).
2. Figure out how to have a dynamically changing model based on the individual footage frames + depth maps. Maybe modify the code from the model generator?
3. Figure out how to export above models to VR + and / or batch convert it to frames for stereoscopic video.
4. Integrate the Midas v3 depth map generation code with the anti-flicker code (this one is a longshot and may not be doable).
Feel free to reply here or email me ([email protected]). I'll more reliably notice the emails. Work on this could go towards a future video!
Last edited by a moderator: