19 Sep 2024
|
Mike Boland
AWE Talks: XR's Photorealistic Holy Grail
AWE USA 2024
Welcome back to AWE Talks, our series that revisits the best of AWE’s conference sessions. With AWE USA 2024 recently concluded, we have a fresh batch of footage to keep us busy for weeks to come.
We continue the action this week by looking at one of the most coveted endpoints for XR experiences: having both interactive and photorealistic scenes. XR veteran Matt Miesnieks is working on just that at Dejavu.
See the summarized takeaways below, along with the full session video. Stay tuned for more video highlights each week and check out the full library of AWE USA 2024 sessions on AWE’s YouTube Channel.
Speakers
Matt Miesnieks, Dejavu
Key Takeaways & Analysis
– Matt Miesnieks is a bit of a legend in the XR sector... when he talks, you should listen.
– One of the lessons he picked up at 6D.ai (sold to Niantic) is the need for photorealism.
– Though digital object and scene scans were impressive, users expect more realism.
– For example, his biggest client AirBnB consistently relayed this feedback from users.
– But of course, photorealism is hard, given file sizes and processing requirements.
– This has advanced to some degree with NeRFs and Gaussian Splats, but still falls short.
– One issue is the bottleneck of the capture device, which is often a smartphone.
– More substantial rigs are meanwhile cost-prohibitive and can't scale to the degree needed.
– These problems will likely be solved by software, which is where Dejavu comes in.
– It wants to enable not just photorealistic experiences but 6DoF immersive experiences.
– This is a tall order but several market factors are converging to fuel it.
– For example, Vision Pro has demonstrated viability in photorealistic/immersive scenes.
– The challenge again is cost, with its "Environments" each costing $2-$10m to produce.
– Deja Vu's mission is therefore to apply software to bring that cost down over time.
– In that sense, it's a long-term play that rides Moore's Law and other factors to its endpoint.
– Meanwhile, it's off to a good start with a working demo of a high-quality interactive scene.
– Its secret sauce combines several methods for constructing and compressing 3D scenes.
– Current specs include 14K native resolution, dynamic lighting, and 6DoF interaction.
– That will only improve... and the mission is in good hands with Miesnieks's XR aptitude.
“[Vision Pro Environments are] just crazy expensive, he said. "There's no way today to just wave your phone around, or even wave a high-end camera around and automatically generate something this high quality. But that's where the world's headed and long term what Deja Vu is going to be building out and betting on.”
For more color, see the full video below...
Want more XR insights and multimedia? ARtillery Intelligence offers an indexed and searchable library of XR intelligence known as ARtillery Pro. See more here.
Speakers
Matt Miesnieks, Dejavu
Key Takeaways & Analysis
– Matt Miesnieks is a bit of a legend in the XR sector... when he talks, you should listen.
– One of the lessons he picked up at 6D.ai (sold to Niantic) is the need for photorealism.
– Though digital object and scene scans were impressive, users expect more realism.
– For example, his biggest client AirBnB consistently relayed this feedback from users.
– But of course, photorealism is hard, given file sizes and processing requirements.
– This has advanced to some degree with NeRFs and Gaussian Splats, but still falls short.
– One issue is the bottleneck of the capture device, which is often a smartphone.
– More substantial rigs are meanwhile cost-prohibitive and can't scale to the degree needed.
– These problems will likely be solved by software, which is where Dejavu comes in.
– It wants to enable not just photorealistic experiences but 6DoF immersive experiences.
– This is a tall order but several market factors are converging to fuel it.
– For example, Vision Pro has demonstrated viability in photorealistic/immersive scenes.
– The challenge again is cost, with its "Environments" each costing $2-$10m to produce.
– Deja Vu's mission is therefore to apply software to bring that cost down over time.
– In that sense, it's a long-term play that rides Moore's Law and other factors to its endpoint.
– Meanwhile, it's off to a good start with a working demo of a high-quality interactive scene.
– Its secret sauce combines several methods for constructing and compressing 3D scenes.
– Current specs include 14K native resolution, dynamic lighting, and 6DoF interaction.
– That will only improve... and the mission is in good hands with Miesnieks's XR aptitude.
“[Vision Pro Environments are] just crazy expensive, he said. "There's no way today to just wave your phone around, or even wave a high-end camera around and automatically generate something this high quality. But that's where the world's headed and long term what Deja Vu is going to be building out and betting on.”
For more color, see the full video below...
Want more XR insights and multimedia? ARtillery Intelligence offers an indexed and searchable library of XR intelligence known as ARtillery Pro. See more here.
