nobbis

letztes Jahrdiscuss

ROS 1 had schema evolution (rosbag migration) 15 years ago

letztes Jahrdiscuss

Understood. Thanks for the clarification.

letztes Jahrdiscuss

Wait, you renamed Forge (https://forge.dev) released last week by World Labs, a startup that raised $230M.

Is this "I worked with some friends and I hope you find useful" or is it "So proud of the World Labs team that made this happen, and we are making this open source for everyone" (CEO, World Labs)?

https://x.com/drfeifei/status/1929617676810572234

nobbis

vor 3 Jahrendiscuss

> The big difference of course, based on reading this, is that NeRF models the whole light transmission function ("radiance field") whereas this seems to model only the boundary conditions (what the light "hits").

The title of the paper is "3D Gaussian Splatting for Real-Time Radiance Field Rendering". Each rendered pixel weights the contribution of unbounded view-dependent Gaussians. So, no, that's not a difference.

nobbis

vor 3 Jahrendiscuss

IANAL but I assume so. "Employee of a competing company" (whatever that means) isn't a protected class.

nobbis

vor 3 Jahrendiscuss

And Matterport will never see them coming, as their employees are banned from using Luma:

Competitors: No employee, independent contractor, agent, or affiliate of any competing 3-D capture company is permitted to view, access, or use any portion of the Service without express written permission from Luma AI. By viewing, using, or accessing the Service, you represent and warrant that you are not a competitor of Luma AI or any of its affiliates, or acting on behalf of a competitor of Luma AI in using or accessing the Service.

https://lumalabs.ai/legal/tos

(Disclaimer: another banned competitor)

nobbis

vor 4 Jahrendiscuss

Rising rates don't make the dollar stronger. There's no empirical relationship between interest rates and exchange rates.

nobbis

vor 4 Jahrendiscuss

Yes, although not complete noise – probably empty. Haven't checked but assume there's regularization of the NeRF parameters.

nobbis

vor 4 Jahrendiscuss

DreamFusion doesn't directly build a 3D model from those generated images. It starts with a completely random 3D voxel model, renders it from 6 different angles, then asks Stable Diffusion how plausible an image of "X, side view" it is.

It then sprinkles some noise on the rendering, makes Stable Diffusion improve it a little, then adjusts the voxels to produce that image (using differentiable rendering.)

Rinse and repeat for hours.

nobbis

vor 4 Jahrendiscuss

Key step in generating 3D – ask Stable Diffusion to score views from different angles:

  for d in ['front', 'side', 'back', 'side', 'overhead', 'bottom']:
    text = f"{ref_text}, {d} view"

https://github.com/ashawkey/stable-dreamfusion/blob/0cb8c0e0...

nobbis

vor 4 Jahrendiscuss

Took a week, as predicted: https://twitter.com/DvashElad/status/1575614411834011651

nobbis

vor 4 Jahrendiscuss

We rely on Wasm for Metascan (https://metascan.ai/explore) which allows us to use the same rendering code on iOS, the desktop, and the web.

Accessing multithreading is limited as SharedArrayBuffer requires cross-origin isolation to mitigate Spectre. Apart from that, it works great.

nobbis

vor 4 Jahrendiscuss

Explains the huge remodel of her property here in South Lake.

nobbis

vor 4 Jahrendiscuss

iPhone/iPad's LiDAR Scanner.

nobbis

vor 4 Jahrendiscuss

6D.ai wasn’t released until 2018. You may be thinking of Abound, which you wanted to license in March 2017. I don’t know of anyone else that had real-time meshing in 2017.

nobbis

vor 4 Jahrendiscuss

Interesting - I’ve never observed it kicking in.

Would love to know in which circumstances it’s used. I assume you work for Apple to know this so understand if you can’t share more.

nobbis

vor 4 Jahrendiscuss

Correlation != causation.

Try out our app, Metascan, to see an example of using TSDF with a multi-resolution GPU hashtable that only stores voxel data near surfaces. Or just skim the original voxel hashing paper from 2013 to understand the technique.

Storing voxel data in an array is a lot simpler. So if it’s not the focus of the research, then why would academics engineer something more complex?

nobbis

vor 4 Jahrendiscuss

ORB-SLAM2 is SLAM only (similar to ARKit w/o VIO) so only reconstructs feature points - no surfaces. This research is about surface reconstruction.

nobbis

vor 4 Jahrendiscuss

ARKit doesn’t use LiDAR for camera pose tracking.

TSDF memory isn’t an issue since Niessner et al. (2013).

nobbis

vor 4 Jahrendiscuss

Their research doesn't just integrate depth maps into a TSDF - it uses NN's to incorporate surface priors.

I don't recall you having similar real-time meshing functionality in 2016-2017, Andrew. Can you show what you had?

As far as I'm aware, Abound was the first to demo real-time monocular mobile meshing: on Android in early 2017 (e.g. https://www.youtube.com/watch?v=K9CpT-sy7HE), and iOS in early 2018 (e.g. https://twitter.com/nobbis/status/972298968574013440).

Submissions

Tinder co-founders win $441m after valuation dispute with Match

Forge: Mixed Reality for Mobile

Crowdsourced Robot Servants

Immersive AR/VR with Digital Light Fields

comments