Trivia: Claude Shannon proposed the idea of predicting the next token (letter) using statistics/probabilities in the training data corpus in 1950:
"Prediction and Entropy of Printed English"
https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf
This really resonates. I am in the CS academia and I feel like I am losing "friends" who used to share the same values and social identity with me at an unprecedented speed. I wonder if it is possible to gather people like this and regroup our social circles.
It could in theory. The model generates a depth image per frame, so each pixel becomes a small 3D point. It also assumes that the 3D scene is static. From this, you can then simply register all the frames into a huge 3D point cloud by unprojecting the pixels to 3D and render it anyway you like (using a classical 3D renderer) and it will be consistent.
Though, a problem is that if the generated video itself has inconsistent information, e.g., the object changes color between frames, then your point cloud would just be "consistently wrong". In practice this will lead to some blurry artifacts because you blend different inconsistent colors together. So when you turn around you will still see the same thing, but that thing is uglier and blurrier because it blends between inconsistent coloring.
It will also be difficult to put a virtual object into the generated scene, because you don't have the lighting information and the virtual object can't blend its color with the environment well.
Overall cool idea but obviously more interesting problems to be solved!
It's interesting to see the trend of the attitude towards GenAI in Hacker News through out the years. This is totally vibe based and I don't have numbers to back it up, but back in 2022-2023, the site was dominantly people who mostly treat GenAI as a curious technology without too much attachment, and some non-trivial amount of folks who are very skeptical of the tech. More recently I see a lot more people who see themselves as evangelists and try very hard to boost/advocate the technology (see all the "LLM coding changes my life" posts). It seems that the tide has turned back a little bit again since we now see this kind of posts surfacing.
For me, I kind of wish this site to go back to the good old days where people just share their nerdy niche hacker things and not filling the first page with the same arguments we see on the other parts of the internet over and over again. ; ) But granted I was attracted by the clickbait title too, so I can't blame others.
Discussed in another thread https://news.ycombinator.com/item?id=41234415
As someone who has worked on diffusion model, it's a clear reject and not a very interesting architecture. The idea is to train a diffusion model to fit to low dimensional data using two MLPs: one accounts for high-level structure and one accounts for low level details. These kind of "global-local" architecture is very common in computer vision/graphics (with the paper mentioned none of the relevant work), so the novelty is low. The experiments also do not clearly showcase where exactly this "dual" structure brings benefits.
That being said, it's very hard to tell it apart from a normal poorly-written paper from a quick glance. If you tell me it's written by a graduate student, I would probably believe it. It is also interesting in a way that maybe for low-dimensional signals there are some architecture tweaks we can do to modify the existing diffusion model architectures to make things better, so maybe not 100% BS.
I am a graphics researcher who has been following your blog posts on your GPU vector graphics engine. I've been benefitted by your posts and I think some of the vector graphics renderer designs you discussed in your blog are genuinely novel and worth publishing as academic papers if written and evaluated properly. Please consider submitting them to a graphics venue like SIGGRAPH, EGSR, High Performance Graphics, or Journal of Computer Graphics Techniques if you find free time to do it. It will greatly benefit the community. ; )