Show HN: Realtime LCM tool(twitter.com)
twitter.com
Show HN: Realtime LCM tool
https://twitter.com/TitusTeatus/status/1723961348101677353
4 comments
LCMs are actually exactly SD architecture. LCM is initialized from a regular SD unet and finetuned on a new objective. We are already compiling to get to these times. A lot of other people getting sub-100ms times are using fewer inference steps than we do, at a quality tradeoff.
> already compiling
Hmm, well if you mean torch.compile, y'all should still check out stable-fast, which is claiming ~16ms/iter on a 4090, twice that of torch.compile:
https://github.com/chengzeyi/stable-fast#rtx-4090-512x512-ba...
Hmm, well if you mean torch.compile, y'all should still check out stable-fast, which is claiming ~16ms/iter on a 4090, twice that of torch.compile:
https://github.com/chengzeyi/stable-fast#rtx-4090-512x512-ba...
some people even claim they get ~50ms on (powerful) consumer hardware (e.g. RTX 4090).
we still have not been able to replicate those results; also because we want to do it in a distributed way
we still have not been able to replicate those results; also because we want to do it in a distributed way
They might be "cheating" with methods that significantly degrade quality.
i worked on the gpu/infra side of this, so feel free to AMA
Ultimately the LCM is just a SD Unet trained with a new objective, so a lot of SD optimizations are transferable to LCMs
i also worked a bit on the infrastructure!
there's a very cool consistent hashing and probability stuff going on for the GPU routing logic but again, i'll write it after this fire ends similar to a post-mortem.
there's a very cool consistent hashing and probability stuff going on for the GPU routing logic but again, i'll write it after this fire ends similar to a post-mortem.
will write an essay about it later, but servers exploding even with experimental alpha users.
I'm excited about what H200s or B100s will mean for technology like this one.
I'm excited about what H200s or B100s will mean for technology like this one.
Can you use custom images?
yea!
there was another video i put on my twitter account but it's already 24 hours old; the UI has changed a bit.
here's the tweet: https://twitter.com/asciidiego/status/1723804366434627611
also, how's infisical doing?
there was another video i put on my twitter account but it's already 24 hours old; the UI has changed a bit.
here's the tweet: https://twitter.com/asciidiego/status/1723804366434627611
also, how's infisical doing?
AITemplate, stable-fast, or even a torch.compile could get that down to 60ms, I bet. Though I'm not sure the example implementations would work on a non SD architecture.