nravic · HackerTrans

nravic

no profile record

comments

vor 11 Monatendiscuss

I think even easier in fact - what's happening behind the scenes w/ an LLM is far more opaque

letztes Jahrdiscuss

This is super interesting! We do something similar I think by taking a checkpoint after model initialization. I'm curious what you think about our approach, here's some benchmarks: https://docs.cedana.ai/articles/performance-of-cedanas-gpu-i...

We do some on-the-fly optimizations as well (like compiling into CUDA graphs or fusing together calls) which ends up resulting (for some inference engines) faster token throughput too.

vor 2 Jahrendiscuss

How were you handling GPU state w/ pytorch? We added some custom code around CRIU to enable GPU checkpointing fwiw: https://docs.cedana.ai/setup/gpu-checkpointing/

vor 2 Jahrendiscuss

We do this using CRIU right now! https://github.com/cedana/cedana

In fact one of our customer's use cases is exactly what you describe, allowing users to "hibernate" container workspaces.

vor 2 Jahrendiscuss

I'm probably the cofounder of the guy you spoke with! Here's our repo: https://github.com/cedana/cedana

vor 3 Jahrendiscuss

what kind of pannier bags do you use to haul groceries? All the ones I've used have been too small for my purposes.

vor 3 Jahrendiscuss

IIRC (and in my experience) KL divergence doesn't account for double counting. Wrote a paper where I ended up having to use a custom metric instead: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4...

vor 3 Jahrendiscuss

This is so cool! Out of curiosity, could you walk through your decisionmaking around using Nix and how it fits in with Rivet?

vor 3 Jahrendiscuss

how often are these batch jobs run? I'm curious to know what the absolute maximum sync frequency can be.

vor 3 Jahrendiscuss

Self plug: run llama.cpp as an inference server on a spot instance anywhere: https://cedana.readthedocs.io/en/latest/examples.html#runnin...

vor 4 Jahrendiscuss

GrapheneOS uses WebUSB to flash your phone.

Pretty wild first experience doing it for me - I'm used to the hassle of sideloading a bootloader and then flashing.

vor 4 Jahrendiscuss

Hey! I'd love to learn more about the position - the hook @ jpl email on your website is bouncing though. Do you mind sending me contact details so we can chat?

Thanks :)

vor 5 Jahrendiscuss

Do people out of college even do Leetcode anymore? Surely there's a better use of time once you're in a paying job.

vor 6 Jahrendiscuss

If you're looking at aerospace/defense startups, this is usually the case.

It's not unheard of for them to employ people who's primary role is grant writing to try and get (for example) SBIR funding

vor 7 Jahrendiscuss

Zotero is my go-to

vor 7 Jahrendiscuss

Julia's REPL is beautiful

vor 7 Jahrendiscuss

Yeah, spiking neural networks are a tough nut to crack. Check out Nengo if you're interested in learning more

vor 7 Jahrendiscuss

What are Protonmail's problems?

vor 7 Jahrendiscuss

very visceral reaction there

vor 7 Jahrendiscuss

I've followed the same path, out of curiosity do you mind if I PM you asking a few questions? I'm a couple months out of undergrad and I'd like to pick your brain a bit.