cshimmin

vorige maanddiscuss

Interesting, I wonder if the rugby thing is a common bias. I did find myself in the weights, as the top result. But apparently there are also Australian rugby versions of me!

cshimmin

2 maanden geledendiscuss

That is simply not true. The naive “glorified auto-complete / stochastic parrot” argument may have some merit when applied to generic pre-trained models, which only learn from unsupervised next-token prediction. But the post training through reinforcement learning the frontier models undergo is very sophisticated and they genuinely learn to do novel things that are purely the work of the model being trained (and the work of the GPUs they burn along the way of course).

cshimmin

2 maanden geledendiscuss

https://xcancel.com/pj_lambert/status/2057477629528150369

cshimmin

2 maanden geledendiscuss

It's on the page, if you click the little info icon in the upper-right. Here's the text but there's some nice graphics there too:

  Snake Game, training entirely in the browser. Built on tinygrad: the rollout / targets / train graphs are TinyJits authored in Python, then compiled once to WGSL and replayed here under WebGPU.

  Observation: flat 10×10 board (100) + 4-dim prev-action one-hot = 104 dims. fc_pi.weight is zero-init so the opening policy is uniform over the legal actions; fc_v uses tinygrad's default Kaiming init.

  Per rollout: T=24 × N=384 parallel snakes (9,216 transitions), then K=3 epochs × 4 mini-batches of PPO updates. GAE γ=0.99, λ=0.95; AdamW wd=0.01; ratio clip ε=0.1; grad-norm 0.5; Huber value β=1, val_coef=1; entropy bonus 0.008333333333333333.

  Action mask + value clip + KL early stop. The 4-dim prev_a obs tail lets fc_pi zero the U-turn logit (the env silently overrides same-axis reversals anyway). Value loss is max(huber(v_new−td), huber(v_clip−td)) at ε=0.2. Approx-KL is sampled after each epoch and breaks the loop at 1.5·kl_target.

cshimmin

3 maanden geledendiscuss

Very helpful! Naïve question (I haven’t had a chance to read TFA at all and diffusion/flow models are not my area of expertise). Doesn’t learning the integral/solution of the diffusion process in a single pass just take us back to like OG generative CNN that we had before diffusion models took over? Surely the answer is “no” but would love to hear your framing as to why.

cshimmin

3 maanden geledendiscuss

Not gonna lie, the first thing I noticed was that the first author was in an anesthesiology department. Your guidelines for sniff-testing are not unreasonable, and can definitely be helpful to people who are unfamiliar with the research area. But I quite intentionally did not appeal to any of those. As a (somewhat) subject matter expert, it's important to _ignore_ things like ad hominem judgement, and instead address the paper on its self-contained merits. And more importantly, to share my assessment of those with the lay public.

cshimmin

3 maanden geledendiscuss

When I was a research physicist I spent a lot of time looking at the effects of ionizing radiation in pictures, although mostly in the context of digital images. The mechanisms are a bit different for photo emulsions, but to me the reason I'd discount radiation is because they're specifically filtering for features that exhibit the expected point spread function (which is a geometric property of the telescope's optical assembly itself). I guess you could test by exposing emulsion plates to ionizing radiation and seeing how often you get PSF-like images by chance. Also, their search is for +/- 1 day of nuclear testing, which seems weird. Certainly radiation from fallout wouldn't make sense on the day before testing. It would have been useful to see +1 day and -1 day separately. Or 0-2 days. The way it's chosen makes me suspect they couldn't find a signal in those windows, and therefore it's probably just statistical noise that they've massaged out of the data.

But to me the biggest flag is that these images are from 50 minute exposures. The objects don't appear as streaks, so they are either very, very short flashes (much shorter than 50 min), or they are very far away. The authors interpret this to mean the objects should be in geosynchronous orbit, which doesn't make sense; objects in geosync would still appear to move relative to the star background over the course of 50 min. Yet this is the entire basis for their "shadow deficit" window calculation. You could constrain the duration vs distance by looking at the effect it would have on smearing the PSF, which would be interesting.

Overall it seems pretty unscientific. If you go looking through enough statistically noisy data for signals in enough places, you'll eventually find it.

cshimmin

3 maanden geledendiscuss

It kinda sounds like a post-doc, in that it provides an on-ramp to working in the industry/institution. But without having to waste your time getting a PhD.

cshimmin

3 maanden geledendiscuss

The 6 is part of 3.6, the model version. 35B parameters, A3B means it's a mixture of experts model with only 3B parameters active in any forward pass.

cshimmin

3 maanden geledendiscuss

Incidentally, I recently learned the origin of the term. Cyber - short for cybernetic - is from the greek κυβερνήτης (kybernetes), meaning helmsman. The original use of cybernetics is in the context of automated control systems, so steering a rudder was a good analogy. It is also the origin for the name k8s.

cshimmin

5 maanden geledendiscuss

Yeah perhaps a better term for Loser is Abstainer. Because the Sociopaths also can certainly lose at the game of maximum capitalist profit. Loser/Abstainer just chooses not to play the game.

cshimmin

7 maanden geledendiscuss

It's basically just a way for the LLM to lazy-load curated information, tools, and scripts into context. The benefit of making it a "standard" is that future generations of LLMs will be trained on this pattern specifically, and will get quite good at it.

cshimmin

9 maanden geledendiscuss

Was this after you and two zany friends made a scheme to divert a fraction of a penny from each of your employer's transactions into a bank account that you control? And then you gave it all back but the building burned down and Milton made off with the cash?

cshimmin

vorig jaardiscuss

So… your argument is that it’s not counterculture unless it’s mainstream culture? And that one should only credit derivative works once they become mainstream, rather than the original inspiring works because they were too obscure?

I don’t think anyone is trying to “gotcha” you. You’ve just got a bad take.

cshimmin

vorig jaardiscuss

If I understood correctly, the global lock is so that notify events are emitted in order. Would it make sense to have a variant that doesn't make this ordering guarantee if you don't care about it, so that you can "notify" within transactions without locking the whole thing?

cshimmin

vorig jaardiscuss

P-1 AI | Hybrid (SF) and Remote (US) roles | Full time

P-1 is trying to do for physical engineering (mechanical, aerospace, electrical), what Devin, Cursor et al. are doing for the software world. Founded by ex Airbus execs and DARPA program managers, we've just raised a $23 million seed round. Our goal is to solve engineering AGI to enable humans to scale engineering complexity by orders of magnitude.

We have several roles open including:

* Research Scientist - Applied AI

Looking for creative people to innovate on agentic AI systems for specialized engineering domains.

* ML Engineer - Training and AI Infra (ad should post tomorrow)

We're doing a lot of LLM fine-tuning and reinforcement learning. We need someone who can make our GPU cluster sing!

* Software Engineer - AI Evals and Test

We believe it's essential develop rigorous, domain-specific evaluations in order to ensure we're making progress on real world issues -- not just hill-climbing on general purpose benchmarks.

... and more!

Apply online at: https://jobs.ashbyhq.com/P-1%20AI

cshimmin

vorig jaardiscuss

some of us have to wear glasses anyway :/

cshimmin

vorig jaardiscuss

Thank you for finally making this make sense to me.

cshimmin

vorig jaardiscuss

almost as though the AIs were trained on a corpus of text written by... humans

cshimmin

vorig jaardiscuss

It’s not unreasonable to suspect they are doing the same. The article starts with a description of a lawsuit NY Times brought against OpenAI for similar reasons. The big difference is that research presented here is only possible with open weight models. OAI and Anthropic don’t make the base models available, so it’s easier to hide the fact that you’ve used copyrighted material by instruction post-training. And I’m not sure you can get the logprobs for specific tokens from their APIs either (which is what the researchers did to make the figures and come up with a concrete number like 42%)

Submissions

How Elsevier finds its peer reviewers

LHC sees hint of boson heavier than Higgs

ATLAS and CMS physics results from Run 2

How to Build Your Own Particle Detector

Computing for the [physics] Energy Frontier: Snowmass Study 2013

comments