> Now people barely bring it up at all. It’s like a lion has escaped the zoo and it’s gulping down schoolchildren, but when people suggest zoo improvements, all the agenda items are like, “We should add another Dippin’ Dots kiosk”. If you bring up the loose tiger, everyone gets annoyed at you, like “Of course, no one likes the tiger”.
> As much as people like to use NumPyro and sometimes even PyMC to generate JAX code, I think it may be easier in the end to just write JAX directly. That way, nothing gets between you and JAX and you don’t have to figure out how to filter JAX through middleware. When you do that, the models can be organized very much like in Stan.
^much truth. Nascent libraries like distreqx make it much easier to work at a lower level while maintaining some of the log density affordances that PPLs provide.
Author reputation and citation patterns provide plenty(?) of signal without journal/reviewer/editor endorsement. But you could still imagine introducing “badges” that provide similar additional signal to what publication in a top journal provides today. Academic societies issue a fixed number of badges to top preprints each year. But ditch the song and dance around peer review.
> The defining - and best - feature of preprint servers is that they do not try to litigate the rigor of the science in a paper, or try to decide whether it will ultimately prove important, before posting it. They just post it. We should embrace this, and fight the temptation to reinstate gatekeeping criteria and systems that disempower authors without providing any value.
Most scientific "products" are not bound books or pdf. The software, datasets, proofs, algorithms, etc contained within them (or in the dreaded "replication materials") are usually more valuable than the text that scientists write around them. We use the text merely to communicate the contribution and its value to other humans. I suspect AI will outperform the best humans at this communication task very soon. Is the purpose of a dissertation really to demonstrate that a human has the capacity to write effective prose? There are many scientists who are brilliant but are terrible at this task. Do we really wish to shun them and their ideas?
In the future, the best science will be produced by those that wield AI tools most effectively. Academics need to figure out how to assess scientific work within this context. This is not a good solution.
I don't understand why this framework has become so popular. You can't instantiate or program around `FlowSpec`s. The type safety is terrible (you're supposed to just haphazardly assign to `self` whenever you need to persist something across steps). `step`s just feel like glorified jupyter cells.
There's been so much great innovation in the ML ops space in the last few years and metaflow puts you in a straightjacket that prevents you from tapping into any of it.