Pre-training mean exposing an already-trained model to more raw text like PDF extracts etc (aka continued pre-training). You wouldn't be starting from scratch, but it's still pre-training because the objective is just next token prediction of the text you expose it to.
Post-training means everything else: SFT, DPO, RL, etc. Anything that involves things like prompt/response pairs, reward models, or benefits from human feedback of any kind.
> Sometimes a law is just on its face and unjust in its application. For instance, I have been arrested on a charge of parading without a permit. Now, there is nothing wrong in having an ordinance which requires a permit for a parade. But such an ordinance becomes unjust when it is used to maintain segregation and to deny citizens the First-Amendment privilege of peaceful assembly and protest.
> I hope you are able to see the distinction I am trying to point out. In no sense do I advocate evading or defying the law, as would the rabid segregationist. That would lead to anarchy. One who breaks an unjust law must do so openly, lovingly, and with a willingness to accept the penalty. I submit that an individual who breaks a law that conscience tells him is unjust, and who willingly accepts the penalty of imprisonment in order to arouse the conscience of the community over its injustice, is in reality expressing the highest respect for law.
I always have to go back to read this part again because I feel like it's so unexpected. You don't really hear anyone saying quite the same thing today.
Writing a calculus book that's more rigorous than typical books is hard because if you go too hard, people will say that you've written a real analysis book and the point of calculus is to introduce certain concepts without going full analysis. This book seems to have at least avoided the trap of trying to be too rigorous about the concept of convergence and spending more time on introducing vocabulary to talk about functions and talking about intersections with linear algebra.
I tried and failed to find some kind of concrete methodology that they used to get to the number 30 months. I'm still waiting for quadratic algebra to make my knowledge of linear algebra obsolete.
I like the subway analogy. I'm sure I've heard some version of it before, but maybe because I was younger I didn't really get it. It really is a little strange to tell kids who have never really directed their own lives before to start doing it all of a sudden.
You're right, I was confusing TensorRT with Dynamo. It looks like the relationship between Dynamo and vLLM is actually the opposite of what I was thinking -- Dynamo can use vLLM as a backend rather than vice versa.
It sounds like you might be confusing different parts of the stack. NVIDIA Dynamo for example supports vLLM as the inference engine. I think you should think of something like vLLM as more akin to GUnicorn, and llm-d as an application load balancer. And I guess something like NVIDIA Dynamo would be like Django.
This project appears to make use of both vLLM and Inference Gateway (an official Kubernetes extension to the Gateway resource). The contributions of llm-d itself seems to mostly be a scheduling algorithm for load balancing across vLLM instances.
I do see other graphs that tell a different story. Namely, that confidence is a monotonically increasing function of competence. If the data supports the idea that there is a valley of despair where confidence decreases as competence increases, I must be missing it.
This just made me realize that we're 10 years past the 100th anniversary of Einstein publishing about general relativity. Which made me realize that we're a quarter of the way through the 21th century...
Also, I think a list made today would have to include some of the early work on deep learning that happened in the 20th century. Which goes to show that sometimes you don't know what's important until much later on.
I see several people complaining about this in this thread. Are you talking about their old interface or the new one they released a couple years ago? I find their new one to be decent. It's a little complicated sometimes, but I think it's hard to build a site that allows you to do so many things without being a little complicated.
Leftists may claim that their activism is motivated by compassion or by moral principles, and moral principle does play a role for the leftist of the oversocialized type. But compassion and moral principle cannot be the main motives for leftist activism. Hostility is too prominent a component of leftist behavior; so is the drive for power. Moreover, much leftist behavior is not rationally calculated to be of benefit to the people whom the leftists claim to be trying to help. For example, if one believes that affirmative action is good for black people, does it make sense to demand affirmative action in hostile or dogmatic terms? Obviously it would be more productive to take a diplomatic and conciliatory approach that would make at least verbal and symbolic concessions to white people who think that affirmative action discriminates against them. But leftist activists do not take such an approach because it would not satisfy their emotional needs. Helping black people is not their real goal. Instead, race problems serve as an excuse for them to express their own hostility and frustrated need for power. In doing so they actually harm black people, because the activists’ hostile attitude toward the white majority tends to intensify race hatred.
Post-training means everything else: SFT, DPO, RL, etc. Anything that involves things like prompt/response pairs, reward models, or benefits from human feedback of any kind.