Does this mean a huge hiring uptick in the US/layoff reversal? I do think this law caused some of the bad market. Will undoing it get us back to where we were?
Curious where you sourced the parts? In Canada, shipping kills it for me. When I priced out the robot + electronics + $100 in shipping, I am around $700 - far cry from the $100 on the "sticker".
In the US? I don't think what you are saying is supported by real data. My understanding is that US works did see an improvement in incomes in the last decade but Canadian workers did not.
What makes life better for everyone is competition. Canada's stagnation can be be summed up in a single phrase - lack of competition. Generally, the US has been a free-for-all when it comes to competition and hence its populace enjoys some of the best living standards.
I'll also relate my experience traveling the subway in Asia vs. Manhattan. Asian transit seems like space-age compared to what we have in the West. I think UBI won't save us as the income must come from somewhere. Hiking taxes kills incentives. The better way is to have more freedom/efficiencies in my humble opinion.
In the limit, this is like arguing against the use of wheels. Automation improves labor productivity. Economies that have not invested in capital, have seen labor productivity and incomes stagnant. This is a current debate going on in Canada (especially as compared to the productivity and income gains in the last decade). Canada does have strong Unions, so I wonder if this is related.
Another thing that seems troubling is how a small group of people can hold a majority of the country by the bXlls. Given how this is an election year, I can see this turning into huge fiasco. The rest of the economy is collateral damage.
GGUF files seems to be proliferating. I think some folks (like myself) make an incorrect assumption that the format has more portability/generalizability than it appears to have. Hence, the horror!
I've also been trying to figure out GGUF and the other model formats going around. I'm horrified to see there is no model architecture details in the file! As you say, it seems they are hard-coding the above architectures as constants. If a new hot model comes out, one would need to update the reader code (which has the new model arch implemented). Am I understanding this right?
I'm also a bit confused by the quantization aspect. This is a pretty complex topic. GGML seems to use 16bit as per the article. If was pushing it to 8bit, I reckin I'd see no size improvement the GGML file? The article says they encode quantization versions in that file. Where are they defined?
I am not good at investing (lost money every single time I've tried). Liquid cash gets spent. I am paying off my 5 year fixed mortgage as fast as I can, as I will be up for renewal at the end of next year.
Some people don't have the financial savvy or time to optimize. One size does not fit all.
Immigrants. 95% (made up stat) may be poor but the 5% who are not, can afford to. In Canada, there is a recent narrative which I think is correct that immigrants are driving our inflation. We expanded our population of 30 million by about a million in the last year. That's a million new household formations, and some people bought a buttload of cash.
I liked the last line the most .. I think we've all gotten accustomed to zero interests, and this has changed our behavior. My father to a large extend screwed up his (and our family's) life because he lived in an era of high rates. He did not understand the world had changed. We could have bought a house for cash in 1995 but he chose to rent. After that, it was always a bubble, with no buying opportunity like 1995 ever.
It makes me wonder if all of us have similarly not realized the world is a different place post rate hikes. There is this inevitable dogma that "rates will go back down". A lot of people are making that bet and I wonder if it is just history again.
What I don't understand is what is driving the US economy today. It seems to be firing on all if not most cylinders. People I know who got laid off are finding work (I hear negative experiences too and feel for those people). Hiring in tech seems like it is picking up.
US mortgages are unique I believe. Canada does 5 year fixed and you refinance at the current rate.
I believe the US mortgages work because of Fannie Mae/Mac and a market in mortgage backed securities ( MBOs? I thought they caused the 2008 crisis but I think they are still a thing with better risk management, I dunno?).
I work at an industry research lab. Key challenge for LLMs is the legality and massive resources needed to train. I have research colleagues that are convinced that even OpenAI may be on shaky legal ground. A lot of non-profit and academic liasoning helps to muddy the issue (academics have fair-use exceptions).
If you don't see the potential of the tech and the rapid advances, I can't help you. But the issue around deployment is more legal (and perhaps not enough GPUs to go around).
I'm a hobbyist in this area, and think the field is somewhat early in its development. MCUs and cyber-physical systems was like this until the Arduino happened. Arduino may not have been the first to do exactly that but it was just good enough to cause an (re?)explosion in electronics as a hobby. During its hayday, I would say Arduino was a core element of the Maker movement.
So what needs to happen to make this a reality for semi-con? First off, we need cheap, cheap fabrication. I actually looked at public funding in Canada and how that was going to the big name Universities who had their own in-house fab labs (at older process nodes). The costs of someone not in the inside was nuts. The actual cost should be in the 100s of dollars to fabricate a design (considering the marginal costs).
There are people that do this at home but it doesn't work either due to chemicals being pretty dangerous and the need for a bunch of equipment. I bet the amount of money the EU spent on its first metaverse townhall (or whatever it was called .. the thing very few people attended) or a tiny fraction Canada wastes on silly things promoting youth culture or whatever, they could fund a lab that is actually open to the public, with the express mission of promoting hobbyists and education. This will NEVER happen because (a) it needs a professor who is on the inside with a kid-like passion in this tech and a commitment to bringing it to the masses (I see some profs like this at schools like MIT but it is so rare at large, competitive schools like the big ones in Canada), and (b) it does not have an instant payoff for the govt. They don't want dabblers and vague educational outcomes. They want workers with degrees.
I am convinced before I am dead, advances in robotics and fabrication will simply the process (or use home equipment such as future laser printers for printing stencils). I'd love to spend my retirement fabricating my own CPUs :D
Edit:
Let me add: I don't mean the cutting edge process node. I mean the kind of process node that was used to make the very first chips (but less toxic, repeatable, cheaper equipment). If it is possible for synthetic biology, it must be doable for semicon :D
Very insightful!! A 175B parameter model with 2 bytes per weight, and say 2 bytes per gradient (not sure if single precision gradients makes sense?) comes in at 700GB, which is beyond a single 8x80GB beefy machine!! I recall reading with tech such as RDMA, you can communicate really fast between machines .. I assume if you add a switch in there, you are toast (from a latency perspective). Perhaps using 2 such beefy machines in a pair would do the trick .. after all .. model weights aren't the only thing that needs to be on the GPU.
I saw a reference that said GPT-3, with 96 decoder layers, was trained on a 400 GPU cluster, so that seems like the ballpark for a 175B parameter model. That's 50 of the hypothetical machines we talked about (well .. really 100 for GPT-3 since back in those days, max was 40 or 48 GB per GPU).
I also wonder why NVIDIA (or Cerebras) isn't beefing up GPU memory. If someone sold a 1TB GPU, they could charge a 100grand easy. As I understood it, NVIDIA's GPU memory is just HBM-6 .. so they'd make a profit?
If you want simple data, lets look at the price of bottled water or soda at an airport (I usually bring an empty with me but not the point). It is priced the freakin same across every eatery. No collusion?
I'm convinced some people have a special gift in terms of # of hours of sleep required. Some famous people like Churchill and Thatcher just needed 4 or 5 hours of sleep. I envy such people. I am in my mid 40s and also awake at 2am but I need 8 hours of sleep. Powered by coffee at the moment :D
I only see one solution to right the wrongs of the past (with respect to the housing bubble). Tax wealth annually. Tax home sales the way the US does it.
As a society, we need to acknowledge how zero interest rates were directly responsible for the housing bubble. The people that caused that should be shamed publicly.