CoT is widely known technique - what became fully novel was the level of training embedding CoT via RL with optimal reward trajectory. DeepSeek took it further due to their compute restriction to find memory, bandwidth, parallelism optimizations in every part (GRPO - reducing memory copies, DualPipe for data batch parallelism between memory & compute, kernel bypasses (PTX level optimization), etc.) - then even using MoE due to sparse activation and further distillation. They operated on the power scaling laws of parameters & tokens but high quality data circumvents this. I’m not surprised they utilized synthetic generation from OpenAI or copied the premise of CoT, but where they should get the most credit is their infra level & software level optimizations.
With that being said, I don’t think the benchmarks we currently have are strong enough and the next frontier models are yet to come. I’m sure at this point U.S LLM research firms now understand their lack of infra/hardware optimizations (they just threw compute at the problem), they will begin paying closer attention. Now their RL-level and parent training will become even greater - whilst the newly freed resources to solve for sub-optimizations that have been traditionally avoided due to computational overhead
This is somewhat untrue as well. HFT because constrained similarly have to optimize on this level akin to HFT crypto doing optimizations not within solidity, nor yul but on opcode in huff. That’s the issue with these big tech companies. Just endless budget and throw bad code into larger distributed clusters to overcompensate.
In residential solar, panels are largely subsidized by federal ITC or financed by banks claiming the tax equity which covers the maintenance, upfront costs, etc; hence the zero upfront residential solar model. Will this be similar?
Ah. I can’t believe this still happens in this day of age. About a decade ago, I was working for a startup and we were getting dominated in our growing space by a much larger, well funded rival. Our competitive intelligence team browsed through their git, and the rival actually exposed access to their customer, pricing and sales agent database by leaving their credentials in one of their branches. The team went to our legal department asking if they can be protected by the company, and if they can use this intel. The team then worked with the product team to integrate all their pricing engines to our POS to undercut their pricing and sent marketing blasts to their leads with targeted marketing campaigns. Long story short that company is now defunct, and it definitely undermined their growth.
I can’t speak for their intent. It has less to do with the altruism or the “mission” and more on their personal well being. My dad survived multiple cancers and after he got his strength back, he wanted to build back up his strength. It’s the structure of the system that helps him. He was telling me about their productivity KPIs, and he sees them as fitness goals - as he builds pallets per hour, etc.
My mom spent the better part of her life as the “COO” of their business, so after being retired for so long, she seeks the mental stimulation of work. They’ve never worked in a corporate office, so seeing the structure and thinking behind Amazon amazes her. They’ve always had their own warehouses, distribution centers, trucking, etc. but they built that from the ground up and never scaled because they were “learning on the job”. Never seen what good looks like.
They both keep to themselves. No one in their section know about their backgrounds.
My parents post-retirement after not working for 10+ years both found roles at an Amazon Fulfillment Center - one at the ship dock and the other in returns. They are both in their 70s and built a nice nest for themselves, neither need it for the money. In fact, my dad was a machine learning professor, multiple publications, and they ran a small medium sized business for 20+ years (at their peak they were running 80M USD annual revenue). They both do it for the mental health and for the physical exercise. They meet a lot of people just like them, some of the workers are living in 3M+ USD homes - although most are not, it’s mostly kids that went to state schools and climb the FC ladder.
This book should be geared for one of those introduction courses, like “Introduction to Engineering”, “AI 101”, etc. I do agree with the vision of the authors that the book is a holistic view of data science, as I personally believe data science is not all about maths, programming; but consider the principles that surround it as a science.
This is also quite practical for large consultancy firms. Most of the chapters, I’ve had clients discuss with me (such as Responsible AI). Personally, I think it could have went away from the applications as it was too high level.
I think about this quite a bit as my parents likely fit this category in the early 90s in Silicon Valley. At peak, they were bootstrapped a company from nothing to eventually at peak with ~40 employees at 100M USD annual revenue, no idea on income as it was a fairly large operation (distribution, warehousing, engineering team, sales team, operations, etc) They exited out of business within 6 years and retired in their 40s.
My family grew up relatively poor and extremely frugal. My dad was formerly a professor in machine learning, but decided to enter the private sector. He didn’t speak much English if at all, and entered the field when it was still immature.
After he was laid off, and with little options left, they decided to use their remaining savings and likely a loan from family & friends to bootstrap a company. My parents never wanted a business, but they had to out of survival.
They never discussed the business with us, so I don’t fully understand the operating model behind their company, but it involved with semiconductors/hardware, etc.
What I think about is was this simply a business or during that time a “startup”. It was in a hyper growth period on relatively emerging technology, they were learning as they went, and exited quickly.
Recently, though my dad unretired in his 70s working at a FANG… Amazon warehouse worker. He says he does it for the exercise and $20/hour.
I've actually decided to take some time off work because of my addiction issues. It's more so the cycle. I'll stop taking Zolpidem/Zopiclone for several weeks/month (the first several days -- terrible shakes, chills, nightmares, several days without sleeping etc. Zopiclone/Benzo withdrawal is far worse) After a while, I'll eventually adjust but I have insomnia/poor sleep so I'll eventually cave in due to sleep deprivation then I'll pick back up.
For the past year, I would roughly take 90mg of either Zolpidem/Zopiclone in a single night. On average, I roughly go through 100 pills a week (I have pharmacies that sell to me...I know sketch).
It wasn't always like this, when I first started taking medication after going to Stanford Sleep Clinic, I would just take a pill or two a night and have fantastic sleep. It was like this for several years but eventually my tolerance increased and my work stress/insomnia got worse with age.
I want to reiterate that I've destroyed/damaged every meaningful relationship in my life. I state this all out because I've realized that people have a strange fascination with Ambien.
I’ve been taking Zolpidem for about 13 years now and suffer from addiction. I’ve sought treatment for this but it doesn’t work as now I cannot sleep without taking something. If you haven’t taken Z-type sleeping aids, I would definitely suggest not to start, as from experience the consequences outweigh the benefits. I’ve lost a considerable amount of friends/reputation/etc. from the abuse. Every experience that you hear about from sleeping aids, I’ve been through it (from the loopy fun to binge eating to weird sex to psychosis to crashing my car)
For me, it's about chasing after an opportunity that will potentially change my career and life. I grew up with nothing, so I chase after everything. My parents were illegals and watching the struggle was very impactful. Even if I decided to go back and take care of them, I would get an earful.
As an Asian-American, I'm struggling through the same dilemma as well. I've been given a huge opportunity to move to Asia and to initiate a program at a Fortune 100 company, but recently my parents have gotten very ill. My dad was recently diagnosed with cancer, and has had multiple surgeries. My mom has a myriad of health issues, and I feel painfully selfish by not helping them. They would never burden me, and pushing me to go pursue my career dreams.
I think this is an experience that most people will face in their lifetime. I don't necessarily know if this is an Asian experience, and the discussion of family burdens because in our culture the parents would never say a thing about their illness. But, I don't think what they realize is that it puts even more pressure on the children on accepting the right path.
But, this is not just Amazon specific. Even when discussing on renewable energies (e.g solar), this is the same argument that most conservative lawmakers and utilities make about applying a tax on residential solar. You have users of the grid that are not paying the same as others, but it's still utilizing the same grid as everyone else, therefore increasing the cost for everyone else that are not discounted, etc.
It is. You’re right. I never take Ambien recreationally.
Here’s the path I went down:
It quickly becomes this cyclical pattern where if you don’t take it, then you don’t sleep for days, so you’re forced to use it. And when I mean forced, I can’t go to work sleep deprived, especially in my line of work. You’ll build a tolerance quickly, then you begin ramping up your dosage, and this helps for awhile. But like a lot of long term ambien users know, you don’t fall asleep as easily anymore. Then the side effects really begin hitting you because you’re awake with a 30-60 mg of Ambien in you.
And if you have a history of drug abuse like I do, you do the dumbest thing possible and drink wine with ambien. This will knock you out for sure, but before that happens, you’ll begin doing extremely destructive things.
I’ve taken a considerable amount of drugs in my life, but Ambien is something else. It’s not a black and white drug as people say to just take and fall asleep. That’s how my first 6 months were like, but after 7 1/2 more years, it becomes something entirely different.
I binge eat when I'm on Ambien, and have intense food cravings. That night I didn't have food, and there was a McDonalds about 5 minutes away from where I lived. I crashed into a stop sign three blocks into it.
Even as an anonymous poster, there are stories that I am too ashamed to even post about. If anyone sees this, and had some inclination to try Ambien recreationally, don't even try it. What's worse is that even after destroying every relationship I've ever had because of Ambien, I still can't quit it.
I've been taking Ambien on and off for the past 8 years. Stretches where I've taken Ambien for a consecutive year.
I've done exactly what the author talks about: write long incoherent but beautifully written poetry, dive deep into studies and journals about the most random subjects, and have auditory hallucinations. But, what she doesn't talk about is the horror stories that come with it.
I've destroyed countless relationships. I've had friends that refuse to pick up any of my calls after 12am. I've crashed my car into a tree because of sleep eating. I've withdrawn from the medication, and don't sleep for 3-4 days. I've taken Ambien, then drank an entire bottle of wine, and the next morning to find out that I took 6 more Ambiens. I should be dead.
If you haven't taken Ambien in the past, and are interested, I would just avoid it at all costs.
Seems somewhat accurate. I grew up in the East bay, been robbed twice (both at knife) off the Fruitvale, West Oakland stops in high school. Bay Area was much different 10-15 years ago, you wouldn't see random people walking their dogs and drinking a lattee in Oakland (and some parts of SF) back then.
I used to be scared to walk through the Mission, now its just so much different.
(Worked many years in product pricing for startup to IPO company)
To even begin on a baseline of how you're going to price a product, you need to understand the unit economics of your product. I think too many people fall into the economics trap and apply a theoretical approach into revenue management, when in reality it's more about understanding your business model, how you expect to grow, and the finances; then you apply economic models.
Understanding the financial aspect will allow to create at least a cost based pricing model or margin based pricing; from there it can grow into a value based model where you figure out the value proposition for each customer/market segmentation.
But, obviously all of this is developed mid-stage, and early stage just figure out what gets the customers in the door.
There are a lot of good resources online about this subject.
With that being said, I don’t think the benchmarks we currently have are strong enough and the next frontier models are yet to come. I’m sure at this point U.S LLM research firms now understand their lack of infra/hardware optimizations (they just threw compute at the problem), they will begin paying closer attention. Now their RL-level and parent training will become even greater - whilst the newly freed resources to solve for sub-optimizations that have been traditionally avoided due to computational overhead