Microbatch size is a hyperparameter, it can be set to 1 and work just as effectively. With gradient accumulation it's equivalent even. Large batch sizes are used to increase parallelism, and sometimes to reduce variance in the loss signal (at the cost of increased bias).
Batch size is frequently limited by compute bottlenecks well before memory.
No one wants to be around an alcoholic. But it's not to be nice, but to understand them, their struggles, and their perspectives.
Anecdote - I was at a concert with my brother, outside in the smoking area. A homeless woman came by, didn't say anything but was kinda just looking around. My brother is the only one who spoke to her, he offered his cigarette butt. Which is exactly what she was looking for.
Never would have occured to me- but I would guess it made her feel more seen and human.
He wasn't the only one who saw the problem computationally. Famously, the mathematician Michael Waterman sat on the other-side of the race for the human genome.
not really. early deep learning models were run on single consumer-grade GPUs. the inflection occured _right_ when parallel computing became fast enough to do backprop in a reasonable amount of time with performance better than tree methods.
at that time all the compute resources in the world would not have been enough to train the models from even the last ~6 years or so, probably more.
3x increase in compute for a 1.5x increase in tdp is pretty good considering the underlying process had barely changed. In anycase, consumer GPUs aren't a good metric as they operate with different economic constraints.
H100 to GB200 saw a 50x increase in efficiency, for example.
> I wanted to conjure models I could play with in my hand. I wanted a museum where I could walk around inside the epithelium during an immune response. I wanted to put ideas into physical space, like on a pinboard—TLRs go here, with the other innate armament; CD4+ T cells are there, in the adaptive world—but I wanted it to be as searchable, copy-pasteable, shareable, and composable as text.
I recently dealt with this. Right before my free trial expired, I went to cancel. A support rep offered another 3 months free. The first red flag was not receiving a log of our conversation. Three months go buy and sure enough, when I went to cancel, they told me I had already committed to a year. Incredible. After some persistence and vague legal threats I managed to receive a full cancelation and refund.
For oregon the population center is almost exactly Detroit. There is a large population in the willamette valley south of portland (and bend is captured by this too).
I have no idea where you guys are seeing these prices. Chalice, on the absolute high end that I've seen, tops out at 250/oz.
Nectar/Green Cross are my gotos, both of them are well below the prices yall are discussing. Green cross in particular has better stuff than the other venues as well. Maybe try shopping in Salem instead of Portland? Dunno, I never go into the city for weed.
go to greencross in bend, they have the exact prices i'm referring to. Dont go to the stores that purchase from bulk retailers, shops that have their own farms typically have higher quality and lower prices.
try this place in salem, really nice stuff and great prices. "Reserve" is their top-of-the-plant pickings that go for $160 oz. Not a big difference between their bulk (75-100) and reserve.
At 49C you need a dewpoint of almost 70 (!!) to get 20% relative humidity. thats nasty -- during the heatwave in portland the dewpoint was between 63-68 most days. Usually its ~55. For context, the dewpoint on the east coast during the storm season usually is between 70-75.
Batch size is frequently limited by compute bottlenecks well before memory.