It is unclear this model should be on that leaderboard because we don't know whether it has been trained on mteb test data.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
Though not relevant to the Atlantic ocean, it is worth noting that the Australian coral reef has in fact recovered from the disastrous state it was after the bleaching and damage events in 2016/2017. Reef coverage is in fact at the highest levels ever recorded in some areas.
It does make one wonder why this more recent good news has not found the same level of exposure as when the situation was bleak back then. Perhaps good news just doesn't gather as many clicks.
I believed the author of the blog was referring to tech roles specifically (not sales or similar).
I can say from personal experience that, at least in Australia, tech workers for mining companies that work in city offices are paid fairly similarly to other non-FAANG tech workers (e.g. banks etc). I also just checked levels.fyi for a few big mining companies and verified that this is the case.
Engineers (mining, geo, tech, whatever) that work out in the field do make quite a bit more. MAYBE around what FAANG would pay, but FAANG still pays more after a few years of refreshers and/if one manages to climb the ladder.
I think the author is quite off with their estimates of what people make in "heavy industries". At least as far as my experience goes for AU. FAANG/HFT still beats everything, and by a vast margin.
Though I agree with the idea that MLPs are theoretically more "capable" than transformers, I think seeing them just as a parameter reduction technique is also excessively reductive.
Many have tried to build deep and large MLPs for a long time, but at some point adding more parameters wouldn't increase models' performance.
In contrast, transformers became so popular because their modelling power just kept scaling with more and more data and more and more parameters. It seems like the 'restriction' imposed on transformaters (the attention structure) is a verg good functional form for modelling language (and, more and more, some tasks in vision and audio).
They did not become popular because they were modest with respect to the parameters used.
The reason why people care about privacy is not necessarily because giving up privacy has some directly observable negative effect. But, simply, living without freedom sucks.
I don't want you to know my personal information not because you could/would do something nefarious with it. I don't want you to have it simply because it's none of your business.
It's the first mobile app I have ever written and I enjoyed the process quite a bit!
My main goal was to deliver better identification accuracy than similar apps.
However I also wanted to provide useful plant information along with the identification and naively thought that this would have had to be a solved problem - surely there would be some online DB with all plants data neatly organized (I'd be even happy to pay for it!), in particular plant care information - but alas!
Lack of differentiability is actually a very important feature of the underlying optimization problem.
You might think that it doesn't matter because ReLU is, e.g., non-differentiable "only at one point".
Gradient based methods (what you find in pytorch) generally rely on the idea that gradients should taper to 0 in the proximity of a local optimum. This is not the case for non-differentiable functions, and in fact gradients can be made to be arbitrarily large even very close to the optimum.
As you may imagine, it is not hard to construct examples where simple gradient methods that do not properly take these facts into account fail to converge. These examples are not exotic.
Only tangentially related, but I have been trying to enroll for Apple's developer program for almost 3 months now.
Understanding what the problem is is essentially impossible. Going to a physical store doesn't help, calling their customer service has them telling you to go to www.apple.com/support (???), and writing for support has them rotate you through 4 different, and decreasingly useful, representatives.
The last response I got I was told the issue had to be handled by yet a different representative and it would take an "indefinite amount of time". Which may be a nice way of them saying it's never going to happen.
It really is demoralizing when you realize there is nothing you can do really, even in cases when you have done nothing wrong.
Would you be able to elaborate on your approach to TDD with spark sessions? You can persist them, which is only useful if you are doing multiple tests in a run.
But I find myself running one given test, making some code changes, and then wanting to run it again, over and over. Instantiating a local spark session takes several seconds every iteration. Enough for me to often want to "alt tab" into something else instead of waiting. It's very disruptive.
I did not know about Fugue but will definitely give it a try. Looks almost too good to be true.
Adding to the answers already given (complexity related to the computation of full Hessians). First order methods, i.e., methods that only require gradients, are the methods of choice when extreme solution accuracy is not required, which is the case in many practical settings (including ML).
One of Newton's method major selling points is that once it's close to a local minimum, under the right smoothness assumptions it essentially converges faster and faster to an exact optimizer - in practice you get "one more correct significant digit each iteration" once you are close enough. It's called Newton's method region of quadratic convergence, see [0] Theorem 14.1, p.3
This is not correct and proves the point made in the parent's comment.
Australia had much more severe bush fires in the 70s [0].
Note that many Australian plants are have evolved to adapt to bush fires, which means they must have been a staple of Australian ecology for many millions of years.
I see a lot of complaints about this but find little information about what the actual problem is.
The abc link referenced mentions that Murdoch's reach through physical newspapers is indeed outsized, but acknowledges that only ~11% of the population uses this as primary news source. There is nothing indicating anything resembling a monopoly in radio, TV, and most importantly digital.
"The Australian" belongs to News Corp, but is generally considered a respectable outlet. Opinion pieces are clearly distinguishable, and they also welcome contributions by left wing politicians (Tanya Plibersek and Jim Chalmers are examples I remember seeing relatively recently).
What evidence do you have to support your claim? Many countries have been subjected to sanctions for years, yet their military power hasn't diminished significantly (NK, Iran, ...).
The misery of the poor and disenfranchised, which you dismiss as people not "having anything anyways", has however been much exacerbated.
Have had to do lots of technical / latex-y drawings in the past as academic, and found Ipe [0] to be incredibly useful.
It is very minimalistic and easy to use. The images produced are simple to embed in latex documents, and they fit the style/font of the rest of the document. I think this tool should be a lot more popular than it is.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
[0] see excel spreadsheet linked here https://blog.voyageai.com/2024/09/18/voyage-3/