If these values really meant anything, then Anthropic should stop working with Palantir entirely given their work with ICE, domestic surveilance, and other objectionable activities.
PyTorch was partly inspired by the python Autograd library (circa 2015 [1]) to the point where they called their autodiff [2] system "autograd" [3]. Jax is the direct successor of the Autograd library and several of the Autograd developers work on Jax to this day. Of course, for that matter, PyTorch author Adam Paszke is currently on the JAX team and seems to work on JAX and Dex these days.
Direct indexing is WAY more valuable to US citizens living in the EU than US citizens living in the USA because of the painful intersection of MiFID II rules and US tax law (PFIC tax cancer makes buying EU domiciled funds a non-starter). Brokers will not sell US domiciled ETFs to US citizens living in the EU unless they can opt out of the consumer disclosure rules (e.g. by becoming an elective professional client of their broker under MiFID II rules). So these 2 million US expats have no choice but to manage a portfolio of individual stocks or pay exorbitant AUM fees. The first half-way decent direct indexing product that accepts US expats residing in the EU will make a killing!
Igor Markov, along with Sat Chatterjee, seem to be pursuing a bizarre vendetta (after Sat failed to take over their project) against the lead authors of the chip placement work, not some sort of intellectually honest critique.
This was covered previously in the press and on social media, with statements from a variety of prominent researchers (e.g.
[1][2][3]).
"The AlexNet paper is generally recognized as the paper that sparked the field of Deep Learning"
Uh not really. That would have been "A fast learning algorithm for deep belief nets" in 2006.
Also weird how this list completely ignores speech recognition. Deep learning's success in speech recognition predates AlexNet and motivated Google to create TPUs [1], and, more generally, invest in deep learning.
Going from 25% top-one error to 22% top-one error is a massive jump on ImageNet and very meaningful in a lot of applications. That said, there is no reason to believe attention-based models are the only way forward on image classification. The humble ResNet-50 can get near 22% top-1 error when trained properly.
Whether we need attention or not is a more interesting question on seq2seq models on text data.
Julia provides no meaningful advantages. PyTorch and JAX are too good. For typical deep learning workloads, Julia will not easily have a speed advantage. Everything goes down to the same cuDNN kernels anyway.
Julia seems like an attempt at a better matlab, but the machine learning world moved to python first.
(Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)
There are plenty of free, open access journals that are reputable.
JMLR http://www.jmlr.org/ is quite successful. There are some fields, such as machine learning, that are not dominated by for-profit journals. Why is this possible in some fields and not others? My answer would be that it is possible in all fields, but incumbency advantages can be very strong and coordination across academic volunteers can be more difficult when they are individually less secure.
In 2009 NVIDIA wouldn't give Geoff Hinton's group the time of day, let alone a single free GPU to try out to see if they wanted to buy more. Geoff Hinton spoke at a NIPS keynote in 2009 and told 1000 people in the audience to all go out and buy a specific NVIDIA card if they wanted to do work in this area and still NVIDIA didn't bat an eye.
I'm sure many AI experts at these places view it negatively. These are not monolithic organizations, but massive companies filled with real people with varying opinions.
The singularity cult is real. The geek rapture will occur when they blog about someone else making God in a Box hard enough to drown out the normal charlatans. Please fund their blogging, it is the most impactful way you can spend your money.
Also perhaps machine learning experts are paid well enough in industry research labs in tech companies that the difference isn't perceived as that large given the diminishing marginal utility of money.