This is the correct answer. A key reason Rebco is so much better than older alternatives is that it can be cooled to superconductivity using only liquid nitrogen (77K), as opposed to liquid helium, which is much harder to work with.
Longtime DVC user here - this is going to be so helpful. We use DVC for all of our model and data versioning, but what's been missing is the ability to cleanly integrate that into our CI workflow. Looks like that's solved now! The cml.yaml syntax also looks quite nice, very easy to follow. Looking forward to trying this out.
Great anecdote from Jeff Dean's twitter [1] about how they thought to try this experiment, given that doctors didn't know such a thing was possible:
"Funny story. We had a new team member joining, and @lhpeng suggested an orientation project for them of "Why don't you just try predicting age and gender from the images?" to get them familiar with our software setup, thinking age might be accurate within a couple of decades,and gender would be no better than chance. They went away and worked on this and came back with results that were much more accurate than expected, leading to more investigation about what else could be predicted."
This is not "AI that builds AI". The actual research behind AutoML is called NASNet (https://arxiv.org/pdf/1707.07012.pdf), and all it is simply: we found two good neural network layers (called NASNet normal cells / reduction cells in the paper) that work well on many different image datasets. It's a very cool research result. But it's not something that will replace AI researchers.
The AI Grant fellowship has been an awesome experience so far; I highly recommend applying if you're on the fence. The grant itself is useful (and this round includes $20k of Google cloud credits!) but equally valuable is the community that Nat has curated around the project. There are a lot of great folks working on AI from all sorts of places just a Slack message away. Now that Daniel Gross (YC's AI-focused partner) is involved as well, I'd expect that community to continue to grow.
This post doesn't even mention the easiest way to use deep learning without a lot of data: download a pretrained model and fine-tune the last few layers on your small dataset. In many domains (like image classification, the task in this blog post) fine-tuning works extremely well, because the pretrained model has learned generic features in the early layers that are useful for many datasets, not just the one trained on.
Even the best skin cancer classifier [1] was pretrained on ImageNet.
"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train."
Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models that use BatchNorm nowadays, so generally tuning there is not an issue either. Many layers of 3x3 conv with stride 1 is still magical.
Basically: deep NNs can work pretty well with little to no tuning these days. The defaults just work.
- Labeled data is very expensive. Historically attempts to learn on synthetic data has failed because ConvNets are very good at detecting small visual artifacts in the synthetic data and using those for classification during training. At test time on real data, those artifacts aren't present so model fails. A technique that can beat state-of-the-art (admittedly on a very narrow Eye Gaze dataset, but still) by only training on labels from synthetic data and testing on real data is important.
- They present a useful new idea to improve GAN training: using a history of "fake" images, rather than only the latest fake images from the generator. Ask anyone who has tried to train a GAN: the training is really unstable, each network only cares about beating the latest version of its "opponent". They show good improvements by saving many previous fake outputs to make the generator more robust. This reminds me of Experience Replay from DeepMind for RL.
- It's a published paper from Apple! Great that they are starting to contribute back to the research community.
This paper builds off of DeepMind's previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today.
The feeling among researchers I've spoken to is not that NTMs aren't useful. DeepMind is simply operating on another level. Other researchers don't understand the intuitions behind the architecture well enough to make progress with it. But it seems like DeepMind, and specifically Alex Graves (first author on NTMs and now this), can.
Highly recommend this video on the Deep Visualization Toolbox to anyone interesting in understanding more about how convnets work through visualization:
What's fantastic about this is that even though the class is at Stanford, everyone from around the world can follow along just as easily. Stanford is opening up more and more of its opportunities to the general public, which is awesome. Another one I'd recommend taking advantage of is Stanford's Hacking Hours:
This is a weekly on-campus meetup where people hack on their projects in a collaborative environment. It's open to the general public, and anyone nearby with code to write is welcome to come work on it at Stanford.
Interesting to note that you started this reply with "I think." Would it have carried more weight if you started with "perhaps" instead, or stripped it altogether? Maybe. Either way it's fun to think about.
For the curious: the reason that this property with 24 holds is because 24 = 2^3 * 3. For any prime number p:
p^2 - 1 = (p+1)(p-1)
And p+1 and p-1 must both be multiples of 2 because p is odd. Furthermore, one of p+1 or p-1 is also a multiple of 4 (because they are both multiples of 2 and only 2 apart). So, we can see where the 2^3 factor comes from in the magic number 24. The remaining factor, 3, comes from the fact that p is prime and not a multiple of 3, so either p+1 or p-1 must be a multiple of 3 (otherwise p-1, p, and p+1 would be three consecutive numbers, none of which are divisible by 3, which is impossible).
As a result, for any prime p > 3, (p+1)(p-1) is divisible by 24, so p^2 - 1 is also divisible by 24.
My aunt used to work at a school that did not have this policy in place. The principle had a debilitating head injury that put her in a coma; she is now unfit to work for the rest of her life. It was tremendously sad for everyone involved, but it has also severely strained the school financially. Under Massachusetts law, the school is required to pay a large percentage of her original salary now that she is disabled. Had they taken out an LTD insurance policy this wouldn't have been a problem.
What is most interesting to me is how the test shows the value we enjoy of living in the information age.
Without instant access to rote information, knowing some of these facts was probably rather important a hundred years ago. Schools today can avoid burdening the student with memorizing so many facts and focus more on developing conceptual understanding, which – coupled with free, rapid access to factual information – is much more valuable.
For anyone trying to determine how they can help: make a phone call. Politicians care about phone calls infinitely more than tweets or Change.org signatures. If enough people call the DoR (617-887-MDOR) and their MA representative with complaints, it's very possible that something could change.