How is £$165/year not nutty? That is more than I pay per year for Amazon Prime, Spotify, Disney+, MLB or The Economist. All for basically being able to just write text in a browser and store it.
Ah yes, can't wait for a 250 page book that should've been a blog post full of high-level content that you can easily find online, padded out with tenuously linked anecdotes.
I’ve been interested in making an open source version of the Seeing AI app (https://www.microsoft.com/en-us/ai/seeing-ai) with a more limited scope. Contact me if this sounds inline with your ideas.
Can’t figure out a way to contact you from your About section —- I’m a ML engineer with an early interest in algorithmic trading, would be interested in brainstorming ideas.
I’m looking for collaborators for re-implementing “modern” machine learning and deep learning models/papers. Modern is in quotes as I’d actually like to focus less on the super recent, and more on those around ~5 years old, as the compute required is usually more feasible. As well as the implementation (which will be open sourced, well written and documented), I’d also like https://distill.pub/ style articles to go along with the implementations.
I’d also like to get into algorithmic trading, but this is something I’m at the very early stages of researching into.
If any of that sounds interesting, contact details are in my profile.
Here's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950
I'm not sure how to interpret the argument of the article or the results in the appendix here.
The first table shows AdamW having the best results, which follows the argument of the article. However, the following three tables all have plain Adam producing the best results.
The way the article is written it seems to be championing AdamW, but the results just seem to conclude that AMSGrad is bad and Adam is the best with AdamW having negligible performance increase over Adam in a single task.
It's bloat due to 'introns' (useless statements that don't effect the output, like x = x * 1). And yes, just adding a fitness function to shorten program length isn't optimal. I've found it easier to evolve successful programs (letting the bloat happen) and then keep removing statements from correctly generated programs whilst checking if the output is the same. Probably not optimal either but I feel like it gives better results.