There's a lot of interest in various ML communities on more efficient training and inference. Both vision and NLP have had a growing focus on these problems in recent years.
I think you make a good observation that much of ML progress is driven by tinkering with existing models, though instead of describing it as more "alchemy than science" it's probably more accurate to say it's very experimental right now. Being very experimental is neither unscientific nor unusual in the development of knowledge. James Watt worked as an instrument maker (not a theoretician) when he invented the Watt steam engine in 1776 [1], and at the time the idea of heat as Phlogiston [2] was still more prevalent than anything that looks like modern thermodynamics. Theory and practice naturally take turns outpacing each other, which is part of why we need both.
I'd also caution against the belief that experimental work doesn't require "particularly demanding thought". There are many things one can tweak in current ML models (the search space is exponential) and, as you point out, the experiments are expensive. Having a solid understanding of the system, great intuition, and good heuristics is necessary to reliably make progress.
For those who are interested in the theory of deep learning, the community has recently made great strides on developing a mathematical understanding of neural networks. The research is still very cutting edge, but the following PDF helps introduce the topic [3].
It's a good point and the study does some investigation of the question in Section 7 [1]. They find the trend seems to generalize across multiple speaker identities. Personal experiences appear more effective than facts at fostering respect for a wide range of different speakers.
I think this example misleads one's intuition for the following reason: in the proposed scenario, you'd only see the coin come up heads 10 times in a row about 1 in 1024 times you ran the experiment. While your conclusion would likely be incorrect, you almost never run into that scenario.
For example, if you conducted a study every week for 20 years, you'd both be extremely prolific and expect to have drawn about one wrong conclusion.
The example is a case of an absurd premise (i.e., a fair coin comes up heads 10 times in a row) leading to an absurd conclusion (that the coin is biased). Of course, this is exactly the guarantee the hypothesis test provides: under robust assumptions, you'll draw the wrong conclusion only rarely.
I don't have any experience programming without visual aids; and you've probably already thought of this aspect though I'll mention it just in-case: if I were facing a similar situation, I would look into long-term disability insurance and see if there's anything I can do on that front. That way, I could still pursue being productive with blindness or deafblindness, but would have less economic pressure while doing so.
Many employers offer a combination of short-term and long-term disability that will provide income (in some cases until you retire) in the event that you can no longer work.
I think this article omits the most important distinction between Bayesian and Frequentist statistics: subjective vs. frequentist interpretations of probability. In my own opinion, neither is "true", they're both just different tools for different purposes.
Bayesian inference is great when you have to make a decision and there are many theorems that illustrate this (for example, the arguments around coherence [1] and the complete class theorems [2]). In fact, Bayesian techniques are often useful for creating estimators with great frequentist properties! However, Bayesian interpretations of probability, and thereby the meaning of Bayesian statements, are inherently tied to the beliefs of an individual. That means that Bayesian statements usually aren't "true" in the objective / non-relative sense that we often expect from science. On the other hand, frequentist statements tend to have more of an objective flavor. The trick is: all our mathematical models have short comings and ways in which they're wrong when applied to any particular situation -- so neither really has a claim to being true.
The frequentist perspective often looks at worst case risk and tends to give a more global understanding of a procedure in terms of "how does this procedure shake out in all reasonably possible scenarios?". So, frequentist methods tend to be a bit more risk-averse which is often useful but can cost you for being to pessimistic. Ultimately, the real win is to know your tools well and to pick the right one for the job.
Oh yeah there are a few Bayesian methods which work great, Bayes-UCB is another. Personally though I think KL-UCB or just plain old UCB would be the ones I'd choose. Like I said earlier, I think these techniques are like programming languages: choose the one you know well enough to get the job done with it.
Actually, in the frequentist paradigm you could choose to run a sequential hypothesis test which will end when you've acquired sufficient data[1]. Or, if you want to get fancy you could use a multi-armed bandit approach which is probably optimal in many situations in perhaps a more robust way than many Bayesian methods[2]. Really both can work well. My advice is, use whichever you know well enough to utilize effectively!
In that case you'd want to use "git add -p" which allows you to pick only parts of a file to stage for a commit. It can be crucial in crafting a really solid project / commit history.
For even more complex cases you could use "git add -i"; however, that command can be tricky to work with and I find it's usually not to helpful to get that far into the weeds.
In my experience, making "gl commit" equivalent to "git commit -a" seems like a bad idea. When leading a team, especially of newer coders, one of the most effective ways I've found of keeping the git logs and the code base sane more generally, is to force people to review their own code at the time when they commit it. Individually adding each file with "git add -p" achieves this, and "git commit -a" squashes it.
Reading the methodology used to develop gitless yesterday was interesting, but if I recall correctly, I think it left something out. They looked at how often a software design allowed users to complete their intention, but the tool ideally not only should allow users to complete their intention, but should encourage users towards practices that increase the quality of the end product. While people may struggle with staging at first, I think it in the end encourages better software, which is my biggest concern.
I could be wrong, but I think the article that the Torvalds quote came from was purely satirical [1]. If you look at the tags it says it was posted under satire. Got me at first too!
If you're interested in clustering text documents, the canonical algorithm would be latent Dirichlet allocation, which is a topic modeling algorithm. You can find latent Dirichlet allocation in sklearn; however, you're more looking for something that returns a raw similarity score it sounds like, in which case it might be interesting to check out word2vec. Perhaps checkout this stack overflow answer: https://stackoverflow.com/questions/22129943/how-to-calculat...
For people interested in the use of neural nets to evaluate board position, it might be worthwhile reading up on TD-gammon which this work builds Off of
I think part of the problem is that our notion of productivity is tied up with producing, perhaps in some absolute and immediate sense. If we expand the idea to encouraging less work at the same level of efficiency I don't think it would be quite as controversial as discouraging productivity.
Because a lot of people really like it. One of the really cool things about lisp syntax is that it exposes the programs abstract syntax tree, making manipulating the program itself easier. It really appeals to a certain way of thinking about programs, and sure it's not how everyone thinks about them out how people should all the time, but it's a great tool to have on your box.
It's also important to remember that an author dying does not prevent him or her from capturing the value. If the copyright survives them, they can sell the rights to someone for a higher price before they die.
So I'm not an expert in the mathematics they're using specifically, but my high level understanding is that convolutional neural nets are able to extract features because they satisfy certain invariance properties. For example, a face is still a face even if it's shifted to the right, made a little smaller, or tilted a bit. Therefore, the feature of faces should be invariant to these transformations. This paper develops the mathematical framework to better help understand when such invariance properties pop up in a wider class of deep neural net architectures, and thereby when these architectures can identify broader classes of features.
I agree, but I'd expand on that and say that BPL and Probabilistic programming in general benefits hugely from the fact that it is interpretable and thus useful in ways that deep learning is not quite as useful. On top of that, the integration of causal concepts allows the program to generalize to completely novel situations in ways that it's unclear whether it nor other techniques can. For example, can we teach a neural net that if the sun didn't rise tomorrow people would still go to work at 9am? (As long as they weren't too freaked out of course :-) )
The three body problem though is famous for having no solution in terms of basic functions. I think for your planets model you must have been making different assumptions than in the article.
Interesting article. It seems to me like what you really want here is to do this in a lisp/scheme. Then you can use the full power of lisp (it's the exact same syntax as the behavior tree), leverage lisp's built-in control flow constructs, and just implement nodes as functions with side-effects that return either true, false, or null or something.
I think you make a good observation that much of ML progress is driven by tinkering with existing models, though instead of describing it as more "alchemy than science" it's probably more accurate to say it's very experimental right now. Being very experimental is neither unscientific nor unusual in the development of knowledge. James Watt worked as an instrument maker (not a theoretician) when he invented the Watt steam engine in 1776 [1], and at the time the idea of heat as Phlogiston [2] was still more prevalent than anything that looks like modern thermodynamics. Theory and practice naturally take turns outpacing each other, which is part of why we need both.
I'd also caution against the belief that experimental work doesn't require "particularly demanding thought". There are many things one can tweak in current ML models (the search space is exponential) and, as you point out, the experiments are expensive. Having a solid understanding of the system, great intuition, and good heuristics is necessary to reliably make progress.
For those who are interested in the theory of deep learning, the community has recently made great strides on developing a mathematical understanding of neural networks. The research is still very cutting edge, but the following PDF helps introduce the topic [3].
[1]: https://en.wikipedia.org/wiki/James_Watt
[2]: https://en.wikipedia.org/wiki/Phlogiston_theory
[3]: https://www.cs.princeton.edu/courses/archive/fall19/cos597B/...