Baidu claims deep learning breakthrough with Deep Speech(gigaom.com)

7 pointsby sherjilozair12 lat temu1 comments

Baidu claims deep learning breakthrough with Deep Speech

https://gigaom.com/2014/12/18/baidu-claims-deep-learning-breakthrough-with-deep-speech/

1 comments

I read the paper - it's interesting and definitely improves on prior efforts. But I wouldn't call it a "breakthrough" - a few percent better accuracy on some datasets (with no real discussion of other measures of performance), and the algorithm they use is dead simple: a recurrent neural network with rectified linear units (as opposed to Long Short Term Memory). It sounds to me like the major improvements they made were to use a ton of data, and a ton of processing power - the interesting part of the paper is largely about data partitioning to take advantage of multiple GPUs, not about a novel learning algorithm or network architecture.

Not to discredit work by what I'm sure is a very effective machine learning research team - this paper is probably important, but as an incremental improvement on prior algorithms that takes advantage of modern hardware, not a dramatically new approach.

I guess the "breakthrough" is showing that pure deep learning (without fancy acoustic models, etc) can perform well - which is pretty cool.