Baidu Hires Andrew Ng(technologyreview.com)
technologyreview.com
Baidu Hires Andrew Ng
http://www.technologyreview.com/news/527301/chinese-search-giant-baidu-hires-man-behind-the-google-brain
17 comments
Andrew Ng on why he decided to leave his day-to-day responsibilities at Coursera for Baidu: http://blog.coursera.org/post/85921942887/a-personal-message...
Usual fluff. Only relevant part is,
With the MOOC movement healthy and growing, I now plan to dedicate more of my time toward AI (Artificial Intelligence) and machine learning. I will be joining Baidu as their Chief Scientist to help build out a research organization, headquartered in Silicon Valley. Both education and AI have been longstanding passions for me. I believe some of my skills will allow me to make a contribution to the latter.
With the MOOC movement healthy and growing, I now plan to dedicate more of my time toward AI (Artificial Intelligence) and machine learning. I will be joining Baidu as their Chief Scientist to help build out a research organization, headquartered in Silicon Valley. Both education and AI have been longstanding passions for me. I believe some of my skills will allow me to make a contribution to the latter.
Wow, that was unexpected. I admire Ray Kurzweil but Andrew has always struck me as that rare sort of person who can both see the big picture and pick out the stepping stones to getting there. That is incredibly valuable and I did not expect that Google would let him be poached by a rival search company.
I think Andrew had already severed his official ties with Google when he started Coursera, since he was probably the person to recommend Google hire Geoff Hinton to replace him (see [1]). If Andrew had returned to Google, I don't think he would have as great of a leadership role as the one that Baidu is offering to him, since there are several other AI leaders that are just as prominent, if not more so, at Google (like Hinton and Kurzweil).
[1] http://www.wired.com/2013/03/google_hinton/
[1] http://www.wired.com/2013/03/google_hinton/
> since there are several other AI leaders that are just as prominent, if not more so, at Google (like Hinton and Kurzweil).
Norvig > Kurzweil
Norvig > Kurzweil
Norvig himself has a different take on that:
http://www.youtube.com/watch?v=zihTWh5i2C4#t=86
http://www.youtube.com/watch?v=zihTWh5i2C4#t=86
Nothing of a "different take" out there. And even if there would have been, it's only natural to speak praisingly about somebody you introduce.
I just watched Norvig's 100K student talk. Quite impressive and mind blowing to think about.
The most interesting thing about this announcement is the huge emphasis on GPUs at Baidu driven by Ren Wu:
http://on-demand.gputechconf.com/gtc/2014/presentations/S465...
Combine this with Andrew Ng, whose own student Adam Coates matched Google's cat detector network with 64 consumer grade GPUs, and we are in for interesting times...
http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC...
http://on-demand.gputechconf.com/gtc/2014/presentations/S465...
Combine this with Andrew Ng, whose own student Adam Coates matched Google's cat detector network with 64 consumer grade GPUs, and we are in for interesting times...
http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC...
In the presentation you linked by Baidu's Ren Wu on page 6 it says:
Deep Learning:
AI Brain - World's first RI
Might just be "Research Institute":
'The Institute of Deep Learning (IDL) is the first formally established research institution projected to be Baidu's “Artificial Intelligence Lab”.'
source: http://idl.baidu.com/en/IDL-about.html
'The Institute of Deep Learning (IDL) is the first formally established research institution projected to be Baidu's “Artificial Intelligence Lab”.'
source: http://idl.baidu.com/en/IDL-about.html
Ah yes I think you're right - I guessed the I stood for intelligence and couldn't come up with anything.
[deleted]
I've seen the acronym used for Restricted Intelligence, though I'm not sure that its the intended meaning here.
Adam Coates has done many more impressive things than that. The "cat detector" was an uninteresting piece of research and not very impressive at all. The image net results were bad and nets trained on one or two gpus were doing much more interesting things.
In Hinton's Coursera class, which was very good, he made a strong point about GPUs being the way to go (at least for now) in building deep learning networks.
Why are they still using GPUs? Maybe hardware companies should start making AI targeted processors instead. Could they boost performance 10 times with a dedicated processor?
GPUs out perform distributed computing currently for deep learning training, but it will be interesting to see how MPI and CUDA will be combined in the future (there are already people working on that, but its still early).
A more detailed article by MIT Technology Review: http://www.technologyreview.com/news/527301/chinese-search-g...
Thanks. We changed the url to that from http://www.marketwatch.com/story/baidu-opens-silicon-valley-....
As a Chinese I have to say baidu dose have a bad reputation even in this country. Even my friends that works at baidu feels confused after heard this.
Baidu devotes a lot in algorithm and machine learning. But when it comes to Chinese market we can see that technology does not even play an important role in one companies' success (marketing and government relations plays a more important role).
I am wondering how long will baidu keep investing in deep learning as it has so many troubles in international market (where technology really matters).
Baidu devotes a lot in algorithm and machine learning. But when it comes to Chinese market we can see that technology does not even play an important role in one companies' success (marketing and government relations plays a more important role).
I am wondering how long will baidu keep investing in deep learning as it has so many troubles in international market (where technology really matters).
baidu sucks. And we Chinese know it. I guess it has to keep investing in relatively advanced tech to keep its high tech image for China market in U.S. stock market. This at least is one of the results baidu wants.
Baidu's deep learning based image search is actually quite impressive: http://shitu.baidu.com/
When I put in an image URL or search term, I would get:
抱歉，您需要检索的图片我们无法下载，请从本地上传图片进行检索。
百度识图测试版感谢您的使用，我们会继续努力提升效果。
Google Translate: Sorry, you need to retrieve the pictures we could not download, please be retrieved from the local to upload pictures. Baidu knowledge map Thank you for using the beta version, we will continue our efforts to enhance the effect.
When I tried to upload an image, it would just hang, sometimes freezing Chrome.
Eventually I got this: http://shitu.baidu.com/i?ct=2&tn=shituresult&pn=0&rn=10&quer...
It recognized and returned other cats of a similar color, which is pretty cool. Just wish the way there was a little cleaner.
Maybe Baidu's expanded Deng's saying. Now it doesn't matter if a cat is mangy and covered in fleas, as long as it catches mice.
Google Translate: Sorry, you need to retrieve the pictures we could not download, please be retrieved from the local to upload pictures. Baidu knowledge map Thank you for using the beta version, we will continue our efforts to enhance the effect.
When I tried to upload an image, it would just hang, sometimes freezing Chrome.
Eventually I got this: http://shitu.baidu.com/i?ct=2&tn=shituresult&pn=0&rn=10&quer...
It recognized and returned other cats of a similar color, which is pretty cool. Just wish the way there was a little cleaner.
Maybe Baidu's expanded Deng's saying. Now it doesn't matter if a cat is mangy and covered in fleas, as long as it catches mice.
shitu, not the best choice of word.
It makes perfect sense in Mandarin, and since it's a Chinese site, I'll call it a reasonable choice of word.
I'm sure we could form a pejorative from it. Like 你是土, pinyin is just so ambiguous.
你是吐, for when you want to look for a spitting image of yourself ;)
It means "learn picture" in Chinese. It's in the title. 識圖 even though the pinyin is not that pleasant to people who knows English, Baidu is a Chinese search engine and most of the users are Chinese speaking users. Furthermore, as a Chinese speaking person I can well make fun of it by saying "shxt" every time the engine spits out the right image!
Well, to be fair, Baidu is going to use simplified characters, not traditional ones, given their mainland focus.
Yes I am aware of that. I just happen to be a Traditional Chinese writer :D but either writing system shouldn't matter.
I don't know why you got donwvoted, your comment is perfectly valid. Plenty of case studies about Anglo-Saxons making silly linguistic faux pas in other countries, including China. China has a huge domestic market but the companies there will eventually also have business case studies warning against "shitty" linguistic mistakes ;)
This, however, is a site in Mandarin targeted at Chinese. "Shitu" is not intended for Western markets, it just happens to appear in Latin script in a URI.
Wow, I just finished his Coursera ML class this week. I hope that course continues to be offered. I wish I could tell him thanks for teaching it! The way he balances cutting-edge research with popularization of hard concepts reminds me of Richard Feynman. Best of luck to him.
Yeah, what coincidence. I was just watching one of the Coursera ML videos on Linear Algebra less than 15 minutes ago. He's such a good teacher, he makes ML look so approachable.
It will definitely still be offered -- that was one of the questions we asked him at All Hands yesterday :).
[deleted]
I just finished this also. Great course.
Hm, leaving Coursera to work at the most censored search engines of all times? Not a good day for humanity...
"For now, relatively few people are versed in the tricks needed to get deep learning to work well"
How does one exactly become one of these people? Is any of this information public yet?
How does one exactly become one of these people? Is any of this information public yet?
The other responses to your question tell only part of the story. Yes, the academic groups that work in deep learning publish papers describing their methods. But these papers are rarely sufficient to be able to recreate the models they built.
There's a lot of other knowledge/expertise/intuition that's required to make working implementations. There have been some deep learning tutorials at recent conferences that might be more in-depth. (See my previous comment [1] for details.)
Another good way to learn is to look at some open source implementations, such as caffe from berkeley [2] or overfeat from NYU [3].
In addition to showing how to choose architectures or set params, they also have tricks for speeding things up. This is actually very important, as they can make orders of magnitude difference (training in hours vs days).
[1] https://news.ycombinator.com/item?id=7742192
[2] http://caffe.berkeleyvision.org/
[3] https://github.com/sermanet/OverFeat
There's a lot of other knowledge/expertise/intuition that's required to make working implementations. There have been some deep learning tutorials at recent conferences that might be more in-depth. (See my previous comment [1] for details.)
Another good way to learn is to look at some open source implementations, such as caffe from berkeley [2] or overfeat from NYU [3].
In addition to showing how to choose architectures or set params, they also have tricks for speeding things up. This is actually very important, as they can make orders of magnitude difference (training in hours vs days).
[1] https://news.ycombinator.com/item?id=7742192
[2] http://caffe.berkeleyvision.org/
[3] https://github.com/sermanet/OverFeat
Excellent response. Thank you.
Could you clarify your question? Are you asking how does one become an AI/ML expert? The traditional path is to study CS/EE-AI-ML and become a researcher.
Most of the academic/industry papers are available. Just browse the nips/icml/sigir/etc conference websites.
But then you'll need to apply that knowledge - ideally by working in the industry. Or you can go try your luck in kaggle and other competitions.
Most of the academic/industry papers are available. Just browse the nips/icml/sigir/etc conference websites.
But then you'll need to apply that knowledge - ideally by working in the industry. Or you can go try your luck in kaggle and other competitions.
> Is any of this information public yet?
Most Machine Learning research that is not done by top secret labs is available publicly. Start looking at papers by Geoffrey Hinton. Could also take a look at word2vec if you are into NLP.
Most Machine Learning research that is not done by top secret labs is available publicly. Start looking at papers by Geoffrey Hinton. Could also take a look at word2vec if you are into NLP.
China is in the process of becoming a real superpower. They are making what US made last century: attracting brains.
http://www.universityworldnews.com/article.php?story=2014032...
http://www.universityworldnews.com/article.php?story=2014032...
Andrew, is in my opinion, one of the smartest people alive today. Call it what you'd like, but I'd love to see his work going into an American company.
I don't think the nationality of the company matters.
There are good reasons to want America to succeed that have nothing to do with nationalism. In particular, China has a very poor record on human rights, and is the main supporter of North Korea. This doesn't make the company bad, but it's not an irrelevant fact.
And the US has better records on human rights with all the lawful torturing and countless wars going on?
I'm pretty sure that China has caused less deaths or generally harm to innocents than the US has in the last 20 years.
I'm pretty sure that China has caused less deaths or generally harm to innocents than the US has in the last 20 years.
As sibling comments say, there is no way that America can attempt to occupy any sort of moral high ground on human rights given the events of the last decade. Where have you been?
Perhaps this would be a wake up call for all the politicians driving the US down in terms of IT progress. Or, perhaps all they care about is money and it won't change a thing.
Cheering for a country based upon an image of that country is the very definition of nationalism.
>China has a very poor record on human rights
Guantanamo
Guantanamo
Really? China's regular court system has even less transparency and integrity than the Guantanamo courts, let alone the regular U.S. court system. I'd have more chance of a fair trial at Guantanamo than in any court in China if the party authorities had decided that you were guilty.
When China isn't a single-party authoritarian state, maybe then we can talk.
When China isn't a single-party authoritarian state, maybe then we can talk.
The US is a single party state. The party's name is "capitalism."
I share that opinion and giving the fact that Hinton is on Google tells me that this would balance quite nicely the search engine industry
I agree with you completely , i always wanted to work with him or for him , just to see how his thought process worked.
He is the reason that i want to get a Graduate degree in machine learning.
I 100% agree that Andrew is one of the smartest people I've ever met, but I for one wouldn't mind if Baidu started to challenge Google in the deep learning space. International competition will be better for American companies than Google's current monopoly on the research and experts.
It never ceases to amaze me how many americans completely overlook this in their blind patriotism. Baidu today may well be the only company able to challenge Google's strength on the search engine land, and an effective monopoly (i.e. if Google got their way with China and toppled Baidu) would be sad for mankind. Here's hoping instead that a breath of american mindset at Baidu can slowly make it reach out of China and who knows, challenge Google with (non-censored as in China, and non-NSA'd as in Google) english results.
Given Baidu's long history of being "cooperative" with Government, I cannot support such a company to be an effective competitor to Google. Besides, deep learning is about far more than just search.
Every company has to be cooperative with its government, or relocate/close.
It just happens that Google isn't based in China, therefore they were able to make the decision not to expand to China.
But in the US, Google will have to accept basically anything the government demands, as long as it is backed by the courts. (which isn't that hard since the government just invents secret courts based on secret laws that back almost 100% of the governments demands)
It just happens that Google isn't based in China, therefore they were able to make the decision not to expand to China.
But in the US, Google will have to accept basically anything the government demands, as long as it is backed by the courts. (which isn't that hard since the government just invents secret courts based on secret laws that back almost 100% of the governments demands)
You can say the same about Google.
It looks like the trend of search engines becoming more like recommender systems will continue.
[deleted]
Baidu has a long history of being one of the most proactive and restrictive online censors in the search arena so how does he reconcile this contradiction?
Careface.
Money.
I think it's natural for Baidu to offer this kind of search capability as a means to hunt down restricted information disguised in non-text format.
They should have hired Chuck Norris instead to shoot all the people that block the internet in China or make the internet slow. With slow I mean slowing gmail down by factor 40.
Every Bozo downvoting this has obviously never lived in China
I expect the downvotes have little to do with true/untrue, but rather that the comment is off topic and doesn't advance the conversation.
[deleted]
By completely leaving Coursera, this is a vouch of confidence for MOOcs in general. sarcasm Horray for worthless online degrees that have an attrition rate past 90% of total registered students and a near-zero overall completion rate. I'm sure MOOCs will have a great future henceworth now that the CEO of a very visible MOOC company has left.
The attrition rate is due to the lack of cost involved in opting out if you find yourself unable to keep up with the course / too many courses. Being able to try and see if you can do a course (despite the other commitments in your life) is actually very attractive because most people cannot assess their goals or time budget that accurately.
From the MOOCs' point of view, this is a worthwhile feature because while it does increase the false positive rate tremendously, it also increases the true positive rate : more people sign up for the course because the barrier to entry is low and end up completing it.
Also, it does a wonderful job of introducing a new product to the market : I do not know a single friend in my circle who has not tried out Coursera. And guess what, I would happily pay up if they charge a reasonable fees in the future!
From the MOOCs' point of view, this is a worthwhile feature because while it does increase the false positive rate tremendously, it also increases the true positive rate : more people sign up for the course because the barrier to entry is low and end up completing it.
Also, it does a wonderful job of introducing a new product to the market : I do not know a single friend in my circle who has not tried out Coursera. And guess what, I would happily pay up if they charge a reasonable fees in the future!
MOOCs are great for people who want to learn on their own (although no different than classes in college, some are more valuable than others). I'm not sure of the value these bring in getting a job, but the learning value makes them very worthwhile.
> MOOCs are great for people who want to learn on their own
MOOC + in person coaching could solve the problem. We don't need professors, we need coaches (experts in education, psychologists) to keep people motivated and not drop off. Also, in large cities study groups could generate the same feeling as going to a real class.
What we are missing in MOOCs is the motivational effect of meeting people (in real life) who are serious about learning and seeing them work hard. That is what makes us believe we can do it too.
MOOC + in person coaching could solve the problem. We don't need professors, we need coaches (experts in education, psychologists) to keep people motivated and not drop off. Also, in large cities study groups could generate the same feeling as going to a real class.
What we are missing in MOOCs is the motivational effect of meeting people (in real life) who are serious about learning and seeing them work hard. That is what makes us believe we can do it too.
MOOCs are even better for people who have to learn on their own. Which is almost anyone, most of the time. You usually don't have time or money for degrees in multiple fields, so most "continued learning" if not in very rigid programs, is essentially "learning on your own".
The "Mooc" movement is actually going as strong as ever. The goals of these courses is not to have high completion rates, but to give anyone a chance of learning as much as he wants.
Sometimes though, moocs are held to standards and goals they never ever set themselves, like replacing universities, which none of them actually want.
Sometimes though, moocs are held to standards and goals they never ever set themselves, like replacing universities, which none of them actually want.