Fewer than 10k people have the skills necessary for AI research?
9 comments
Exploding demand is the real catalyst producing AI talent shortages. At both the Fortune 100 end of the spectrum, as well as the 3-person garage startup level.
Think about Spotify's Discover Weekly feature and how it multiplies user engagement. Collaborative filtering of 100M users can possibly be done by a CS undergrad on their NVidia 1050 gaming laptop in a couple of hours. Metadata analysis of 1B songs is more of a grad level problem for someone with access to the departmental research cluster. But machine learning audio tempo, frequency and pitch similarity for thousands of new songs added daily? That requires a corporate R&D budget and dedicated cloud GPU resources. And at least one luminary for the "breakthrough" that takes you from 95% to 97-99% human level understand which will make the technology world competitive.
The great thing about AI that perhaps many overlook is that genius is not required to build a startup with unicorn status valuation ;) Look at the recent YC article on Toutiao, where a handlful of engineers built a news content generation algorithm in a few months. If your input data quality is very high at the outset, even standard sci-kit out of the box t-SNE classification can prove "unreasonably effective"!
The Hidden Forces Behind Toutiao: China’s Content King
http://blog.ycombinator.com/the-hidden-forces-behind-toutiao...
Think about Spotify's Discover Weekly feature and how it multiplies user engagement. Collaborative filtering of 100M users can possibly be done by a CS undergrad on their NVidia 1050 gaming laptop in a couple of hours. Metadata analysis of 1B songs is more of a grad level problem for someone with access to the departmental research cluster. But machine learning audio tempo, frequency and pitch similarity for thousands of new songs added daily? That requires a corporate R&D budget and dedicated cloud GPU resources. And at least one luminary for the "breakthrough" that takes you from 95% to 97-99% human level understand which will make the technology world competitive.
The great thing about AI that perhaps many overlook is that genius is not required to build a startup with unicorn status valuation ;) Look at the recent YC article on Toutiao, where a handlful of engineers built a news content generation algorithm in a few months. If your input data quality is very high at the outset, even standard sci-kit out of the box t-SNE classification can prove "unreasonably effective"!
The Hidden Forces Behind Toutiao: China’s Content King
http://blog.ycombinator.com/the-hidden-forces-behind-toutiao...
All depends on how you define "tough A.I. problems". Basically for all N>0 there exists a definition of "tough" such that less than N people are capable of "Solving tough A.I. problems"
For any reasonable definition, just having a Masters in a subject certainly does not qualify you to do tough AI research. For that matter just having a PhD in a subject doesn't really qualify you to do high quality research in that subject.
That being said there is a lot of 'easy' AI problems out there to be tackled. Take just about any random domain that you even a basic understanding of, collect some data, apply some interesting sounding algorithms from scikit-learn to that data, tweak some parameters, does anything interesting fall out? If no try some new data and/or new algorithms. Repeat until something interesting happens. Congratulations you've just done some AI research, probably good enough to be published somewhere, and all you needed was high school level math and some basic programming skills.
For any reasonable definition, just having a Masters in a subject certainly does not qualify you to do tough AI research. For that matter just having a PhD in a subject doesn't really qualify you to do high quality research in that subject.
That being said there is a lot of 'easy' AI problems out there to be tackled. Take just about any random domain that you even a basic understanding of, collect some data, apply some interesting sounding algorithms from scikit-learn to that data, tweak some parameters, does anything interesting fall out? If no try some new data and/or new algorithms. Repeat until something interesting happens. Congratulations you've just done some AI research, probably good enough to be published somewhere, and all you needed was high school level math and some basic programming skills.
Frankly, I think that there are fewer than 10k people who have the skills for quality research in pretty much any field (n.b., the all-too-frequent p-hacking researcher is not "quality").
I don't think that this is news.
I don't think that this is news.
Having skills is not enough. Is more a question of having enough resources, and interesting questions crossing the road in your way.
And (fortunately) many great discoveries are a question of pure luck. I wonder how this fits in their model.
And (fortunately) many great discoveries are a question of pure luck. I wonder how this fits in their model.
The news is the surge in demand for this specific skill set. If you haven't been paying attention to AI/CS, this might come as news.
Judging just by the number of people doing the CS231n course online http://cs231n.stanford.edu/, the number of would-be practitioners who have the skills is definitely increasing. CS231n is a graduate level class and is conceptually harder than a "flavour of the month" app. Add to that the number of articles being published on DIY experiments with AI, I think the 10K number seems small. Do these qualify as "serious" research? Maybe not, but they are not all trivial.
Ya know, part of me wants to say "what a load of bollocks. Doing AI research doesn't take that much that's special." And in truth, it doesn't take a lot of what most of us would consider "special". I mean, if you can get through multi-variable caculus, linear algebra and probability / stats, you pretty much have the mathematical tools you need. So in that regard, I do think it's bollocks.
But...
I would argue that AI research (depending on how you define that. AI is a BIG field) is more demanding that "flavour of the month smartphone app" development. And in the past, I scoffed when I heard people say things like "70% of so-called developers can't complete FizzBuzz." Then I started interviewing a lot of candidates for developer roles at the $dayjob and whaddaya know... it seems that most of the candidates we get can't successfully code FizzBuzz. And that's a pretty low bar.
So I dunno... part of me believes there are plenty of people who could contribute to AI research, but part of me is shocked by how few people can even write the most trivial computer program.
Are there fewer than 10k people with a master's in the subject in the entire world?
I don't think having a Masters has much to do with anything. We get folks all the time with a Masters in C.S., even from relatively prestigious schools, and they can't even code up FizzBuzz.
Anyway, I wish I had a better answer for you, but recent experiences have me questioning just how many really competent people there are out there.
But...
I would argue that AI research (depending on how you define that. AI is a BIG field) is more demanding that "flavour of the month smartphone app" development. And in the past, I scoffed when I heard people say things like "70% of so-called developers can't complete FizzBuzz." Then I started interviewing a lot of candidates for developer roles at the $dayjob and whaddaya know... it seems that most of the candidates we get can't successfully code FizzBuzz. And that's a pretty low bar.
So I dunno... part of me believes there are plenty of people who could contribute to AI research, but part of me is shocked by how few people can even write the most trivial computer program.
Are there fewer than 10k people with a master's in the subject in the entire world?
I don't think having a Masters has much to do with anything. We get folks all the time with a Masters in C.S., even from relatively prestigious schools, and they can't even code up FizzBuzz.
Anyway, I wish I had a better answer for you, but recent experiences have me questioning just how many really competent people there are out there.
I definitely agree that it's a lot more demanding than smartphone apps - or most of the rest of what we call software development.
I'm trying to make the switch from backend development to machine learning right now, and even though I have a supportive employer I've begun thinking about going back to school because the math involved seems steeper and harder to avoid.
Disclaimer: My math level is high school since I self-taught my way into software engineering.
I'm trying to make the switch from backend development to machine learning right now, and even though I have a supportive employer I've begun thinking about going back to school because the math involved seems steeper and harder to avoid.
Disclaimer: My math level is high school since I self-taught my way into software engineering.
I did a linguistics degree before doing my computational linguistics PhD 2005-2010. In Australia we don't do any graduate coursework, so the last maths class I took was when I was 15 or 16. I'm now pretty good at these things. You can see my work here: https://explosion.ai
You might be interested in the machine learning library I wrote for spaCy: https://github.com/explosion/thinc . The backward propagation can be understood quite simply with callbacks.
The main problem I had, and I think you're having, is that the notation around calculus is just hopeless. It's even worse when you're trying to map between the equations and backpropagation, because what you need is slightly different. I also find the notations for functions really bad.
The other thing you need to get used to are the ways linear algebra overload the operators. This is just data, not concepts. Personally I find it much easier to just work in the Einstein summation: https://obilaniu6266h16.wordpress.com/2016/02/04/einstein-su... . Once it's debugged, you can translate into faster calls to numpy.tensordot.
Don't go back to school. School's fucked. It'll take you three years to do about 200 hours of learning. Hiring a tutor would be vastly better if you need some help.
You might be interested in the machine learning library I wrote for spaCy: https://github.com/explosion/thinc . The backward propagation can be understood quite simply with callbacks.
The main problem I had, and I think you're having, is that the notation around calculus is just hopeless. It's even worse when you're trying to map between the equations and backpropagation, because what you need is slightly different. I also find the notations for functions really bad.
The other thing you need to get used to are the ways linear algebra overload the operators. This is just data, not concepts. Personally I find it much easier to just work in the Einstein summation: https://obilaniu6266h16.wordpress.com/2016/02/04/einstein-su... . Once it's debugged, you can translate into faster calls to numpy.tensordot.
Don't go back to school. School's fucked. It'll take you three years to do about 200 hours of learning. Hiring a tutor would be vastly better if you need some help.
What are the concepts you are most struggling with..could you list out a few..check out metacademy.org
I never took statistics or linear algebra. Only AP calculus. So when people refer me to academic papers (or even Wikipedia most of the time) I'm not able to understand it.
I can study on my own, and have been for stats/probability. But started asking myself, why not just get a degree and make it quicker. I'd have to leave my job for a number of years, but then I'd be able to study all day instead of nights/weekends.
I can study on my own, and have been for stats/probability. But started asking myself, why not just get a degree and make it quicker. I'd have to leave my job for a number of years, but then I'd be able to study all day instead of nights/weekends.
There's deeplearningbook.org, which starts with the basic maths and then goes into considerable detail on cutting edge work.
It assumes a math background implicitly. I feel like you need some context already to get value out of it. I wouldn't bother reading it without doing something hands on or going through another course like cs231n or fastai first. It's an excellent book regardless of the above points.
Well, it's like saying "inventing the new best sorting or searching algorithm doesn't take that much that's special".
You can only make this claim if you've done it.
You can only make this claim if you've done it.
I'm just talking about general prerequisite skills. There's a stock set of "stuff" that, generally speaking, one needs to know to do research in this area... specifically some areas of math & stats. That's not to say that making a "BIG" advance doesn't require a "special" level of insight, or creativity, or whatever.
But in terms of basic skills that are required to work in the field, the prerequisites aren't anything particularly demanding. When you look at most of what's published in the ML area, the bulk of what's required to understand it is contained in Calculus, Probability/Statistics and Linear Algebra. Maybe Topology for some things. But we're mostly not talking even anything as advanced as Abstract Algebra, Galois Theory, Group Theory, etc.
But in terms of basic skills that are required to work in the field, the prerequisites aren't anything particularly demanding. When you look at most of what's published in the ML area, the bulk of what's required to understand it is contained in Calculus, Probability/Statistics and Linear Algebra. Maybe Topology for some things. But we're mostly not talking even anything as advanced as Abstract Algebra, Galois Theory, Group Theory, etc.
Using my analogy again: if you do research in sorting/searching algorithms, even improving the performance of the best algorithm by a tiny bit would require quite a "special level of insight, or creativity, or whatever." Making a "BIG" advance would require genius level abilities, think a few dozens of people in the entire world.
We are not talking about understanding what has been published in the ML area. We are talking about publishing your own, new ideas in peer-reviewed journals/conferences, and/or pushing state of the art results. That's what "doing research in AI" is all about.
Same applies to doing research in all those other fields you mentioned (calculus, statistics, group theory, etc). There's a big gap between people who can merely understand new ideas, and people who can come up with them.
We are not talking about understanding what has been published in the ML area. We are talking about publishing your own, new ideas in peer-reviewed journals/conferences, and/or pushing state of the art results. That's what "doing research in AI" is all about.
Same applies to doing research in all those other fields you mentioned (calculus, statistics, group theory, etc). There's a big gap between people who can merely understand new ideas, and people who can come up with them.
I think we're talking about two different things here. I'm just talking about the fundamental prerequisites to do research in the field. I get the feeling that you're talking about something other, more ephemeral "thing".
if you do research in sorting/searching algorithms, even improving the performance of the best algorithm by a tiny bit would require quite a "special level of insight, or creativity, or whatever." Making a "BIG" advance would require genius level abilities, think a few dozens of people in the entire world.
Perhaps so, but we're not talking about research on sorting/searching algorithms. We're talking about AI. I would argue that the bar for publishing novel research is lower here, as the field is broader and correspondingly less developed (in areas).
I mean, right now, ML in particular is heavily based on empirical observation and experiment and lacks theory in many areas. So just trying some new combination of hyper-parameters that look interesting, achieving a "better than the latest published result" in some benchmark, and writing up a paper can constitute original research in this area. No, it's not the same as a major breakthrough in theoretical understanding, but then again not everything is. Which is at least part of my overall point.
It's probably also worth nothing that not all "research" happens in academic settings and not all researchers publish (or are driven to publish to the extent that academics are... especially tenure track folks). I'm thinking of corporate settings where simply achieving a useful result that helps the company make more money is considered a win.
All of that said, I'm open to the possibility that there are, indeed, fewer than 10k people who can do AI research, as I'm starting to wonder if there are fewer than 10k people who can write FizzBuzz successfully. But acquiring the basic tools necessary to do research in the AI/ML domain isn't so hard as to be inaccessible to a large number of people.
if you do research in sorting/searching algorithms, even improving the performance of the best algorithm by a tiny bit would require quite a "special level of insight, or creativity, or whatever." Making a "BIG" advance would require genius level abilities, think a few dozens of people in the entire world.
Perhaps so, but we're not talking about research on sorting/searching algorithms. We're talking about AI. I would argue that the bar for publishing novel research is lower here, as the field is broader and correspondingly less developed (in areas).
I mean, right now, ML in particular is heavily based on empirical observation and experiment and lacks theory in many areas. So just trying some new combination of hyper-parameters that look interesting, achieving a "better than the latest published result" in some benchmark, and writing up a paper can constitute original research in this area. No, it's not the same as a major breakthrough in theoretical understanding, but then again not everything is. Which is at least part of my overall point.
It's probably also worth nothing that not all "research" happens in academic settings and not all researchers publish (or are driven to publish to the extent that academics are... especially tenure track folks). I'm thinking of corporate settings where simply achieving a useful result that helps the company make more money is considered a win.
All of that said, I'm open to the possibility that there are, indeed, fewer than 10k people who can do AI research, as I'm starting to wonder if there are fewer than 10k people who can write FizzBuzz successfully. But acquiring the basic tools necessary to do research in the AI/ML domain isn't so hard as to be inaccessible to a large number of people.
Are you serious about FizzBuzz? Just searched it up and it's pretty simple lol. I'm just an informatics engineer of an average South American College ;)
Yes, dead serious. I used to experience a lot of disbelief when I heard this kind of stuff myself. And then I started doing a lot of reviewing resumes and interviewing people, and I've been fairly stunned.
I think the skills to develop training sets and develop real applications are even rarer.
Well, where I took my PhD degree (I'm not in the US), I was the only one working on Text-mining and Information Retrieval (besides my professor), and I think there was only three or four other PhD students working on AI - one on SOM (Self-Organizing maps), one on automatic translation, and the others I can't recall exactly on what they were working on.
So, although 10K seems too few to me, I would say that the magnitude may be right..
So, although 10K seems too few to me, I would say that the magnitude may be right..
One way to estimate: how many people have published in top conferences (CVPR, ICML, NIPS, etc) as a first author?
Those are the people who can do serious AI research.
Those are the people who can do serious AI research.
Of the 5-10k CS professors out there , only a fraction are researching AI. There are postdocs, grad students, undergrad researchers. Students are not likely to make any ground breaking discoveries. So yeah, under 10k sounds reasonable. CS is not as big as some people think. There have been less than 50k CS pHd's conferred in all human history and less than 500k master degrees.
With AI though, the jobs probably pay $300-500k per year, so there are going to be a lot of people flat out lying about credentials and abilities.
And some people's idea of "research" is running scripts against Azure, AWS, or some grad student's thesis code. "Research" most likely does not involve a large amount of computing.
With AI though, the jobs probably pay $300-500k per year, so there are going to be a lot of people flat out lying about credentials and abilities.
And some people's idea of "research" is running scripts against Azure, AWS, or some grad student's thesis code. "Research" most likely does not involve a large amount of computing.
Of the 5-10k CS professors out there , only a fraction are researching AI. There are postdocs, grad students, undergrad researchers. Students are not likely to make any ground breaking discoveries. So yeah, under 10k sounds reasonable.
What about people working in private industry, as opposed to academia?
What about people working in private industry, as opposed to academia?
> There have been less than 50k CS pHd's conferred in all human history and less than 500k master degrees.
Do you have a source for those figures?
Do you have a source for those figures?
https://nces.ed.gov/programs/digest/d16/tables/dt16_324.10.a...
https://nces.ed.gov/programs/digest/d16/tables/dt16_323.10.a...
This is US only and not for all years.
This is US only and not for all years.
Thanks!
"Solving tough A.I. problems is not like building the flavor-of-the-month smartphone app. In the entire world, fewer than 10,000 people have the skills necessary to tackle serious artificial intelligence research, according to Element AI, an independent lab in Montreal." https://www.nytimes.com/2017/10/22/technology/artificial-intelligence-experts-salaries.html?