Not the autoworkers of our generation(imsky.co)
imsky.co
Not the autoworkers of our generation
http://imsky.co/blog/not-autoworkers/
12 comments
We do have programs that write programs. Who's using them?
I like your optimism, but the average guy with an MBA definitely can't throw a MVP together, especially not in 2013 when the low hanging fruit have been steadily picked off. The bar is continuously being raised, and it'll be cheaper for the MBA to hire a developer to throw that MVP together, using highly complex and abstract components.
If I'm getting the second half of your reply right, you're saying it's the availability of high-quality low-complexity zero-cost solutions to mundane problems that's going to put software developers out of a job. But that's exactly what I address in my post - these things were going to be automated anyway. And don't be disappointed by a user base attracted to glossy boxes - some of them will get curious eventually.
I like your optimism, but the average guy with an MBA definitely can't throw a MVP together, especially not in 2013 when the low hanging fruit have been steadily picked off. The bar is continuously being raised, and it'll be cheaper for the MBA to hire a developer to throw that MVP together, using highly complex and abstract components.
If I'm getting the second half of your reply right, you're saying it's the availability of high-quality low-complexity zero-cost solutions to mundane problems that's going to put software developers out of a job. But that's exactly what I address in my post - these things were going to be automated anyway. And don't be disappointed by a user base attracted to glossy boxes - some of them will get curious eventually.
"We do have programs that write programs. Who's using them?"
If you count things like Weebly, the answer is: a "whole bunch of people", heading rapidly toward "everyone". Not that long ago, every Weebly customer would have meant a programming job for someone. That kind of work is gone.
Then take a close look at AWS, Heroku, Rackspace...those were all systems engineering jobs. Startups don't have to hire people to do that now. What used to be thousands of jobs is now maybe a tenth of that.
"the average guy with an MBA definitely can't throw a MVP together"
You're wrong. It happens all the time. It happens so often that's a cliche. You can't go to a meetup in the valley without being assaulted by ten brohans looking for "feedback" on their "MVP".
The important thing to realize is that the need for software engineers doesn't scale with the maximum complexity of software projects; it scales as a function of the average complexity: think of a bell curve, where the X-axis is "complexity", and "need for engineers" is the area under the curve to one side of some vertical line. We keep moving that line further and further toward greater complexity. So far, the distribution has shifted to keep up, but there's no guarantee that will keep happening.
Are we going to need people to write complex, novel software in 2050? Absolutely. Will we need more people than we need to write complex, novel software today? That's not a question with a predictable answer. It might well turn out that the "average" programmer (say, the ones who know how to use Rails and jQuery and are fond of complaining about interview questions involving algorithms) is a doomed creature.
If you count things like Weebly, the answer is: a "whole bunch of people", heading rapidly toward "everyone". Not that long ago, every Weebly customer would have meant a programming job for someone. That kind of work is gone.
Then take a close look at AWS, Heroku, Rackspace...those were all systems engineering jobs. Startups don't have to hire people to do that now. What used to be thousands of jobs is now maybe a tenth of that.
"the average guy with an MBA definitely can't throw a MVP together"
You're wrong. It happens all the time. It happens so often that's a cliche. You can't go to a meetup in the valley without being assaulted by ten brohans looking for "feedback" on their "MVP".
The important thing to realize is that the need for software engineers doesn't scale with the maximum complexity of software projects; it scales as a function of the average complexity: think of a bell curve, where the X-axis is "complexity", and "need for engineers" is the area under the curve to one side of some vertical line. We keep moving that line further and further toward greater complexity. So far, the distribution has shifted to keep up, but there's no guarantee that will keep happening.
Are we going to need people to write complex, novel software in 2050? Absolutely. Will we need more people than we need to write complex, novel software today? That's not a question with a predictable answer. It might well turn out that the "average" programmer (say, the ones who know how to use Rails and jQuery and are fond of complaining about interview questions involving algorithms) is a doomed creature.
> Not that long ago, every Weebly customer would have meant a programming job for someone.
Isn't that kind of like saying that every pirated album == a lost sale?
Isn't that kind of like saying that every pirated album == a lost sale?
Right, that's the fallacy being made constantly here - as if Weebly and Google Maps are taking jobs from developers, when really there would have been no need for a site or a mapping component in their absence.
"...as if Weebly and Google Maps are taking jobs from developers, when really there would have been no need for a site or a mapping component in their absence."
There were once a great many programmers who did custom website development. That market doesn't exist anymore. It's not a "fallacy" if it actually happened.
What you're really trying to argue is that Weebly and Google Maps and the others expanded the market for professional software development -- that all of those people who did simple websites are now doing harder things. That's true so far, but there's no guarantee that it will continue to be true.
What's happening here is that you're arguing sound-bite economics, and we're arguing a point that's about five steps ahead of where you are: eventually, automation will reach a point where the average need of a customer will be satisfied without the assistance of a programmer. That we have not yet reached that point is not an effective counterargument. It has happened in nearly every other industry, and thus far, programming appears to be following the same curve.
It's the same reason that we no longer pay artisan woodworkers to make most of our furniture.
There were once a great many programmers who did custom website development. That market doesn't exist anymore. It's not a "fallacy" if it actually happened.
What you're really trying to argue is that Weebly and Google Maps and the others expanded the market for professional software development -- that all of those people who did simple websites are now doing harder things. That's true so far, but there's no guarantee that it will continue to be true.
What's happening here is that you're arguing sound-bite economics, and we're arguing a point that's about five steps ahead of where you are: eventually, automation will reach a point where the average need of a customer will be satisfied without the assistance of a programmer. That we have not yet reached that point is not an effective counterargument. It has happened in nearly every other industry, and thus far, programming appears to be following the same curve.
It's the same reason that we no longer pay artisan woodworkers to make most of our furniture.
> There were once a great many programmers who did custom website development. That market doesn't exist anymore.
This is not true; just try doing a search for "web developer" in any major city. There will be dozens, if not hundreds of jobs listed on almost any day of the year.
This is not true; just try doing a search for "web developer" in any major city. There will be dozens, if not hundreds of jobs listed on almost any day of the year.
A website is not a program, let's be fair. It's at most a minimum amount of logic wrapped around content. Automating it is easy, and it has long ago lost its prestige.
Re MBA guys - maybe we have a different understanding of MVP here. I don't mean PowerPoint-ware - I mean version 0.1 of a product solving real problems. MBAs that are working on something more complex than yet another social network will immediately look for an engineer.
I'm not missing the average/maximum-complexity point at all, but I have a different view on it than you do. The average complexity won't stay constant, it'll keep increasing, and the average programmer will have to adapt. Will that mean a reduction in demand? I doubt it - we've yet to see the full extent of how software can be used. On the flip side, I constantly see clients looking for someone to fix or alter their "automated" installations - the average programmer will surely be in enough demand from these cases alone.
Re MBA guys - maybe we have a different understanding of MVP here. I don't mean PowerPoint-ware - I mean version 0.1 of a product solving real problems. MBAs that are working on something more complex than yet another social network will immediately look for an engineer.
I'm not missing the average/maximum-complexity point at all, but I have a different view on it than you do. The average complexity won't stay constant, it'll keep increasing, and the average programmer will have to adapt. Will that mean a reduction in demand? I doubt it - we've yet to see the full extent of how software can be used. On the flip side, I constantly see clients looking for someone to fix or alter their "automated" installations - the average programmer will surely be in enough demand from these cases alone.
>A website is not a program, let's be fair. It's at most a minimum amount of logic wrapped around content. Automating it is easy, and it has long ago lost its prestige.
Previously, most people would say that it requires programmers to make a website. Now you're demoting producing a website to something that doesn't even require programmers, saying it's been automated for several years.
I think this proves exactly the point of the GP.
Re Re MBA guys - I don't know how many MBAs are working on something more complex than another social network, because many successful businesses recently have been quite easy to build a MVP for with not a lot of programming skills.
Examples of succesful companies with not difficult to program MVPs: AirBnB, Instagram, Twitter, Groupon.
I believe many MBAs could build something like these given some time. If we want to broaden the category from MBAs to just STEM students, I know very few STEM students at my college (which is ~97% STEM majors) who couldn't build more than a MVP for one of these sites given 2-3 months.
Previously, most people would say that it requires programmers to make a website. Now you're demoting producing a website to something that doesn't even require programmers, saying it's been automated for several years.
I think this proves exactly the point of the GP.
Re Re MBA guys - I don't know how many MBAs are working on something more complex than another social network, because many successful businesses recently have been quite easy to build a MVP for with not a lot of programming skills.
Examples of succesful companies with not difficult to program MVPs: AirBnB, Instagram, Twitter, Groupon.
I believe many MBAs could build something like these given some time. If we want to broaden the category from MBAs to just STEM students, I know very few STEM students at my college (which is ~97% STEM majors) who couldn't build more than a MVP for one of these sites given 2-3 months.
@lightcatcher:
>Previously, most people would say that it requires programmers to make a website.
It hasn't required a programmer to build a website in many years now - HTML and CSS were declarative to begin with. It's only with the advent of CMS's - which are definitely NOT all automated (i.e. Joomla or WordPress) - that programmers entered the picture.
> Re Re MBA guys - I don't know how many MBAs are working on something more complex than another social network, because many successful businesses recently have been quite easy to build a MVP for with not a lot of programming skills.
Good point, but are they doing it? Facebook, Twitter, Instagram, and Airbnb all had technical founders. The true complexity of these startups lies in their server software - how to optimize at scale and so on. An MBA aware of this complexity deserves all the success he or she can get.
>Previously, most people would say that it requires programmers to make a website.
It hasn't required a programmer to build a website in many years now - HTML and CSS were declarative to begin with. It's only with the advent of CMS's - which are definitely NOT all automated (i.e. Joomla or WordPress) - that programmers entered the picture.
> Re Re MBA guys - I don't know how many MBAs are working on something more complex than another social network, because many successful businesses recently have been quite easy to build a MVP for with not a lot of programming skills.
Good point, but are they doing it? Facebook, Twitter, Instagram, and Airbnb all had technical founders. The true complexity of these startups lies in their server software - how to optimize at scale and so on. An MBA aware of this complexity deserves all the success he or she can get.
"It hasn't required a programmer to build a website in many years now - HTML and CSS were declarative to begin with. It's only with the advent of CMS's - which are definitely NOT all automated (i.e. Joomla or WordPress) - that programmers entered the picture."
Well, first off, you're just shifting the goalposts and redefining what it means to "program" to suit your needs: it's inconvenient for your argument that there's now software that does what programmers used to do, so you just call those things something else.
But it doesn't matter, because you can't run from the core of the argument: it isn't limited to static HTML & CSS. Want web forms, but don't know how to write software or use databases? There's Wufoo, and dozens of others. Want to send email? Mailgun. Mailchimp. Want to accept payments? Paypal, Stripe. Want to show an interactive map with stuff on it? Google maps has you covered. Want to implement search on your website, but don't know anything about search? Swiftype will do that. So will Google site search. Need a sophisticated customer analytics system. OK. That'll be one line of javascript, please.
And what's more: it's only a matter of time before someone rolls all of these tools together into a pointy clicky interface that eliminates the token integration work (assuming they haven't already). There's no part of your argument that is safe from the trends that have been driving our industry.
"The true complexity of these startups lies in their server software - how to optimize at scale and so on."
True, but like I said before: almost nobody needs that. There's always going to be the need for complex software...it's just not clear how many people will be asking for it.
Well, first off, you're just shifting the goalposts and redefining what it means to "program" to suit your needs: it's inconvenient for your argument that there's now software that does what programmers used to do, so you just call those things something else.
But it doesn't matter, because you can't run from the core of the argument: it isn't limited to static HTML & CSS. Want web forms, but don't know how to write software or use databases? There's Wufoo, and dozens of others. Want to send email? Mailgun. Mailchimp. Want to accept payments? Paypal, Stripe. Want to show an interactive map with stuff on it? Google maps has you covered. Want to implement search on your website, but don't know anything about search? Swiftype will do that. So will Google site search. Need a sophisticated customer analytics system. OK. That'll be one line of javascript, please.
And what's more: it's only a matter of time before someone rolls all of these tools together into a pointy clicky interface that eliminates the token integration work (assuming they haven't already). There's no part of your argument that is safe from the trends that have been driving our industry.
"The true complexity of these startups lies in their server software - how to optimize at scale and so on."
True, but like I said before: almost nobody needs that. There's always going to be the need for complex software...it's just not clear how many people will be asking for it.
Even if you limit it to static HTML and CSS, there are still many people who do that as a job. The only thing that has really been automated away is when someone is willing to use bog standard templates.
And in that respect, that kind of optimization has been happening since the dawn of the personal computer. Lotus 1-2-3 was the defining application of the PC, allowing average users to do programming-like operations that were once only in reach of programmers.
And in that respect, that kind of optimization has been happening since the dawn of the personal computer. Lotus 1-2-3 was the defining application of the PC, allowing average users to do programming-like operations that were once only in reach of programmers.
@timr:
> Well, first off, you're just shifting the goalposts and redefining what it means to "program" to suit your needs: it's inconvenient for your argument that there's now software that does what programmers used to do, so you just call those things something else.
Definitely not. I think "website automation" is among the simplest kinds of software automation and historically it has not required a significant amount of programming.
> But it doesn't matter, because you can't run from the core of the argument: it isn't limited to static HTML & CSS. Want web forms, but don't know how to write software or use databases? There's Wufoo, and dozens of others. Want to send email? Mailgun. Mailchimp. Want to accept payments? Paypal, Stripe. Want to show an interactive map with stuff on it? Google maps has you covered. Want to implement search on your website, but don't know anything about search? Swiftype will do that. So will Google site search. Need a sophisticated customer analytics system. OK. That'll be one line of javascript, please.
That's a stretch. Wufoo, PayPal, Stripe, Mailgun, Google Maps, and MailChimp are not going to cover 100% of use cases, nor do they all come free. You're also ignoring that many of these services are alternatives, not integrations (MailChimp, PayPal, Stripe, Mailgun, Google Maps), of older services (AWeber, Authorize, MapQuest) that filled the same function, and a few of them are filling a need that simply didn't exist before prior automation tools (MailChimp for email lists, Stripe for payment processing). It's not a zero-sum game, nor do the same players stay on top throughout time.
> And what's more: it's only a matter of time before someone rolls all of these tools together into a pointy clicky interface that eliminates the token integration work (assuming they haven't already). There's no part of your argument that is safe from the trends that have been driving our industry.
I actually mentioned that in the post: "For every API that wraps around a business process, there's an application yet to be written (and an API around that application in due time)." You seem to think there's some end point, where most software will just be done and most activity automated. I doubt that will ever be the case.
> True, but like I said before: almost nobody needs that. There's always going to be the need for complex software...it's just not clear how many people will be asking for it.
You seem to imply that in the future, most non-complex (i.e. average) software will just either exist or be generated on demand. That seems far-fetched - there are programs orders of magnitudes simpler than "software generators" that routinely need to be adjusted and tuned to produce correct output (as will all the automated APIs in the future).
> Well, first off, you're just shifting the goalposts and redefining what it means to "program" to suit your needs: it's inconvenient for your argument that there's now software that does what programmers used to do, so you just call those things something else.
Definitely not. I think "website automation" is among the simplest kinds of software automation and historically it has not required a significant amount of programming.
> But it doesn't matter, because you can't run from the core of the argument: it isn't limited to static HTML & CSS. Want web forms, but don't know how to write software or use databases? There's Wufoo, and dozens of others. Want to send email? Mailgun. Mailchimp. Want to accept payments? Paypal, Stripe. Want to show an interactive map with stuff on it? Google maps has you covered. Want to implement search on your website, but don't know anything about search? Swiftype will do that. So will Google site search. Need a sophisticated customer analytics system. OK. That'll be one line of javascript, please.
That's a stretch. Wufoo, PayPal, Stripe, Mailgun, Google Maps, and MailChimp are not going to cover 100% of use cases, nor do they all come free. You're also ignoring that many of these services are alternatives, not integrations (MailChimp, PayPal, Stripe, Mailgun, Google Maps), of older services (AWeber, Authorize, MapQuest) that filled the same function, and a few of them are filling a need that simply didn't exist before prior automation tools (MailChimp for email lists, Stripe for payment processing). It's not a zero-sum game, nor do the same players stay on top throughout time.
> And what's more: it's only a matter of time before someone rolls all of these tools together into a pointy clicky interface that eliminates the token integration work (assuming they haven't already). There's no part of your argument that is safe from the trends that have been driving our industry.
I actually mentioned that in the post: "For every API that wraps around a business process, there's an application yet to be written (and an API around that application in due time)." You seem to think there's some end point, where most software will just be done and most activity automated. I doubt that will ever be the case.
> True, but like I said before: almost nobody needs that. There's always going to be the need for complex software...it's just not clear how many people will be asking for it.
You seem to imply that in the future, most non-complex (i.e. average) software will just either exist or be generated on demand. That seems far-fetched - there are programs orders of magnitudes simpler than "software generators" that routinely need to be adjusted and tuned to produce correct output (as will all the automated APIs in the future).
"MBAs that are working on something more complex than yet another social network will immediately look for an engineer."
Put bluntly, I think that both Groupon and Zappos started as glorified Wordpress installs.
I don't think that the average complexity for business programs is going to change--at their core, most all of them have the business proposition "Take my X so I can get Y in time Z". Everything else is window dressing.
Put bluntly, I think that both Groupon and Zappos started as glorified Wordpress installs.
I don't think that the average complexity for business programs is going to change--at their core, most all of them have the business proposition "Take my X so I can get Y in time Z". Everything else is window dressing.
Has anyone ever considered groupon as a technical challenge? Most of their personnel were salesmen, were they not?
>It's not that one day programs will write programs that write--oh wait, that's already happened
Hi, I write software for a living. Just replace the most tedious part, please: debugging.
Say what? You can't even give me a probability [1] that my program has an infinite loop in it?
[1] http://en.wikipedia.org/wiki/Chaitin%27s_constant
Hi, I write software for a living. Just replace the most tedious part, please: debugging.
Say what? You can't even give me a probability [1] that my program has an infinite loop in it?
[1] http://en.wikipedia.org/wiki/Chaitin%27s_constant
What makes you think that you can give a probability that a program has an infinite loop in it?
As for your request:
https://ece.uwaterloo.ca/~lintan/publications/r2fix-icst13.p...
But bugs aside, part of the allure of having a program generate a program is that it is much less likely to produce bugs in the first place. That said, we still have a ways to go before we achieve human-like synthetic developers.
As for your request:
https://ece.uwaterloo.ca/~lintan/publications/r2fix-icst13.p...
But bugs aside, part of the allure of having a program generate a program is that it is much less likely to produce bugs in the first place. That said, we still have a ways to go before we achieve human-like synthetic developers.
>What makes you think that you can give a probability that a program has an infinite loop in it?
I can't. That's why I basically rummage through the code base, using all kinds of heuristics, most of which I make up on the spot. Sometimes it takes me two minutes to find that off-by-one, and other times two days.
Interesting paper. I don't think it will help that much though:
>Ideally, we want to automatically generate patches for all bug reports. Realistically, it is impossible. [...] Even among bugs that can be fixed, some are too complex to be fixed automatically [...] Therefore, a realistic goal is to automatically generate patches for relatively simple bugs [where] the bug reports contain useful information, e.g., the buggy file and the symptom
I can't. That's why I basically rummage through the code base, using all kinds of heuristics, most of which I make up on the spot. Sometimes it takes me two minutes to find that off-by-one, and other times two days.
Interesting paper. I don't think it will help that much though:
>Ideally, we want to automatically generate patches for all bug reports. Realistically, it is impossible. [...] Even among bugs that can be fixed, some are too complex to be fixed automatically [...] Therefore, a realistic goal is to automatically generate patches for relatively simple bugs [where] the bug reports contain useful information, e.g., the buggy file and the symptom
Yeah... my last project at the company I worked for was basically to automate myself out of a job. They kept one developer to maintaining everything, the rest of us were kicked off the assembly line.
Hint: you should be constantly automating yourself out of your current job and into your new one.
I feel like neither the article, nor any of the comments here so far, mention a very obvious point.
The automation of tasks that formerly required writing software, are automated by someone writing new software.
This should be the normal, everyday experience of the software developer. Take the software you wrote yesterday, and write higher level software on top of it today to accomplish things you couldn't before.
So automating the software jobs of yesterday out of existence will require the software jobs of tomorrow.
As for the computers writing their own software, that strikes me as indicating the Singularity has arrived, in which case all human jobs will be obsolete.
The automation of tasks that formerly required writing software, are automated by someone writing new software.
This should be the normal, everyday experience of the software developer. Take the software you wrote yesterday, and write higher level software on top of it today to accomplish things you couldn't before.
So automating the software jobs of yesterday out of existence will require the software jobs of tomorrow.
As for the computers writing their own software, that strikes me as indicating the Singularity has arrived, in which case all human jobs will be obsolete.
>The automation of tasks that formerly required writing software, are automated by someone writing new software.
This.
User timr writes:
"Then take a close look at AWS, Heroku, Rackspace...those were all systems engineering jobs. Startups don't have to hire people to do that now. What used to be thousands of jobs is now maybe a tenth of that."
Let's take this point and unravel it a bit. AWS is hugely disruptive. It takes work previously done by operations folks (like me) and puts useful tools directly in the hands of users (programmers).
This is a huge change that increases a programmer's efficiency. (I prefer the term developer, but I'll keep it to stay consistent). In effect, it has removed one of the major barriers to entry, that is, getting your code online and accessible from the internet.
So automation is the thrust of the argument; robots on the assembly line replacing auto-workers jobs. Someone had to build those robots.
In the programmers case, that someone is a programmer.
So, what can we learn from all this?
Change is good. We need these disruptions to setup the next growth phase. Software is different. The barriers to entry are being removed. Productivity increases as a result.
Be on the positive side of the change. Predict, plan, strategize, and stay relevant.
Happy Labor Day.
This.
User timr writes:
"Then take a close look at AWS, Heroku, Rackspace...those were all systems engineering jobs. Startups don't have to hire people to do that now. What used to be thousands of jobs is now maybe a tenth of that."
Let's take this point and unravel it a bit. AWS is hugely disruptive. It takes work previously done by operations folks (like me) and puts useful tools directly in the hands of users (programmers).
This is a huge change that increases a programmer's efficiency. (I prefer the term developer, but I'll keep it to stay consistent). In effect, it has removed one of the major barriers to entry, that is, getting your code online and accessible from the internet.
So automation is the thrust of the argument; robots on the assembly line replacing auto-workers jobs. Someone had to build those robots.
In the programmers case, that someone is a programmer.
So, what can we learn from all this?
Change is good. We need these disruptions to setup the next growth phase. Software is different. The barriers to entry are being removed. Productivity increases as a result.
Be on the positive side of the change. Predict, plan, strategize, and stay relevant.
Happy Labor Day.
Great point. The nature of the work changes as old complexity is automated away, and new growth possibilities arise as a result.
Good point - what we do is not the visible coding; only a small 10-20% of our time is doing that. It's the 80-90% that is thinking, the considering and discarding of solutions that is valuable and not easily automated.
A good point, but an obvious one to economically-literate readers. The counterpoint to it is that it requires fewer developers to automate a process than it requires to write it. In the short run, unemployment may increase, but in the long run, it'll stabilize.
> The counterpoint to it is that it requires fewer developers to automate a process than it requires to write it.
Not by definition. Rather, only once it becomes easier to automate than to do by hand, we automate.
Not by definition. Rather, only once it becomes easier to automate than to do by hand, we automate.
The first part is technically true - the automation team can exceed the original team, but this may be a special case of "engineers untangling a mess of functionality accumulated over the years." The second part has an exception of its own - there doesn't necessarily have to be a complexity threshold, developers "over-automate" simple processes all the time (e.g. Microsoft).
Interesting that one of his examples, Wordpress, has contributed to lots of software jobs as well. Writing themes, plugins, etc can be fairly lucrative and probably more interesting than making a blog from scratch over and over.
The whole point of writing software is to make it so you don't have to keep doing the same thing over and over. That doesn't mean there aren't new things to do. Software really is eating the world. If we run out of software jobs, it's probably because we've automated literally everything else already.
The whole point of writing software is to make it so you don't have to keep doing the same thing over and over. That doesn't mean there aren't new things to do. Software really is eating the world. If we run out of software jobs, it's probably because we've automated literally everything else already.
That's a good point - there can be serious career opportunities in managing, supporting, and modifying ostensibly "automated" solutions. The core may be automated, but the ecosystem doesn't have to be.
It will be too arrogant on our part to think we can't be automated.
If you have used eclipse over the years you will see the kind of gradual automation happening there. Its not the kind of automation that produces code automagically all alone by itself. But software like eclipse pulls the barrier to entry too low to a point even people who understand nothing much about Java or in general about software can comfortable contribute to make a living out of it. And eclipse is only getting better with time. These days you can pretty much code complex Java application without even reading a book. I guess very soon, software like eclipse will fill-in-code-blocks as we express a intention. And then our job will be to stitch those blocks.
This is what is dangerous. And this kind of thing has the potential to drive the supply up very quickly. Thereby the wages decline.
There is no 1-1 mapping, between a automaker and programmer. But the theme is very common.
I am even damn sure we will soon make software a commodity to a point when people can think in terms of abstract software blocks. Just like how a common man can today think of a car in terms of engine, clutch, gear etc.
If you have used eclipse over the years you will see the kind of gradual automation happening there. Its not the kind of automation that produces code automagically all alone by itself. But software like eclipse pulls the barrier to entry too low to a point even people who understand nothing much about Java or in general about software can comfortable contribute to make a living out of it. And eclipse is only getting better with time. These days you can pretty much code complex Java application without even reading a book. I guess very soon, software like eclipse will fill-in-code-blocks as we express a intention. And then our job will be to stitch those blocks.
This is what is dangerous. And this kind of thing has the potential to drive the supply up very quickly. Thereby the wages decline.
There is no 1-1 mapping, between a automaker and programmer. But the theme is very common.
I am even damn sure we will soon make software a commodity to a point when people can think in terms of abstract software blocks. Just like how a common man can today think of a car in terms of engine, clutch, gear etc.
> If you have used eclipse over the years you will see the
> kind of gradual automation happening there
The programmers job is not to do the things that eclipse can do. It's doing the things that he could do on paper, or just in his mind.
Automating programming is like automating composers, writers and actors. Possible, but hardly desireable.
The programmers job is not to do the things that eclipse can do. It's doing the things that he could do on paper, or just in his mind.
Automating programming is like automating composers, writers and actors. Possible, but hardly desireable.
Aren't programmers more like the engineers who made the cars? Their work product is not just the design but the assembly line and the process, and those things help produce parts and cars. The autoworkers are more analogous to the people who work within software-controlled workplaces. That would be the people who pack boxes at Amazon.com, or do data entry, or customer support.
Software people are so arrogant. Auto workers arent and weren't unskilled manual labor.
Both articles misses the mark. There are still autoworkers out there, just not in Detroit (OK, there are less of them, but they're still there). Autoworking didn't fail, Detroit (ie "big auto") failed. And they failed because they spend all their energy building moats (trade policy, emissions policy, unions etc) rather than improving and innovating (like the asian auto companies excelled at and the european ones partly kept up with), so when the house of cards came down, it came down hard.
As long as we (software engineers) don't sequester ourselves wholesale in a homogenous monolith but continuously keep up with innovation, we should be well equipped to respond to changes in the market in time. The first article touches on this in the last paragraph, but I'm not convinced it's for the "right reasons".
As long as we (software engineers) don't sequester ourselves wholesale in a homogenous monolith but continuously keep up with innovation, we should be well equipped to respond to changes in the market in time. The first article touches on this in the last paragraph, but I'm not convinced it's for the "right reasons".
The rote, unskilled work in auto assembly was easily automated (apply a spot weld here, insert a part there) but a lot of the skilled work was as well. The precision parts in automobiles used to be made by hand by machinists. This was highly skilled work, and it's now almost entirely been replaced by CNC. Machinists today, to the extent they still exist, are much more like programmers. They are still used for small and one-off jobs, but mass production of machined parts is programmed directly from the designs now, so you don't need a machinist in the picture at all for that. No amount of keeping up with innovation would have helped there.
If those people were let go (or retrained) continuously as they became redundant, instead of being employed in a unsustainable behemoth which then collapses all at once, they are much better positioned to re-qualify and re-join the workforce (maybe in the same, maybe in a different company) in a higher-skilled position. This could happen multiple times over a career.
I'd like to talk about the claim that high-skill work is less likely to be automated than low-skill work. The author says:
>On almost all points, Baugues misses the mark. First, there is a qualitative difference between an auto worker (unskilled manual work) and a software developer (skilled knowledge work) that made automating the former inevitable.
As a rebuttal, I would point to some passages from Martin Ford's book on automation, The Lights in the Tunnel:
>A common misconception about automation is the idea that it will primarily impact low paying jobs that require few skills or training. To illustrate that this is not necessarily the case, consider two very different occupations: a radiologist and a housekeeper....
>In fact, we can reasonably say that software jobs (or knowledge worker jobs) are typically high paying jobs. This creates a very strong incentive for businesses to offshore and, when possible, automate these jobs....
>As a result, we can expect that, in the future, automation will fall heavily on knowledge workers and in particular on highly paid workers....
In general, I think knowledge jobs are at greater risk of automation than manual labor jobs. Robots are expensive. And they also don't scale as well as software. It's no coincidence that the largest software companies dwarf the largest robot companies. Ultimately, because software can scale, it becomes ridiculously cheap when you deploy it to millions or billions of people.
I agree with your overall point that creativity and design are hard to automate. However, software jobs are being and will be automated. What matters in the end is whether automation increases or decreases the overall demand for development. I.e., whether software automation is a complement to labor or a supplement to labor. So far, it's been a complement and software productivity has skyrocketed over the last couple of decades. This trend seems likely to continue, but who knows what the future holds 100 years from now.
P.S. Another industry that evolved similarly is agriculture. At first, tools and animals made farmers more productive, and farmer employment rose. But eventually, productivity rose so high that farmers started to saturate demand. Today, only 2% of Americans are farmers, despite farmers being the most productive of all time. So what's the moral here? What can software learn from farming? I think the moral is that quantity supplied is a function of both SUPPLY of labor and DEMAND for labor, and you can't ignore one when you predict the future.
P.P.S. You probably realize that the weakness of the agriculture comparison is that the hunger for food is far more easily sated than the hunger for software. The first has a biological limit, whereas the second is ostensibly unlimited. Therefore, you might reason, higher productivity in software is less likely to saturate demand and reduce employment. However, I would caution that some software demands might be easily saturated. Consider a company that wants its own website and app. If websites and apps become 100x cheaper (as a result of automation/higher productivity), would businesses demand 100x as many? Probably not.
P.P.P.S. Not all manual labor is created equal. Repetitive, indoor labor is much easier to automate than non-repetitive or outdoor labor.
>On almost all points, Baugues misses the mark. First, there is a qualitative difference between an auto worker (unskilled manual work) and a software developer (skilled knowledge work) that made automating the former inevitable.
As a rebuttal, I would point to some passages from Martin Ford's book on automation, The Lights in the Tunnel:
>A common misconception about automation is the idea that it will primarily impact low paying jobs that require few skills or training. To illustrate that this is not necessarily the case, consider two very different occupations: a radiologist and a housekeeper....
>In fact, we can reasonably say that software jobs (or knowledge worker jobs) are typically high paying jobs. This creates a very strong incentive for businesses to offshore and, when possible, automate these jobs....
>As a result, we can expect that, in the future, automation will fall heavily on knowledge workers and in particular on highly paid workers....
In general, I think knowledge jobs are at greater risk of automation than manual labor jobs. Robots are expensive. And they also don't scale as well as software. It's no coincidence that the largest software companies dwarf the largest robot companies. Ultimately, because software can scale, it becomes ridiculously cheap when you deploy it to millions or billions of people.
I agree with your overall point that creativity and design are hard to automate. However, software jobs are being and will be automated. What matters in the end is whether automation increases or decreases the overall demand for development. I.e., whether software automation is a complement to labor or a supplement to labor. So far, it's been a complement and software productivity has skyrocketed over the last couple of decades. This trend seems likely to continue, but who knows what the future holds 100 years from now.
P.S. Another industry that evolved similarly is agriculture. At first, tools and animals made farmers more productive, and farmer employment rose. But eventually, productivity rose so high that farmers started to saturate demand. Today, only 2% of Americans are farmers, despite farmers being the most productive of all time. So what's the moral here? What can software learn from farming? I think the moral is that quantity supplied is a function of both SUPPLY of labor and DEMAND for labor, and you can't ignore one when you predict the future.
P.P.S. You probably realize that the weakness of the agriculture comparison is that the hunger for food is far more easily sated than the hunger for software. The first has a biological limit, whereas the second is ostensibly unlimited. Therefore, you might reason, higher productivity in software is less likely to saturate demand and reduce employment. However, I would caution that some software demands might be easily saturated. Consider a company that wants its own website and app. If websites and apps become 100x cheaper (as a result of automation/higher productivity), would businesses demand 100x as many? Probably not.
P.P.P.S. Not all manual labor is created equal. Repetitive, indoor labor is much easier to automate than non-repetitive or outdoor labor.
Ford's point is a good one, but it's a bit dishonest. A radiologist's job is processing information - it's easy to move overseas or stick into an algorithm. A housekeeper's job is managing a house and the people living in it - a hotelier is similarly difficult to automate.
Regarding your PPS - it's not so much that demand for software is unlimited, it's that software can be used for far more purposes than food or natural resources. A single algorithm can be used in multiple industries, and given that the world doesn't develop evenly as a whole, there will be software opportunities in the real world (machinery), in processing the real world (commerce), in the virtual world (media), and in processing the virtual world (analytics), and these opportunities will increasingly require more and more knowledge and creativity.
Regarding your PPS - it's not so much that demand for software is unlimited, it's that software can be used for far more purposes than food or natural resources. A single algorithm can be used in multiple industries, and given that the world doesn't develop evenly as a whole, there will be software opportunities in the real world (machinery), in processing the real world (commerce), in the virtual world (media), and in processing the virtual world (analytics), and these opportunities will increasingly require more and more knowledge and creativity.
There might not be ways to automate everything, but there will certainly be changes to how we do things that facilitate the replacement and downsizing of some otherwise 'creative' jobs. A housekeeper can be replaced by modifying the nature of the house - a fridge that keeps itself stocked and roombas that actually do a proper job, for example.
My point is that the qualitative difference between a creative and non-creative job is mitigated to some extent by all the 'non-creative' stuff we have to do every day. So even knowledge workers are not immune to 'dumb' automation. An example of this would be remote work and off-shoring, which has opened up software development projects to global competition.
However, the argument still stands - knowledge workers aren't unionized manual labour and if robots are going to replace everything, someone has to program the robots. And, there's easily going to be more 'robots' tomorrow than today for some time to come. So what could end the 'world automation' gravy train we are on right now?
IMHO, the average knowledge worker has a much clearer and more present danger that the chance they might be making themselves redundant - and one that would affect pretty much everyone, not just knowledge workers. They should more concerned about that other thing that destroys demand: financial destruction. The redundancy threat that knowledge workers should be concerned about are the sins of our owners coming home to roost; another depression.
My point is that the qualitative difference between a creative and non-creative job is mitigated to some extent by all the 'non-creative' stuff we have to do every day. So even knowledge workers are not immune to 'dumb' automation. An example of this would be remote work and off-shoring, which has opened up software development projects to global competition.
However, the argument still stands - knowledge workers aren't unionized manual labour and if robots are going to replace everything, someone has to program the robots. And, there's easily going to be more 'robots' tomorrow than today for some time to come. So what could end the 'world automation' gravy train we are on right now?
IMHO, the average knowledge worker has a much clearer and more present danger that the chance they might be making themselves redundant - and one that would affect pretty much everyone, not just knowledge workers. They should more concerned about that other thing that destroys demand: financial destruction. The redundancy threat that knowledge workers should be concerned about are the sins of our owners coming home to roost; another depression.
Knowledge workers are definitely not immune to automation, I've acknowledged that in the post. They do set the bar quite high for software and intelligent agents, however.
Regarding your last point, I think a knowledge worker smart enough to automate themselves out of a job will be smart enough to capitalize on that discovery. Moreover, there will always be tasks people will prefer done manually, knowledge-based or otherwise.
Regarding your last point, I think a knowledge worker smart enough to automate themselves out of a job will be smart enough to capitalize on that discovery. Moreover, there will always be tasks people will prefer done manually, knowledge-based or otherwise.
What if the problems to solve are becoming easier to solve, because we're more plugged-in into an infrastructure of information? That eliminates work.
Standards are a good example. We love them, but, in a way, they eliminate the creation of middleware jobs that transform the data from one format to another.
Standards are a good example. We love them, but, in a way, they eliminate the creation of middleware jobs that transform the data from one format to another.
Good point, but it touches on a refrain I've seen in the comments: that it might be better to keep the world in an inefficient state. Standards rise up after the complexity of interoperability becomes too much to handle.
As to your point about information, the volume of data these days requires increasingly sophisticated analysis. There seems to be a shift in sentiment around Big Data, that algorithms are still too unsophisticated to provide useful output, and that's a good sign that not only is there a lot of work to be done in this area, but a large volume of information doesn't automatically guarantee quality insight.
As to your point about information, the volume of data these days requires increasingly sophisticated analysis. There seems to be a shift in sentiment around Big Data, that algorithms are still too unsophisticated to provide useful output, and that's a good sign that not only is there a lot of work to be done in this area, but a large volume of information doesn't automatically guarantee quality insight.
Actually the radiologists job has proved difficult to either move overseas (requires US credentials) or automate (CAD still generates a ton of false positives). A radiologist still makes a considerable salary and is the R in the ROAD to success. Plus advances in continuous imaging and interventional procedures make it unlikely that they will see any significant decline.
Exactly right, it seems many people assume that automation or offshoring will be a smooth and seamless process with no trade-offs. That's not the case at all, offshoring and automation present their own kind of problems.
What's dishonest? I don't think I follow.
Do you mean that housekeeper and radiologist were cherrypicked and that in reality laborers are at higher risk of automation than knowledge workers?
Do you mean that housekeeper and radiologist were cherrypicked and that in reality laborers are at higher risk of automation than knowledge workers?
No, this part:
"A common misconception about automation is the idea that it will primarily impact >low paying< jobs that require few skills or training. To illustrate that this is not necessarily the case, consider two very different occupations: a radiologist and a housekeeper."
A hotelier's job, the well-compensated version of a housekeeper's job, is also difficult to automate. On the flip side, a good deal of information processing jobs, no matter how well compensated, can be automated. I think the point was made rather bluntly, without elaboration.
A hotelier's job, the well-compensated version of a housekeeper's job, is also difficult to automate. On the flip side, a good deal of information processing jobs, no matter how well compensated, can be automated. I think the point was made rather bluntly, without elaboration.
The point was made just fine. He said that there is not one definitive compensation class that will be primarily affected. It doesn't seem like you've refuted that.
@nrivadeneira: Yes, the point Ford made works, but it works because of the _nature of the jobs_, not because of any inherent property of theirs that influences their compensation (implied by Ford to be the knowledge/labor distinction). Clearly you and I can think of high-income location-bound jobs (bodyguard) and of low-income information-processing jobs (library clerk).
Thanks for clarifying!
Regards agriculture: there remain countries in which the vast majority (over 90%) of the population is engaged in agriculture. Much agricultural productivity has come through vastly increased inputs and capital, and not all countries have access to or can afford these.
The excess labor from agriculture was absorbed into other fields (so to speak). Not always painlessly -- working conditions in early Industrial Revolution factories were terrible (see Toynbee and Carlyle for descriptions, as well as, of course, Dickens). But once the kinks were worked out, we ended up with carpenters and welders and artists and writers and dog walkers.
My own view is that we're starting to run a bit far in that direction, but the coming shortage of raw materials and energy will likely address that problem for us, if not how many would prefer it be done.
The excess labor from agriculture was absorbed into other fields (so to speak). Not always painlessly -- working conditions in early Industrial Revolution factories were terrible (see Toynbee and Carlyle for descriptions, as well as, of course, Dickens). But once the kinks were worked out, we ended up with carpenters and welders and artists and writers and dog walkers.
My own view is that we're starting to run a bit far in that direction, but the coming shortage of raw materials and energy will likely address that problem for us, if not how many would prefer it be done.
I agree that, long-term, high-skill work is at least as likely to be automated as low-skill work. There's no shortage of occupations that were once high-skill and now are largely mechanized. We might not be able to foresee how that can happen with software, but there's a long list of industries that weren't automated until they were, to the surprise of those employed within those industries.
In a small way, one can argue that increasingly easy-to-use frameworks are already having that effect. Right now, with the market for software and software developers growing so rapidly, the efficiency gained from using frameworks is a great thing for the individual programmer. The time she doesn't spend building animations or site looks or CMS's can easily be applied to other problems, and she benefits directly from the increased productivity. But it's risky to assume that the demand for software will always outpace the supply, as this has never been the case for any other labor market I can think of. Markets seek and tend to eventually find equilibrium in these areas.
In a small way, one can argue that increasingly easy-to-use frameworks are already having that effect. Right now, with the market for software and software developers growing so rapidly, the efficiency gained from using frameworks is a great thing for the individual programmer. The time she doesn't spend building animations or site looks or CMS's can easily be applied to other problems, and she benefits directly from the increased productivity. But it's risky to assume that the demand for software will always outpace the supply, as this has never been the case for any other labor market I can think of. Markets seek and tend to eventually find equilibrium in these areas.
Good point, but we've always been automating software, and so far the market for it has been booming. The idea of "peak software" seems to involve assumptions that just don't work in the case of software development. It's worth thinking about software as more than just web development problems - think energy, education, entertainment, sports, media, recreation, art, manufacturing (big and small), virtual reality, and so on. The solution space is enormous, practically infinite.
Thanks for bringing that up, excellent article.
Regardless of which article is correct (this one or the one it is referencing) in predicting the future of software employment, I think we'd all do well to keep an eye out for the future and be agile and ready enough if a Detroit-like situation does arrive. It should never be taken as a given that we'll always be highly sought after and well paid. With that in mind, I find Baugue's article much more useful considering I've never thought of an article that can be reduced to "Things are great, I don't think anything will change." to be of much help.
That's a gross oversimplification of what I wrote. If you take only Baugues's conclusion ("don't be reckless, save and network, things may not always be so great") and ignore his premises, you're reducing the article to a platitude. My intent was to show how fears of automation are nothing new and should not stop developers from pursuing their careers.
Autowork in the 1930s and beyond may have been semi-skilled, but before widespread automation, it was a craft, and cars were expensive. This de-skilling has always been happening in software. Back in the 80s, word processors were written in assembly. Today, they can be written in JavaScript.
It's a pretty complacent article. How long until software development has good quality automation - of a kind that drastically reduces the number of jobs? I suggest it's not as far away as the author thinks.
Software generation is automated but when people are talking about software development they mean (technical) design. Design isn't a commodity or product but a one-off process and therefore by definition cannot be automated. The underlying topic isn't about automation of software development - it's about buy vs build, and as ever that depends on specific requirements and circumstances.
That's the wrong question. The very nature of software is automate jobs including software jobs and has since the very beginning. A programmer today is easily one or more orders of magnitude more productive as a programmer 20 years ago. We're all producing much more software with fewer people.
The question is when will the demand for software be low enough that this ongoing process will reduce the number of programmer jobs.
The question is when will the demand for software be low enough that this ongoing process will reduce the number of programmer jobs.
We've had the tools to automate software development for years and there's been almost no movement in that direction.
There is something that can slaughter the job security of software engineers, but it's not technological advancement.
Before I get to that, my personal forecast is: income expectancy for intelligent and hardworking people will continue to increase but so will variance. Eventually, the variance will become intolerable enough (sans regularization) that basic income will appear to be the only solution; even the smartest people will see a personal need for something like it. There will be a lot of struggling and fighting over implementation (e.g. do college graduates get more? how about families? should society incentivize or penalize reproduction?) but everyone will agree that it's necessary. That's probably about 30-50 years from now.
Okay, so what's the #1 threat to software developers' job security? Not robots. We have robots that write code (compilers) and our jobs are still challenging. Instead, it's upper-level complacency in society. An elite that would rather hold position and keep relative rank than grow society and gain absolute prosperity (but at a lesser rate than others, losing rank). Historically, this is the norm for elites. Many people, especially in positions of power (because power selects such people) would rather reign in hell than serve in heaven. If they can conspire toward self-protection and social mediocrity, they will. The good news is that, in a large and heterogeneous society, it's actually very hard to hold any conspiracy together for very long. This is why conspiracy theories overstate the power of such organizations; conspiracies (lower-case 'c') exist all over the place (e.g. Davos assholes) but none have the power to run the world unilaterally. It's too chaotic and large. Societies and nations have set themselves back centuries when their elites conspired toward mediocrity, but I don't think the world elite is capable of doing so-- especially not now.
If the global business elite figures out a way to conspire toward mediocrity, the people like us are fucked, because excellence will cease to matter and being good at what one does will just get one the reputation of a troublemaker. However, the likelihood that anyone could pull this together in a world with unprecedented distribution of technical literacy is incredibly low.
As long as there are people looking to do things in this world, and the world continues to advance technologically, there will be a high level of demand for people who are technically competent. We might be applying our skills differently and changing industries, but there will be work for us, and society would have to reach an advanced (that is, certainly plausible but unlikely) state of degradation for us not to do that work on at least half-decent (e.g. middle-class) terms.
Before I get to that, my personal forecast is: income expectancy for intelligent and hardworking people will continue to increase but so will variance. Eventually, the variance will become intolerable enough (sans regularization) that basic income will appear to be the only solution; even the smartest people will see a personal need for something like it. There will be a lot of struggling and fighting over implementation (e.g. do college graduates get more? how about families? should society incentivize or penalize reproduction?) but everyone will agree that it's necessary. That's probably about 30-50 years from now.
Okay, so what's the #1 threat to software developers' job security? Not robots. We have robots that write code (compilers) and our jobs are still challenging. Instead, it's upper-level complacency in society. An elite that would rather hold position and keep relative rank than grow society and gain absolute prosperity (but at a lesser rate than others, losing rank). Historically, this is the norm for elites. Many people, especially in positions of power (because power selects such people) would rather reign in hell than serve in heaven. If they can conspire toward self-protection and social mediocrity, they will. The good news is that, in a large and heterogeneous society, it's actually very hard to hold any conspiracy together for very long. This is why conspiracy theories overstate the power of such organizations; conspiracies (lower-case 'c') exist all over the place (e.g. Davos assholes) but none have the power to run the world unilaterally. It's too chaotic and large. Societies and nations have set themselves back centuries when their elites conspired toward mediocrity, but I don't think the world elite is capable of doing so-- especially not now.
If the global business elite figures out a way to conspire toward mediocrity, the people like us are fucked, because excellence will cease to matter and being good at what one does will just get one the reputation of a troublemaker. However, the likelihood that anyone could pull this together in a world with unprecedented distribution of technical literacy is incredibly low.
As long as there are people looking to do things in this world, and the world continues to advance technologically, there will be a high level of demand for people who are technically competent. We might be applying our skills differently and changing industries, but there will be work for us, and society would have to reach an advanced (that is, certainly plausible but unlikely) state of degradation for us not to do that work on at least half-decent (e.g. middle-class) terms.
So good, thanks!.
To pick on the "We'll never be automated example":
It's not that one day programs will write programs that write--oh wait, that's already happened. It's not that tools will come around that making throwing up a simple website with ecommerce functionality ea--shit, that happened too.
It's that the core functions of software, from a business perspective, are pretty much nailed down now. It's that the average dudebro with an MBA and half a brain can throw together an MVP and start getting market feedback. It's that the homeless guy who lives down the street from me can publish an ebook from his acer.
The thing that we all love so much, the removal of gatekeepers and democratization of information capital? Yeah, so, that thing is what will put us out of jobs. And that's fine.
The thing the author misses is not that there won't be a few gigs around for people who need to optimize their database backends and red-black tree all the things and skiplist their twitters--it's that the vast majority of things that used to require people to find a coder to help them are now a few buttons away.
Oh, and also that we've cultivated a user base so ignorant and attracted to simple glossy black boxes that they can't appreciate what it is to have a real engineer along for the ride anyways, nor how to usefully communicate with them if they had one. That's going to sting too.