Domain Knowledge or a Lack Thereof(jacquesmattheij.com)
jacquesmattheij.com
Domain Knowledge or a Lack Thereof
http://jacquesmattheij.com/domain-knowledge-or-a-lack-thereof
111 comments
Yes
And some are almost proud to not know anything beyond their field.
Newsflash: You can be the best writer in the world, but what are you going to write about then if you know nothing besides writing?
Programmers often complain about managers knowing nothing about the field. But it's the same thing when the programmer doesn't know anything.
There is a lot of money in doing specialist applications. Most of these are made by people who know a lot about their field but only a little bit of programming.
And some are almost proud to not know anything beyond their field.
Newsflash: You can be the best writer in the world, but what are you going to write about then if you know nothing besides writing?
Programmers often complain about managers knowing nothing about the field. But it's the same thing when the programmer doesn't know anything.
There is a lot of money in doing specialist applications. Most of these are made by people who know a lot about their field but only a little bit of programming.
This. The same programmers that complain about clueless micromanagement are usually also the ones with the "I don't want to know all this business shit" attitude.
Guess what? You're being micromanaged by MBA's because otherwise, despite all your l33t coding skills, you're f-ing useless.
Guess what? You're being micromanaged by MBA's because otherwise, despite all your l33t coding skills, you're f-ing useless.
I'm a junior in an Information Systems program right now. I have to admit, when I first started school, and I looked at my curriculum, It was pretty disappointing to find out that I'd be taking more business and economics classes than actual computer-related courses.
Now that I'm over 80 credit hours in, I have to say that my Micro and Macroeconomics classes were two of the most interesting courses in my entire program.
Now that I'm over 80 credit hours in, I have to say that my Micro and Macroeconomics classes were two of the most interesting courses in my entire program.
> usually also the ones with the "I don't want to know all this business shit" attitude
Very easily identified early in school as the ones who say "I don't want to know all this history shit" or "I don't want to know all this writing shit"
Very easily identified early in school as the ones who say "I don't want to know all this history shit" or "I don't want to know all this writing shit"
Two of the most painful experiences in my life (ok- slight exaggeration) came in engineering school. One was watching the first time we were asked to make powerpoint presentations (they were transparencies, actually, but same diff). The other was reading the first papers we wrote in our "writing for engineers" class.
In both cases, sitting there and watching was excruciating in that "holy crap this is awful, just make it stop for the sake of the poor guy" kind of way.
But you know what? Some of us got better at it - the ones who made an effort. This made me realize that we do not force enough liberal arts on engineers in school. In fact, I was prohibited from taking many art classes - I got no credit towards my degree for them because they were perceived as "soft". That is crazy.
In both cases, sitting there and watching was excruciating in that "holy crap this is awful, just make it stop for the sake of the poor guy" kind of way.
But you know what? Some of us got better at it - the ones who made an effort. This made me realize that we do not force enough liberal arts on engineers in school. In fact, I was prohibited from taking many art classes - I got no credit towards my degree for them because they were perceived as "soft". That is crazy.
I ended up realizing in my junior year of high school that liberal arts were enjoyable. Graduated with a degree in CS and a minor in creative writing and I'm very glad I did it.
I also think that reading and posting on places with high quality discussion like HN and reddit circa 2008 can really improve your communication skills.
I also think that reading and posting on places with high quality discussion like HN and reddit circa 2008 can really improve your communication skills.
A liberal arts education gives you an advantage over other engineers. "You mean this guy can actually read and write?"
Especially true if the business ever needs to write a technical proposal to win a contract. Very important to have engineers who can read and write then.
Especially true if the business ever needs to write a technical proposal to win a contract. Very important to have engineers who can read and write then.
Most liberal arts educations are a quite inefficient way to teach people to write. Many who leave such educations are not that much better off than when they started.
A liberal arts education gives you an advantage over other engineers. "You mean this guy can actually read and write?"
There are a couple of questionable assumptions baked into that statement, IMO. First, that a "liberal arts education" (by which I assume you mean earning a 4 year degree from a liberal arts university) definitely means you will come out able to read and write to a superior degree. Maybe if you major in English or Philosophy, otherwise, I wouldn't take that as a given.
Also, there seems to be an implicit assumption that the only way to gain a solid grasp of reading and writing well is through a liberal arts education (again, I'm assuming you mean a formal liberal arts education). I'm not sure I buy that. The best way to learn to write, IMO, is to write a lot, read a lot, and solicit feedback on your writing. None of that requires a formal liberal arts education.
Now if you aren't necessarily referring to formal / university education, then we probably don't disagree. I certainly agree with the point that having solid reading and writing skills is a valuable thing!
There are a couple of questionable assumptions baked into that statement, IMO. First, that a "liberal arts education" (by which I assume you mean earning a 4 year degree from a liberal arts university) definitely means you will come out able to read and write to a superior degree. Maybe if you major in English or Philosophy, otherwise, I wouldn't take that as a given.
Also, there seems to be an implicit assumption that the only way to gain a solid grasp of reading and writing well is through a liberal arts education (again, I'm assuming you mean a formal liberal arts education). I'm not sure I buy that. The best way to learn to write, IMO, is to write a lot, read a lot, and solicit feedback on your writing. None of that requires a formal liberal arts education.
Now if you aren't necessarily referring to formal / university education, then we probably don't disagree. I certainly agree with the point that having solid reading and writing skills is a valuable thing!
This. The same programmers that complain about clueless micromanagement are usually also the ones with the "I don't want to know all this business shit" attitude.
Ridiculous. I know this stereotype exists, but I don't know many programmers who actually hold that attitude. The "business shit" is intellectually easier and not at all uninteresting. The payoff in learning it is quite high: in a functional organization, you get more respect and money for a relatively small investment. There might be some clueless 23-year-olds out there who reflexively dislike "business" because of Marxist-academic indoctrination, but that's a rare and transient state.
If anything, the MBA micromanagers try to encourage this attitude by taking the "I will handle all of this" approach with the business side and the "big picture". They make it their turf by assertion, then wonder why the engineers under them seem not to give a shit.
Engineers tune out when they don't have any power or control. It's a survival mechanism. If you can't do anything about it, don't waste cycles on it.
Ridiculous. I know this stereotype exists, but I don't know many programmers who actually hold that attitude. The "business shit" is intellectually easier and not at all uninteresting. The payoff in learning it is quite high: in a functional organization, you get more respect and money for a relatively small investment. There might be some clueless 23-year-olds out there who reflexively dislike "business" because of Marxist-academic indoctrination, but that's a rare and transient state.
If anything, the MBA micromanagers try to encourage this attitude by taking the "I will handle all of this" approach with the business side and the "big picture". They make it their turf by assertion, then wonder why the engineers under them seem not to give a shit.
Engineers tune out when they don't have any power or control. It's a survival mechanism. If you can't do anything about it, don't waste cycles on it.
I work with at least one guy who just straight-up isn't interested in managing, ever. As the team has expanded, he's moved two tiers down the org chart, but he isn't interested in learning the administrative portion at all. It's not about control, some people just seem to be allergic to meetings, dealing with stakeholders, etc. It's a perfectly valid viewpoint to have: some people just want to write code.
The guy you mention likely is a solitary worker (I'm one as well).
We tend to prefer not working in teams, other people slow us down and cause confusion in our work.
But America's insistence on a "team" based work culture really hampers those like me. We can get 10x the amount of work done individually as we can as part of a team if left to our devices - but this is frowned upon, indeed looked at as a social "problem".
I * sincerely* blame MBA culture for this detrimental attitude.
We tend to prefer not working in teams, other people slow us down and cause confusion in our work.
But America's insistence on a "team" based work culture really hampers those like me. We can get 10x the amount of work done individually as we can as part of a team if left to our devices - but this is frowned upon, indeed looked at as a social "problem".
I * sincerely* blame MBA culture for this detrimental attitude.
It's kind of an interesting viewpoint; how do you handle being 'crowded out' of a project? For example, if you started as the sole worker on something internal, but then it became customer-facing, and the team necessarily had to expand to get the work done. I can appreciate saying that you only get 10% of the work done if you're on a team, but you don't honestly think you can achieve as much as a ten man team? What about when more than 10 people are required?
I guess my question is, as a self-identified 'solitary worker', would you take it upon yourself to leave a project if it reached a point where team expansion was necessary? Is there any good way to manage someone like you to get you to contribute to a larger project, or if it a lost cause?
I guess my question is, as a self-identified 'solitary worker', would you take it upon yourself to leave a project if it reached a point where team expansion was necessary? Is there any good way to manage someone like you to get you to contribute to a larger project, or if it a lost cause?
There is a difference between working on a team where everyone works on the same bits and you have to work around their projects and their timelines, and being part of a team where each 'member' works in a semi-siloed, semi-independant capacity.
I much prefer the latter to the former since I don't have to work about anyone else makes changes to stuff without my knowledge but I'm still on the team, still met with the others to discuss what we are working on and how it impacts X or Y.
I much prefer the latter to the former since I don't have to work about anyone else makes changes to stuff without my knowledge but I'm still on the team, still met with the others to discuss what we are working on and how it impacts X or Y.
I can see the siloed approach working to a point, but there are two big drawbacks:
- no exposure to new skills and techniques. This particular employee is pretty fond of cut-and-paste, and we've been refactoring to fix a lot of that. As we refactor, the ground is sort of shrinking beneath his feet, because he isn't willing to work on other people's code. If you aren't willing to learn new techniques and read other people's code, your portion of the codebase is going to accumulate technical debt while the rest moves forward.
- no big-picture perspective. I love to stick my nose into every architectural discussion, and I think it's beneficial for us to plan big decisions as a team, to try and find the best solution. From experience, a siloed employee won't be interested in getting involved in these large-scale discussions, because they haven't kept up with the rest of the codebase, and they don't have an interest in it.
Essentially, having all your team hived off from each other, doing their own thing:
- gives you a very low bus factor. If one guy moves on, dies, whatever, suddenly you've got a dark area of your codebase with built up debt and no knowledge.
- reduces the cohesiveness of your design. At the best, the high level plan will be dictated by management or a single architecture, and each employee builds their part according to spec. This seems to reduce autonomy and job satisfaction, and it presumes none of the coders have any good ideas about architecture (wrong).
If you have techniques or suggestions for getting around these issues, I'd be glad to hear them. It is a big problem that some very good technical people aren't necessarily very social.
- no exposure to new skills and techniques. This particular employee is pretty fond of cut-and-paste, and we've been refactoring to fix a lot of that. As we refactor, the ground is sort of shrinking beneath his feet, because he isn't willing to work on other people's code. If you aren't willing to learn new techniques and read other people's code, your portion of the codebase is going to accumulate technical debt while the rest moves forward.
- no big-picture perspective. I love to stick my nose into every architectural discussion, and I think it's beneficial for us to plan big decisions as a team, to try and find the best solution. From experience, a siloed employee won't be interested in getting involved in these large-scale discussions, because they haven't kept up with the rest of the codebase, and they don't have an interest in it.
Essentially, having all your team hived off from each other, doing their own thing:
- gives you a very low bus factor. If one guy moves on, dies, whatever, suddenly you've got a dark area of your codebase with built up debt and no knowledge.
- reduces the cohesiveness of your design. At the best, the high level plan will be dictated by management or a single architecture, and each employee builds their part according to spec. This seems to reduce autonomy and job satisfaction, and it presumes none of the coders have any good ideas about architecture (wrong).
If you have techniques or suggestions for getting around these issues, I'd be glad to hear them. It is a big problem that some very good technical people aren't necessarily very social.
@alanctgardner2 Agreed, there are limitations. If that guy is a fan of cut-n-paste then there are bigger issues at hand than just social interaction.
Working in a siloed environment doesn't mean not learning new skills or techniques. And I don't mean that you are only working on the same project/chunk of code the entire time. We would often switch projects so that what I'm working on today is completely different that last week and has been touched/edited/tweaked by just about everyone on the team at some point.
Having a Big Picture perspective is vital be being able to work individually. You have to know how your work will ultimately impact everyone else and how their will impact you.
I don't mean that everyone is working on completely separate projects but that Employee 1 might be working on a feature enhancement on version 4.5.3 while Employee 2 is working on a refactor of the back end code and I might be working on implementing a change control request on version 4.5.4.
Perhaps this is a better way to explain: everyone is working on the same Project but on individual and separate Tasks
Working in a siloed environment doesn't mean not learning new skills or techniques. And I don't mean that you are only working on the same project/chunk of code the entire time. We would often switch projects so that what I'm working on today is completely different that last week and has been touched/edited/tweaked by just about everyone on the team at some point.
Having a Big Picture perspective is vital be being able to work individually. You have to know how your work will ultimately impact everyone else and how their will impact you.
I don't mean that everyone is working on completely separate projects but that Employee 1 might be working on a feature enhancement on version 4.5.3 while Employee 2 is working on a refactor of the back end code and I might be working on implementing a change control request on version 4.5.4.
Perhaps this is a better way to explain: everyone is working on the same Project but on individual and separate Tasks
That just seems like every team ever, except for pairs programming I can't imagine having two people in the same part of the codebase at the same time. Is there an alternative you can think of that isn't just poor management, or ambiguous descriptions leading to conflicts?
Also, you can reply even during the cooling-off period by hitting the 'link' link.
Also, you can reply even during the cooling-off period by hitting the 'link' link.
Majors bugs that involve both the front and back end developers.
Major feature enhancements in which multiple devs touch the same set of files.
There are probably ways around these issues, just some instances where I've had to sit next to someone and work out a bug/problem.
Major feature enhancements in which multiple devs touch the same set of files.
There are probably ways around these issues, just some instances where I've had to sit next to someone and work out a bug/problem.
" would you take it upon yourself to leave a project if it reached a point where team expansion was necessary?"
Not every project requires team expansion and teams vary in strength and productivity. But in general, yes I would leave if I were required to stand in front of 9 "team members" every morning and tell them my deeds and troubles. At this point, the challenges have been solved and the single hardest problem in the project is coordinating everyone. I get bored and frustrated at this point causing me to seek a more interesting project.
Software is fundamentally a creative discipline - creativity rarely occurs in a committee.
A 10 man team can certainly do more work then I can (in general). But if I'm unable to work properly in a 10 man team, I SHOULDN'T be on that team. Not all of us are effective in the traditional agile environment - give us a nook somewhere and let us solve problems independently. You'll get far more value out of us that way then adding us as the 11th member of that team.
Not every project requires team expansion and teams vary in strength and productivity. But in general, yes I would leave if I were required to stand in front of 9 "team members" every morning and tell them my deeds and troubles. At this point, the challenges have been solved and the single hardest problem in the project is coordinating everyone. I get bored and frustrated at this point causing me to seek a more interesting project.
Software is fundamentally a creative discipline - creativity rarely occurs in a committee.
A 10 man team can certainly do more work then I can (in general). But if I'm unable to work properly in a 10 man team, I SHOULDN'T be on that team. Not all of us are effective in the traditional agile environment - give us a nook somewhere and let us solve problems independently. You'll get far more value out of us that way then adding us as the 11th member of that team.
I work with at least one guy who just straight-up isn't interested in managing, ever. As the team has expanded, he's moved two tiers down the org chart, but he isn't interested in learning the administrative portion at all.
I just want to point out that there are different aspects of "managing". There's tactical, command and control, administrivia type stuff, and there's "solving problems, acquiring resources and providing high level coordination" and then there's "strategic decision making" kind of management. One can certainly lack interest in one of those, but might find the others interesting if exposed to that part of things.
Personally, I consider most of what most managers do, to be "administrivia" which is of very low value, and I'd have no interest in ever doing that. But when a manager it tasked with actually identifying and solving problems, making strategic decisions, etc., that part can actually be as intellectually challenging as programming and can actually be fun in it's way.
It's not about control, some people just seem to be allergic to meetings, dealing with stakeholders, etc. It's a perfectly valid viewpoint to have: some people just want to write code.
That's absolutely true as well. I was largely that way when I was younger, but over time I've grown more interested in the other aspects of things.
I just want to point out that there are different aspects of "managing". There's tactical, command and control, administrivia type stuff, and there's "solving problems, acquiring resources and providing high level coordination" and then there's "strategic decision making" kind of management. One can certainly lack interest in one of those, but might find the others interesting if exposed to that part of things.
Personally, I consider most of what most managers do, to be "administrivia" which is of very low value, and I'd have no interest in ever doing that. But when a manager it tasked with actually identifying and solving problems, making strategic decisions, etc., that part can actually be as intellectually challenging as programming and can actually be fun in it's way.
It's not about control, some people just seem to be allergic to meetings, dealing with stakeholders, etc. It's a perfectly valid viewpoint to have: some people just want to write code.
That's absolutely true as well. I was largely that way when I was younger, but over time I've grown more interested in the other aspects of things.
If you can't do anything about it, don't waste cycles on it.
I think the point is that you can do something about it - it just requires stepping out of the engineering midset for moment now and then. I've seen this fatalistic attitude frequently, both in software developers and "real" engineers (mechanical, electrical, etc). It's not healthy for the engineer or the organization, and I can't quite figure out where the attitude comes from.
I think the point is that you can do something about it - it just requires stepping out of the engineering midset for moment now and then. I've seen this fatalistic attitude frequently, both in software developers and "real" engineers (mechanical, electrical, etc). It's not healthy for the engineer or the organization, and I can't quite figure out where the attitude comes from.
Engineers are culturally unusual in being very tolerant of criticism (from other engineers). That's because our compilers, nominally subordinate, yell at us on a daily basis about minor syntactic errors.
With managers, you can't take the same approach. You cannot simply tell your boss that his managerial style is nukular buttfail. That gets you fired.
With managers, you can't take the same approach. You cannot simply tell your boss that his managerial style is nukular buttfail. That gets you fired.
I've known a few programmers who really just were not interested in the business side of things. And I've known a few who were really interested in it. On balance, I think this stereotype exist for a reason, but - as with all stereotypes - it is flawed when treated as a general truth.
Engineers tune out when they don't have any power or control. It's a survival mechanism. If you can't do anything about it, don't waste cycles on it.
Yeah, I've definitely seen this. But, on the flip side, I've seen manager types who want the technical people to get move involved in understanding the business issues and develop more of an ability to contribute at that boundary (to the extent that such a thing actually exists) between the "business world" and the "IT world".
If you folks will indulge me a bit of (related) self-promotion, I've written two recent blog posts[1][2], touching on a specific technique and methodology that I see gaining some steam, which deals with operating at that "chasm" between the worlds. In the first part, I argue that, in the future to come, developers are going to need to learn more about the business world AND that managers are going to have to learn more about technology.
[1]: http://fogbeam.blogspot.com/2013/02/so-what-is-capability-ca...
[2]: http://fogbeam.blogspot.com/2013/01/why-capability-cases-are...
Engineers tune out when they don't have any power or control. It's a survival mechanism. If you can't do anything about it, don't waste cycles on it.
Yeah, I've definitely seen this. But, on the flip side, I've seen manager types who want the technical people to get move involved in understanding the business issues and develop more of an ability to contribute at that boundary (to the extent that such a thing actually exists) between the "business world" and the "IT world".
If you folks will indulge me a bit of (related) self-promotion, I've written two recent blog posts[1][2], touching on a specific technique and methodology that I see gaining some steam, which deals with operating at that "chasm" between the worlds. In the first part, I argue that, in the future to come, developers are going to need to learn more about the business world AND that managers are going to have to learn more about technology.
[1]: http://fogbeam.blogspot.com/2013/02/so-what-is-capability-ca...
[2]: http://fogbeam.blogspot.com/2013/01/why-capability-cases-are...
I use to hold that attitude as did a lot of the people I worked with. Just said 'tell me what you want the thing to do and I'll make it happen', didn't really care about the business side of things.
That changed a few years ago (can't remember the exact reason why) when I started learning about the industry I was working in (because it was something I wanted to know about) and saw my work getting better and coming back with fewer and fewer change requests and a couple of Oh, that was a great way of doing X. You get it!
Made a world of difference.
That changed a few years ago (can't remember the exact reason why) when I started learning about the industry I was working in (because it was something I wanted to know about) and saw my work getting better and coming back with fewer and fewer change requests and a couple of Oh, that was a great way of doing X. You get it!
Made a world of difference.
Your point about writers is a real problem. Way too many literary novels where the main character is a writer.
Too many people, especially worker bees like engineers and janitors, tend to focus only on their part of the job and not how it fits into the big picture.
Too many people, especially worker bees like engineers and janitors, tend to focus only on their part of the job and not how it fits into the big picture.
> The profession of programmers who only know how to program should disappear like the profession of scribes who only know how to write.
Everybody "knows" how to write.
There's a difference between literacy and mastery.
Without fluency in the solution domain, the problem domain will simply remain that: a problem without a solution.
The Other Jacques's point was that as developers we often see lots of problem domains and that learning about them is useful.
I tell my customers that part of my job is becoming an "instant expert" in the part of the world they've called me in to help with.
The only subject I can think of with similar professional requirements would be law, where deep knowledge of the solution domain is required and quick acquisition of the problem domain is a predictor of success.
Everybody "knows" how to write.
There's a difference between literacy and mastery.
Without fluency in the solution domain, the problem domain will simply remain that: a problem without a solution.
The Other Jacques's point was that as developers we often see lots of problem domains and that learning about them is useful.
I tell my customers that part of my job is becoming an "instant expert" in the part of the world they've called me in to help with.
The only subject I can think of with similar professional requirements would be law, where deep knowledge of the solution domain is required and quick acquisition of the problem domain is a predictor of success.
I hope we'll have specialized careers someday (Medical CS, Economics CS, etc...).
In many careers, it's the other way round - most physics I know taught themselves how to program, for example, but they're lacking some tools that would probably help them a lot (and physics are very close to CS). I've seen accountants do a lot of programming in Excel macros and Access too.
Doctors and Economists might have less interest in learning, but academy-minded ones will really appreciate the modeling they can do with the use of computers (I think I saw a medical paper where they reinvented some basic CS concept :) ).
Some of the domain training comes from practical experience - I started out working for the credit report sector, and then moved on to the insurance sector, so I've picked along some domain knowledge in those areas. Other colleagues have worked in logistics, and now know about their problems, etc...
In many careers, it's the other way round - most physics I know taught themselves how to program, for example, but they're lacking some tools that would probably help them a lot (and physics are very close to CS). I've seen accountants do a lot of programming in Excel macros and Access too.
Doctors and Economists might have less interest in learning, but academy-minded ones will really appreciate the modeling they can do with the use of computers (I think I saw a medical paper where they reinvented some basic CS concept :) ).
Some of the domain training comes from practical experience - I started out working for the credit report sector, and then moved on to the insurance sector, so I've picked along some domain knowledge in those areas. Other colleagues have worked in logistics, and now know about their problems, etc...
> (I think I saw a medical paper where they reinvented some basic CS concept :)
Perhaps more shocking:
https://fliptomato.wordpress.com/2007/03/19/medical-research...
Perhaps more shocking:
https://fliptomato.wordpress.com/2007/03/19/medical-research...
Yes, reading HN it often seems like programming skill is the only game in town. It is obviously important, but it's nice to see something on the importance of domain knowledge as well.
So coding skill is one dimension along which you have to be good. Domain knowledge is another dimension.
A third dimension is knowing the code base you're working in (which takes time, since most production systems tend to be big). To be a really effective developer, you need to be good along all these dimensions.
So coding skill is one dimension along which you have to be good. Domain knowledge is another dimension.
A third dimension is knowing the code base you're working in (which takes time, since most production systems tend to be big). To be a really effective developer, you need to be good along all these dimensions.
> Yes, reading HN it often seems like programming skill is the only game in town
I always felt most of us on HN were well rounded individuals--interested in most areas of knowledge. Looking at the random articles that get upvoted along with ones on programming, it seems to coincide with that premise. Sure, we talk quite a bit about programming, but it's a just a tool for automating the repetitive work we do and to make ours and other's lifes easier. It's a means to an end and also something that happens to interest us at the same time.
I always felt most of us on HN were well rounded individuals--interested in most areas of knowledge. Looking at the random articles that get upvoted along with ones on programming, it seems to coincide with that premise. Sure, we talk quite a bit about programming, but it's a just a tool for automating the repetitive work we do and to make ours and other's lifes easier. It's a means to an end and also something that happens to interest us at the same time.
Programmers are not like scribes, they are like authors and writers - which are still here. Programming is a creative discipline, it's not just formatting one written format (specs) into another (code). Of course, the bad ones might make it seem like it...
"the bad ones might make it seem like it..."
Or the ones with highly uncreative managers who lack sufficient domain knowledge themselves.
Or the ones with highly uncreative managers who lack sufficient domain knowledge themselves.
If I were to get all novice programmers to learn something else, it would be some practical anthropology. How to observe. How to interview. How to see and record behaviors, triggers, social interactions, power dynamics. How to discover motivations, attitudes, and they way people understand the world. The one thing all of our software has in common is that it is supposed to make a difference in somebody's life.
If I could demand a second thing, it would be the basics of business. A large portion of software is directly about business, and almost all of it is somehow shaped by how business works. E.g., I'm volunteering at a non-profit this month, but the developer I sit next to is hip deep in a risk management problem.
If I could demand a second thing, it would be the basics of business. A large portion of software is directly about business, and almost all of it is somehow shaped by how business works. E.g., I'm volunteering at a non-profit this month, but the developer I sit next to is hip deep in a risk management problem.
"Ah yes, the tragedy of studying computing to the exclusion of everything else. Graduates with computer science degrees have an in-depth understanding of... text editors."
That makes a lot of sense. To me is that because we are using "languages". With a language you write. And your problems are writing problems: syntax, commas, naming, verbs, etc etc.
What if we just get rid of "languages" ? We actually do not "write a skyscraper". We make a project for a skyscraper. Or we "draw" a skyscraper.
Are languages the best tool of choice to solve the problem in connection with the information (and its calculation)?
(not that I have a definite answer, just a question).
That makes a lot of sense. To me is that because we are using "languages". With a language you write. And your problems are writing problems: syntax, commas, naming, verbs, etc etc.
What if we just get rid of "languages" ? We actually do not "write a skyscraper". We make a project for a skyscraper. Or we "draw" a skyscraper.
Are languages the best tool of choice to solve the problem in connection with the information (and its calculation)?
(not that I have a definite answer, just a question).
Languages exist to convey complex information, nothing else comes close to them in that ability.
As an exercise, try to convey the idea "I'm diabetic and I need some insulin" via !language. Charades? Interpretive dance? Drawing?
And yes, we do "write a skyscraper". That's what blueprints are for.
As an exercise, try to convey the idea "I'm diabetic and I need some insulin" via !language. Charades? Interpretive dance? Drawing?
And yes, we do "write a skyscraper". That's what blueprints are for.
You can draw a skyscraper because at it's core a skyscraper is an object that exists strictly in a 3 dimensional world.
Therefor you can convey a lot of important information about the skyscraper by showing it's shape and dimensions in an autocad drawing.
In reality however I expect that most skyscraper projects involve many many pages of writing and formulas in addition to the drawings.
OTOH a computer at it's core is a machine that sequentially follows instructions. So any description of a computer program must be able to reduce to this.
Since around the dawn of computer (or at least in the 10+ years I have been involved) there have been attempts to remove the requirement for computer programming, but in reality nobody has come close to anything as powerful as a programming language.
The most effective progress has been made via abstraction, where you allow one instruction to be used to imply many. For example making a REST API call with 1 line implies a lot of operations on a lower level (TCP/IP etc).
I think this is fundamentally because any system that you propose to replace a programming language will either be:
1) Turing complete, in which case it could be classified as a programming language itself and will require knowledge of algorithms/abstractions etc to operate effectively. This is where you end up with flowchart driven development tools where the flowchart ends up literally resembling a plate of spaghetti for any reasonably complex system.
2) Not Turing complete, in which case you lose a lot of power as there are things the system will not allow you to express and thus lose the ability to deal with many edge cases. Since you are developing software and not just using it, the reason for the development activity in the first place is probably to deal with some edge case than an off the shelf program does not.
When I was younger I used to play with various "code free" game development tools and ran into the same fundamental problem.
For example, I have a shooting game and when the player is hit with some projectile he takes some amount of damage based on the projectile he was hit with. This was usually very easy to do.
BUT then say I want to add an "armour" powerup to the game. If a player with the armour powerup is hit then he takes damage but at a reduced rate that is proportional to his armour value. The armour value is also reduced based on itself and the armour piercing value of the projectile which hits. However if the armour piercing value of the projectile is below some value proportional to the armour and health values combined then no damage is taken at all. Oh and the armour powerup cannot stack with certain other powerups except under some specific circumstances.
That type of logic was impossible to express with these tools, or required finding creative ways to abuse the tooling in ways not intended which would in turn make other functionality more difficult/impossible.
In reality however I expect that most skyscraper projects involve many many pages of writing and formulas in addition to the drawings.
OTOH a computer at it's core is a machine that sequentially follows instructions. So any description of a computer program must be able to reduce to this.
Since around the dawn of computer (or at least in the 10+ years I have been involved) there have been attempts to remove the requirement for computer programming, but in reality nobody has come close to anything as powerful as a programming language.
The most effective progress has been made via abstraction, where you allow one instruction to be used to imply many. For example making a REST API call with 1 line implies a lot of operations on a lower level (TCP/IP etc).
I think this is fundamentally because any system that you propose to replace a programming language will either be:
1) Turing complete, in which case it could be classified as a programming language itself and will require knowledge of algorithms/abstractions etc to operate effectively. This is where you end up with flowchart driven development tools where the flowchart ends up literally resembling a plate of spaghetti for any reasonably complex system.
2) Not Turing complete, in which case you lose a lot of power as there are things the system will not allow you to express and thus lose the ability to deal with many edge cases. Since you are developing software and not just using it, the reason for the development activity in the first place is probably to deal with some edge case than an off the shelf program does not.
When I was younger I used to play with various "code free" game development tools and ran into the same fundamental problem.
For example, I have a shooting game and when the player is hit with some projectile he takes some amount of damage based on the projectile he was hit with. This was usually very easy to do.
BUT then say I want to add an "armour" powerup to the game. If a player with the armour powerup is hit then he takes damage but at a reduced rate that is proportional to his armour value. The armour value is also reduced based on itself and the armour piercing value of the projectile which hits. However if the armour piercing value of the projectile is below some value proportional to the armour and health values combined then no damage is taken at all. Oh and the armour powerup cannot stack with certain other powerups except under some specific circumstances.
That type of logic was impossible to express with these tools, or required finding creative ways to abuse the tooling in ways not intended which would in turn make other functionality more difficult/impossible.
> The profession of programmers who only know how to program should disappear like the profession of scribes who only know how to write.
I think it would be better to say that the new programmers would replace the old as writers replaced scribes. Because we do have people who only write, it's just not quite the same.
I think it would be better to say that the new programmers would replace the old as writers replaced scribes. Because we do have people who only write, it's just not quite the same.
Writers are not merely new scribes. Writers are people who use the written word for some specific purpose. Very coarsely, I claim that novelists are storytellers, poets are artists (who use words as the medium, like sculptors use marble or bronze), biographers are historians, textbook authors are teachers, journalists are investigators. For all these subtypes of "writer," the process of getting words on paper (or into a Word document, or an Emacs buffer!) is simply a means to an end. They certainly don't "only write" — at least not the ones who produce anything remotely worth reading.
> Writers are not merely new scribes.
Indeed, that's why I made the comparison :) The new and old programmers he describes are not the same either.
Indeed, that's why I made the comparison :) The new and old programmers he describes are not the same either.
I guess it all depends on the student in question. Me, I majored in Computer Science, but I love all areas of science and study them on my own. Same with History, Art, Philosophy, Music (played violin & clarinet) and Literature. I was probably one of the few odd students in Computer Science at my school that actually loved the creative writing and art courses he took. I am not a great artist, but I can talk to someone who knows way more about art than me and relate to what interests them to a point. Such things lead to forming connections with people and exchanging great ideas across knowledge gaps between them. It also helps you to form better analogies and metaphors for explaining difficult, technical concepts.
I would have actually gone into teaching History if the United States school system was not as limited as it is these days in what one can teach students. I probably read and study history as much as I do about software development. However, I love Computer Science and have no regrets about going into it instead.
Any student that does not strive to be well-rounded in as many areas of knowledge as they can outside of their field is really missing out on life. Entrepreneurship in general is all about knowing what will make life better and it's hard to know what needs improving if one does not have experience in areas outside of their field.
A person does a great disservice to themselves when they choose to isolate their realm of knowledge to a narrow field, no matter which field it is.
I would have actually gone into teaching History if the United States school system was not as limited as it is these days in what one can teach students. I probably read and study history as much as I do about software development. However, I love Computer Science and have no regrets about going into it instead.
Any student that does not strive to be well-rounded in as many areas of knowledge as they can outside of their field is really missing out on life. Entrepreneurship in general is all about knowing what will make life better and it's hard to know what needs improving if one does not have experience in areas outside of their field.
A person does a great disservice to themselves when they choose to isolate their realm of knowledge to a narrow field, no matter which field it is.
One of the big detractors that made programming very difficult for me was the fact that I couldn't find any practical application to what I was learning. As a result I felt unmotivated and dragged my feet for a very long time. Double clicking on Eclipse and following book instructions on how to make an array in Java for a class wasn't particularly interesting or fun to me. I always chucked it up as not bothering because I wasn't a 'math person', and this was probably something that only people who liked math would do. (I know I know)
It was only years later after I began delving into Linux and found a need to make something a bit more powerful than a bash script that I began developing an interest in programming again.
I'm sure there are many other capable and willing folks out there right now who feel the same way that I did. They want to learn but they don't see the point of it. Most courses at least don't really teach any real world applicability to what is being studied.
It was only years later after I began delving into Linux and found a need to make something a bit more powerful than a bash script that I began developing an interest in programming again.
I'm sure there are many other capable and willing folks out there right now who feel the same way that I did. They want to learn but they don't see the point of it. Most courses at least don't really teach any real world applicability to what is being studied.
Exactly. Programming is a means to an end. One of the great joys of programming is making something that is useful, especially if it is useful to somebody else (points 1 and 2 here: http://henrikwarne.com/2012/06/02/why-i-love-coding/)
I had a similar experience, but from a slightly different perspective. I always found programming for the sake of programming to be sufficient motivation to spend time writing code. But when it comes to learning math, I have a really hard time caring about math in the abstract. Sitting me in a math class and having me memorize equations and formulas and operations, and solve a few word problems here and there (two boats leave point A, one going west, one going east, after one hour, blah, blah, blah) leaves me bored shitless for the most part. But if I have an application that matters to me, for the math I'm learning, I'm FAR more interested and willing to put in the time to learn the stuff.
Now that I'm trying to get deeper into Machine Learning stuff, I find myself motivated to go pick up books on Statistics and Linear Algebra and dig in, when I had very little interest in that stuff before.
Now that I'm trying to get deeper into Machine Learning stuff, I find myself motivated to go pick up books on Statistics and Linear Algebra and dig in, when I had very little interest in that stuff before.
I found cryptography, for example RSA, a great application of math and a neat example to see how math has real world applications.
[deleted]
Scribing was easy from the onset. But programming as it is today is not easy. Unless it evolved into plug-and-play-the-building-blocks, the profession of programming is going to stay.
And yet another cheap stab at many people reading HN. HN is filled with negativity --if not downright hatred-- towards nerds lately.
Items of zero value? A certain sci-fi author would beg to differ with you. Even someone that would have zero knowledge of anything else but would be mastering programming is still a technopriest. Even with zero knowledge about anything else but computers one still understand more about today's world than most people out there.
I'm not advocating refusing to learn other domains (I'm the kind of person who, when writing a book, decides to learn typography and typesetting software so I can typeset the book myself) but your cheap stab at young programmers (I'm 40) is uncalled for.
The world is going to shift to even more 0's and 1's. Many things are going to disappear and be replaced by 0's and 1's.
If there's one thing common to every single profession out there is that people need to read, write and communicate. And you know what? The tools allowing to read, write and communicate are written by the technopriests.
It's even more "scary" than that: the very infrastructure in place allowing people to communicate (like electricity and Internet) are, themselves, run using software. Not to mention very basic need like the entire infrastructure providing clean water to fullfill one of the most basic Maslow need.
There's no need for someone taking care of a network to understand what's inside the packets of information running through the network. All they need to know is that if there's no more Internet packets travelling worldwide, then there's no western civilization anymore.
The very foundation the entire society now relies on is software. If a worlwide EMP blast would render all computers useless tomorrow the world would probably turn to a mad max horror scenario in a few hours. No more water. No more electricity. No more food. Our logistics would be so screwed that "advanced" countries would fall into chaos. It's probable that in some places people would die after a few days because of the lack of drinkable water and it's very probable that in a few weeks many would die due to lack of food. Not to mention the nuclear facilities which would leak radioactivity left and right and break havoc.
Doctors aren't forced to understand how computers works to do their job. Most doctors are utterly clueless when it comes to technology: they're bound to live in fear of being hacked and not understanding what's going on when they connect to their online banking account and not understand why this or that mainstream article tell them to disable Java in their Mac. I've got doctor friends: they simply don't understand the world they're living in. We're already in the technopriest age. And they're asking me questions, lots of them.
The most computer-wise knowledgable doctor I know was thinking that the latest state of the art way to diagnose using computers was still to use an "expert system", filled by doctors. That's not how it works. These "experts" aren't able to process the amount of information a computer can process and can't find the correlation a computer software can find. Which is why now machines are trained with gigantic amount of information and suggesting exams and deceases to look for that are way more relevant than what doctors would suggest.
I think you got it totally backwards: it's a lot of the other professions that are going to disappear. Replaced by robots, machine learning algorithms, software and... The technopriests.
The ones still able to somehow understand the world they're living in and still able to write the world they're living in.
P.S: regarding mainstream economics I'd like to point out that Krugman, Nobel Prize winner, recently said that it wasn't a bad idea to mint a one trillion coin to deal with the U.S. debt SNAFU. Sure. Why not mint 26 of them and be done with the U.S. public debt!? That is the kind of economics most economics and most universities do teach worldwide: Keynesian economics. I happen to think Keynes was all wrong and keynesians have never been able to predict the consequences of their interventions more than a few months/years forward. There are economists from other schools who predicted 15 years before it happened, before the first euro was even circulating, that the euro would lead to: too many houses in spain, too many public servants in France, too many industries in Germany and who explained by which mechanism Greece and Spain would eventually default. Not a single Keynesian has ever been able to do a prediction that precise. That's all I need to know about "economics": that the mainstream is all wrong.
I do also think you put way too much faith in doctors: these people really aren't nearly half as expert as you think in their field. And they are the ones getting replaced today by sofware and robots. There's simply no way a surgeon can be as precise and have as many axis of rotation compared to today robotic arms (explained to me by a surgeon friend). There's also simply no way a doctor can read blood samples and other exams results and be able to correlate that with hundreds of thousands of other results and hence make more correct deduction as to what's going on.
If we eventually end up one day with true AI then everybody becomes meaningless because that day there ain't going to be doctors nor economists anymore.
Until that day programmers (even the one knowing nothing besides 0s and 1s and how 0s and 1s move around) are going to be more and more like the technopriests.
Items of zero value? A certain sci-fi author would beg to differ with you. Even someone that would have zero knowledge of anything else but would be mastering programming is still a technopriest. Even with zero knowledge about anything else but computers one still understand more about today's world than most people out there.
I'm not advocating refusing to learn other domains (I'm the kind of person who, when writing a book, decides to learn typography and typesetting software so I can typeset the book myself) but your cheap stab at young programmers (I'm 40) is uncalled for.
The world is going to shift to even more 0's and 1's. Many things are going to disappear and be replaced by 0's and 1's.
If there's one thing common to every single profession out there is that people need to read, write and communicate. And you know what? The tools allowing to read, write and communicate are written by the technopriests.
It's even more "scary" than that: the very infrastructure in place allowing people to communicate (like electricity and Internet) are, themselves, run using software. Not to mention very basic need like the entire infrastructure providing clean water to fullfill one of the most basic Maslow need.
There's no need for someone taking care of a network to understand what's inside the packets of information running through the network. All they need to know is that if there's no more Internet packets travelling worldwide, then there's no western civilization anymore.
The very foundation the entire society now relies on is software. If a worlwide EMP blast would render all computers useless tomorrow the world would probably turn to a mad max horror scenario in a few hours. No more water. No more electricity. No more food. Our logistics would be so screwed that "advanced" countries would fall into chaos. It's probable that in some places people would die after a few days because of the lack of drinkable water and it's very probable that in a few weeks many would die due to lack of food. Not to mention the nuclear facilities which would leak radioactivity left and right and break havoc.
Doctors aren't forced to understand how computers works to do their job. Most doctors are utterly clueless when it comes to technology: they're bound to live in fear of being hacked and not understanding what's going on when they connect to their online banking account and not understand why this or that mainstream article tell them to disable Java in their Mac. I've got doctor friends: they simply don't understand the world they're living in. We're already in the technopriest age. And they're asking me questions, lots of them.
The most computer-wise knowledgable doctor I know was thinking that the latest state of the art way to diagnose using computers was still to use an "expert system", filled by doctors. That's not how it works. These "experts" aren't able to process the amount of information a computer can process and can't find the correlation a computer software can find. Which is why now machines are trained with gigantic amount of information and suggesting exams and deceases to look for that are way more relevant than what doctors would suggest.
I think you got it totally backwards: it's a lot of the other professions that are going to disappear. Replaced by robots, machine learning algorithms, software and... The technopriests.
The ones still able to somehow understand the world they're living in and still able to write the world they're living in.
P.S: regarding mainstream economics I'd like to point out that Krugman, Nobel Prize winner, recently said that it wasn't a bad idea to mint a one trillion coin to deal with the U.S. debt SNAFU. Sure. Why not mint 26 of them and be done with the U.S. public debt!? That is the kind of economics most economics and most universities do teach worldwide: Keynesian economics. I happen to think Keynes was all wrong and keynesians have never been able to predict the consequences of their interventions more than a few months/years forward. There are economists from other schools who predicted 15 years before it happened, before the first euro was even circulating, that the euro would lead to: too many houses in spain, too many public servants in France, too many industries in Germany and who explained by which mechanism Greece and Spain would eventually default. Not a single Keynesian has ever been able to do a prediction that precise. That's all I need to know about "economics": that the mainstream is all wrong.
I do also think you put way too much faith in doctors: these people really aren't nearly half as expert as you think in their field. And they are the ones getting replaced today by sofware and robots. There's simply no way a surgeon can be as precise and have as many axis of rotation compared to today robotic arms (explained to me by a surgeon friend). There's also simply no way a doctor can read blood samples and other exams results and be able to correlate that with hundreds of thousands of other results and hence make more correct deduction as to what's going on.
If we eventually end up one day with true AI then everybody becomes meaningless because that day there ain't going to be doctors nor economists anymore.
Until that day programmers (even the one knowing nothing besides 0s and 1s and how 0s and 1s move around) are going to be more and more like the technopriests.
I think you two are coming two extreme ends of the spectrum. He's coming from the "clueless one-dimensional engineers" end of the scale, and you're coming from the "we are digital gods" end.
The truth lies somewhere down the middle, though I think you're a bit more extreme on your end of the scale than he is.
It's hard to take your views seriously when you describe your kind as "technopriests", and claim that people understanding nothing but computers "understand more about today's world than most people out there". This is a laughable claim.
I'm sorry, I can't get over this. Technopriest? This is the real world, not Snow Crash.
> "Many things are going to disappear and be replaced by 0's and 1's."
Yes, and they are being replaced by 0's and 1's in specific spaces where those doing the replacing must understand non-computer things about the space. Want to automate the entire traffic control system (LA just finished doing this)? Sorry, you have to know a great deal about transportation.
Want to automate food production? Sorry, you must know a great deal about both manufacturing and food. You can know all you want about computers, but unless you know these other topics you're useless. Well, you could be coding to a spec from someone who actually has a broad base of knowledge - a cog in the machine at that point.
The future belongs to people who have a deep understanding of computers and [insert relevant field]. Those who understand just computers have only a future pounding keyboards and not making decisions.
You know what, you're right - there will be "technopriests" in the future - but they will be the ones with a broad base of knowledge on top of technological mastery. They will not be the ones who understand tech and nothing else - those will be the digital equivalent of janitors.
Though they might fancy themselves technopriests while the world passes them by.
The truth lies somewhere down the middle, though I think you're a bit more extreme on your end of the scale than he is.
It's hard to take your views seriously when you describe your kind as "technopriests", and claim that people understanding nothing but computers "understand more about today's world than most people out there". This is a laughable claim.
I'm sorry, I can't get over this. Technopriest? This is the real world, not Snow Crash.
> "Many things are going to disappear and be replaced by 0's and 1's."
Yes, and they are being replaced by 0's and 1's in specific spaces where those doing the replacing must understand non-computer things about the space. Want to automate the entire traffic control system (LA just finished doing this)? Sorry, you have to know a great deal about transportation.
Want to automate food production? Sorry, you must know a great deal about both manufacturing and food. You can know all you want about computers, but unless you know these other topics you're useless. Well, you could be coding to a spec from someone who actually has a broad base of knowledge - a cog in the machine at that point.
The future belongs to people who have a deep understanding of computers and [insert relevant field]. Those who understand just computers have only a future pounding keyboards and not making decisions.
You know what, you're right - there will be "technopriests" in the future - but they will be the ones with a broad base of knowledge on top of technological mastery. They will not be the ones who understand tech and nothing else - those will be the digital equivalent of janitors.
Though they might fancy themselves technopriests while the world passes them by.
Is an incoherent rant like this an uncharacteristic indulgence for a technopriest? Or are you just a fan of technopriests, not being one yourself?
Everyone in our increasingly automated gift economy lives in fear of being cut off from the support of the layers of technology they rely upon for all their needs, all the way up your vaunted Maslow hierarchy.
You shouldn't romanticize this state of affairs, and you should not crow about the special role of the priestly class. In my opinion you should think soberly between now and your 50th birthday about how you can contribute to a global society that is both more stable and more technologically advanced, and try to forget about the futurism that was peddled to you ( and all of us) by Wired magazine in 1994.
Everyone in our increasingly automated gift economy lives in fear of being cut off from the support of the layers of technology they rely upon for all their needs, all the way up your vaunted Maslow hierarchy.
You shouldn't romanticize this state of affairs, and you should not crow about the special role of the priestly class. In my opinion you should think soberly between now and your 50th birthday about how you can contribute to a global society that is both more stable and more technologically advanced, and try to forget about the futurism that was peddled to you ( and all of us) by Wired magazine in 1994.
I didn't read it as a stab at nerds or young programmers but more like an advice that domain knowledge still is important beyond pure programming skills. Maybe the comment was formulated a littel bit over tthe top, but well, I'm doing that quite regularly.
I completely agree that the world is running on software right now, but the world isn't software ITSELF (well, not yet).
On the other hand, people being cracks in there field claiming that there field is the only one that counts are just arrogant. No mater if they a programmers, accountants, mech. engineers, doctors, car mechanics or what not. And yes, software written by people without the slightest idea of the purpose of said software tends to be not so good.
I completely agree that the world is running on software right now, but the world isn't software ITSELF (well, not yet).
On the other hand, people being cracks in there field claiming that there field is the only one that counts are just arrogant. No mater if they a programmers, accountants, mech. engineers, doctors, car mechanics or what not. And yes, software written by people without the slightest idea of the purpose of said software tends to be not so good.
> without the slightest idea of the purpose of said software
In the OPs specific example, the purpose of the network is to carry packets, and your job is to make sure packets are delivered. Whether they carry orders to launch nukes, or donations to charity isn't the purpose.
In the OPs specific example, the purpose of the network is to carry packets, and your job is to make sure packets are delivered. Whether they carry orders to launch nukes, or donations to charity isn't the purpose.
The point is that you need to be well acquainted with the problem that you're trying to solve. If you're developing a new network protocol then yes, you don't need to know what kind of information is going to flow through the network, you need to know about network protocols. On the other hand, if you're developing a business application then you better know how that business works.
I love the "takedown" of Keynesian/mainstream economics in the middle of this rant.
Any other centuries-old disciplines your knowledge of Big-O notation and compilers qualifies you to discredit? Oh, modern medicine? Good for you.
This guy takes the "hubris" as a virtue line way too seriously.
Any other centuries-old disciplines your knowledge of Big-O notation and compilers qualifies you to discredit? Oh, modern medicine? Good for you.
This guy takes the "hubris" as a virtue line way too seriously.
The trillion dollar coin wasn't supposed to deal with US debt, it was a trick to get around the debt cap rule that Congress imposed on the government.
If they actually spent the trillion dollar coin, that would likely cause devaluation of the dollar; as long as the coin remains locked up in the treasury it's just an accounting trick to get around their own arbitrary rule and wouldn't cause devaluation, because everyone buying dollars knows it's just a stupid accounting trick and not actual money.
If they actually spent the trillion dollar coin, that would likely cause devaluation of the dollar; as long as the coin remains locked up in the treasury it's just an accounting trick to get around their own arbitrary rule and wouldn't cause devaluation, because everyone buying dollars knows it's just a stupid accounting trick and not actual money.
In science and engineering, it's common to hear of "Computational X" where X is "Plasma Physics", "Geodynamics", "Ocean Science", "Electrical Engineering", "Aerodynamics", or any of a myriad of other disciplines. In these fields, the primary task is formulating Science/Engineering questions that can be effectively answered using computation, then writing the software to do so.
There is also the more generic field "Computational Science and Engineering" (CS&E) [1], which is primarily the development of methods and software that is applied in many specific disciplines. I work primarily on the library development side of CS&E, working with many applications to understand their needs, often finding ways to satisfy those needs with better software reuse and better ultimate flexibility, then implementing any components that belong in libraries. Additionally, we do basic research by recognizing unexploited common themes and designing new algorithms that will benefit existing and potential users.
There is currently a great deal of demand for Computational X, for CS&E through the postdoc level, and for CS&E in industry and national labs. It is recognized that a versatile practitioner of CS&E can effectively assist Computational X for many values of X, since there really is a lot in common. Once you've put in 10000 hours of CS&E, plus 1000 hours in a handful of disciplines, you should be able to get orientedy quickly in new applications.
At the faculty level, support for CS&E is highly variable. At some universities, X departments have the view that all their faculty should "X first, computation second", while the mathematics department wants to see theorems and isn't overly concerned with following through to see the method applied to obtain a new science or engineering result. When hiring committees say "we need CS&E, but some other department should have the CS&E faculty", it drives a lot of talent away from teaching roles (e.g., into industry), perpetuating the relative lack of early education in CS&E fundamentals. On the other hand, the trend is for universities to have more direct interest in CS&E (sometimes recognizing it with a dedicated department or inistitute). It helps that computation stimulates many interdisciplinary collaborations, leading to more diverse funding possibilities.
[1] We have a big meeting next week: http://www.siam.org/meetings/cse13/
There is also the more generic field "Computational Science and Engineering" (CS&E) [1], which is primarily the development of methods and software that is applied in many specific disciplines. I work primarily on the library development side of CS&E, working with many applications to understand their needs, often finding ways to satisfy those needs with better software reuse and better ultimate flexibility, then implementing any components that belong in libraries. Additionally, we do basic research by recognizing unexploited common themes and designing new algorithms that will benefit existing and potential users.
There is currently a great deal of demand for Computational X, for CS&E through the postdoc level, and for CS&E in industry and national labs. It is recognized that a versatile practitioner of CS&E can effectively assist Computational X for many values of X, since there really is a lot in common. Once you've put in 10000 hours of CS&E, plus 1000 hours in a handful of disciplines, you should be able to get orientedy quickly in new applications.
At the faculty level, support for CS&E is highly variable. At some universities, X departments have the view that all their faculty should "X first, computation second", while the mathematics department wants to see theorems and isn't overly concerned with following through to see the method applied to obtain a new science or engineering result. When hiring committees say "we need CS&E, but some other department should have the CS&E faculty", it drives a lot of talent away from teaching roles (e.g., into industry), perpetuating the relative lack of early education in CS&E fundamentals. On the other hand, the trend is for universities to have more direct interest in CS&E (sometimes recognizing it with a dedicated department or inistitute). It helps that computation stimulates many interdisciplinary collaborations, leading to more diverse funding possibilities.
[1] We have a big meeting next week: http://www.siam.org/meetings/cse13/
What sort of backgrounds do people working in this field have? More specifically, I have a bachelor's degree in materials science and currently work doing run-of-the-mill web development (were the typical background of my peers is a bachelor's in CS). What steps might I take to move into computational engineering?
There are a couple journals dedicated to computational materials science. Two active fields are phase field models and molecular dynamics.
Phase field models are mesoscale continuum (PDE) models based on the Cahn-Hilliard and Allen-Cahn equations, and are useful for things like predicting crystal grain growth and reorganization under strain, as well as fracture and many other uses. In phase-field modeling, the interesting physics is a consequence of the free energy functional, which is phenomenological and different for each material, so software should make it as easy as possible to turn specification of the free energy into an accurate and efficient massively parallel simulation. One such approach is outlined in this paper [1] from some colleagues. The resulting discrete systems are stiff and have positivity properties that must be preserved, so scalable implicit solvers (like multigrid) as well as methods for variational inequalities are necessary.
In molecular dynamics, there are many parallel scalability and expressivity challenges. Fast algorithms for long-range interactions (such as the fast multipole method) lead to less regular computational, but huge asymptotic (O(n) versus O(n^2)). Since better algorithms become more important as our hardware allows us to keep scaling up the problem sizes, Fast methods need to be applied in more places.
There is also a lot of activity in multiscale modeling and numerical homogenization, where explicit microscale models (often kinetic) are used to provide missing components of meso/macro-scale continuum models.
[1] http://dx.doi.org/10.1016/j.commatsci.2011.07.028
Phase field models are mesoscale continuum (PDE) models based on the Cahn-Hilliard and Allen-Cahn equations, and are useful for things like predicting crystal grain growth and reorganization under strain, as well as fracture and many other uses. In phase-field modeling, the interesting physics is a consequence of the free energy functional, which is phenomenological and different for each material, so software should make it as easy as possible to turn specification of the free energy into an accurate and efficient massively parallel simulation. One such approach is outlined in this paper [1] from some colleagues. The resulting discrete systems are stiff and have positivity properties that must be preserved, so scalable implicit solvers (like multigrid) as well as methods for variational inequalities are necessary.
In molecular dynamics, there are many parallel scalability and expressivity challenges. Fast algorithms for long-range interactions (such as the fast multipole method) lead to less regular computational, but huge asymptotic (O(n) versus O(n^2)). Since better algorithms become more important as our hardware allows us to keep scaling up the problem sizes, Fast methods need to be applied in more places.
There is also a lot of activity in multiscale modeling and numerical homogenization, where explicit microscale models (often kinetic) are used to provide missing components of meso/macro-scale continuum models.
[1] http://dx.doi.org/10.1016/j.commatsci.2011.07.028
The lack of this kind of domain knowledge - either from being unwilling to go find it, or not being given the time and the resources to do so - is a big problem for tools development in games. People doing the scheduling and planning tend to assume that if you put enough smart programmers in a room and give them detailed specifications, they'll be able to solve all the problems faced by some artists and writers in another room, even though they don't know anything about art or writing. Sometimes this results in the kind of obvious mistakes that you might otherwise assume only happen when you're writing software for the government.
In a sense it's kind of sad: In building traditional software, understanding the customer is one of the hardest steps because you can't simply walk over and observe them going about their daily work, or ask them questions on a regular basis to understand their issues - you're separated from them by a sales process and probably some automated support tickets and a PR guy who won't let you have open discussion in public. People building tools for game development can have their customers sitting a few desks away!
In a sense it's kind of sad: In building traditional software, understanding the customer is one of the hardest steps because you can't simply walk over and observe them going about their daily work, or ask them questions on a regular basis to understand their issues - you're separated from them by a sales process and probably some automated support tickets and a PR guy who won't let you have open discussion in public. People building tools for game development can have their customers sitting a few desks away!
> In building traditional software, understanding the customer is one of the hardest steps because you can't simply walk over and observe them going about their daily work
One of the nicest things about the Agile fashion at the moment is that my customer (or a reasonable representative of him) is a couple of desks away and I really can go and ask him about what he wants. It's incredible how much time is saved by avoiding doing things he doesn't care about and how much happier he is with the end result.
One of the nicest things about the Agile fashion at the moment is that my customer (or a reasonable representative of him) is a couple of desks away and I really can go and ask him about what he wants. It's incredible how much time is saved by avoiding doing things he doesn't care about and how much happier he is with the end result.
I always asked my tools programmers (and do myself) to spend some time regularly sitting by an artist or designers while she works, and observe. HUGE difference.
> Sometimes this results in the kind of obvious mistakes that you might otherwise assume only happen when you're writing software for the government.
My personal guess is that the reason there too is lack of domain knowledge. That many government projects are ran by handing out specs to the people tasked with implementing them.
My personal guess is that the reason there too is lack of domain knowledge. That many government projects are ran by handing out specs to the people tasked with implementing them.
This particularly resonates with me. I have been working in a bank as a graduate developer for the last year and a half, getting by without much care about how instruments are priced, traded, recorded, audited, etc. To me, I was moving a button around and colouring things using WPF. To me, the "Yield" column was just another column with numbers that were supposed to be 2 decimal places and turns red if negative. All the while I was dreaming I wished I worked at Google on 'technology'.
Which, in hindsight, was a little misguided. There is so much value in learning and understanding the domain in which we are working in because it makes us so much more effective. The realisation came when I became frustrated in meetings where we would discuss ideas on financial models and I could not add any value to the discussion.
I then imagined working at a company like Google, let's say, doing stuff I wouldn't normally be interested in. Would I bother learning the domain knowledge? I couldn't say yes with 100% conviction.
I love programming but I also want to be effective in what I do. To me now, understanding the problems we are solving is probably just as important as writing maintainable code.
Which, in hindsight, was a little misguided. There is so much value in learning and understanding the domain in which we are working in because it makes us so much more effective. The realisation came when I became frustrated in meetings where we would discuss ideas on financial models and I could not add any value to the discussion.
I then imagined working at a company like Google, let's say, doing stuff I wouldn't normally be interested in. Would I bother learning the domain knowledge? I couldn't say yes with 100% conviction.
I love programming but I also want to be effective in what I do. To me now, understanding the problems we are solving is probably just as important as writing maintainable code.
My take is that one of the main improvements in programming languages will be to reduce the gap between domain experts and programming experts.
Now we should face three cases: 1) the programmer needs to learn the domain; 2) the domain expert needs to know how to develop; 3) something in between.
Most of developing is done in 1). There are some brilliant cases of 2). I once talked with a biologist doing some multi sequence alignment that told me that some routine in C was quite slow and was far better rewrite that routine from scratch in assembly. I was quite speechless.
In order to create better programming languages we could: 1) express domains in a better way. Actually, just be able to express domains would be really cool (and not so easy). 2) provide higher levels of abstraction.
Now we should face three cases: 1) the programmer needs to learn the domain; 2) the domain expert needs to know how to develop; 3) something in between.
Most of developing is done in 1). There are some brilliant cases of 2). I once talked with a biologist doing some multi sequence alignment that told me that some routine in C was quite slow and was far better rewrite that routine from scratch in assembly. I was quite speechless.
In order to create better programming languages we could: 1) express domains in a better way. Actually, just be able to express domains would be really cool (and not so easy). 2) provide higher levels of abstraction.
Every programming language from the dawn of time up to the present has made that claim at some point. This time it would be different.
I think it is just a people problem. Programmers tend to seclude themselves from their customers. Whole IT departments tend to seclude themselves from their users.
When you are looking for underwater welders for the off-shore industry, what is better, teaching a welder to dive or teaching a diver to weld?
(that's a serious question with a real answer).
It is much easier for a programmer to acquire the required domain knowledge than it is (usually) for a domain specialist to learn how to program. But you can only do so if you're prepared to 'dive in'.
I think it is just a people problem. Programmers tend to seclude themselves from their customers. Whole IT departments tend to seclude themselves from their users.
When you are looking for underwater welders for the off-shore industry, what is better, teaching a welder to dive or teaching a diver to weld?
(that's a serious question with a real answer).
It is much easier for a programmer to acquire the required domain knowledge than it is (usually) for a domain specialist to learn how to program. But you can only do so if you're prepared to 'dive in'.
My guess would be the first, teaching a welder to dive.
I base that on the fact that the core value, the welds, will be higher-quality and that's more important than the diving execution.
The diving might be off, but it shouldn't be that hard to learn well enough to survive and be effective. Of course, I'm neither a diver nor a welder.
Did I win?
I base that on the fact that the core value, the welds, will be higher-quality and that's more important than the diving execution.
The diving might be off, but it shouldn't be that hard to learn well enough to survive and be effective. Of course, I'm neither a diver nor a welder.
Did I win?
> Did I win?
Yep. But a drowned welder is also not much ;)
Yep. But a drowned welder is also not much ;)
Funnily enough, I know someone who was an underwater welder. From the conversation with him, it was a multi-month certification course, so I went ahead and looked it up online.
https://www.commercialdivingacademy.com/underwater-welding.d...
That's just one program, but it appears diving is the first thing you learn.
Edit: If you look at it from the context of a normal welder, they have an understanding and control of their environment before they start welding. You don't realize it because you've been understanding your environment since you were born. To do underwater welding, you'd first have to understand how you operate in that work environment, and then they can teach you how to start welding in it.
https://www.commercialdivingacademy.com/underwater-welding.d...
That's just one program, but it appears diving is the first thing you learn.
Edit: If you look at it from the context of a normal welder, they have an understanding and control of their environment before they start welding. You don't realize it because you've been understanding your environment since you were born. To do underwater welding, you'd first have to understand how you operate in that work environment, and then they can teach you how to start welding in it.
> I think it is just a people problem. Programmers tend to seclude themselves from their customers. Whole IT departments tend to seclude themselves from their users.
My experience is that it generally has been managers shielding the programmers rather than the programmers shielding themselves.
My experience is that it generally has been managers shielding the programmers rather than the programmers shielding themselves.
Maybe that is a little bit of the mark right now, but wouldn't that domain knowledge be a great opportunity for a non-tech co-founder to add value to a start-up? Just guessing, but there are a lot of domains out there that are quite diffcult to learn but that are also (or maybe because they are difficult to learn) ripe to be changed by software.
I could imagine the right match of domain knowledge and the capability to learn how to develop in a non-tech guy and programming skills and the capability to learn the domain in a tech guy would be rather powerfull combination. And pretty rare one, I guess. At least for me, the actual programming part is... difficult. ;-)
On another point, this applies pretty much things like lean or six sigma, too. Both are quite usefull (or can be, that depends!), but people tend to see them as domain by itself. In practice, and that's only my personal opinion, this comes out short. If you add this to existing expertiece in a certain domain, then it works!
I could imagine the right match of domain knowledge and the capability to learn how to develop in a non-tech guy and programming skills and the capability to learn the domain in a tech guy would be rather powerfull combination. And pretty rare one, I guess. At least for me, the actual programming part is... difficult. ;-)
On another point, this applies pretty much things like lean or six sigma, too. Both are quite usefull (or can be, that depends!), but people tend to see them as domain by itself. In practice, and that's only my personal opinion, this comes out short. If you add this to existing expertiece in a certain domain, then it works!
In order to create better programming languages we could:
3. Create another EUP paradigm. It's been 34 years since VisiCalc came out, and despite it's success, I don't see a lot of effort to get end-users to program, especially in models, like the spreadsheet, where they don't know that's what they're doing. ;)
If you're interested in that sort of thing, ping me (see profile). My company does R&D in this space.
3. Create another EUP paradigm. It's been 34 years since VisiCalc came out, and despite it's success, I don't see a lot of effort to get end-users to program, especially in models, like the spreadsheet, where they don't know that's what they're doing. ;)
If you're interested in that sort of thing, ping me (see profile). My company does R&D in this space.
The line I found most insightful:
>>"Software is much more intertwined with the institution commissioning it and the people using it ..."
Given that, I'm surprised at the emphasis on 'reading' as the primary source of gaining domain knowledge. There are just 2-3 mentions of speaking/collaborating with the domain experts that are driving you to build the software itself. If you are lucky enough to work with acknowledged domain experts, spend time talking to them. Pick their brain. Two techniques I use: a) forward interesting articles to them with a question or an analogous concept in my domain b) resist the urge to show them a 'better' way. E.g.: In the HealthCare domain, I learnt people don't really want quicker or lesser clicks. They would prefer a set, stable workflow. If you cater to senior-citizens, keyboard shortcuts are less important than instructions in large font.
I still suck at engaging domain experts. So, I'd love to learn more techniques to hold an interesting conversation with domain experts. Ideas?
Given that, I'm surprised at the emphasis on 'reading' as the primary source of gaining domain knowledge. There are just 2-3 mentions of speaking/collaborating with the domain experts that are driving you to build the software itself. If you are lucky enough to work with acknowledged domain experts, spend time talking to them. Pick their brain. Two techniques I use: a) forward interesting articles to them with a question or an analogous concept in my domain b) resist the urge to show them a 'better' way. E.g.: In the HealthCare domain, I learnt people don't really want quicker or lesser clicks. They would prefer a set, stable workflow. If you cater to senior-citizens, keyboard shortcuts are less important than instructions in large font.
I still suck at engaging domain experts. So, I'd love to learn more techniques to hold an interesting conversation with domain experts. Ideas?
I'm surprised about the emphasis too. In terms of ideas, I used to work in a large corporate on the technology side - now work for a software company, so YMMV with these:
One of the first projects I worked on involved re-writing an entire admin system, both the technology and the underlying processes. One technique that worked well for the team, was to work-shadow; we would each spend time watching how a couple of people worked; compared notes and would go back and ask questions if we didn't understand it. Work shadowing doesn't have to be project related either; I visited nearly every department in our division of the over the years (1 every couple of months). I even managed to swing going out with the sales guys and visit clients; the clients were happy to spend an hour talking about their business and problems. Anyway, my experience with large corporates if that if you ask - people are generally happy to help you out - but I think many people just censure themselves from asking in the first place.
One other technique that I think works quite well is to get the strong end users seconded to beef up the QA team. Again, this doesn't work in all environments. But in the context of developing an internal system, it worked really well. For an external facing system, we did get a sales guy seconded onto the team too - which again really helped build up the knowledge of the whole team.
A variant on reading is podcasts and forums - I've used both of those to get to grips with whats going on in an industry.
For the software company, most of the above doesn't work, we're adapating Steve Blank's mantra of "get out of the building" and just trying to talk to as many people (especially customers) as we can. If they agree to talk to you, they're generally going to be helpful.
One of the first projects I worked on involved re-writing an entire admin system, both the technology and the underlying processes. One technique that worked well for the team, was to work-shadow; we would each spend time watching how a couple of people worked; compared notes and would go back and ask questions if we didn't understand it. Work shadowing doesn't have to be project related either; I visited nearly every department in our division of the over the years (1 every couple of months). I even managed to swing going out with the sales guys and visit clients; the clients were happy to spend an hour talking about their business and problems. Anyway, my experience with large corporates if that if you ask - people are generally happy to help you out - but I think many people just censure themselves from asking in the first place.
One other technique that I think works quite well is to get the strong end users seconded to beef up the QA team. Again, this doesn't work in all environments. But in the context of developing an internal system, it worked really well. For an external facing system, we did get a sales guy seconded onto the team too - which again really helped build up the knowledge of the whole team.
A variant on reading is podcasts and forums - I've used both of those to get to grips with whats going on in an industry.
For the software company, most of the above doesn't work, we're adapating Steve Blank's mantra of "get out of the building" and just trying to talk to as many people (especially customers) as we can. If they agree to talk to you, they're generally going to be helpful.
I can read a book faster than I can speak or hear words.
In addition, books can usually be found, on almost any topic, which are intended to bring a novice up to a basic level of conversancy. Reading one (or two) such books will make discussions with the experts much more productive.
Plus: it's fun to learn about new things.
In addition, books can usually be found, on almost any topic, which are intended to bring a novice up to a basic level of conversancy. Reading one (or two) such books will make discussions with the experts much more productive.
Plus: it's fun to learn about new things.
Oh, no argument on the fact that reading is extremely useful. I read too. In expressing the balance between reading and collaborating, I felt that your article emphasized too much on reading - but I fully recognize that you do collaborate.
>>"I can read a book faster than I can speak or hear words."
I disagree that this is an argument in favor of reading. Reading is easier but I think its not always as high in quality as collaborating.
>> "Reading one (or two) such books will make discussions with the experts much more productive."
Totally agree. I try my best to read as much with the end goal being picking the brains of my colleagues.
>>"I can read a book faster than I can speak or hear words."
I disagree that this is an argument in favor of reading. Reading is easier but I think its not always as high in quality as collaborating.
>> "Reading one (or two) such books will make discussions with the experts much more productive."
Totally agree. I try my best to read as much with the end goal being picking the brains of my colleagues.
> I felt that your article emphasized too much on reading
You've mistaken me for Jacques Mattheij.
You've mistaken me for Jacques Mattheij.
Ooops. Mea culpa. Sorry.
No worries. It's kinda flattering.
Plus "Jacques" is an uncommon name in the anglosphere, apart from South Africa.
Plus "Jacques" is an uncommon name in the anglosphere, apart from South Africa.
I've held programming jobs in: Water Treatment Process Control, Workflow Management, Clinical Trials Management, Electrical Power Monitoring and Simulation, and Medical Imaging. In each case my natural curiosity led me to learn as much as I could about the field. I always enjoyed talking to the domain experts at each company and asking them as many questions as they could handle. There are usually one or two people who relish answering these questions and I especially like the ones who talk fast, mock astonishment at my ignorance, and proceed to try to fix it.
In water treatment, I learned about oxic/anoxic bacteria and how they're grown and recycled in the process to treat waste, how to write chemical metering logic in a PLC because I was the only one left on-site to do it, how motors are controlled with variable frequency drives, how proportional-integral-derivatives are used to prevent oscillation in feedback based controls.
In clinical trials management, I learned how double-blind studies are made, the process of randomization and dispensing drugs, medical coding and the life cycle of the trial.
In power systems simulation, I learned about three-phase power transmission, resistive and capacitive loads, active and reactive power, the challenges of hooking up a lot of solar panels our existing grids, how data centers are build and cooled and powered, microgrids, power meters, intelligent breakers and more.
I feel privileged that my skills at software development have exposed me to a variety of fields and that I know how several important processes in our word work. And I got to take home a nice salary while gaining that exposure. I do lament that, unless I pick a field and stick with it, I'll never get to work on the difficult programming problems that require a decade or more of experience and knowledge.
In water treatment, I learned about oxic/anoxic bacteria and how they're grown and recycled in the process to treat waste, how to write chemical metering logic in a PLC because I was the only one left on-site to do it, how motors are controlled with variable frequency drives, how proportional-integral-derivatives are used to prevent oscillation in feedback based controls.
In clinical trials management, I learned how double-blind studies are made, the process of randomization and dispensing drugs, medical coding and the life cycle of the trial.
In power systems simulation, I learned about three-phase power transmission, resistive and capacitive loads, active and reactive power, the challenges of hooking up a lot of solar panels our existing grids, how data centers are build and cooled and powered, microgrids, power meters, intelligent breakers and more.
I feel privileged that my skills at software development have exposed me to a variety of fields and that I know how several important processes in our word work. And I got to take home a nice salary while gaining that exposure. I do lament that, unless I pick a field and stick with it, I'll never get to work on the difficult programming problems that require a decade or more of experience and knowledge.
HN is understandably filled with programmers who value programming above all else and believe domain knowledge can be picked up quickly. It would be true for someone hired as a programmer who implement a published algorithm or port existing software to newer systems or different platform; in other words, improving existing systems. I believe it requires equal depth if not not more for domain specific knowledge to innovate in non-computational fields. For my field of bioinformatics/computational biology, many open questions cannot be answered because of computational hurdles. Reading textbooks won't lead anyone to them, nor would reading survey papers. It requires expert knowledge in both biology and computer science to even know what is feasible and interesting. We want to find the driver mutations for cancer, discover mechanism of gene regulation, uncover subtle interaction of heredity and environment. Computation is intimately involved because observation is indirect, noisy, and massive. Only with knowledge in both fields can answer questions that domain experts care about but cannot do because of computational/quantitative obstacles.
I find researchers (I am in academia) in both fields cross-talk too much. Computational/quantitative experts try to shoe-horn biological questions into their small corner of research area, while biologists don't care about computational efficiency, statistical robustness/efficiency, and the mathematical implication of their simple methods; no one cares about software quality. This is unfortunate. It is also unfortunate hiring committees do not look favorably towards multi-disciplinary researchers because one is not a trained geneticist/cancer researcher. I definitively do not encourage computer scientists go into support roles in academia; they are poorly paid and get little respect.
I find researchers (I am in academia) in both fields cross-talk too much. Computational/quantitative experts try to shoe-horn biological questions into their small corner of research area, while biologists don't care about computational efficiency, statistical robustness/efficiency, and the mathematical implication of their simple methods; no one cares about software quality. This is unfortunate. It is also unfortunate hiring committees do not look favorably towards multi-disciplinary researchers because one is not a trained geneticist/cancer researcher. I definitively do not encourage computer scientists go into support roles in academia; they are poorly paid and get little respect.
I am currently working on a web software project in the medical/health space. Of course its really good to know how the people that use the software work and what they expect and i am eager to learn those things, but i will not dive into a sector i am not really interested and absorb any and all information i can find, since i probably wont need it for the next project and it personally does not interest me a lot.
For technical domain knowledge i agree, but it also happens that this is often of personal interest to me anyway.
For technical domain knowledge i agree, but it also happens that this is often of personal interest to me anyway.
I agree about the importance of gathering as much domain knowledge as possible.
One thing that I have always had issues with when gathering requirements around something for an unfamiliar industry is that people who are in that industry will often neglect to mention things that are both important and non-obvious/counter-intuitive to someone unfamiliar with the industry.
This is probably because they are so familiar with the quirks of the way that the particular industry works that such things seem so obvious as to not need mentioning (and they probably don't see how it could possibly be any other way). Basically similar to how we are often amazed by assumptions some people have about the way that computers/software works.
The paradox of this though is that to deliver truly good software for a particular industry one must really know that industry as well as somebody who has years of experience in that area. At which point you start to feel that you could do their job too.
Another alternative is to have a very tight feedback loop with someone who has tons of domain knowledge and is the target user. Basically sit next to them as you develop the software and have them review every single change you make.
One thing that I have always had issues with when gathering requirements around something for an unfamiliar industry is that people who are in that industry will often neglect to mention things that are both important and non-obvious/counter-intuitive to someone unfamiliar with the industry.
This is probably because they are so familiar with the quirks of the way that the particular industry works that such things seem so obvious as to not need mentioning (and they probably don't see how it could possibly be any other way). Basically similar to how we are often amazed by assumptions some people have about the way that computers/software works.
The paradox of this though is that to deliver truly good software for a particular industry one must really know that industry as well as somebody who has years of experience in that area. At which point you start to feel that you could do their job too.
Another alternative is to have a very tight feedback loop with someone who has tons of domain knowledge and is the target user. Basically sit next to them as you develop the software and have them review every single change you make.
I think in order to create good software that fits the industry's current best practices then you do need a really in-depth domain-specific knowledge. If that's the goal then learn everything possible about that industry.
To create good solutions that are also innovative, I think that programmers only really need enough knowledge in that domain to be dangerous - to know more or less what happens but with enough gaps to not really know what's doable and what isn't. But, this knowledge is only useful if you can combine it with bits and pieces from several completely separate domains.
If innovation is the goal then I think there is a point where domain knowledge actually hurts you.
Of course there still needs to be at least one specialist to keep everything on track though - and I completely agree with you about having someone with deep knowledge on board. I think it should be someone who has been in that industry and will stay in that industry.
I can't see the point in programmers trying to become experts in areas that they're not interested in working in for the long haul. It's easier to learn (and actually apply) a little about a lot than a lot about a little.
To create good solutions that are also innovative, I think that programmers only really need enough knowledge in that domain to be dangerous - to know more or less what happens but with enough gaps to not really know what's doable and what isn't. But, this knowledge is only useful if you can combine it with bits and pieces from several completely separate domains.
If innovation is the goal then I think there is a point where domain knowledge actually hurts you.
Of course there still needs to be at least one specialist to keep everything on track though - and I completely agree with you about having someone with deep knowledge on board. I think it should be someone who has been in that industry and will stay in that industry.
I can't see the point in programmers trying to become experts in areas that they're not interested in working in for the long haul. It's easier to learn (and actually apply) a little about a lot than a lot about a little.
Actually I think domain knowledge is overrated. A good hacker can pick up the domain knowledge needed to contribute to a project very quickly.
Now, if you're designing a product to solve a problem in a particular domain you need the knowledge ahead of time, sure. But if you're a shit-hot coder and a fast learner, brought in to make things work, who cares?
Now, if you're designing a product to solve a problem in a particular domain you need the knowledge ahead of time, sure. But if you're a shit-hot coder and a fast learner, brought in to make things work, who cares?
Because, as the article clearly mentions, there's a difference between making something work and making something work right.
I disagree that it's important unless you're the person designing the solution. If you're not (and in many/most teams you won't be) it's far less important.
I guess the key is how often are we just handed a spec and asked to code it up. In my roles, I've never worked like that, I've nearly always had some impact on design - but hey, that's just my experiences.
I would agree with your original point though, that domain knowledge can be learned quickly by a good hacker. Which leads to a more interesting question; what techniques can we use to learn a new domain?
I would agree with your original point though, that domain knowledge can be learned quickly by a good hacker. Which leads to a more interesting question; what techniques can we use to learn a new domain?
I agree that domain knowledge can be learned by decent hacker quickly up the degree necessary to write good code for that domain.
Sure, if the problem is solved, then you probably don't need to know anything. But implementing other people's solutions is only satisfying for so long.
I don't disagree with that, but there are plenty of people that make a career out of it. In the case of ex-colleagues who find a corner at a big multinational and get told what to do, someone else translates domain knowledge for them, and they come up with tech solutions to the presented requirements.
And then there are people like me who get called in because we're good at picking up large, existing, broken codebases quickly and somebody's in the sort of trouble that's best summed up as "Oh $&%£ we've got three months to deliver, the product's less than half-done and it's full of weird intermittent bugs". I'm finding it satisfying enough for now :)
And then there are people like me who get called in because we're good at picking up large, existing, broken codebases quickly and somebody's in the sort of trouble that's best summed up as "Oh $&%£ we've got three months to deliver, the product's less than half-done and it's full of weird intermittent bugs". I'm finding it satisfying enough for now :)
I should add that I'm not claiming 'shit-hot coder' status as mentioned in my original post there. I like to think I'm pretty good, but blowing ones own trumpet is more than a little gauche :)
This is a very true post, but I don't think the solution is at the university, at least for undergrads. Schools barely have enough time in 4 years to create entry level programmers. Lopping off half the coursework in exchange for domain knowledge leaves grads unprepared.
My observation is if you can learn to program well, you can learn any field of knowledge. A programmer can learn consumer products easier than a brand manager can learn Python. Similarly a CS major can learn bond math easier than a Finance major can learn database design. (Yes, these are generalizations)
What's this mean? That bridging the Tech/Client divide is on the shoulder of the technologist. In addition to learning the latest frameworks, we have to learn our customers business. In addition to tech credentials, we should add theirs too.
My observation is if you can learn to program well, you can learn any field of knowledge. A programmer can learn consumer products easier than a brand manager can learn Python. Similarly a CS major can learn bond math easier than a Finance major can learn database design. (Yes, these are generalizations)
What's this mean? That bridging the Tech/Client divide is on the shoulder of the technologist. In addition to learning the latest frameworks, we have to learn our customers business. In addition to tech credentials, we should add theirs too.
The degree gives you a tour of the foundations of the solution domain.
Mastery of the solution domain and knowledge of various problem domains cannot be taught in a 3 or 4 year degree.
Mastery of the solution domain and knowledge of various problem domains cannot be taught in a 3 or 4 year degree.
Agree completely.
Programmers should learn more about other domains and domain experts should learn how to program. Programming is becoming nearly as basic a skill as writing.
Does anyone here find the work of Lave and Wenger ('situated learning') of any use in their attempts to understand a problem area for a customer?
http://www.infed.org/biblio/communities_of_practice.htm
Basically, some organisations make their processes easily visible to most employees, and some don't. The ones that do can train people more quickly and improve their processes more quickly. Organisations that make 'peripheral participation' easy might be better customers for developers than those that don't. You might not have to make many sails or use a height gauge too often to 'see' what the issues are, just interview knowledgeable people carefully and cross reference the results with observation.
As a concrete example, the original author invested a considerable amount of (quite expensive) time in learning about sail making and lathe operating. In some of the 'business applications' I have to use daily, it is pretty obvious that the customer was a manager higher up and that the interface we have to use to populate the database that generates the wonderful reports has not really had much time spent on it!
http://www.infed.org/biblio/communities_of_practice.htm
Basically, some organisations make their processes easily visible to most employees, and some don't. The ones that do can train people more quickly and improve their processes more quickly. Organisations that make 'peripheral participation' easy might be better customers for developers than those that don't. You might not have to make many sails or use a height gauge too often to 'see' what the issues are, just interview knowledgeable people carefully and cross reference the results with observation.
As a concrete example, the original author invested a considerable amount of (quite expensive) time in learning about sail making and lathe operating. In some of the 'business applications' I have to use daily, it is pretty obvious that the customer was a manager higher up and that the interface we have to use to populate the database that generates the wonderful reports has not really had much time spent on it!
So I don't work in a scientific field, and I'm not a "programmer" or "software engineer". I am a business analyst. I am a business analyst who CAN code. My code isn't always pretty, and I recycle a crap ton of boilerplate and hack the hell out of django apps. And I read the heck out of HN and have spent 5 years slowly refining my design chops. But most importantly, I know top-to-bottom the ins and outs of my business. So when our call center shopped around for a software solution to provide a call center full of agents realtime access to their stats, we were given a 3 month time frame and quoted $100k+ for a solution. So I wrote and deployed a web based results dashboard of realtime stats over two weekends. And we were just quoted a price on a quality management tool for $225k. Guess what's next for me? I've been an agent on the phones. I've sold in a retail environment. I've been the corporate trainer dealing with the LMS. Domain knowledge plus tech chops has proven a powerful combination for me.
I believe this is actually the major problem faced by outsourcing projects. When I was in the cable industry, I saw several projects fail when the "Systems Engineers" (who were in charge of writing the specifications) were unable to convey enough information to the developers.
There's a real limit to the amount of "truth" you can put into a specification, and at some point you have to trust the developer to "do the right thing". If they don't have domain knowledge in the industry they're writing for, they don't stand a chance ... and so your project fails even when everyone works hard and has good intentions.
There's a real limit to the amount of "truth" you can put into a specification, and at some point you have to trust the developer to "do the right thing". If they don't have domain knowledge in the industry they're writing for, they don't stand a chance ... and so your project fails even when everyone works hard and has good intentions.
I agree, but this is something I've found resistance to in the past. For example, last year I was building an automated shipping system for a client and I wanted to spend a few days in the shipping department - this was flat out refused as the management didn't see it how it would be useful, since the process was fully documented. Turns out, much had to be rewritten because the shipping staff were (predictably) doing 'undocumented' things to speed up their workflow, the new shipping system messed this up and made them slower.
Jacques is right-on. For a lot of the work in my life, most of the effort was coming up to speed on a domain rather than just technology. Much of my work has been in 'artificial intelligence' (for some definition; never liked that term because it is too general) so domain knowledge is central if you are trying to get human level or better performance in some narrow domain.
Having to spin up on new domains also keeps work and life interesting.
Having to spin up on new domains also keeps work and life interesting.
Turning the tables around: perhaps the software (or even physical products) that we use where we think, "this is stupid! why would anyone build it this way?" were similarly built by people with no domain expertise in the product.
Introduce a software engineer with an interest in the subject matter into that environment, and provided it isn't too big (like government), it might be self-correcting.
Introduce a software engineer with an interest in the subject matter into that environment, and provided it isn't too big (like government), it might be self-correcting.
OP might be right if they choose to write software without collaboration with clients or subject matter experts.
IMHO, with collaboration, it's more a matter of providing a hyper-rational process by which the purpose and priorities of the target audience(s) are focused and honed, and then producing the software as feedback, automation, and a usable compliment to their needs.
IMHO, with collaboration, it's more a matter of providing a hyper-rational process by which the purpose and priorities of the target audience(s) are focused and honed, and then producing the software as feedback, automation, and a usable compliment to their needs.
This article is talking about what I learned as classic Systems Analysis and Design. It is an interdisciplinary intentional practice of seeking to fully understand the business processes involved.
See http://en.m.wikipedia.org/wiki/Systems_analysis_and_design as a starting point.
See http://en.m.wikipedia.org/wiki/Systems_analysis_and_design as a starting point.
So have you ever tried what the articles suggest? Ever spent a week working on the 'shop floor'?
In my experience of public sector software projects, the programmers seem to be kept as far away from the end users as possible.
In my experience of public sector software projects, the programmers seem to be kept as far away from the end users as possible.
As a independent contractor / systems analyst, yes I have. However I agree, most solutions providers don't bother.
I think domain knowledge is important, but sometimes it can be a trap that hinders inovation. As Pierre Bayard put very nicely in "How to talk about books you haven't read", when we don't have domain knowledge we open ourselves to questions that we wouldn't make otherwise.
Reminds me of the old "what the customer actually wanted" comic:
http://imgur.com/NXHbZ
http://imgur.com/NXHbZ
Perhaps we should all work in compilers ;)
Another metaphor for necessity of Domain Knowledge is translation. Give an average translator a technological text full of idioms from this technology domain and you will laugh your ass off.
I remember reading reports about some SAP installation process (written by idiots) full of stupid special terms such as "System Landscape" and "User Privileges" translated by an English teacher.)
Why we have this as a top news? Isn't it obvious that you must know the slang before you're trying to speak, let alone translate?
Well, most of coders use a slightly different strategy - jump right into IDE and pile it up, using all that containers and design patterns they have heard of (without understanding, why bother?) until it satisfies the manager (compiles) who know absolutely nothing (this is why everything must be statically typed). Then test it like a black-box - correct input produces correct output (never try incorrect or unexpected) and then ship and forget.)
I remember reading reports about some SAP installation process (written by idiots) full of stupid special terms such as "System Landscape" and "User Privileges" translated by an English teacher.)
Why we have this as a top news? Isn't it obvious that you must know the slang before you're trying to speak, let alone translate?
Well, most of coders use a slightly different strategy - jump right into IDE and pile it up, using all that containers and design patterns they have heard of (without understanding, why bother?) until it satisfies the manager (compiles) who know absolutely nothing (this is why everything must be statically typed). Then test it like a black-box - correct input produces correct output (never try incorrect or unexpected) and then ship and forget.)
Sadly in some domain it's not possible. I've had cases where to understand a particularly complex business logic case it turned out nobody had any idea how that case should be resolved. Lawyers / jurists, employees, managers, etc.
We tried to go up the chain up to someone, anyone, who could explain how that case should be resolved but nobody could tell.
Why? Legalese. And just like most legalese documents: there's more than one interpretation possible because the very concept of legalese (based on thousands years old tradition) is inherently flawed and open to interpretation.
So, somewhere, some politicians, probably out of bikeshedding fashion, decides to vote an amendment exceedingly badly written and exceedingly complex. Then it becomes law. And law has to be followed. So that case has to be taken into account by the software but isn't (yet).
Everything is all and well until that specific case appear. The day it appears all bets are of. It's not working and nobody knows why. Nobody knows what is not working and why it's not working, nor how it should work. The law is bogus and the business case cannot be resolved without contradicting with the law(s).
Trying to determine this ended up with the programming team receiving a PDF with tens and tens of pages of legalese text that nobody up the chain (including lawyers) could explain to us. Probably that the politicians who voted these changes couldn't even themselves explain it.
We simply couldn't make the changes.
How did it end up? Someone very high up saying: "That's what should be done in this particular case" and then everybody covering its arse.
And there came the technopriests: fixing the production DB...
We tried to go up the chain up to someone, anyone, who could explain how that case should be resolved but nobody could tell.
Why? Legalese. And just like most legalese documents: there's more than one interpretation possible because the very concept of legalese (based on thousands years old tradition) is inherently flawed and open to interpretation.
So, somewhere, some politicians, probably out of bikeshedding fashion, decides to vote an amendment exceedingly badly written and exceedingly complex. Then it becomes law. And law has to be followed. So that case has to be taken into account by the software but isn't (yet).
Everything is all and well until that specific case appear. The day it appears all bets are of. It's not working and nobody knows why. Nobody knows what is not working and why it's not working, nor how it should work. The law is bogus and the business case cannot be resolved without contradicting with the law(s).
Trying to determine this ended up with the programming team receiving a PDF with tens and tens of pages of legalese text that nobody up the chain (including lawyers) could explain to us. Probably that the politicians who voted these changes couldn't even themselves explain it.
We simply couldn't make the changes.
How did it end up? Someone very high up saying: "That's what should be done in this particular case" and then everybody covering its arse.
And there came the technopriests: fixing the production DB...
> How did it end up? Someone very high up saying: "That's what should be done in this particular case" and then everybody covering its arse.
This is usually how it ends up.
I've been close to similar situations (but while creating something, not changing something that already exists), but usually, if nobody understands how this is to be done, the fact that you can't code it is second in importance.
This is usually how it ends up.
I've been close to similar situations (but while creating something, not changing something that already exists), but usually, if nobody understands how this is to be done, the fact that you can't code it is second in importance.
Then these graduates go into the working world, and they typically can't help solve problems in economics, physics, applied mathematics, or medicine except by coding up some system to someone else's specs. The more dedicated ones make an effort to understand what they're doing. They exceedingly rare, gifted, and motivated ones might learn enough about an industry to make real contributions.
I hope that our field eventually evolves to the point that programming skill becomes an accessory. People will then learn to program just like they learn to write, and use the craft to help them solve their actual problems. The profession of programmers who only know how to program should disappear like the profession of scribes who only know how to write.