Living on the edge of academia(drmaciver.com)
drmaciver.com
Living on the edge of academia
http://www.drmaciver.com/2008/12/living-on-the-edge-of-academia/
12 comments
Something I've always wondered about- is there no value in failed results? I'd think people would benefit from knowing what approaches have failed. How is that information communicated?
It basically isn't, which I suspect is a huge collective waste of effort. I have several times had what I thought was a reasonably obvious idea that was "sure to work" which I was baffled that I couldn't find in the literature. Then, after spending a lot of time pursing it, I find it didn't work (for non-obvious reasons). I suspect I am one in a line of many people that have gone through that process.
I occasionally wish there was a Journal of Negative Results. However, there is a real problem in that it is nearly impossible to properly interpret a negative result. Something might not work because you have a sloppy experimental technique or because you have a typo in the implementation of your algorithm.
There are real current efforts to create such publications. This is quite relevant in pharmacology and other areas where biostatistics is applied, since there are often fewer incentives to publish the results of, say, a drug trial that fails to find a significant effect.
Hence, The Journal of Negative Results in Biomedicine (yes, it is actually called that): http://www.jnrbm.com/
Hence, The Journal of Negative Results in Biomedicine (yes, it is actually called that): http://www.jnrbm.com/
I like empirical papers describing experiences of applying ideas in real situations, in which they tend to expose unsuccessful approaches they have tried before finding one that works. It is not only useful but also enjoyable to read.
I'd like to see more of those papers, especially from industry. But to write such kind of paper, you usually have to devote to a successful project all the way through, which usually takes a few years or more, and then you can write just one paper. Not very efficient way to produce publications, and you might have to write a paper in your spare time, for your employer wants you to write code instead....
I'd like to see more of those papers, especially from industry. But to write such kind of paper, you usually have to devote to a successful project all the way through, which usually takes a few years or more, and then you can write just one paper. Not very efficient way to produce publications, and you might have to write a paper in your spare time, for your employer wants you to write code instead....
It would seem embarrassing to write up in a paper; but excellent filler (and justified) in a thesis. The thesis goes online, google indexes it; now it's available as lit.
[deleted]
And I bet you didn't document your failure... like everyone else before you.
"Something I've always wondered about- is there no value in failed results?"
Peter Norvig has some great comments on the problem of NOT communicating failed results in his article on interpreting research.
http://norvig.com/experiment-design.html
Peter Norvig has some great comments on the problem of NOT communicating failed results in his article on interpreting research.
http://norvig.com/experiment-design.html
[deleted]
It is called publication bias. http://en.wikipedia.org/wiki/Publication_bias
Coming from someone who's considering going back to school (grad school) for Bioinformatics/Comp Bio, I have always thought that Bioinformatics wasn't like "traditional" academia because so much of it is "Run n' Gun" programming/statistical analysis of the latest data-set or with a new mining algorithm. Or at least, this is what I envision to be (if you will, the dawn of a major field, like the 1960-1970's of Computer Science research).
Would you care to shed more light onto doing Comp. Bio research as a graduate student for someone who's seriously considering taking on 6-8 years of indentured servitude into the field?
Would you care to shed more light onto doing Comp. Bio research as a graduate student for someone who's seriously considering taking on 6-8 years of indentured servitude into the field?
Most scientific and engineering research follows the same pattern: you spend a lot of time setting up an elaborate thingamajig to test a hypothesis, then you run the experiment, and spend months analyzing the data. Then you discover that you've failed. Lather, Rinse, Repeat. Data-mining research just uses computer software as the thingamajig, so maybe you do a few more failure cycles per unit time than the guys down the hall (who are building a nuclear accelerator to find the anti-bozon -- but hey, at least they get to play with lasers!)
As far as practical advice for someone contemplating a PhD in bioinformatics, mine would be: don't get a PhD in bioinformatics. There are no jobs, and a PhD is generally just a fantastic waste of time (unless you want to be a professor, and are comfortable with the numerous, well-documented drawbacks of that lifestyle). If you feel that you must explore the field, go far enough to get an MS, then quit.
I realize that's harsh, so I'll add some slightly softer, less-valuable advice: if you must get a PhD in bioinformatics, at least have the good sense to do it in a computer science department, so that you can go back into software when you find there are no jobs in bioinformatics. It's a dangerous mistake to get a PhD in bioinformatics from a biology department (ahem...do as I say, not as I do.)
Failing all of that advice, if you really, really want to be in bioinformatics, despite all good advice to the contrary, then make sure that you spend equal amounts of time at the bench and at the computer. If you want to be respected in the field, you have to have experimental results to back up your computer models, and the best way to do that is to run the experiments yourself. Run away from any program or professor who won't let you work at the bench.
As far as practical advice for someone contemplating a PhD in bioinformatics, mine would be: don't get a PhD in bioinformatics. There are no jobs, and a PhD is generally just a fantastic waste of time (unless you want to be a professor, and are comfortable with the numerous, well-documented drawbacks of that lifestyle). If you feel that you must explore the field, go far enough to get an MS, then quit.
I realize that's harsh, so I'll add some slightly softer, less-valuable advice: if you must get a PhD in bioinformatics, at least have the good sense to do it in a computer science department, so that you can go back into software when you find there are no jobs in bioinformatics. It's a dangerous mistake to get a PhD in bioinformatics from a biology department (ahem...do as I say, not as I do.)
Failing all of that advice, if you really, really want to be in bioinformatics, despite all good advice to the contrary, then make sure that you spend equal amounts of time at the bench and at the computer. If you want to be respected in the field, you have to have experimental results to back up your computer models, and the best way to do that is to run the experiments yourself. Run away from any program or professor who won't let you work at the bench.
Thank you for posting such helpful advice. I think the last part of your advice is the most helpful - that you need to really do wet lab benchwork, something I've tried to avoid during my undergrad, thinking that computers are the way of future and lab work is for suckers. But alas, in the realm of biology, the micropipetters and mice-killers win again.
But I appreciate you for also being very adamant about the "dark side" of academia; it would have deterred me if someone else didn't also warn me once about the "dark side" of going corporate (http://news.ycombinator.com/item?id=66390). I appreciate again your advice and will think it over very carefully when I decide as to what exactly I am going to do next.
But I appreciate you for also being very adamant about the "dark side" of academia; it would have deterred me if someone else didn't also warn me once about the "dark side" of going corporate (http://news.ycombinator.com/item?id=66390). I appreciate again your advice and will think it over very carefully when I decide as to what exactly I am going to do next.
"...computers are the way of future and lab work is for suckers. But alas, in the realm of biology, the micropipetters and mice-killers win again."
Doing biology is the only way to develop biological intuition. Sometimes this means pipetting and killing mice.
Ernst Mayr speaks to this:
"About two years ago, three years ago, for maybe the 20th time I went over the whole business of the species concept. What is a species? I looked at the major figures in the evolutionary synthesis, and I looked at Robzhansky and myself, and Huxley and Stebbins, all of us had reasonable species concepts, and the only person that had a species concept that I thought was quite absurd was the paleontologist G.G. Simpson. And then I said to myself, "Well, he can't have been a naturalist in his youth if he had such a peculiar, unworkable species concept." So I went to Simpson's biography and what did I find? I found that in college he was an English major. He had never been a naturalist as a youngster. He never collected anything, and he discovered geology in his senior year in college, and from there he went to stratigraphy and finally to paleontology. Not surprisingly, not having been a naturalist, he has no idea what a species is and he never had. I argued with him about the species concept year after year, but lacking that background, he was unable to see it, and that is the thing. Being a naturalist -- having had that background of being a naturalist -- gives you a view of nature that cannot be acquired just learning from books."
(Go to bottom of http://www.achievement.org/autodoc/page/may1int-5 for a video of him saying it while sitting next to E. O. Wilson.)
Doing biology is the only way to develop biological intuition. Sometimes this means pipetting and killing mice.
Ernst Mayr speaks to this:
"About two years ago, three years ago, for maybe the 20th time I went over the whole business of the species concept. What is a species? I looked at the major figures in the evolutionary synthesis, and I looked at Robzhansky and myself, and Huxley and Stebbins, all of us had reasonable species concepts, and the only person that had a species concept that I thought was quite absurd was the paleontologist G.G. Simpson. And then I said to myself, "Well, he can't have been a naturalist in his youth if he had such a peculiar, unworkable species concept." So I went to Simpson's biography and what did I find? I found that in college he was an English major. He had never been a naturalist as a youngster. He never collected anything, and he discovered geology in his senior year in college, and from there he went to stratigraphy and finally to paleontology. Not surprisingly, not having been a naturalist, he has no idea what a species is and he never had. I argued with him about the species concept year after year, but lacking that background, he was unable to see it, and that is the thing. Being a naturalist -- having had that background of being a naturalist -- gives you a view of nature that cannot be acquired just learning from books."
(Go to bottom of http://www.achievement.org/autodoc/page/may1int-5 for a video of him saying it while sitting next to E. O. Wilson.)
Glad to help. I was being a little tongue-in-cheek, obviously, but I meant what I said. The job market is very bad for biologists of all disciplines right now, and computational biology isn't the discipline with the greatest demand. If you decide to go on, being able to work at the bench is essential to your career.
But don't take my word as proof -- the Science Magazine Career forums are literally filled with sad stories of PhDs and post-docs in the biological sciences who can't get jobs. I wish it had existed when I started my program:
http://scforum.aaas.org/
(One more thing: I just read the link to the other comments, and I don't disagree with what they wrote. Most software jobs suck, too. But the key insight here is that most jobs suck -- there is no magic bullet to career happiness. The corollary to that insight is that you need to fail fast; you must quickly work through the things that suck, and find the things that don't.
Because a PhD ties you to one thing for so long, it is almost the antithesis of failing fast. For this reason, I would not recommend a PhD to anyone, unless they explicitly wanted to become a professor, and knew exactly what they were getting themselves into. It's not the kind of decision that rewards indecisiveness.)
But don't take my word as proof -- the Science Magazine Career forums are literally filled with sad stories of PhDs and post-docs in the biological sciences who can't get jobs. I wish it had existed when I started my program:
http://scforum.aaas.org/
(One more thing: I just read the link to the other comments, and I don't disagree with what they wrote. Most software jobs suck, too. But the key insight here is that most jobs suck -- there is no magic bullet to career happiness. The corollary to that insight is that you need to fail fast; you must quickly work through the things that suck, and find the things that don't.
Because a PhD ties you to one thing for so long, it is almost the antithesis of failing fast. For this reason, I would not recommend a PhD to anyone, unless they explicitly wanted to become a professor, and knew exactly what they were getting themselves into. It's not the kind of decision that rewards indecisiveness.)
I'm trying to decide between a CS MS with a machine learning specialty or a bioinformatics specialty. Would you say machine learning is more employable? Bioinformatics probably interests me more.
My feeling is that, at the moment, there are a huge number of academic bioinformatics jobs available. (More than machine learning.) Be aware, however, that these academic jobs are probably not attainable with an MS, and the market tends to change quickly. In industry, my feeling is that machine learning is more employable. I think that a CS MS who is a great programmer and knows machine learning cold would have very good employment prospects.
[deleted]
I'll second what timr says, I think a PHD in Bioinformatics is possibly a waste of your time. I got into the field in 2000 immediately after undergrad. Since then, I've been working in industry and did my masters in bioinformatics at a top notch school. I've always had a pretty good feel for where new technology is going, how things are going to advance, etc... but I called bioinformatics completely wrong. I really thought it was going to be much more important than it has actually been. Over the last year or so I started looking for a new job in the field and they are REALLY hard to come by (especially industry jobs in the US). In the end, I managed to get a good position (much more medical informatics that bioinformatics actually). I am very aware that I got super lucky. In fact, I was VERY close to pulling the plug on the whole field and trying my hand at a startup (mostly because I hang out here too much :)).
A few years ago, you could count on finding a job at one of the major Pharmas or in a software shop that serves one of them. But many of the pharmas are cash strapped and have been cutting "luxuries" like bioinformatics and software. My experience is that many biotechs are so small and narrowly focused, they often don't benefit from too much bioinformatics work. That leaves academia, but even there, the supply of PHD scientists is way higher than the number of jobs (this is actually a more general problem than in just bioinformatics).
I've often regretted that I didn't just go for a straight CS masters so that I could beef up some of those CS concepts you always avoid when your programming for fun.
I think you need to decide what your ultimate career goal is. If you want to be a PHD scientist getting grants and doing work at a university, then you have to go the PHD route. But if you think you want to work in industry, I would strongly urge a different route, one that doesn't paint you into a corner too much.
A few years ago, you could count on finding a job at one of the major Pharmas or in a software shop that serves one of them. But many of the pharmas are cash strapped and have been cutting "luxuries" like bioinformatics and software. My experience is that many biotechs are so small and narrowly focused, they often don't benefit from too much bioinformatics work. That leaves academia, but even there, the supply of PHD scientists is way higher than the number of jobs (this is actually a more general problem than in just bioinformatics).
I've often regretted that I didn't just go for a straight CS masters so that I could beef up some of those CS concepts you always avoid when your programming for fun.
I think you need to decide what your ultimate career goal is. If you want to be a PHD scientist getting grants and doing work at a university, then you have to go the PHD route. But if you think you want to work in industry, I would strongly urge a different route, one that doesn't paint you into a corner too much.
If you graduated college, consult with your college librarian. The chance is, most journals have online subscription deals and they are accessible via VPN/proxy through your school's subscriptions. If this doesn't apply, go to a local University's library or research hospital in your area, most of these subscriptions are validated through universities' ip addresses - so any computer on campus will do. Trust me, I do this all this time even though I graduated already.
The ACM's copyright form states that author's are legally allowed to put a copy of the paper on their personal website. Really, you don't need to pay any subscription. If you know the author's names for a paper, chances are one of them will have it on their personal page.
Most of this is findable in Google within 30 seconds. Often when I search for a paper's title, I get a pdf from the author's site before an official ACM or IEEE listing.
Most of this is findable in Google within 30 seconds. Often when I search for a paper's title, I get a pdf from the author's site before an official ACM or IEEE listing.
CiteseerX is also a good source of papers though not as good as an ACM subscription.
Some university libraries have a public terminal or two (mine does), which has access to all the electronic journals they are subscribed to (ACM, IEEE, JSTOR, Springerlink etc).
They don't advertise this; you have to ask.
They don't advertise this; you have to ask.
I end up needing a lot of research papers for the work that I'm doing. My solution is kind of lo-fi crowd sourcing. I'm usually in a few channels on Freenode and when I hit a paper that I'd like to have and can't hit with a few well-crafted Google searches I just ask if anyone around has access to them. Usually gets me the copy I need within 10 minutes or so.
Now, what's really annoying is market research. Friggin' Forester and their hundreds of dollars per paper. Hrmf.
Now, what's really annoying is market research. Friggin' Forester and their hundreds of dollars per paper. Hrmf.
The process the author describes is almost identical to the one I've experienced, despite the fact my area of research is different than his (machine learning), but his analysis of the results drastically differs from mine
Its true, 90% of the stuff doesn't work/isn't applicable and I have spent more money on buying papers than on rent in the past year, and I spend many days working on stuff that often has little or no positive effect. But its still the most cost effective research you can do, after you factor in the value of your time.
Every new idea I develop on my own takes far longer to develop and test, and trying to get better results than everyone else is a lot easier when you're basing your progress on the cutting edge instead of something that is a decade old.
Its true, 90% of the stuff doesn't work/isn't applicable and I have spent more money on buying papers than on rent in the past year, and I spend many days working on stuff that often has little or no positive effect. But its still the most cost effective research you can do, after you factor in the value of your time.
Every new idea I develop on my own takes far longer to develop and test, and trying to get better results than everyone else is a lot easier when you're basing your progress on the cutting edge instead of something that is a decade old.
There is a lot of value in failed results, especially in science. What doesn't work is critical, if for nothing else, than for protocol and experiment design. Freeing up negative data is a huge motivation behind Open Notebook Science(1) and a subject of much discussion on the science blogosphere(2)
1. http://en.wikipedia.org/wiki/Open_notebook_science 2. http://network.nature.com/groups/harvardpublishingforum/foru...
1. http://en.wikipedia.org/wiki/Open_notebook_science 2. http://network.nature.com/groups/harvardpublishingforum/foru...
The solution to the "Academic Firewall of Doom" is to get a degree, and then you can generally go back to your alma mater and get some kind of library account that will let you access the stuff behind the firewall (generally from within the library only, of course). Of course its been over a decade since that loophole worked for me, might have been closed up by now. Or it might have expanded.
I recall some kind of problem when I attempted to do this at Dalhousie, but I can't remember what it was. Maybe I'll try again soon.
For journal access, consider some part-time lecturing / mentoring at your nearest university. I mentor a group of business students once a week, and aside from the extra money and overall enjoyment, get a university email and athens account - as I'm undertaking market research for a new startup, this is saving me A LOT of money.
I pay a fee to have access to my local university library. (As a graduate of said university I get half price, but full price (160GBP) is still reasonable.) This gives me book-borrowing rights, and reference access to printed copies of journals, (but not electronic access).
It's a current tragedy in computer science research: authors release a matlab blob (no source code) or no implementation at all. When asked, I've encountered these answers:
1) An implementation is not important. We developed an algorithm!
2) Uh... the student who made the program already left and we can't find it.
3) [the student] I can no longer find my notes on which parameters I used to make the figures of the paper.
Very rarely, nearly never, does one encounter a proper software release.
Why? Many reasons--the key point is that only a handful of research institutions have technical staff on board to collect and curate software produced in house.
1) An implementation is not important. We developed an algorithm!
2) Uh... the student who made the program already left and we can't find it.
3) [the student] I can no longer find my notes on which parameters I used to make the figures of the paper.
Very rarely, nearly never, does one encounter a proper software release.
Why? Many reasons--the key point is that only a handful of research institutions have technical staff on board to collect and curate software produced in house.
oh, that's so cute! it's almost like he's pretending that academia isn't a soul-less abyss controlled by self-aggrandizing profiteers.
Ugh, seriously. Academia isn't the place for people with 'Big Ideas' anymore (or was it ever?)
You find most of the innovation and creativity out in the startups and small businesses.
You find most of the innovation and creativity out in the startups and small businesses.
This is why it amuses me when people stereotype academic scientists as impractical dreamers -- anyone who has stuck with the process of completing a PhD has demonstrated him or herself to possess a truly super-human tolerance for drudgery.
(Public service announcement: Hire a PhD! They need the work!)