Outsourcing code reviews?
4 comments
Code reviews can bottleneck development, so I was thinking it'd be nice if I could just commit at end of day... Then in the morning, I'd come in to an email box magically full of corrections, criticisms, etc.
This sounds like open source development. There's a reason why people joke that the fastest way to get a patch reviewed is to commit it. :-)
But more seriously: Finding people who are good at reviewing your code is very difficult. The more obscure the code you're writing -- that is, the more you actually need your code to be reviewed -- the fewer the number of people who would be competent to review it; so any attempt to put together a generic "group of reviewers" isn't likely to get the most important code reviewed.
If I were you, I'd think about what specific things you're doing in which you're likely to make mistakes -- generally speaking, this means sorts of code which you don't have much experience writing -- and then go out and look for experts to review for those specific bits of code.
I'm available for contract work reviewing cryptographic code. :-)
This sounds like open source development. There's a reason why people joke that the fastest way to get a patch reviewed is to commit it. :-)
But more seriously: Finding people who are good at reviewing your code is very difficult. The more obscure the code you're writing -- that is, the more you actually need your code to be reviewed -- the fewer the number of people who would be competent to review it; so any attempt to put together a generic "group of reviewers" isn't likely to get the most important code reviewed.
If I were you, I'd think about what specific things you're doing in which you're likely to make mistakes -- generally speaking, this means sorts of code which you don't have much experience writing -- and then go out and look for experts to review for those specific bits of code.
I'm available for contract work reviewing cryptographic code. :-)
Finding people who are good at reviewing your code is very difficult.
I think this is generally true even without the emphasis on my particular code. Most folks suck even more at reviewing than they do at coding. And of course it's always hard to find really good developers.
I think there might be attractive qualities for a high quality developer to take on such a project though.. For instance:
Short off-time hours.. like someone in Seattle could review pretty interesting code after their stupid day of meetings that they got sucked into by being too good at their job. Hell, they might be able to do it _in_ the meetings.
Pay... I'd happily pay well over good dev rates for a good reviewer (or set of reviewers). People telling me how I'm dumb and how to improve is great.
Joy of working outside of Java / C# / C++ / VB / ASP.NET corporate code.
Freedom to rip stuff apart without fearing for one's job or worrying about office politics. Some of us have a lot of pent up rage :)
I think this is generally true even without the emphasis on my particular code. Most folks suck even more at reviewing than they do at coding. And of course it's always hard to find really good developers.
I think there might be attractive qualities for a high quality developer to take on such a project though.. For instance:
Short off-time hours.. like someone in Seattle could review pretty interesting code after their stupid day of meetings that they got sucked into by being too good at their job. Hell, they might be able to do it _in_ the meetings.
Pay... I'd happily pay well over good dev rates for a good reviewer (or set of reviewers). People telling me how I'm dumb and how to improve is great.
Joy of working outside of Java / C# / C++ / VB / ASP.NET corporate code.
Freedom to rip stuff apart without fearing for one's job or worrying about office politics. Some of us have a lot of pent up rage :)
The more obscure the code you're writing -- that is, the more you actually need your code to be reviewed
The only part that should be obscure is why the program is even useful and why it has some features at all. The code and design should be entirely understandable given a 1-page overview document. I'm certainly not talking about cryptographic code as you mentioned.
I'd actually guess that's the case for most code written in the world. Given certain business needs, the code should be reasonably understandable for someone who wants to understand.
The only part that should be obscure is why the program is even useful and why it has some features at all. The code and design should be entirely understandable given a 1-page overview document. I'm certainly not talking about cryptographic code as you mentioned.
I'd actually guess that's the case for most code written in the world. Given certain business needs, the code should be reasonably understandable for someone who wants to understand.
We've been doing this under our "Code Audit" service for clients since 2003.
If it's allowed, could you give me a reference to how I could contact your firm? Are you in the US?
I do reviews for a client and might be interested in taking on another. You can see my contact info and some of my code at http://github.com/darius.
Is all of the code you are writing open source? If not, then you may not want to release it to third parties, and they may not want to read it.
Also, getting a reasonable code review on a large project pretty much requires the reviewer to be versed in your codebase. Depending on what you're doing, it may be more costly (in time/money) to bring someone up to speed.
Also, getting a reasonable code review on a large project pretty much requires the reviewer to be versed in your codebase. Depending on what you're doing, it may be more costly (in time/money) to bring someone up to speed.
Is all of the code you are writing open source? If not, then you may not want to release it to third parties, and they may not want to read it.
I was speaking of closed source parts. Open-source has a reasonable enough review process already. Although I don't want the code publicly available, I don't want to waste my life being paranoid.
Depending on what you're doing, it may be more costly (in time/money) to bring someone up to speed.
Very true. Fortunately, I control the requirements, and I'm almost compulsive about simplicity. I want to be able to keep the code small and be able to rewrite parts quickly. I'm very much into quantity over quality (http://www.codinghorror.com/blog/archives/001160.html), so if it's useful, I'd be happy to start from scratch just to bring reviewers up with me.
That said, I want to be able to bring other developers up to speed on the project later, and it might be time well invested trying to make the project easier to get up to speed on. That'd also reduce the dependency on particular people that are in some part of the world I'll likely never see.
I was speaking of closed source parts. Open-source has a reasonable enough review process already. Although I don't want the code publicly available, I don't want to waste my life being paranoid.
Depending on what you're doing, it may be more costly (in time/money) to bring someone up to speed.
Very true. Fortunately, I control the requirements, and I'm almost compulsive about simplicity. I want to be able to keep the code small and be able to rewrite parts quickly. I'm very much into quantity over quality (http://www.codinghorror.com/blog/archives/001160.html), so if it's useful, I'd be happy to start from scratch just to bring reviewers up with me.
That said, I want to be able to bring other developers up to speed on the project later, and it might be time well invested trying to make the project easier to get up to speed on. That'd also reduce the dependency on particular people that are in some part of the world I'll likely never see.
I don't especially like the idea of outsourcing coding, but having someone else tear apart my design, style, and logic seems like it could be a good idea. The across-the-world aspect of some outsourcing seems like it could be a benefit.
Code reviews can bottleneck development, so I was thinking it'd be nice if I could just commit at end of day... Then in the morning, I'd come in to an email box magically full of corrections, criticisms, etc.
When I think of the big worries I have about outsourced development, schedule risk, quality risk, and the miscommunication burdens. They all seem to be mitigated by just doing reviews. If they can't review well in a reasonably timely manner (strong preference for a day), just quit using them. No major loss. If the code isn't understandable, there's a real benefit to improving it... it's not just a burden imposed by the process.
Has anyone tried this? What issues did you / might you run into?
Are there companies I could approach about hiring them to do it? I'm happy to pay up for such a service because if it works well it could make me dramatically more productive.
Am I missing something that makes this obviously a stupid idea?