Microsoft changed how it interviews software developers(businessinsider.fr)
businessinsider.fr
Microsoft changed how it interviews software developers
https://www.businessinsider.fr/us/microsoft-new-developer-interview-process-2018-12
331 comments
> But the aha moment for me was that not everyone does well in those fast-paced brainstorming sessions. A lot of people (including me) prefer to sit with a cup of coffee and some data and try to think things through.
This is me. Hard to get that across in an interview, but once people work with me, they're cool with me coming back an hour later in an email with some thoughts on the last meeting topic. They know and respect that that's the way I work[0].
[0] I'm not the only one on the team who does this, so that helps.
This is me. Hard to get that across in an interview, but once people work with me, they're cool with me coming back an hour later in an email with some thoughts on the last meeting topic. They know and respect that that's the way I work[0].
[0] I'm not the only one on the team who does this, so that helps.
Great, now can Netflix, Google and Facebook do this, too? Not because I want to work in these places, but because they influence everyone else and as a senior engineer in the systems space I feel I shouldn't need to study days or weeks for fizzbuzz sorting algorithms questions that are designed to test comp sci recent grads. I have a proven career, and was never suddenly stumped in a project due to not being able to.. sort letters in some obscure order based on arbitrary rules. How about we look at that, and compare the candidate's past experience to the current needs and interview based on that -- You DID look at the CV before bringing the candidate in, didn't you?
The original article [1] describes new process in the context of PM interviews, which are different from software developer interview, no matter how technical PM is.
The changes that I particularly like are: improvement in coordination between interviewers themselves, and stop sharing feedback between themselves until the very end. The latter really affects how next interviewers view the interviewee.
[1] https://blog.usejournal.com/rethinking-how-we-interview-in-m...
The changes that I particularly like are: improvement in coordination between interviewers themselves, and stop sharing feedback between themselves until the very end. The latter really affects how next interviewers view the interviewee.
[1] https://blog.usejournal.com/rethinking-how-we-interview-in-m...
> Under the new process, Microsoft shares the interview questions in advance so that candidates can prepare. During the interview itself, a candidate might run through a real scenario or problem the team is trying to solve.
LOVE IT.
I think best when I have a chance to actually think, not when I’m participating in some odd job interview game show.
LOVE IT.
I think best when I have a chance to actually think, not when I’m participating in some odd job interview game show.
Here's what they are doing, from the linked blog post:
> Our dev teams had taken to working with candidates to solve a bug or feature as part of the interview process. It was a collaborative effort with the candidate and the team working together to solve a real problem.
Sounds like a great idea.
Except it's so variable. What if you get lucky and you get a easy bug, whereas someone else gets something much harder. Let's standardize that by giving everyone the same task to work on.
And what if we want to hire people who aren't fluent in your project's programming language? That would be a big handicap for them. Let's deal with that by having minimally sized "projects" in all the languages, and allowing the candidate to choose their preferred language.
And we don't want the interviews to be biased against people who don't already have domain knowledge in a specific field. So let's make the task generic enough to be approachable by any good programmer.
Congrats, if you did all of the above, you've reinvented leetcode style interviews.
There's some cool stuff that MS has put into practice. I like the focus on reading/understanding existing code, instead of just writing code. I also like the relaxed pacing and "open book" approach.
But for the most part, I don't think this is really as revolutionary as people think it is.
> Our dev teams had taken to working with candidates to solve a bug or feature as part of the interview process. It was a collaborative effort with the candidate and the team working together to solve a real problem.
Sounds like a great idea.
Except it's so variable. What if you get lucky and you get a easy bug, whereas someone else gets something much harder. Let's standardize that by giving everyone the same task to work on.
And what if we want to hire people who aren't fluent in your project's programming language? That would be a big handicap for them. Let's deal with that by having minimally sized "projects" in all the languages, and allowing the candidate to choose their preferred language.
And we don't want the interviews to be biased against people who don't already have domain knowledge in a specific field. So let's make the task generic enough to be approachable by any good programmer.
Congrats, if you did all of the above, you've reinvented leetcode style interviews.
There's some cool stuff that MS has put into practice. I like the focus on reading/understanding existing code, instead of just writing code. I also like the relaxed pacing and "open book" approach.
But for the most part, I don't think this is really as revolutionary as people think it is.
When I interviewed at Redmond 15 years ago, it was the recruiter who asked me a bunch of techie questions then decided to bring me out for a whole day of onsite interviews. During that I recall it was only a director of SW who asked me some mentally crushing questions about subtleties in how the Windows kernel fiddles with files. As I said, I was a UNIX developer who didn't know those bottlenecks in the Windows kernel. Still he persisted. He did take me for a spin in his awesome BMW sports car after though!
My thoughts on Redmond were that it rains all the time and the inside of the Microsoft buildings is pretty depressing.
My thoughts on Redmond were that it rains all the time and the inside of the Microsoft buildings is pretty depressing.
Good riddance to all those mostly useless brain-teaser questions.. They require countless hours of practice for most people to get good at them - and I find it semi-tragic that many highly intelligent people are wasting their time on that instead of on complex real-world problems (not to mention on learning things that would make them happier and more well-rounded).
> ... Second, we run through a real problem the team is trying to solve — improving satisfaction, increasing retention, boosting usage of a service or feature. The fact that it’s a real problem that we’re working on helps foster a collaborative conversation.
maybe I'm reading too much into this but ... for some reason, none of these questions are highly programming-centric (e.g. implement a sort algorithm, build a B-Tree, or even just implement the singleton pattern in C#, or whatever). all of the questions are human behavior-centric. and I wonder if maybe Microsoft has concluded that the really important challenges are not technical challenges, but marketing ones.
maybe I'm reading too much into this but ... for some reason, none of these questions are highly programming-centric (e.g. implement a sort algorithm, build a B-Tree, or even just implement the singleton pattern in C#, or whatever). all of the questions are human behavior-centric. and I wonder if maybe Microsoft has concluded that the really important challenges are not technical challenges, but marketing ones.
I noticed that the two people quoted are both PMs. So does this have any relevance for software engineers?
https://blog.usejournal.com/rethinking-how-we-interview-in-m... is the original post by a PM at Microsoft. It doesn't have any useful information about a pure technical interview for software engineering.
>(Note: I started at Microsoft when we were still asking questions about why manhole covers were round, how many ping pong balls would fill a 747, and how to reverse a linked list. In 20 years here, I’ve yet to have to write the code to reverse a linked list (copy-paste anyone?) or fill a 747 with any kind of ball.)
The manhole cover question is actually a valid industrial design or UX question even if it's not appropriate to ask for software interviews. The linked-list question is the equivalent of Fizzbuzz for data structures. And the 747 question is a valid Fermi problem. The exact number doesn't matter but the steps required to estimate the answer are similar to other estimation problems when mapping out a business strategy and trying to determine if it's profitable. Or maybe the company is interested in salvaging sunken planes: https://www.iusmentis.com/patents/priorart/donaldduck/
>(Note: I started at Microsoft when we were still asking questions about why manhole covers were round, how many ping pong balls would fill a 747, and how to reverse a linked list. In 20 years here, I’ve yet to have to write the code to reverse a linked list (copy-paste anyone?) or fill a 747 with any kind of ball.)
The manhole cover question is actually a valid industrial design or UX question even if it's not appropriate to ask for software interviews. The linked-list question is the equivalent of Fizzbuzz for data structures. And the 747 question is a valid Fermi problem. The exact number doesn't matter but the steps required to estimate the answer are similar to other estimation problems when mapping out a business strategy and trying to determine if it's profitable. Or maybe the company is interested in salvaging sunken planes: https://www.iusmentis.com/patents/priorart/donaldduck/
Although I’m a product person (CPO) I’ve hired (alongside a CTO) maybe 100+ engineers in my time and I’m hoping my perspective is useful/helpful.
The process that my CTO and I have found works best is as follows;
A 20min phone interview (to screen for BS’ers).
A technical test that’s based on the sort of problems they will be solving in their job. How would you do X in language Y, no boiler plate code, and it doesn’t have to be perfect. (So no fizz buzz, conways game of life etc.) We also stipulate to spend no more than 2 hours on it, and if their successful they get that time back. We even offer a discount (on the thing we sell) to unsuccessful people who don’t make it. The test serves as not only a good high level demonstration of ones skils but, more importantly, gives us all something to talk around. It also shows us how they approach key things like testing, deployment, maintainability etc etc.
Then there is a 2 hours face to face interview. Ahead of this the test is peer reviewed by the engineering team (or part of it) The interview is made up of one one hour technical section with the CTO/Senior, and one with a designer/product manager. A joint decision is made.
Then, if they are successful, we have a probation period where we (the candidate and us) work out if it’s all going as we’d hoped. If not, we part ways. You most likely wouldn’t marry someone before a first date. A job is the same. You both need time to see if you like each other and your profile pics match real life.
It’s not perfect, but no system is. Sure we’ve hired a few people we’ve had to let go, but for the most part it’s a great system. We’ve managed to hire not just great engineers but also build a team and culture that fits and works.
Hiring is time consuming, hard and at times emotional. There is no shortcut.
I’ve interviewed and worked at big tech companies and they all think there is some secret sauce, a magic code to hiring. That is, if you ask super hard questions only the “best” make it through - utter tosh. Some of the most amazing engineers I’ve worked with would fail these type of stupid questions (for a while load of reasons) but quietly write amazing code.
Also, and more importantly imo, to be an amazing engineer you need to be great (equally) at teamwork and communication, not just be super book smart.
I read a while back (or imagined it?) that MSFT only hired people with Msc’s/PHd’s and that it created a probelem where there wasn’t any intellectual diversity. These tests cause this too. You get cookie cutter people who are smart but often can’t work in teams and can’t communicate well.
The process that my CTO and I have found works best is as follows;
A 20min phone interview (to screen for BS’ers).
A technical test that’s based on the sort of problems they will be solving in their job. How would you do X in language Y, no boiler plate code, and it doesn’t have to be perfect. (So no fizz buzz, conways game of life etc.) We also stipulate to spend no more than 2 hours on it, and if their successful they get that time back. We even offer a discount (on the thing we sell) to unsuccessful people who don’t make it. The test serves as not only a good high level demonstration of ones skils but, more importantly, gives us all something to talk around. It also shows us how they approach key things like testing, deployment, maintainability etc etc.
Then there is a 2 hours face to face interview. Ahead of this the test is peer reviewed by the engineering team (or part of it) The interview is made up of one one hour technical section with the CTO/Senior, and one with a designer/product manager. A joint decision is made.
Then, if they are successful, we have a probation period where we (the candidate and us) work out if it’s all going as we’d hoped. If not, we part ways. You most likely wouldn’t marry someone before a first date. A job is the same. You both need time to see if you like each other and your profile pics match real life.
It’s not perfect, but no system is. Sure we’ve hired a few people we’ve had to let go, but for the most part it’s a great system. We’ve managed to hire not just great engineers but also build a team and culture that fits and works.
Hiring is time consuming, hard and at times emotional. There is no shortcut.
I’ve interviewed and worked at big tech companies and they all think there is some secret sauce, a magic code to hiring. That is, if you ask super hard questions only the “best” make it through - utter tosh. Some of the most amazing engineers I’ve worked with would fail these type of stupid questions (for a while load of reasons) but quietly write amazing code.
Also, and more importantly imo, to be an amazing engineer you need to be great (equally) at teamwork and communication, not just be super book smart.
I read a while back (or imagined it?) that MSFT only hired people with Msc’s/PHd’s and that it created a probelem where there wasn’t any intellectual diversity. These tests cause this too. You get cookie cutter people who are smart but often can’t work in teams and can’t communicate well.
> He thinks it would be useful to figure out if there ways to either shorten the process entirely, or at least spread it out over multiple days and gives candidates more time to reflect and think.
Please don't. I can take one day off work for a full-day interview; I'm not going to burn through four fucking vacation days for you to vacillate over whether or not I'm a "culture fit" or whatever.
Please don't. I can take one day off work for a full-day interview; I'm not going to burn through four fucking vacation days for you to vacillate over whether or not I'm a "culture fit" or whatever.
IMO those interviews test whether you are smart "enough" and capable
of collaborating to solve a problem. There's not that much knowledge
required and good answers tend to be quite succint.
Depending on what you do, maybe most of the time interview-like problems won't rise up, but not everyone is ok taking the risk of poor algorithm design on production. While it won't happen that a single engineer will need to do everything without any design or code reviews, if you keep "lowering" the bar you risk breaking the review failsafe. Maybe scaling the review process can enable teams having more engineers working "safely" as the hard problems will still be reviewed by the "elite" engineers those companies try hard to get.
I hope this takes some pressure away on pretending to have "real" interviews needlessly. There's too many problems to be solved out there to be overly picky on who can work on them.
It would be nice to hear back from MS after they a few years.
Depending on what you do, maybe most of the time interview-like problems won't rise up, but not everyone is ok taking the risk of poor algorithm design on production. While it won't happen that a single engineer will need to do everything without any design or code reviews, if you keep "lowering" the bar you risk breaking the review failsafe. Maybe scaling the review process can enable teams having more engineers working "safely" as the hard problems will still be reviewed by the "elite" engineers those companies try hard to get.
I hope this takes some pressure away on pretending to have "real" interviews needlessly. There's too many problems to be solved out there to be overly picky on who can work on them.
It would be nice to hear back from MS after they a few years.
Sounds like a good approach. It's nice to see some meaningful progress in interview processes. Kudos to Microsoft.
Article used incorrect title, the original blog posts describes the interview Microsoft has for technical PM and not software developers. I bet that developer interview hasn't changed much.
I think a dumbest question thread might be quite amusing?
I’ll go first, because I’m quite sure I’ll win...
This was for a front-end mid weight software engineer position. The candidate was smart and a 100% hire (I’ve since hired him 3x in different companies)
Me; “great thanks for your time and we will be in touch later today”
Co-interviewer; “I have one more question”
Candidate; “sure, go ahead”
Co-interviewer; “Imagine your uncle has just died and left you his pizzeria in his will, what would you do?”
Me; “what?”
Candidate; “what...? Errr... sell it because I’m a software engineer, not a pizza chef”
I’ll go first, because I’m quite sure I’ll win...
This was for a front-end mid weight software engineer position. The candidate was smart and a 100% hire (I’ve since hired him 3x in different companies)
Me; “great thanks for your time and we will be in touch later today”
Co-interviewer; “I have one more question”
Candidate; “sure, go ahead”
Co-interviewer; “Imagine your uncle has just died and left you his pizzeria in his will, what would you do?”
Me; “what?”
Candidate; “what...? Errr... sell it because I’m a software engineer, not a pizza chef”
Good to see an actual change vs. SWEs that suck at LeetCode-style questions complain about these algorithmic questions...
It's ironic considering how MSFT pioneered the original brain teasers/problem solving questions.
It's ironic considering how MSFT pioneered the original brain teasers/problem solving questions.
The quality of interview depends on quality of questions. A qualified question is one which has many solutions, then the interviewee could pick the best solution on his own.
Has anyone looked at how g loaded these interview quizzes and brain teasers are?
You can afford to be selective when you have more candidates than open positions. Also you want to secure the top talent out there.
[deleted]
I wonder the effectiveness of the MS hiring process every time I use a Microsoft product. Teaching interviewees how to test their software locally for at least 10 mins before deploying it to hundreds of millions of people would be huge.
One of the key issues in recruiting is how to make hiring decisions quickly. Good candidates don't have the patience for your internal politics to resolve over a month or two to figure out whether they're getting an offer or not; they're just going to interview with other places in the meantime who will beat you to the punch.
Take a resume off the pile. Email or text to schedule a call. During the call, talk about relevant experience / past projects, what each side is looking for, etc. Make a decision during the call whether you want to invite the person for a two-hour interview, and schedule it during the call, first offering an evening time slot for the interview. In the first hour, talk about culture, problem-solving and teamwork approaches, engineering attitudes. In the second hour, pull out an actual current problem and solve it collaboratively. If all goes well, send an offer, and offer to set up a dinner to talk about / negotiate the offer.
It should not take more than a few hours to make a decision on someone; it should not require a candidate to take paid time off. Your hiring pipeline's average lead time should be measured in days.