One thing that I always see in these threads is how these interviews are not really a good indicator. I work for a company that does these interviews, and most of us acknowledge that the interviews result in sizable false negatives, but the ones that do come through the process are usually very competent ones.
Also, complexity analysis/algorithm questions that are asked in the interview setting is not rocket science...
"Let’s imagine we perform 100 tests on a website and, by running each test for 2 months, we have a large enough sample to achieve 80% power. 10 out of our 100 variants will be truly effective and we expect to detect 80%, or 8, of these true effects.
If we use a p-value cutoff of 5% we also expect to see 5 false positives. So, on average, we will see 8+5 = 13 winning results from 100 A/B tests."
If we expect 10 truly effective tests and 5 false positives, we'd have 15 tests that rejected the null hypothesis of h_0=h_test. Taking power into account, shouldn't we see 15*0.8, 12 winning results from the results? I.e. wouldn't one of the false positives also have not-enough-power?
Every now and then, these posts come up on HN people mention how bad the interview process at certain companies is and how it should be changed.
I also work for a company that does these types of interviews. While I wholeheartedly agree that asking these algorithms/data structures or other CS fundamental questions is not perfect, precisely because it may induce quite a bit of false negatives, the engineers who does end up successfully completing the process at my company are very competent ones. So from an anecdotal perspective, the process works fairly well in the fact that it does not produce too many false positives.
Also, complexity analysis/algorithm questions that are asked in the interview setting is not rocket science...