Ask HN: What's The Most Expensive Mistake You've Made At Work?
3 comments
Not jumping from company to company has cost me a lot in both past and future earnings.
This !!!
I made the same mistake as junior, thinking that staying at same place for years will get me to somewhere, spend 4 years as junior thinking "ok, it will not be that year, next year i will be mid for sure", nope, changed jobs and found pretty quick that not only my title was unnecessary delayed, i was in fact so grossly underpaid that my next role payed me almost 400% more
I've never switched. Philly is a terrible area for tech jobs. Same company for 10 years. Just barely touching $100k now.
What would you say is the "perfect interval" to jump companies ?
The general wisdom is switching every two to two and a half years in order to maximize lifetime earnings.
I doubt there is one.
The most expensive mistake I made was not properly emphasizing and escalating how important it was to not simply migrate just one of my old kickstart files to automation and use it for everything and to retest the settings each time the OS, BIOS and hardware had changed. I said each role of server needed its own settings and to re-test them each time the OS or hardware changed. Nobody wanted to do this.
I had settings specific to CentOS5+Dell+BIOS versions that controlled CPU power settings. There was a specific need at time time for this. I did communicate this multiple times but I was shrugged off. CentOS 6 brought in a less old kernel that vastly improved intel power management and around the time CentOS 7 was being used the company had grown significantly and there was a team dedicated to doing analytics on power usage and cost. The old kernel settings that had never been updated were costing the company an extra $3 million in power usage per year.
I had settings specific to CentOS5+Dell+BIOS versions that controlled CPU power settings. There was a specific need at time time for this. I did communicate this multiple times but I was shrugged off. CentOS 6 brought in a less old kernel that vastly improved intel power management and around the time CentOS 7 was being used the company had grown significantly and there was a team dedicated to doing analytics on power usage and cost. The old kernel settings that had never been updated were costing the company an extra $3 million in power usage per year.
>costing the company an extra $3 million in power usage per year
at $0.15/kwh, that's ~2300 excess kilowatts used per hour (~$340/hour)
You're saying you had 25,000 Dell servers[0] of a specific CPU combo that happened to have a fluke with RHEL power management?
I suppose that's possible - but it seems highly improbable
Even if you did, it's not a "customize the kickstart" problem, it's "write a post-deployment config script" issue that is far simpler to manage
After working in data center management and automation for over a decade, I can say your case is the first I ever heard of such a thing - and I would've told all my customers what I told you - have a single kickstart per major release of RHEL/CentOS, and do a post-deploy config if necessary :)
----------
[0] https://news.ycombinator.com/item?id=32692023
at $0.15/kwh, that's ~2300 excess kilowatts used per hour (~$340/hour)
You're saying you had 25,000 Dell servers[0] of a specific CPU combo that happened to have a fluke with RHEL power management?
I suppose that's possible - but it seems highly improbable
Even if you did, it's not a "customize the kickstart" problem, it's "write a post-deployment config script" issue that is far simpler to manage
After working in data center management and automation for over a decade, I can say your case is the first I ever heard of such a thing - and I would've told all my customers what I told you - have a single kickstart per major release of RHEL/CentOS, and do a post-deploy config if necessary :)
----------
[0] https://news.ycombinator.com/item?id=32692023
Baseline kickstarts per major RHEL release make sense (and, every once in a very great while ... on the U releases)
But for every BIOS/CPU/etc change? No
That's a phenomenal waste of time and energy
Good enough is, as the saying goes, good enough
But for every BIOS/CPU/etc change? No
That's a phenomenal waste of time and energy
Good enough is, as the saying goes, good enough
Good enough is, as the saying goes, good enough
On CentOS 5.7 and later on Dell you would be correct. Prior to 5.7 there was a bug specific to cstate management and the Dell BIOS that required a work around. This was not optional for CPU latency specific workloads. Good enough was not good enough based on application benchmarks. The bug was worked around with multiple fixes in the 5.7/5.8 timeline. Dell also released a BIOS fix that worked around this as well around the time that CentOS 6 was released but we stayed clear of .0 releases and the BIOS fix had a dependency on the kernel released with C6.
On CentOS 5.7 and later on Dell you would be correct. Prior to 5.7 there was a bug specific to cstate management and the Dell BIOS that required a work around. This was not optional for CPU latency specific workloads. Good enough was not good enough based on application benchmarks. The bug was worked around with multiple fixes in the 5.7/5.8 timeline. Dell also released a BIOS fix that worked around this as well around the time that CentOS 6 was released but we stayed clear of .0 releases and the BIOS fix had a dependency on the kernel released with C6.
Accepting management position instead of focusing on my technical skills.
I ended up having some interesting experience but boy I would have been better off honing my tech skills instead.