Why do think so? Databricks notebook product I have used in couple of companies is pretty solid. I have done any google research but they are generally known to be very high talent dense kind of place to work.
This is good info thanks. I have some cloud Infra experience so I am interesting in knowing how does they keep the data stored and remove the "query" servers when not in use.
Possibly some kind of EBS equivalent storage which is attached to the VM when it's booted up? I wonder that creates more failures at the cost of operational simplicity?
To be absolutely clear, comp is not based on stock appreciation. You get your grant (aka no. of stocks/RSUs) that vest at some cadence. By the time you vest a tranche and share price has increased is an added bonus (though generally people in tech think 4x in 4 years is their birth right.)
As for Netflix, they do give you option of choosing to split your salary every year i.e all cash, all options, some cash and some options as you see fit.
They have apparently hired a ton of great k8 folks who have been tasked with building an internal layer of sorts for themselves so I guess they are moving off it (though not anytime soon).
Right now there is no specific distinction between what we want to run in Cloud vs On Prem. Important thing to note here is we use Cloud as an IaaS only. We have our own stack which sort of prepares the hosts before it is ingested into clusters as usable capacity.
We actually recommend not using custom cloud providers Databases or any other value added services.
Why not completely either way (on prem vs cloud) is something that happened way before I joined the group but I think the main reason is to have a tactical edge in the long run such that we avoid lock in. I guess in some ways it helps us negotiate pricing better.
Imagine moving a certain workload from GCP region to an AWS region as part of a failover drill.
From what I have seen it is actually of the same order or sometimes more. In one of the region/zone we add few hundreds hosts every week but that is after telling them we plan to upscale this in this region upto some big X number.
Instant Elasticity in Cloud is a myth. If you think you are going to get 1k hosts just like that from AWS you will have an unpleasant experience.
I work at one of the decent size tech company and we are split between cloud and on prem. From our experience you have to inform AWS/GCP in advance (sometime way early) if you are looking to meaningfully increase capacity in zone/region.
Sure, auto scaling few hundreds of hosts may be possible but people who run a service which needs few hundreds of hosts run it directly on AWS, they will run it some kind of scheduler+resource manager which will have some kind of operational buffer anyway (as in you would already have those hosts so cloud elasticity is not a factor here).