It's been a while since I thought about this but isn't the reason providers advertise only 3.2tbps because that's the limit of a single node's connection to the IB network? DGX is spec'ed to pair each H100 with a Connect-X 7 NIC and those cap out at 400gbps. 8 gpus * 400gbps / gpu = 3.2tbps.
Quiz 2 is confusingly worded but is, iiuc, referring to intranode GPU connections rather than internode networking.
I had this problem and complained to AT&T support. I know it doesn't make sense to replace the modem because this is a software, not hardware, issue but they very quickly offered to replace the modem and now my DNS isn't getting hijacked. Would recommend trying it!
This is both a good thing and a bad thing, but Pulumi is way more flexible than Terraform. I wanted to have a cloud-provider-specific submodule that created resources (like EKS and GKE) that then exported output values (think kubeconfig), and then I wanted the parent module to pass those in as inputs to a cloud-provider-independent submodule. Terraform couldn't do it without needing to duplicate a ton of code, or without something heinous like Terragrunt (not sure it even would have worked.) Pulumi makes it trivial and in a language I like writing.
Additionally, our applications consume our cloud configuration (eg something that launches pods on heterogenous GPUs needs to know which clusters support which GPUs, our colo cluster has H100s but our Google cluster has A100s etc.) Writing in the same language in the same monorepo makes it very easy to share that state.
I found that using "helm template" to convert every Helm chart into yaml, and then using Pulumi to track changes and update my clusters (with Python transformation functions to get per-cluster configuration) made my life so much better than using Helm. Watching Pulumi or Terraform watch Helm watch Kubernetes update a deployment felt pointlessly complicated.
It's worth noting that just because an H100 has a higher flops number doesn't mean your program is actually hitting that number of flops. Modern TPUs are surprisingly competitive with Nvidia on a perf/$ metric, if you're doing cloud ML they are absolutely worth a look. We have been keeping costs down by racking our own GPUs but TPUs are so cost effective that we need to do some thinking about changing our approach.
I'm not certain but I think part of this is that XLA (for example) is a mountain of chip-specific optimizations between your code and the actual operations. So comparing your throughput between GPU and TPU is not just flops-to-flops.
Ahh this is a good idea, my friend in fact has been asking me to add it forever. I've been hesitating since the data seems so sparse but I think you're both right. Thank you for the feedback!
Myself and a friend have been working on this for quite a while. It's frustrating because it has come so far (showing trail data worldwide interactively on a tiny budget is challenging), and yet it is so far from being something super useful like AllTrails due to low data quality and a lack of relevant features (photos, reviews, etc etc.)
Because they've limited the dynamism of Python so you can't do as much crazy runtime nonsense, their (vaporware) tech stack is able to (theoretically) compile more of your program into GPU kernels.
I built a cross of React + Wiz (a fantastic frontend framework at Google): https://github.com/aschleck/trailcatalog/tree/main/js/corgi . Totally irresponsible and probably full of bugs, but I was so tired of writing business logic in the same place as my view logic with React and now I'm free of it!
Atomic AI is fusing cutting-edge machine learning and structural biology to unlock RNA drug discovery.
We're looking for sharp and thoughtful engineers excited to help us design therapeutics to treat untreatable disease. There are two positions currently open.
* Senior Software Engineer, Data - design and build solutions with our in-house experimental biology team to analyze, store, and serve experimental data and enable new analysis workflows (https://boards.greenhouse.io/atomai/jobs/4726839004)
I wrote this post to try and give candidates more insight into what it feels like to be a technical coding interviewer, what we're looking for, and in the hope that more candidates might see the interview as a conversation (versus simply a written test.) I'd be very curious to hear what others think.
(chiming in here as a founding engineer at Atomic)
So I spent more than 8 years as a SWE at Google, and now work here with both experimental biologists and machine learning scientists. And yes, a lot of the concerns mentioned in this thread are also things I have had anxiety about.
Most obvious to me, being a software engineer at Google felt like being the center of the universe. Coming here, the focus is the scientific research. And yes, the scientists all managed to complete their PhDs so they don't necessarily need me to unblock them every second of their day. But contrary to my expectations, this has been remarkably freeing. I think one particularly important part of our company that makes this work is that, even on the science side, we're multidisciplinary (at a high level, emphasizing both experimental biology and ML.) And so engineering feeling like another arm of that multi-discipline nature is fairly... natural.
The reason I feel it's freeing, and the reason I enjoy working here, is also the greatest challenge. Because the scientists are focused on the science, because they respect me and trust me to figure it out, and because they aren't constantly blocked by me, my job is mostly about dreaming extremely expansively about what I can do to reduce toil and make the scientists more productive. Of course they have feedback and input, but how I use my time and what I build is ultimately my decision because I am the engineer. And I have been able to do some things I am very proud of, like rolling out Bazel and Kubernetes and finding ways to seamlessly bring them into the cloud (we're even multi-cloud now without them even noticing!) On the other hand, it's very challenging because when you work on a product, say Google Photos, as a SWE, you always have some direct tether to the product ("what should we build next? ahhhh, well I guess we could just embed stable difficusion and a million people would immediately play with it".) At Atomic, my tether is very ambiguous. If I do my job successfully, they'll be able to do research more quickly (? effectively?), and eventually we'll be able to produce a therapeutic that hopefully changes the world. Identifying what I can do today to speed up that far outcome in the future is very challenging, but it is a far more interesting challenge than gluing some pre-existing software into my UI or running A/B tests to turn a red button blue.
If, like me, you enjoy being given ownership over incredibly ambiguous problems, please do reach out!
This is sort of a confusing article because it assumes the premise of "you have a fixed hardware profile" and then argues within that context ("Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers".) Of course if you're getting 100% utilization then you'll find better raw pricing (and this article conveniently leaves out staffing costs), but this model misses one of the most powerful parts of cloud providers: autoscaling. Why would you want to waste scientist time by making them wait in a queue when you can just instead autoscale as high as needed? Giving scientists a tight iteration loop will likely be the biggest cost reduction and also the biggest benefit. And if you're doing that on prem then you need to provision for the peak load, which drives your utilization down and makes on prem far less cost effective.
The permissions management (user/group but with cloud roles) is what I miss most. NFS style approaches assume that I will give every user an unprivileged user on their local machine and deny root access so they can't just create a user with a different ID. S3 style ACLs per-object are just very confusing and no one is going to set those. S3 access policies per role are obviously not a solution because that requires centralizing lists of what the permissions are. I just want to be able to assign a chmod and a chown on a file for a cloud user and group... And don't forget about how fantastic the group tooling was.
I also miss being able to write files with /ttl=72h/ anywhere in the path and knowing they will be cleaned up automatically without me doing any more work. In contrast S3 just rots. Need to find an intern to write an SQS pipeline to schedule file deletions :)