To massively increase the reliability to get GPUs, you can use something like SkyPilot (https://github.com/skypilot-org/skypilot) to fall back across regions, clouds, or GPU choices. E.g.,
$ sky launch --gpus H100
will fall back across GCP regions, AWS, your clusters, etc. There are options to say try either H100 or H200 or A100 or <insert>.
Essentially the way you deal with it is to increase the infra search space.
From what I know this idea underpins a few FAANG-level companies' data transfer systems. OP's value = a simple implementation of the idea that's OSS and applied to AI.
> Its revenue run rate has spiked this year and now sits at around $30 million to $50 million, three sources said — with one noting that it had more that tripled compared to the start of the year.
You could finetune then add on a retrieval step, which has the advantage of citing sources. Jury's probably still out on which, or which combination of, methods work best. Likely use case and data size-dependent.
> The training was done with PyTorch FSDP on 8 A100 GPUs in one day.
> We employ SkyPilot managed spot to reduce the cost by leveraging the cheaper spot instances with auto-recovery for preemptions and auto zone switch. This solution slashes costs for training the 7B model from $500 to around $140 and the 13B model from around $1K to $300.
So, this is using for example a2-ultragpu-8g (8x A100-80GB) on GCP using spot instances. You can use SkyPilot to quickly see the price is $12.8 per hour (~$307 for a day):
» sky launch --gpus A100-80GB:8 --use-spot
Check out detailed CLI instructions and SkyPilot YAMLs here if you want to give it a try:
We don't anticipate this will happen for a few reasons. When the usage of SkyPilot (or a SkyPilot-like "intercloud broker" system) is small, it probably doesn't warrant the dominant clouds' attention.
When the usage gets bigger, I'm not sure how providers can restrict access anyway (curious if there are precedents). There are quite a few large multicloud platforms like Snowflake or Databricks heavily utilizing AWS/GCP/Azure already. (Granted, these platforms are not meta-cloud, in the sense of moving their customer workloads transparently across clouds.)
Ultimately we see such a system to grow the pie for the whole cloud market. The incumbents' relative shares may drop, but their absolute volume will grow.