Using Bazel to build a new Rust project(blog.fahhem.com)
blog.fahhem.com
Using Bazel to build a new Rust project
https://blog.fahhem.com/2024/09/bazel-build-new-rust-project/
2 コメント
Out of curiosity, why do people switch to bazel? I don't have much experience with it, so maybe I'm missing out?
To have the same build system for many languages with support for distributed artifacts caching and strong guarantees about build reproducibility if used correctly.
But yes, it’s like a lot of things whose need is born of scale. If you really needed it, it would be obvious to you and if you use it without needing it, it’s going to be painful and difficult to understand.
But yes, it’s like a lot of things whose need is born of scale. If you really needed it, it would be obvious to you and if you use it without needing it, it’s going to be painful and difficult to understand.
Large cross language mono repos is what it was built for and for that particular use case I don’t think there is anything better.
Everything outside of that use case however it probably isn’t the right choice currently but their stated goal is to eventually become the obvious choice in a lot of other scenarios too.
Everything outside of that use case however it probably isn’t the right choice currently but their stated goal is to eventually become the obvious choice in a lot of other scenarios too.
There are a few reasons why people use Blaze-inspired build systems (in addition to Bazel there are Buck and Buck2, Pants, Please, and others). Such build systems are sometimes called “action-cached”, which refers to a design in which the build is expressed in a way to maximize opportunities for caching based on the inputs to an action rather than the mtime like Make.
One big reason is speed: aggressively cached builds are generally much faster. This is very similar to how Turbo gets used in the JS/TS ecosystem, but for many more programming languages. Bazel in particular is very well-supported for transparent remote caching, so your build will often get faster rather than slower as the number of contributors goes up.
Another common reason is that in the presence of more than one or two programming languages, and particularly if one language is furnishing libraries to another, having a unified build rapidly makes it much more manageable to learn one build tool well rather than learn multiple build tools and some orchestration glue on top. A common example here would be the C/C++ extensions that are all but ubiquitous in high-performance numeric and scientific Python applications.
A third example is that C/C++ don’t really have a package manager in the sense that Rust has Cargo. CMake is probably the most popular these days, but it has its own problems and doesn’t interoperate well with other languages.
People seem to like making fun of Bazel et al, and I suppose that’s understandable: it’s difficult to learn and not worth learning until you have the problems it addresses. Add in the default animus for all things FAANG and it’s kind of an easy point to score. But this tendency is what Andre Alexandrescu calls “aggressively intermediate”, when senior people criticize Bazel they do so with valid concrete arguments (and such exist), it’s never “FAANG cargo cult” FUD.
Turbo is popular in the front end for a reason, an imprecise but still useful analogy is that Bazel is “Turbo for backend”.
One big reason is speed: aggressively cached builds are generally much faster. This is very similar to how Turbo gets used in the JS/TS ecosystem, but for many more programming languages. Bazel in particular is very well-supported for transparent remote caching, so your build will often get faster rather than slower as the number of contributors goes up.
Another common reason is that in the presence of more than one or two programming languages, and particularly if one language is furnishing libraries to another, having a unified build rapidly makes it much more manageable to learn one build tool well rather than learn multiple build tools and some orchestration glue on top. A common example here would be the C/C++ extensions that are all but ubiquitous in high-performance numeric and scientific Python applications.
A third example is that C/C++ don’t really have a package manager in the sense that Rust has Cargo. CMake is probably the most popular these days, but it has its own problems and doesn’t interoperate well with other languages.
People seem to like making fun of Bazel et al, and I suppose that’s understandable: it’s difficult to learn and not worth learning until you have the problems it addresses. Add in the default animus for all things FAANG and it’s kind of an easy point to score. But this tendency is what Andre Alexandrescu calls “aggressively intermediate”, when senior people criticize Bazel they do so with valid concrete arguments (and such exist), it’s never “FAANG cargo cult” FUD.
Turbo is popular in the front end for a reason, an imprecise but still useful analogy is that Bazel is “Turbo for backend”.
> Out of curiosity, why do people switch to bazel?
From what I've read, even for C++ projects Bazel is pushed by FANG-driven cargo cult mixed with FOMO.
Even for its internal use, Bazel sounds an awful lot like a promotion-oriented project.
From what I've read, even for C++ projects Bazel is pushed by FANG-driven cargo cult mixed with FOMO.
Even for its internal use, Bazel sounds an awful lot like a promotion-oriented project.
It has caused nothing but problems wherever I've seen it being used.
Yep. The only way to reap the benefits with bazel is to have somebody who continuously deals with the problems.
This is not that different from many other aspects of software engineering, but as on those other areas you need to run a cost-benefit analysis
This is not that different from many other aspects of software engineering, but as on those other areas you need to run a cost-benefit analysis
We've had great success in using Bazel for a large mixed C++/Python and some other languages monorepo.
The initial setup cost is somewhat high, but the caching aspects and configurability make it a real game changer in this space.
If you're building a Rust-only project it's probably better to stick with Cargo, since Cargo caches really well and doesn't have such a large setup cost.
The initial setup cost is somewhat high, but the caching aspects and configurability make it a real game changer in this space.
If you're building a Rust-only project it's probably better to stick with Cargo, since Cargo caches really well and doesn't have such a large setup cost.
Cargo caches well locally, but I had problems getting good performance on CI with cache save/restore.
OTOH bazel remote caching works quite well (didn't try with rust yet though)
OTOH bazel remote caching works quite well (didn't try with rust yet though)
>OTOH bazel remote caching works quite well (didn't try with rust yet though)
I don't think there's a reason it wouldn't work for that.
I don't think there's a reason it wouldn't work for that.
I mean if you're in the situation of the author where you already have a bazel project why add Cargo?
There's a lot of nice things about bazel like being able to embed optimized JS into an iOS app built from transpiled Java code. But like mdhb points out, if you're not doing a lot of stuff with multiple languages you are probably better off with just that language's recommended build tool.
There's a lot of nice things about bazel like being able to embed optimized JS into an iOS app built from transpiled Java code. But like mdhb points out, if you're not doing a lot of stuff with multiple languages you are probably better off with just that language's recommended build tool.
Is it possible to specify locked glibc and cc/linking toolchain in your Bazel project?
In my previous position, we accomplished this using a "dev docker" (which I covered generally in a previous post: https://blog.fahhem.com/2023/12/dev-onboarding-then-now/) that's based around a specific version of Ubuntu (20.04) so the glibc is tied to that.
In another project, I used a hermetic C++ toolchain (you can use gcc with https://github.com/f0rmiga/gcc-toolchain/ or clang/llvm with https://github.com/bazel-contrib/toolchains_llvm) that pulled in its own (locked) version of glibc and lld, etc.
In another project, I used a hermetic C++ toolchain (you can use gcc with https://github.com/f0rmiga/gcc-toolchain/ or clang/llvm with https://github.com/bazel-contrib/toolchains_llvm) that pulled in its own (locked) version of glibc and lld, etc.