I reduced (incremental) Rust compile times by up to 40%(coderemote.dev)
coderemote.dev
I reduced (incremental) Rust compile times by up to 40%
https://www.coderemote.dev/blog/faster-rust-compiler-macro-expansion-caching/
4 comments
I appreciate this detailed explanation! To preface, I took on macro expansion caching because it was a clear solution for the slow builds I saw in some projects (taking the "project-specific" approach I mentioned in the blog). That said, my medium-term vision for CodeRemote isn't to develop a superior closed-source Rust compiler fork; instead, it is to provide remote tooling for faster development (i.e. bring small dev teams' tooling up closer to Google's level):
(a) pre-configured build servers, so a developer doesn't need to be equipped with the latest Mac to be functional
(b) shared cache within a developer team
(c) potential avenues in CI/CD, testing, etc.
I'd love to hear your thoughts on this goal! Does some aspect address a development pain point of yours? I also want to address your points above, and I agree with lots of them:
1. I agree that especially in the wider Rust community, closed source is not a long-term solution. For what it's worth, I have been considering making the source available -- though likely not with a super copyleft license, at least initially. Do you think making the source available would alleviate people's safety concerns?
2. I have thought about the mold/sold linker quite a bit, especially because the author changed his license on sold from a commercial license to MIT. And defaults are indeed very powerful (this is discussed in a recent change to strip debug info by default, https://kobzol.github.io/rust/cargo/2024/01/23/making-rust-b...) because most people aren't bothered to (or don't know to) change them. Generally one expects the defaults to be the best for the general case, but this is decidedly not true for Rust compiler settings.
I think there may be a difference, though, between catering to the general public and catering to specific users/clients. For the folks where link time is truly atrocious, I assume they make the effort to use mold (though again, there is good reason to be skeptical given that sold switched to an MIT license). I assume a similar willingness if/when someone's macro expansions are slow.
3. You're right, this macro caching feature is not complex! I'm sure someone like bjorn3 could code it up rather quickly. It does me no good trying to solo out-perform the rustc experts. But I think there is a lack of people improving the Rust dev experience. That's where I want to operate.
(a) pre-configured build servers, so a developer doesn't need to be equipped with the latest Mac to be functional
(b) shared cache within a developer team
(c) potential avenues in CI/CD, testing, etc.
I'd love to hear your thoughts on this goal! Does some aspect address a development pain point of yours? I also want to address your points above, and I agree with lots of them:
1. I agree that especially in the wider Rust community, closed source is not a long-term solution. For what it's worth, I have been considering making the source available -- though likely not with a super copyleft license, at least initially. Do you think making the source available would alleviate people's safety concerns?
2. I have thought about the mold/sold linker quite a bit, especially because the author changed his license on sold from a commercial license to MIT. And defaults are indeed very powerful (this is discussed in a recent change to strip debug info by default, https://kobzol.github.io/rust/cargo/2024/01/23/making-rust-b...) because most people aren't bothered to (or don't know to) change them. Generally one expects the defaults to be the best for the general case, but this is decidedly not true for Rust compiler settings.
I think there may be a difference, though, between catering to the general public and catering to specific users/clients. For the folks where link time is truly atrocious, I assume they make the effort to use mold (though again, there is good reason to be skeptical given that sold switched to an MIT license). I assume a similar willingness if/when someone's macro expansions are slow.
3. You're right, this macro caching feature is not complex! I'm sure someone like bjorn3 could code it up rather quickly. It does me no good trying to solo out-perform the rustc experts. But I think there is a lack of people improving the Rust dev experience. That's where I want to operate.
The Rust bhild config defaults are pretty fine for the general case. It's just that not everyone has the general case :) In Rust the normal distribution will be quite flattened.
Ha yes that is a diplomatic way to put it. To the commenter's point though, I too question some of the defaults. mold does seem to be objectively better than the default linker. Stripping debuginfo does seem like it would have been a better default, which is why it was made so recently! Pipelined compilation also falls into this latter category, so perhaps because there is (understandably) just a delay until adoption to stable Rust.
I know you mean it as a figure of speech, but I would consider the complexity (and build time) distribution for Rust to be heavy-tailed and skewed right, more so than a flattened normal.
I know you mean it as a figure of speech, but I would consider the complexity (and build time) distribution for Rust to be heavy-tailed and skewed right, more so than a flattened normal.
> Does some aspect address a development pain point of yours?
I've worked in a company with a large codebase, but we did all development on the server. Devex teams automatically set up artefact caching so builds would be fast. I guess this is what you're trying to replicate. I think others have tried it as well. Maybe it would be worth reaching out to Josh Triplett (https://github.com/joshtriplett) who was working on something similar.
In my current company (using Go) or personal projects (using Rust) I don't feel this pain at all. For the codebases I work on, Apple's M series chips are great.
> source available
I don't think this would help. Ultimately, it's a trust issue. Most people, me included, are used to getting Rust through rustup or their package manager. Downloading and using a fork is too much friction, and any security expert will strongly advise against it. Using a compiler binary built by someone else requires a lot of trust.
Being on a fork has other issues - there might be subtle issues like miscompilations that only I experience, instead of benefiting from the bug reports filed by the whole community. The compiler built by the Rust team is tested against every open source crate, there's no way a fork could do that kind of testing. Speed is another issue - 60% of all Rust developers are on the latest stable release (lib.rs/stats), so a majority upgrades asap. Waiting for a rebase and a release is just too late for all these people.
If anyone could have succeeded with a fork, it is the Sealed Rust initiative by Ferrous Systems. They were targeting customers who prefer to stick with one version of a toolchain for years. It would have been technically feasible for them to fork, add their special sauce and pass it on. But even they decided against it, upstreaming all their changes. I think it comes down to the work of maintaining a fork in perpetuity and the distrust of closed source.
> I assume a similar willingness if/when someone's macro expansions are slow.
I think by this time next year most users will see a 30-40% increase in compilation perf thanks to parallel frontend, cranelift and switch to lld. I'm sure no one would say no to additional speedups, but they won't be desperate for increased performance.
> there is a lack of people improving the Rust dev experience
I think the number of people working on the three projects I mentioned, GCC backend, GCC frontend, cargo improvements (like the strip debug info change you mentioned) - that's a lot of improvement. This is the general stuff, so I'm not counting the specific work like making Rust better for embedded or Linux kernel development. Seems like a lot of good work happening in parallel.
I don't want to discourage you, but I'll leave you with this - even if you found enough people to adopt your forked changes (source available or not) and pay for it, that would just be a signal to the open source project that its a really good idea to clean room reimplement your changes. Competing with an incumbent is hard, competing with an open source incumbent when you're not open source - I've never seen it done successfully.
As a Rust user, I'd love to see you contribute this back to Rust and for Rust to become better. But I also understand your need to make a living. Best of luck.
I've worked in a company with a large codebase, but we did all development on the server. Devex teams automatically set up artefact caching so builds would be fast. I guess this is what you're trying to replicate. I think others have tried it as well. Maybe it would be worth reaching out to Josh Triplett (https://github.com/joshtriplett) who was working on something similar.
In my current company (using Go) or personal projects (using Rust) I don't feel this pain at all. For the codebases I work on, Apple's M series chips are great.
> source available
I don't think this would help. Ultimately, it's a trust issue. Most people, me included, are used to getting Rust through rustup or their package manager. Downloading and using a fork is too much friction, and any security expert will strongly advise against it. Using a compiler binary built by someone else requires a lot of trust.
Being on a fork has other issues - there might be subtle issues like miscompilations that only I experience, instead of benefiting from the bug reports filed by the whole community. The compiler built by the Rust team is tested against every open source crate, there's no way a fork could do that kind of testing. Speed is another issue - 60% of all Rust developers are on the latest stable release (lib.rs/stats), so a majority upgrades asap. Waiting for a rebase and a release is just too late for all these people.
If anyone could have succeeded with a fork, it is the Sealed Rust initiative by Ferrous Systems. They were targeting customers who prefer to stick with one version of a toolchain for years. It would have been technically feasible for them to fork, add their special sauce and pass it on. But even they decided against it, upstreaming all their changes. I think it comes down to the work of maintaining a fork in perpetuity and the distrust of closed source.
> I assume a similar willingness if/when someone's macro expansions are slow.
I think by this time next year most users will see a 30-40% increase in compilation perf thanks to parallel frontend, cranelift and switch to lld. I'm sure no one would say no to additional speedups, but they won't be desperate for increased performance.
> there is a lack of people improving the Rust dev experience
I think the number of people working on the three projects I mentioned, GCC backend, GCC frontend, cargo improvements (like the strip debug info change you mentioned) - that's a lot of improvement. This is the general stuff, so I'm not counting the specific work like making Rust better for embedded or Linux kernel development. Seems like a lot of good work happening in parallel.
I don't want to discourage you, but I'll leave you with this - even if you found enough people to adopt your forked changes (source available or not) and pay for it, that would just be a signal to the open source project that its a really good idea to clean room reimplement your changes. Competing with an incumbent is hard, competing with an open source incumbent when you're not open source - I've never seen it done successfully.
As a Rust user, I'd love to see you contribute this back to Rust and for Rust to become better. But I also understand your need to make a living. Best of luck.
> Ultimately, it's a trust issue. Most people, me included, are used to getting Rust through rustup or their package manager.
$corporate usually has a lot less trust issues once there's a contractual relationship with the proprietary provider. But that hurdle also means it's more difficult to get a foot in the door, especially when an open alternative exists.
$corporate usually has a lot less trust issues once there's a contractual relationship with the proprietary provider. But that hurdle also means it's more difficult to get a foot in the door, especially when an open alternative exists.
I’m not sending you my code to compile on your machine.
I understand wanting to make a successful product, and keepin your secret magic secure, but I’m afraid this just sets off so many alarm bells for me.
I understand wanting to make a successful product, and keepin your secret magic secure, but I’m afraid this just sets off so many alarm bells for me.
Thanks for raising this. I hadn't anticipated this as a major concern.
Is your main concern that you can't see the compiler code -- in which case would it help if the the source were available?
Or is it even more than that, that you want to ensure that your build outputs are untampered and verifiably produced by a (deterministic) compiler?
I wonder though, would you trust remote CI/CD servers or Intel's proprietary C++ compiler (https://news.ycombinator.com/item?id=12363973)?
I wonder though, would you trust remote CI/CD servers or Intel's proprietary C++ compiler (https://news.ycombinator.com/item?id=12363973)?
Often during early development I don’t want to share my code with others. Obviously a compiler on my machine could make a secret internet connection, but I guess that would be noticed, and certainly couldn’t be later denied.
Can I be at all sure you aren’t keeping a copy of all source you are ever submitted?
Usually I eventually submit to GitHub, then CI occurs, and all is public. However, only getting fast building on CI doesn’t motivate me that much, I don’t care if I burn GitHub/Microsoft’s CPU cycles as much as I mind about my own :)
Can I be at all sure you aren’t keeping a copy of all source you are ever submitted?
Usually I eventually submit to GitHub, then CI occurs, and all is public. However, only getting fast building on CI doesn’t motivate me that much, I don’t care if I burn GitHub/Microsoft’s CPU cycles as much as I mind about my own :)
Gotcha, I respect that. With a few privacy-minded devs like you, I have offered them a custom compiler toolchain (and happy to make the source available) that they can "self-host". If you are interested in trying it out, I'd be happy to get you set up.
Thanks for the offer. I would mention it somewhere on your webpage, just so people (like me) don't get worried. Right now I'm too busy to try a new compiler, but if I find myself with a little more time, and a little longer rust compile times, in the future, I will remember to look you up!
Hi HN, I’m Kapil from CodeRemote. I am building Rust developer productivity tooling to speed up your edit-compile-run cycle. While I love Rust for its runtime performance and safety (not to mention its sweet syntactic sugar), I have long been frustrated with the resultant compile times.
This frustration hit a tipping point a few months ago, as I was building on a sluggish framework that made heavy use of code generation. From the Rust/Cargo/Rustc books and Rust community’s blog posts, I picked up and applied most of the standard tricks to speed up builds — but with only marginal improvement. Continued frustration led to me profiling and analyzing compile times, and soon I landed at the Rustc (compiler) source code…
I modified the compiler and implemented procedural macro caching, which has considerably sped up incremental builds for projects with macro-heavy dependencies. My fork of the compiler is still private as I’m still thinking through the licensing model; but if you’re interested in trying it out, please hit the “sign up” at the bottom of the blog. I am packaging it into a build/development server for a few Rust dev teams, and have already shared it with a few self-hosters! In this blog post, I discuss in detail my macro caching implementation and results. I’d love to hear your thoughts, and I’m happy to talk further about my compiler modifications.
This frustration hit a tipping point a few months ago, as I was building on a sluggish framework that made heavy use of code generation. From the Rust/Cargo/Rustc books and Rust community’s blog posts, I picked up and applied most of the standard tricks to speed up builds — but with only marginal improvement. Continued frustration led to me profiling and analyzing compile times, and soon I landed at the Rustc (compiler) source code…
I modified the compiler and implemented procedural macro caching, which has considerably sped up incremental builds for projects with macro-heavy dependencies. My fork of the compiler is still private as I’m still thinking through the licensing model; but if you’re interested in trying it out, please hit the “sign up” at the bottom of the blog. I am packaging it into a build/development server for a few Rust dev teams, and have already shared it with a few self-hosters! In this blog post, I discuss in detail my macro caching implementation and results. I’d love to hear your thoughts, and I’m happy to talk further about my compiler modifications.
This directly addresses one of the core gripes about Rust - long compile times.
Very nice to see efforts to address exactly that.
Would be quite interested to learn how this was achieved.
Would be quite interested to learn how this was achieved.
Thanks! I go over some of the details of my procedural macro caching solution in the linked blog, and my code snippet there might be most informative.
- The community has an instinctive distrust of closed source or a compiler from an untrusted source. If you’re familiar with the Trusting Trust attack you’ll understand why. People won’t send their code to you to compile, because they’re afraid of it being modified. People won’t use a closed source compiler, when an open source trusted one is available.
- Dev tools in every language ecosystem are almost always free, but sometimes even free tools don’t get adopted. Look the experience of the guy who built the mold linker(https://github.com/rui314/mold). Far superior to the state of art, improves incremental compiles a lot, widely applicable across ecosystems (C, C++, Rust), CPU architectures and Operating Systems. You don’t even have to modify your compiler, just need to point to his linker. He’s even giving it away for free for personal use. But still, almost no one uses it. The inertia of the established options is really high.
- It’s not complex enough. Think about the complexity involved in the cranelift backend. No one can seriously recreate the efforts of bjorn3. If we could have, we would have. But the idea idea here can be recreated, especially by the experts who already built incremental compilation into rustc.
- But if your solution is truly complex, like the parallel frontend, the burden of maintaining a fork would be too high. You’d have to spend all your time rebasing.
Again I’m not trying to discourage you, just stating the difficulties of making a business in the dev tools space. You would be better off contributing this excellent work to the community and trying a different tack.