Gotcha, I respect that. With a few privacy-minded devs like you, I have offered them a custom compiler toolchain (and happy to make the source available) that they can "self-host". If you are interested in trying it out, I'd be happy to get you set up.
Ha yes that is a diplomatic way to put it. To the commenter's point though, I too question some of the defaults. mold does seem to be objectively better than the default linker. Stripping debuginfo does seem like it would have been a better default, which is why it was made so recently! Pipelined compilation also falls into this latter category, so perhaps because there is (understandably) just a delay until adoption to stable Rust.
I know you mean it as a figure of speech, but I would consider the complexity (and build time) distribution for Rust to be heavy-tailed and skewed right, more so than a flattened normal.
I appreciate this detailed explanation! To preface, I took on macro expansion caching because it was a clear solution for the slow builds I saw in some projects (taking the "project-specific" approach I mentioned in the blog). That said, my medium-term vision for CodeRemote isn't to develop a superior closed-source Rust compiler fork; instead, it is to provide remote tooling for faster development (i.e. bring small dev teams' tooling up closer to Google's level):
(a) pre-configured build servers, so a developer doesn't need to be equipped with the latest Mac to be functional
(b) shared cache within a developer team
(c) potential avenues in CI/CD, testing, etc.
I'd love to hear your thoughts on this goal! Does some aspect address a development pain point of yours? I also want to address your points above, and I agree with lots of them:
1. I agree that especially in the wider Rust community, closed source is not a long-term solution. For what it's worth, I have been considering making the source available -- though likely not with a super copyleft license, at least initially. Do you think making the source available would alleviate people's safety concerns?
2. I have thought about the mold/sold linker quite a bit, especially because the author changed his license on sold from a commercial license to MIT. And defaults are indeed very powerful (this is discussed in a recent change to strip debug info by default, https://kobzol.github.io/rust/cargo/2024/01/23/making-rust-b...) because most people aren't bothered to (or don't know to) change them. Generally one expects the defaults to be the best for the general case, but this is decidedly not true for Rust compiler settings.
I think there may be a difference, though, between catering to the general public and catering to specific users/clients. For the folks where link time is truly atrocious, I assume they make the effort to use mold (though again, there is good reason to be skeptical given that sold switched to an MIT license). I assume a similar willingness if/when someone's macro expansions are slow.
3. You're right, this macro caching feature is not complex! I'm sure someone like bjorn3 could code it up rather quickly. It does me no good trying to solo out-perform the rustc experts. But I think there is a lack of people improving the Rust dev experience. That's where I want to operate.
Thanks for raising this. I hadn't anticipated this as a major concern.
Is your main concern that you can't see the compiler code -- in which case would it help if the the source were available?
Or is it even more than that, that you want to ensure that your build outputs are untampered and verifiably produced by a (deterministic) compiler?
I like the simplicity. I definitely get the payoff for standalone Python scripts, where once the script errors out the memory is cleared.
But do you see a similar payoff for Jupyter notebooks (or similar)?
But usually, at least I don't build an executable while iterating. And if I do, I try to set up the feature flags so the build is minimal for the tests I am running
This is most definitely self-promotion (I am the author), but also extremely relevant if you are looking at incremental build speed-ups: https://news.ycombinator.com/item?id=39743431
Agree, especially when developing, I'd assume speed of optimizer matters more than quality of optimization. I wonder though if LLVM spent less time on that "phase ordering" problem, what is the tradeoff between these two factors?
Hi HN, I’m Kapil from CodeRemote. I am building Rust developer productivity tooling to speed up your edit-compile-run cycle. While I love Rust for its runtime performance and safety (not to mention its sweet syntactic sugar), I have long been frustrated with the resultant compile times.
This frustration hit a tipping point a few months ago, as I was building on a sluggish framework that made heavy use of code generation. From the Rust/Cargo/Rustc books and Rust community’s blog posts, I picked up and applied most of the standard tricks to speed up builds — but with only marginal improvement. Continued frustration led to me profiling and analyzing compile times, and soon I landed at the Rustc (compiler) source code…
I modified the compiler and implemented procedural macro caching, which has considerably sped up incremental builds for projects with macro-heavy dependencies. My fork of the compiler is still private as I’m still thinking through the licensing model; but if you’re interested in trying it out, please hit the “sign up” at the bottom of the blog. I am packaging it into a build/development server for a few Rust dev teams, and have already shared it with a few self-hosters! In this blog post, I discuss in detail my macro caching implementation and results. I’d love to hear your thoughts, and I’m happy to talk further about my compiler modifications.
If you are in a corporate environment, then Jetbrains products are probably decent. What language are you developing in?
Also not free but Sublime with SFTP (if that's still something people use) is an option