Well, check out RWX! We don't do exactly everything like CirrusCI did -- most notably, we aren't open source and we are still managing all of our own infrastructure for running CI right now -- but we have a great "local" story and a validation CLI (`rwx lint`), and a lot of fundamental changes that make us better than other CI systems too :)
There's three main things we do to solve this, all of which relate to the fact that we have our own (OCI-compatible) container runtime under the hood instead of using Docker.
1. We don't gzip layers like Docker does. Gzip is really slow, and it's much slower than the network. Storage is cheap. So it's much faster to transmit uncompressed layers than to transmit compressed layers and decompress them.
2. We've heavily tuned our agents for pulling layers fast. Disk throughput and IOPS are really important so we provision those higher than you typically would for running workloads in the cloud. When pulling layers we modify kernel parameters like the dirty_ratio to values that we've empirically found with layer pulls. We make sure we completely exhaust our network bandwidth and throughput when pulling layers. And so on.
3. This third one is experimental and something we're actively working on improving, but we have our own underlying filesystem which lazily loads the files from a layer instead of pulling tons of (potentially unneeded) files up front. This is similar to AWS's [Seekable OCI](https://github.com/awslabs/soci-snapshotter) but tuned for our particular needs.
I've been slowly working on improving our documentation to explain these kinds of differentiators that our architecture and container runtime provide, but most of it is unpublished so far. We definitely need to do a much better job of explaining _how_ we are faster and better rather than just stating it :).
The other side of this is that we also made _building_ those layers much much faster. We blogged a little bit about it at https://www.rwx.com/blog/we-deleted-our-dockerfiles but just to hit some quick notes: in RWX you can vary the compute by task, and it turns out throwing a big machine at (e.g.) `npm install` is quite effective. Plus we make using an incremental cache very easy, and layers generated from an incremental cache are only the incremental parts, so they tend to be smaller. And we're a DAG, so you can parallelize your setup in a way that is very painful to do with Docker, even when using multi-stage builds. And our cache registry is global and very hard to mess up, whereas a lot of people misconfigure their Docker caches and have cache misses all over their docker builds. And we have miss-then-hit semantics for caching. Okay, I'm rambling now! But happy to go into more depth on any of this!
I hope the author will check out RWX -- they say they've checked out most CI systems, but I don't think they've tried us out yet. We have everything they praise Buildkite for, except for managing your own compute (and that's coming, soon!). But we also built our own container execution model with CI specifically in mind. We've seen one too many Buildkite pipelines that have a 10 minute Docker build up front (!) and then have to pull a huge docker container across 40 parallel steps, and the overhead is enormous.
This is one of the big problems we solved with the RWX CI platform (RWX.com). You can use ‘rwx run’ and it automatically syncs your local changes, so no need to push — and with our automated caching, steps like setting up the environment cache hit so you don’t have to execute the same stuff over and over again while writing your workflow. Plus with our API or MCP server you can get the results directly in your terminal so no need to open the UI at all unless you want to do some in-depth spelunking.
It's really concerning that the biggest, most eye-grabbing part of this posting is the note with the following: "It’s common for critical CVEs to uncover follow‑up vulnerabilities."
Trying to justify the CVE before fully explaining the scope of the CVE, who is affected, or how to mitigate it -- yikes.
I had the same experience -- it was way easier for me to find it with the rainbow highlighting (even though I already knew where to look when it came to the monochrome highlighting!).
The author later asks what color class definitions were. I think this fundamentally gets wrong how syntax highlighting helps humans. I don't have a clue what color anything is in my favored highlighting, but my brain does incredible pattern recognition to help me digest code in it without me consciously knowing what color does what.
So his arguments for why there's a problem don't hold up, but that doesn't mean there is not in fact a problem.
We use a somewhat similar approach in RWX when pulling LFS files[1]. We run `git lfs ls-files` to get a list of the lfs files, then pass that list into a task which pulls each file from the LFS endpoint using curl. Since in RWX the output of tasks are cached as long as their inputs don't change, the LFS files just stay in the RWX cache and are pulled from there on future clones in CI. In addition to saving on GitHub's LFS bandwidth costs, the RWX cache is also _much_ faster to restore from than `git lfs pull`.
Not on iOS, as I understand it. If you "Ask app not to track" on iOS then the app cannot access your IDFA, which was the ID that previously was used to track a device across apps.
Come check out RWX :). We have per second billing but I think it won't even matter for your use case because most of those checks are going to take 0s on RWX. And our UI is optimized for showing you what is wrong at a glance without having to look at logs at all.
It's pretty amazing to see what Blacksmith, Depot, Actuated, etc. have been able to build on top of GitHub Actions. At RWX we got a bit tired of constantly trying to work around the limitations of the platform with self-hosted runners, so we just built an entirely new CI platform on a brand new execution model with support for things like lightning-fast caching out of the box. Plus, there are some fundamental limitations that are impossible to work around, like the retry behavior [0]. Still, I have a huge appreciation for the patience of the Blacksmith team to actually dig in and improve what they can with GHA.
In my experience (and as reflected by the comments on this post already), trying to run complex CI workflow locally is a fairly hopeless enterprise. If you have a fully containerized workflow, you might be able to get close, but even then ensuring you have all of your CI specific environment variables is often not a trivial task, and if your workflow orchestrates things across tasks (e.g. one task uploads an artifact and another task uses that artifact) you'll have a hard time reproducing exactly what is happening in CI. My company (RWX) builds a GitHub Actions competitor and we intentionally do not support local execution -- instead we focused on making it easy to kick off remote builds from your local machine without having to do a `git push` and we made it easy to grab an SSH session before, during, or after a running task to inspect things in the exact build environment that your workflow is running.
> The orange remains in the living room. Moving the plate to the kitchen does not affect the location of the orange, since it was placed below the plate but not attached to it. Therefore, the orange stays where it was originally placed, which is in the living room.
You don't need to visualize it in your mind to understand the relationship between being _below_ and being _moved with_. Keep in mind that many people cannot visualize anything in their mind!
Isn't this exactly what Apple did with the M1 chips when they were first being released? Plus, Apple announced they would do developer kit applications for the Vision Pro at the same time they announced the Vision Pro, so they couldn't have based this decision on the traction with third party developers.
How would the DoS work? As I understand it the issue here only occurs if you try to push the entire project to a _new_ repo. It doesn't affect existing repos.
To be honest, I don’t find this particularly impressive. I didn’t read to the end, but the beginning parts of the game are littered with errors. Basically every command is followed by ChatGPT miscalculating the remaining resources and the player having to correct it. Could the player have just asserted they had different resources? I’d guess they could have. Another poster mentioned “good enough to be used” as a criterion, and this is far afield from that. It’s not good enough to be used for a game, much less any serious business practice.