Absolutely - there are plenty of more cost-effective, cloud-agnostic ways to build something like this. This is just an experiment to explore Lambda networking and push it beyond its intended use cases.
Like most things with AWS, it’s tricky to pin down exactly - and it’ll probably cost more than you’d expect once you factor in Lambda invocations, GB-seconds of execution time, S3 request fees, and especially data‐transfer charges.
I did toy around with Tailscale initially trying to get it to spin up as an exit node but wasn't able to get that functional. I did manage to get Tailscale Funnel to work as the tunnel mechanism to Lambda, but unfortunately the performance was really poor.
I investigated EBS snapshot as an option, but there were two problems. 1) cost as i mentioned initially - for just AOSP source tree alone you are looking at > 250GB and at a cost of $0.05 per GB you are already at > $10/month and 2) EBS snapshots lazy load from S3 which gives TERRIBLE performance which means you end up with far far slower builds. AWS released a feature "EBS Fast Snapshot Restore" to workaround this issue, but it's extremely expensive.
In order to limit costs, everything is pulled from source on each build and there is nothing cached. This strategy takes advantage of the fact that AWS doesn't charge for ingress traffic and unfortunately puts additional load on Google's servers. I've attempted a few different strategies on caching AOSP and Chromium source trees, but since you have to incur the storage costs on an ongoing basis, it's just not very economical.
Yes, building AOSP requires a fairly powerful machine (at least to do it quickly): https://source.android.com/setup/build/requirements. It's definitely possible to do on a local machine with decent specs though.
With rattlesnakeos-stack, it uses spot instances and defaults to a c5.4xlarge which takes about 7-8 hours to build AOSP and Chromium (for an up to date webview) and equates to about ~$1 a build. I typically build on a c5.24xlarge instance which takes about 2.5 hours and costs about ~$2 per build. Unfortunately both AOSP and Chromium are massive projects that require a ton of computing power to build quickly. It's definitely still possible to do on less powerful machines, but it's just going to take a lot longer to do builds.
CalyxOS is an awesome project. I have worked with the lead developer a bit over the past few years and it's been such a pleasure. We share some bits of code between our projects here: https://github.com/AOSPAlliance.
If anyone is interested in building their own custom android OS in the cloud (AWS) with same ability to lock your bootloader like CalyxOS, you can checkout my project I've been maintaining for a few years now called RattlesnakeOS: https://github.com/dan-v/rattlesnakeos-stack.
And if you prefer to not build in the cloud, there is also a really great project called robotnix (https://github.com/danielfullmer/robotnix) which provides a way to build many flavors of OS (AOSP, GrapheneOS, LineageOS, etc).
For anyone interested in doing their own customizable builds of AOSP for Pixel devices check out: https://github.com/dan-v/rattlesnakeos-stack. This doesn't have any of the security hardening features of GrapheneOS, but does have some of the same security properties like verified boot, OTA software updates that included updated firmware/drivers, support for remote attestation, etc.
Agreed. For anyone interested in doing their own monthly signed AOSP builds for Pixel phones with OTA updates, take a look at a project that I built that fully automates the process in AWS: https://github.com/dan-v/rattlesnakeos-stack.
For those that prefer algo, there is an open issue to add Pi-Hole (https://github.com/trailofbits/algo/issues/1258) and in my fork I've added Pi-Hole support: https://github.com/dan-v/algo. It's a really nice setup especially on mobile devices where you typically have to choose between using a VPN or using adblock as they typically require a local VPN hack to function.
If you are looking to maintain a secure device similar to stock Android without google services, a Pixel phone with verified boot using your own signing keys, latest AOSP, and up to date drivers/firmware is a far better option. I've been working on a project that automates the entire process in AWS and it supports all Pixel phones now: https://github.com/dan-v/rattlesnakeos-stack.
For sure. I'd love to have this script be able to run standalone as a local build process that would be portable across platforms and only run the AWS specific bits if you passed a flag for example. Anyways, happy to chat - you can shoot me an email [email protected]
Right now if a spot instance terminates mid-build it doesn't attempt to save any progress. I do have an open item to at least alert users that this has happened though (https://github.com/dan-v/rattlesnakeos-stack/issues/41). Although if Chromium is built it will checkpoint that by saving the built artifact to S3 and would then skip the Chromium build next time. I've found it's much cheaper to do full builds on AWS each time rather than trying to store the source tree for AOSP and Chromium as they are just so large.
If you were looking to just do something similar but running on another platform, knowledge of Go isn't required. The core build process is a shell script, which just has some variables injected through Go templates: https://raw.githubusercontent.com/dan-v/rattlesnakeos-stack/...
I actually recently added a feature to search for the cheapest spot instance price across a number of different regions. Right now for example the cheapest price for the instance type used c5.4xlarge is ~$0.15/hour in us-east-2. This equates to a full AOSP build with Chromium build included (~5.25 hours) costing $0.80 and without Chromium (~1.75 hours) costing $0.30