> Offline is very hard. A lot of bugs happened due to trying to make Horizon v1 work with flaky or very slow connections, and losing transactional consistency as a result. The SOTA here has barely advanced since the 90s, instead the industry has just given up on this and now accepts that every so often there'll be massive outages that cause parts of the economy to just shut down for a few hours when the SPOF fails. Should there be more focus on how to handle flaky connectivity in mission-critical apps safely?
If there's a network partition you have two options: accept reduced availability and keep your consistency, or have better availability and have reduce node consistency. Not much else you can do, that's just life.
Obviously, ways to increase consistency with consensus algorithms etc with 2 phase commits, and you reduce consistency with consensus algorithms. Depends on your requirements.
Appreciate it's just an MVP but I think there's a good niche you can go down. Big Data on AWS is such a pain to set-up (Glue, EMR, RedShift, LakeFormation), with IAM policies and roles a simple data pipeline is around 500 lines of YAML. Would be good if you could add native support for that, so say you have some CSV in S3 you want to convert to parquet, drop null fields, and then make shareable with another AWS account. Would solve a massive problem for me
Just boils down to not optimising until you need to. Start with a 3 tier web app (unless your requirements lead you to another solution), then start with read replicas, load balancing, sharding, redis/RabbitMQ etc
GitHub/Gitlab allow you to rebase your commits when merging. We do that, but has the disadvantage of mucking up things for people who are using that commit. I personally think it’s worth it
No company is going to touch Virtualbox without a 10ft barge pole. The licensing is so easy to get wrong and before you know it you’re writing a cheque to Oracle’s legal department
But wait until your master worker falls over and you have to promote your read replica. RDS handles everything for me, I just have to monitor it and be ready for any alerts from Promthesus.
I mean I’d say cloud providers measuring everything is a pro. I’ve been at companies in the past that have lost track of all their physical tin. One machine had been doing nothing for 5 years
> RDS. It’s slow and super expensive. I don’t want to worry about one-off queries or migrations. If you use only 5% of your server capacity you can afford to do inefficient things at times.
The author has clearly not had to do a database migration at 3AM. AWS is more expensive but so is my time, so it works out better all the time.
Isn’t computer science all about building on abstractions? Where do you draw the line? Should I write a kernel every time because Red Hat /MSFT charges me $100?
How can you protect yourself from file upload threats? It's basically the worst possible threat model -- executing complex user input that conforms to a spec that was written 20 years ago by some proprietary company with no security.
Executing everything on an isolated container with no permissions? Audit trial etc/good logging? If someone comes up with an RCE you're basically done for, you can only mitigate it but not completely stop it.
We use Typescript and jest for the mocks. Probably about 10%, they’re very simple mocks, all they do is return success and in a few unit tests we make them return failure and then test how our app responds.
I can’t really say if it pays off but it has caught some issues, would like to expand them more and implement more logic in then to check arguments etc but haven’t been able to yet.
We also have a pipeline that does e2 work tests and canary releases to hopefully limit the blast radius. If we encounter >10 errors in a 1 minute period we roll back and the team are alerted
If there's a network partition you have two options: accept reduced availability and keep your consistency, or have better availability and have reduce node consistency. Not much else you can do, that's just life.
Obviously, ways to increase consistency with consensus algorithms etc with 2 phase commits, and you reduce consistency with consensus algorithms. Depends on your requirements.