I worked at a company with a world-class QA team. They were amazing and I can't say enough nice things about them. They were comprehensive and professional and amazing human beings. They had great attention to detail and they catalogued a huge spreadsheet of manual things to test. Engineers loved working with them.
However -- the end result was that engineers got lazy. They were throwing code over to QA while barely testing code themselves. They were entirely reliant on manual QA, so every release bounced back and forth several times before release. Sometimes, we had feature branches being tested for months, creating HUGE merge conflicts.
Of course, management noticed this was inefficient, so they formed another team dedicated to automated QA. But their coverage was always tiny, and they didn't have resources to cover every release, so everyone wanted to continue using manual QA for CYA purposes.
When I started my own company, I hired some of my old engineering coworkers. I decided to not hire QA at all, which was controversial because we _loved_ our old QA team. However, the end result was that we were much faster.
1. It forced us to invest heavily on automation (parallelizing the bejesus out of everything, so it runs in <15min), making us much faster
2. Engineers had a _lot_ of motivation to test things well themselves because there was no CYA culture. They couldn't throw things over a wall and wash their hands of any accountability.
We also didn't have a lack of end-to-end tests, as the author alludes to. Almost all of our tests were functional / integration tests, that run on top of a docker-compose set up that simulated production pretty well. After all, are unit tests where you mock every data source helpful at all? We invested a lot of time in making realistic fixtures.
Sure, we released some small bugs. But we never had huge, show stopping bugs because engineers acted as owners, carefully testing the worst-case scenarios themselves.
The only downside was that we were slower to catch subtle, not-caught-by-Sentry bugs, so things like UX transition weirdness. But that was mostly okay.
Now, there is still a use case for manual QA -- it's a question of risk tolerance. However, most applications don't fit that category.
I used to do a similar thing, then I realized it was a potential problem.
Let's say you have an account at AcmeCo. Let's say AcmeCo has a breach and I can see your password hash. Let's say the company uses a weak password hash (e.g. MD5), or no salt and it's easy to reference a rainbow table.
From this rainbow table, I can look up your hash and see that your password is "lulzSecret2$AcmeCo".
Now let's say you're in another leak from BetaCo. Similar situation -- I see that your password is "lulzSecret2$BetaCo2". Maybe the two is because you were forced to rotate your password once.
It doesn't take a genius to guess what your algorithm is.
But we can take it another level. Maybe I'll try all the major banks and guess passwords using your algorithm ("lulzSecret2$bofa", "lulzSecret2$chase"). Most banks require 2fa, but most of the time they keep it to text-based 2fa.
If I know your phone number from one of the breaches (happens all the time), maybe I can hijack your SIM card (this also happens all the time) and boom, I'm into your bank account.
As an aside, this is the first time I've heard about Flightcontrol. Super impressed! The biggest con of something like Vercel is that you can't be on your own AWS VPC. An RDS instance with a public IP address (which Vercel's docs endorse) is a dealbreaker for me.
But... wouldn't a Terraform module accomplish something similar? Our own stack is something like Codepipeline + Fargate + ALB + Cloudwatch + Cloudfront and we basically just forked https://github.com/cloudposse/terraform-aws-ecs-web-app
Reading the docs, I'm a fan of your authentication / session management. When anyone pushes JWTs on me, I get sad. Session revoking is a necessary part of any authentication system and making a Redis call doesn't take that long.
However, the downside of custom session managers is that other services might not be able to read/write the created session. For example, I'm currently to get off of Express and onto Fastify. Unfortunately, @fastify/session isn't perfectly compatible with express-session (although I'm working on it). I would have a similar issue if I introduced Next.js + Blitz... Sometimes, I wish there was a shared protocol for sessions between languages/libraries!
Back at my old job in ~2016, we built a cheap homegrown data warehouse via Postgres, SQLite and Lambda.
Basically, it worked like this:
- All of our data lived in compressed SQLite DBs on S3.
- Upon receiving a query, Postgres would use a custom foreign data wrapper we built.
- This FDW would forward the query to a web service.
- This web service would start one lambda per SQLite file. Each lambda would fetch the file, query it, and return the result to the web service.
- This web service would re-issue lambdas as needed and return the results to the FDW.
- Postgres (hosted on a memory-optimized EC2 instance) would aggregate.
It was straight magic. Separated compute + storage with basically zero cost and better performance than Redshift and Vertica. All of our data was time-series data, so it was extraordinarily easy to partition.
Also, it was also considerably cheaper than Athena. On Athena, our queries would cost us ~$5/TB (which hasn't changed today!), so it was easily >$100 for most queries and we were running thousands of queries per hour.
I still think, to this day, that the inevitable open-source solution for DWs might look like this. Insert your data as SQLite or DuckDB into a bucket, pop in a Postgres extension, create a FDW, and `terraform apply` the lambdas + api gateway. It'll be harder for non-timeseries data but you can probably make something that stores other partitions.
SeedFi | Engineers, Designers, PMs | SF, NYC, Atlanta or REMOTE (USA) | seedfi.com
SeedFi is a fintech startup that builds products for Americans living paycheck to paycheck. We're focused on improving our customers' financial health by helping them build savings and improve their credit score. Our products aim to permanently get people out of debt cycles. So far, the response we've seen has been really amazing: trustpilot.com/review/seedfi.com
SeedFi has about 40 employees and is growing fast. We raised over $34M from top tier VCs like A16Z and from major social impact funds and we've already helped our customers build millions of dollars in savings.
SeedFi is a fintech startup that builds products for Americans living paycheck to paycheck. We're focused on improving our customers' financial health by helping them build savings and improve their credit score. Our products aim to permanently get people out of debt cycles. So far, the response we've seen has been really amazing: trustpilot.com/review/seedfi.com
SeedFi has about 30 employees and is growing fast. We raised over $30M from top tier VCs like A16Z and from major social impact funds and we've already helped our customers build millions of dollars in savings.
To apply, please email your resume to jobs at seedfi dot com.
SeedFi is a fintech startup that builds products for Americans living paycheck to paycheck. We're focused on improving our customers' financial health by helping them build savings and improve their credit score. Our products aim to permanently get people out of debt cycles. So far, the response we've seen has been really amazing: trustpilot.com/review/seedfi.com
SeedFi has over 25 employees and is growing fast. We raised $19M from top tier VCs like A16Z and from major social impact funds. We've already helped our customers build millions of dollars in savings.
To apply, please email your resume to jobs at seedfi dot com.
No polecats smh