it is very funny that most of the (presumably germans) in the comments are saying "it's not that bad and it's fine because you chose a complicated LLC structure"
Super awesome, love the vision, exciting to try something in this space, and agreed with your core observations and conclusions. My only criticism is that the interior / package needs to look way cooler to tempt people across. Working with a decent industrial design firm and making some sexy renders would go a long way!
And totally agreed, both products take a slightly different approach and have different strengths and weaknesses, try them both and see which is a better fit!
the other major difference is the design principles: our core belief is that everyone owns quality, and so we build for the 'no code' user as well. It's a really hard bar to hit, but I think we've done a good job so far - to use reflect you need to be far more technical. 1/3 of our daily users are PMs.
First off, we totally agree on this: "true product Quality is dependent upon an endemic cultural philosophy of an organization."
What we're asking you to consider is how much of that is because the tooling is shutting out the roles who are naturally incentivized to care about product quality?
I think your broader thesis, while amusing, is ignoring some of the biggest and most valuable brands ever created... Apple, Rolex, Mercedes...
We have a free trial - you should try it instead of criticizing theory, I would love to get your feedback on the actual product, here or directly (I'm [email protected]).
Regarding DOM interaction, you're missing the point. All automation that tests the front end code, regardless of how it attaches, is using a path that is different from your end user. The end user interacts with the application visually. That's why we test visually. A decrease in code-based brittleness is just a nice side effect. And as you note, this is a very high-level post outlining one key idea about quality ownership. You may be interested in this, which is one of our front end folks talking about why we believe testing visually is superior: https://www.rainforestqa.com/blog/the-downfall-of-dom-and-th...
We have been selling a QA solution for almost 10 years. In that time we've seen thousands of setups and directly worked with hundreds of teams. Your claim "weaknesses of automated test frameworks [...] are compensated for quite easily and routinely" is, quite simply, not true for the majority of engineering teams - few QA leaders, including proponents of Cypress, would agree with you.
I totally see your point here - it's frustrating that how I presented it undermined the impact of the core point, which is that siloed ownership of quality is an anti-pattern, and an anti-pattern which is often a response to the limitations and design principles of quality tooling.
We are to my knowledge the only product built for this kind of cross-functional ownership, that's why I don't recommend other products.
While I agree the post is too marketing-y - to be honest I didn't expect it to appeal so much to the HN audience - it was written in good faith. We built the product because after 7 years of building a product for one of those silos, we became convinced that the only long-term durable solution was to empower everyone to own quality. Clearly I need to figure out how to strike the balance that you outline in your last sentence, which was the goal!
[author here] You are absolutely right: the optimal setup is a cross-functional team of domain experts collaborating on solving the customer's problem. The article is meant to point out that siloing QA to either just dev or just QA is an anti-pattern that is unstable and will not work, long term, without significant pain.
I think the challenge is that, while we agree on the optimal setup, it's almost never done like this. There's lots of reasons why, but the main one (imo) is organizational scaling. The logic of scaling teams tends towards specialization, especially because developer time is extremely expensive.
There are many ways to address the problem of QA. From our perspective, if they don't broaden the ownership of QA from just developers or just QA engineers (which is where all the current products are targeted), they will exacerbate this specialization problem.
I think the ideal team setup is one you allude to: a cross-functional team with experts from each domain collaborating together. My intent was not to say that can't happen - but that it typically doesn't, because of specialization and organizational politics. The vast majority of software teams have a siloed QA team. Why? That's for another thread :)
Regarding the incentive structure, I think you're right - there's no way to eliminate the friction that comes from competing incentives. Our experience has been that empowering the product organization to make that tradeoff themselves leads to the optimal outcome for the business. The goal isn't to eliminate the tradeoff between speed and quality, but surface it, and put it in the hands of the people who are the business decision makers, which tends to be product.
Re test data - we had seen this bottleneck with our previous product, which was purely about crowd testing. What we've seen since we shipped no code automation is that much of the data seeding by less mature teams can be done through the tests themselves. This is suboptimal, but with automation so cheap and fast, it works. Then over time the engineering team can seed the states that are most often created through the tests.
this has been our experience as well - sadly, however, most (all?) teams specialize and silo as they scale, and so you don't tend to see this setup beyond the very early stage