End-to-end testing in software engineering is something really useful, when accompanied with tons of unit and integration tests. But is it that helpful when it comes to systems and networks?
Context: My company has a complex system of proxies interconnecting to other companies in other cloud environments. We were lately discussing about end-to-end testing. (e.g from our company A in cluster A in cloud A, to external company B cluster B in cloud B. Just uni-directional, because we would not even like to extract the monitoring from company B into cloud A, whatever they have set).
The most senior engineer was against end-to-end testing, while the rest were in favor of it. Of course we are having tons of monitoring on http status codes, latency, etc. So to say, "unit tests", as in the software engineering counterpart.
Why was he saying so? Because for sure we would be "pinging"(or whatever kind of ping is it) to another "demo" service at the other end, which in the end it is not part of a production environment, and the production environment may change, leaving this demo service unattended and prone to errors.
And I understand that we would not like to affect production traffic by "pinging" production services.
He is arguing that with sufficient "unit" tests we can disclose an issue better than with the unreliable end-to-end tests. Other colleague argued that with end-to-end tests we can check everything in case we forgot to monitor some part of it. The system has several series of reverse proxies, backends and load balancers in a rather complex way with HTTPS in every step so I also understand that.
So what would your take be here in this situation? I believe in the SRE books from Google something was suggested in similar scenarios, but I can not find it.
Context: My company has a complex system of proxies interconnecting to other companies in other cloud environments. We were lately discussing about end-to-end testing. (e.g from our company A in cluster A in cloud A, to external company B cluster B in cloud B. Just uni-directional, because we would not even like to extract the monitoring from company B into cloud A, whatever they have set).
The most senior engineer was against end-to-end testing, while the rest were in favor of it. Of course we are having tons of monitoring on http status codes, latency, etc. So to say, "unit tests", as in the software engineering counterpart.
Why was he saying so? Because for sure we would be "pinging"(or whatever kind of ping is it) to another "demo" service at the other end, which in the end it is not part of a production environment, and the production environment may change, leaving this demo service unattended and prone to errors. And I understand that we would not like to affect production traffic by "pinging" production services.
He is arguing that with sufficient "unit" tests we can disclose an issue better than with the unreliable end-to-end tests. Other colleague argued that with end-to-end tests we can check everything in case we forgot to monitor some part of it. The system has several series of reverse proxies, backends and load balancers in a rather complex way with HTTPS in every step so I also understand that.
So what would your take be here in this situation? I believe in the SRE books from Google something was suggested in similar scenarios, but I can not find it.