A couple of months ago I was being interviewed for a QA role by a startup company. When I asked what was their test strategy the interviewer replied that they automate all the scenarios with Cypress. To me it was a red flag for several reasons, thus I started to ask more questions. But the only response I got from the interviewer was - how do I know its all working unless it’s tested this way (through the UI). For some reason, I think this is the byword for testing in many people’s minds - test everything through the UI. All of this happened in 2020, that’s exactly 5 years after Just Say No to More End-To-End Tests was published. Five years in tech is like an eternity. And still, we (as a testing community) tend to put E2E tests as a cornerstone for our test strategy.
I am not saying an end to end test are evil, they still can have its place in today’s test strategies, but they should be used wisely and on an appropriate scale. The biggest issue I had with them is that they are very expensive to debug and they have very high requirements on our (testing) infrastructure to avoid all possible timeouts and hiccups. This includes servers running our backend service, CDN, database, and others. The „how do you know its’ all working“ is, on one hand, the biggest selling point of E2E tests to many people but on the other, the complexity is their biggest drawback. Imagine our E2E „login“ test fails. There are endless possibilities of what could go wrong - CSS issue(button is not visible), API not responding, API interface changed but UI logic was not updated accordingly, maybe just some infrastructure issue, or an issue with a database schema, and so on. Although the „login“ test mitigates very specific business risk, it does so very ambiguously. In 2020 we need our tests to be much more targeted and isolated to decrease their run-time and the time needed for debugging. Put it in other words, we need our tests to be much more effective as we develop and deliver software at a higher pace than ever before.
The microservice architecture demonstrates the rapid pace of software delivery and development while emphasizing testability and reducing complexity as opposite to monolith applications. All of this is possible because microservices are independent and loosely coupled. This means that one microservice is responsible for very specific business capabilities. Let’s carry the same paradigm to testing. So Instead of performing complex E2E tests (which tests application through several layers), try to decouple it and test each layer separately or in smaller and more manageable parts. The practical way to achieve this would be to mock the API and run our front end application against it. As a result, our UI test would be much faster and resilient and we would cover frontend specific risks only. To cover the other risk, we should also introduce contract tests and API tests. This way we would ensure these two systems (front end and backend) understand each other and the backend application works as expected. These practices in place allow us to test specific risks on the appropriate layer instead of testing them all through one layer only. All of this has one important incidental benefit, we can run these test much earlier in the delivery pipeline (shift left), they can run locally during every opened merge requests, instead of waiting for application deployment to the testing environment.