Preface to the critique of UI tests
UI checks (unfortunately, commonly known as „tests“) are not necessarily bad, but they come with costs which may be ignored, since you hope, you will get an return on your investment. This critique aims to the hidden costs, things you may do not have on your radar and is not a „do not automate UI checks“ statement per se.
You are developing an additional new product
It needs design, planning, maintenance, care as if it would be a classic software product. In fact, it is a piece of software with all the dos and don’ts. If you are ignoring certain aspects of basic software engineering, you will introduce technical debts in your test code every week.
UI checks („tests)“ have a strong dependency to the system under test (SUT) in a technical and organizational way.
Which also means, that planning has to be mostly in sync. When you implement a new feature in the SUT or change something, you often need to update the UI tests too. The workload for writing the test code must be able to keep pace with the development of the core product. The ratio may vary, sometimes in an unpredictable way. A feature may take weeks to implement, but much less time to write the UI check. On the other hand, a simple change in the UI can generate tons of work to update the UI checks.
Complexity rises week by week
Increasing code base means you have to refactor your UI-tests. If you fail to do so, you will have big monolith which may compile but will do unpredictable things during test executions and will be hard to read for others.
The journey to Flaky Land
You will painfully find out what flaky means. It works on developer’s machine, but some UI tests are failing on your build server with flying colors, occasionally, sometimes. This means, you have to invest a tremendous amount of time for chasing these flaky tests.
Frontend has to take care of the UI tests, so that it supports the automated check instead of making them difficult to implement.
Developing UI tests means that you have less time for your core product
You may have heard of opportunity costs. Accept it: when you do UI tests, it will prevent you from coding for your core product. While there may be dedicated testers who will do it, it will involve the frontend or backends guys and gals to some extent.
What do you think? What are your experiences? Please leave a comment.