The one who doesn’t make any mistakes, probably doesn’t do anything at all — this maxim matches every area of life, also when it comes to software development. Nonetheless, it’s always worth minimizing the risk and eliminating system failures — and that’s the role of a tester in a project. So, what tests should you carry? Is it better to focus on manual or automated ones? How to decide on what’s the most optimal solution?
Manual and automated tests — what are they?
Manual tests are carried by a human according to the control lists and rules, but without using any tools or automating scripts. Being a manual tester requires specialized testing knowledge and general IT information, but you don’t need to know all the technicalities such as programming languages or automation tools. Such tests are great for exploration, functional and usability testing, although you shouldn’t forget that they can be really time-consuming, especially during constant product growth.
When it comes to automated tests, they have already been explained on our blog — check the article: “Tests automation in a project — when and why?”.
Manual testing vs. automated testing
Let’s analyze some differences between these two types of tests.
The matter of time
We’ve already mentioned that, in contrast to manual tests, automated ones are based upon varied frameworks, e.g. Data Driven, Hybrid, etc. Another thing is the time needed to perform the tests. Manual testing is definitely more demanding — because it’s based on human work, it’s more time-consuming and requires completing repeatable tasks. It carries the risk of falling into a routine, especially when a product is constantly growing.
It seems that in this case, automated tests are better — after all, they’re performed by a machine which is faster and more precise. Nevertheless, you should remember that automated tests also take time to create and maintain in the later stages of the process.
Overall costs
What we should also mention are costs — initial investment in the process of manual testing is lower than in case of automation. It’s necessary to hire a dedicated person (or people in case of larger projects) with adequate competencies.
Automated testing needs a bigger investment at the beginning — in this case you should invest in testing tools development, which translates directly into hiring an automation specialist. Latter costs, however, are much more favorable than in manual testing.
Precision
Manual testing is more prone to errors — after all, nobody’s infallible. Automated testing uses scripts and tools which make the whole process more reliable and accurate. However, you should know that such tests need to be wisely and well written — inaccurate scripts can indicate a false positive result with evidently incorrect system behavior. A person who has a technical background doesn’t necessarily have enough experience in manual testing, which can translate into inaccurately designed tests set. As a result, it can lead to overlooking important paths and not covering all test cases that can be automated.
Error reports
Error reporting in manual tests is time-consuming as well — a tester has to arrange steps for reproduction, provide screenshots, recordings and information on the equipment and software version. To speed up the process and facilitate the management of the reported bugs, a tester can of course use some dedicated software such as Testrail or Jira. In case of test automation, you can take care of a proper bug report configuration — a detected error can be reported without any human interference.
Manual approach is a good idea in testing user interface. It allows observing an application or testing it explorationally. Automated tests can’t do that, but they’re great for performance or regression testing.
Can we actually talk about test automation?
One of the well known speakers and experts in the subject of QA, Michael Bolton, claims that tests cannot be automated. But why did he come up with such a thesis? Well, he maintains that testing is a process of evaluating a product by getting to know it through exploration and experimentation which involves asking questions, research, modelling, observation, conclusion, and more. On the other hand, he defines verifying as a process of evaluation through using algorithmic decision rules to particular product observations. An implication of both definitions is that verifying is a process which can be performed by a tool instead of a human (automated process), while testing can only be supported by tools — which is not an automation in Michael’s opinion. Of course, it’s only his point of view, but I refer to it to let you see the world of tests from a different perspective.
Which tests should you choose?
Now, when you already know the difference between automated and manual tests, you can think which is the most applicable for your product.
Manual testing is perfect in case of exploration, usability and ad hoc tests — they all need both skills and experience as well as intuition and a human eye. Short-term projects should also be supported by manual tests — otherwise cost and time for creating an automated tests infrastructure will be disproportionate to project’s size and duration.
In turn, you can invest in automation when it’s necessary to test a product’s performance or load — in this case it’s easier to simulate a simultaneous activity of many users or a large amount of processed data. Test automation is also recommended for often repetitive tasks such as regression testing. Apart from that, it’s useful in case of functional tests while simulating quick code alterations in a short period of time.
Summing up
Both approaches have their advantages and flaws. To choose correctly, you need to consider many aspects of your application. May I risk saying that actually every project requires both manual and automated testing.
