One can say that a tester’s job is verifying if a code, an application or a system is working correctly and according to established requirements. With this kind of verification, a user can trust a product’s quality and be sure that the number of potential errors is minimized. We can distinguish a few most common testing methods including white- and black-box testing. And today we’ll focus on the latter. What’s black-box testing? What’s behind this mysterious name? When should you use this method? What are its advantages? Read till the end and find out!
Black-box testing — a thorough definition
Black-box testing involves techniques for testing and designing test cases based on module or system specification. They can relate to both functional and non-functional testing.
Black-box techniques (called behavioral) are based on the tests’ basis analysis, e.g. formal documentation with requirements, specifications, user paths, and business processes descriptions. They concentrate on the input and output data of a tested object, with no reference to their inner structure.
When to use black-box testing?
The method is commonly chosen by testers who don’t know the inner structure of a system. It means that a tester “takes a look” at a system or an app from its inner side — taking on the role of an end user. Black-box testing is used by people who aren’t the software’s authors. As mentioned earlier, black-box testing can relate to functional testing, i.e. verifying “what a system should and what it shouldn’t do”. In this case, a tester has information about what features a product should include, but he or she doesn’t know how they work — usually not knowing the algorithms nor inner system’s operations. A practical example of the method is testing the payments based on information from the user path.
Black-box testing’s most important features
So, your knowledge is getting wider and wider, but you still don’t know what’s so special about the black-box testing. Here are some of its important features:
- Test conditions, test cases and test data are driven from the tests’s basis — they can be software related requirements, specifications or user paths;
- Test cases can be used to detect potential discrepancies between the requirements and their implementation or deviations from the requirements;
- The coverage is measured considering tested elements of the tests’ basis and the technique applied for the tests’ basis.
Black-box testing’s undoubted advantage is the repeatability of performed tests. However an obstacle can be a fact that despite testing the software, we don’t have the knowledge about the causes of occurring errors and we’re not able to test all the options thoroughly. In other words, the method allows a more accurate detection of errors, but doesn’t indicate their causes.
5 main techniques of black-box testing
We can divide the black-box testing into: equivalence partitioning, boundary value analysis, testing based on a decision table, state transition testing, and use case based testing.
1. Equivalence partitioning
Equivalence partitioning focuses on dividing data into groups (called equivalence classes) so that each of them involves elements which should be processed in the same way. Equivalence classes can be determined for proper values which should be accepted by a module or system and improper values which should be denied.
If necessary, the classes can be additionally divided into subclasses. However, it’s extremely essential to remember that each single value can be a part of only one equivalence class.
Here’s a visualization of a proper class division:
A condition to gain 100% coverage is using test cases of all identified equivalence classes (including improper ones) which requires selecting at least one value from each class to the tests.
Let’s take a look at a real-life example:
Assume that we want to test ATM withdrawal work. Our ATM realizes withdrawals from 50$ to 2000$. Based on this knowledge, we can define proper and improper classes. A proper class is a range between 50$ and 2000$ because the ATM is able to make such a withdrawal. Nonetheless, there are additionally two improper classes in the given case — for amounts lower than 50$ and higher than 2000$ which are not possible in the tested ATM.
We should select at least one representative from each class to the tests, e.g.:
- 20$ for the first improper class
- 200$ for the proper class
- 3000$ for the second improper class
2. Boundary value analysis
This technique is in some ways an expansion of the equivalence partitioning, but it can be used only for the orderly classes with numerical values. The boundary value will therefore be a minimal value and a maximal value for each equivalence class.
So, let’s consider the following example:
Assume that we examine a river water level. An acceptable and safe level is between 3 and 5 meters. Above this level, the risk of water overflowing the dam increases. On the other hand, below the safe level, we experience drought which has some negative effects, too.
In this case, we can distinguish 3 equivalence classes — a proper class and two improper ones:
- The proper class includes values between 3 and 5 meters,
- The improper class that includes values which are too low — from 0 to 3 meters (without 3),
- The improper class that includes values which are too high — more than 5 meters.
This example identifies two boundary values for each boundary between the neighboring classes. Therefore, for boundary tests between the water value which is too low and the one which is proper we should choose one number ranging <0, 3), e.g. 0.5 meter and second value amounting 3. For boundary tests between the proper water value and the value which is too high, we should choose number 5 and another number higher than 5, e.g. 7.
However, this technique has its variation which tests 3 boundary values (so called three-point technique) — the boundary value, the value just before the boundary value, and the value just after the boundary value.
If in the above case, we assume that the level of water is measured every 0.5 meters, we would test the following water levels: at 2.5, 3, 3.5, 4.5, 5, and 5.5 meters.
Writing a code, we usually use conditions to establish the boundaries which relates to the fact that improper system work is usually more probable for boundary values than the inner equivalence class values, e.g.:
if (x >5)
else if (x < 3)
else
Using this technique is therefore extremely important and useful. We can also apply it for requirements’ testing which relate to the range of numbers, hours, and dates, e.g. testing an advertising banner display in a specified range of time.
3. Decision table testing
This technique is commonly used when dealing with complex business rules which should be implemented in the system. Developing decision tables, a tester identifies input data (conditions) and the resulting output conditions (actions). It all comes down to creating a table with rows and columns with conditions on top and actions at the bottom. A column is related to the decision rule which defines a unique combination of conditions leading to an execution of actions related to this rule. The value in the table field is usually a logic value (true or false), less often the form of numbers (e.g. 0 for false and 1 for true) or a color.
A finished decision table consists of as many columns (test cases) as are needed to cover all the combinations of conditions. The number of test cases can be diminished, e.g. by removing the columns that describe impossible combinations of conditions. In practice, usually a minimal requirement concerning the coverage is creating at least one test case for each decision rule in the table. Normally it’s connected with the necessity to cover all combinations of conditions.
An unquestionable advantage of this technique is a possibility to detect gaps in requirements. It also allows identifying all of the crucial combinations of conditions that could be ignored or overlooked in case of different types of black-box testing. It’s used while dealing with different combinations of conditions.
So, let’s move on to a real-life example:
Let’s assume that we test the process of sending an advertising email to clients of a certain shop. A user has to fill a short questionnaire with marketing agreements and optional contact data for verification. In a simplified example such a table would look like this:
We have 2^n cases to test — n stands for the number of initial conditions. In our case, these are questions about an email, marketing agreements and verifying the email’s correctness. The above analysis shows that we have 8 test cases. The first one in which the email is sent correctly and the remaining 7 in which it isn’t sent at all. As you can see, some conditions can’t appear simultaneously (T2 and T3). That’s why we use an optimization technique, removing the columns which testing doesn’t make any sense.
4. State transition testing
A history of system actions and behaviors followed from the moment of its launch can be presented with a use of a state term. We can use a diagram to illustrate possible system states and transfers between them — a graphic representation of events and actions that they caused. The same event can result in two or more transitions, e.g. clicking the “send” button on the application form can result in both displaying information that the form has been sent correctly and an error message. The state transition table includes all correct and incorrect transitions between the states along with connected events and actions. Diagrams, on the other hand, are usually limited to correct transitions, i.e. a proper system behavior.
This technique, depending on the purpose, can be used for testing states, the correctness of transitions between them, as well as incorrect transitions.
It’s applied widely in cases of applications with drop-down menus in which a user can choose available options or a graphic representation of screens navigation.
Real-life example:
Here’s an exemplary diagram of transitions between the states in case of a game with available menu:
In this case, an initial state is the game’s main menu. The arrows show possibilities of transitions between particular states (arrowheads directed towards one side inform about a one-way transition, and those facing towards both sides about a possibility of two-way transition between neighboring states). We can choose one of the available options from the main menu: leaving the game, loading it, entering help or transfer to options, e.g. change control keys. Opening test cases, a tester should consider not only the correct transitions, but also the incorrect ones, e.g. attempting to click on two buttons at the same time and checking how the app behaves.
5. Use case testing
Use cases describe interactions with software elements, verifying requirements for functionality. At this point, we can introduce the term “actors”, i.e. users, external devices, and other modules or systems, as well as “subjects” to which a case is applied.
Each use case determines a specific action which can be performed by a subject in cooperation with an actor/actors. Testing the interactions can lead to changing the subject’s state.
Use cases can include basic system’s behavior, potential variations of basic behavior, but also exceptional behaviors and error handling. Tests are designed to enable verifying all of the above defined behaviors. So, let’s get to practice.
A real-life example:
Let’s assume that we’re testing the following case — topping up a phone with a specified amount of money. The goal is to verify the function of a phone top-up, and an initial condition — a client logged into the app. Additionally, we open some steps that have to be taken for a successful transaction, e.g.:
- Choosing an option of topping up the phone
- Entering necessary data:
- Top-up amount
- Phone number which we want to top up
- Payment method
- Confirming an operation with a “Top up” button
- Data verification
- Transaction authorization with an SMS code
- User confirmation of the transaction order (the transaction is accepted)
An expected result is informing a user about the phone being topped up and a correct phone to-up with an indicated amount of money.
The above test case we can lead out use cases with alternative versions for point 6, being:
6a. Transaction was not accepted
6b. Authorization failed
6c. Return to point 5, i.e. another authorization trial
Summing up
As you can see, every black-box testing technique is different. The right approach to testing requires a large amount of elasticity, modification, and combining all the techniques to gain the best results and a high quality software.
