Practical examples of using software testing techniques, p. 2

In the first part of the article we presented all software testing techniques along with a thorough description of black box ones. This time, we’ll focus on white box and experience-based testing. Practical examples included!

White box testing — techniques and examples

Code statement testing and coverage

Code statement testing is the simplest white box technique. However, it requires a precise definition of the statement. It’s an element of the code written in a specified programming language which is the smallest indivisible unit of execution.

Example no. 5

Let’s consider a bubble sorting program, which is used to compare neighbouring elements. The algorithm works as follows: in each transition of the inner loop, two following values are compared and, if necessary, swapped places. During one cycle of the inner loop, the highest number in the collection will move to the last position (just as bubbles in a fizzy drink). This way, we get a subset which is partly sorted. We repeat these activities for the set omitted by the elements already arranged.

In the described example, we test a function using a code line coverage criterion.

Step 1. Establishing a subject and elements of tests. The only test element is the BubbleSort() function presented in Listing no. 1.

Listing no. 1.

1. function BubbleSort(int T[])
2. n = element_number_T B1
3. do B2 | B3 | B4
4. for (i = 0; | i<n-1; | i++) do
5. if (T[i]>T[i+1]) then B5
6. Swap places T[i] with T[i+1] B6
7. end if B4
8. end for
9. n = n-1 B7
10. while (n>1)
11. return T
12. end

Step 2. Derivation of test conditions. Test conditions are performed in instructions. In the discussed code, one program line corresponds with a single instruction.

Test conditions:

TC1: instruction no. 2;

TC2: instruction no. 3;

TC3: instruction no. 4;

TC4: instruction no. 5;

TC5: instruction no. 6;

TC6: instruction no. 9;

TC7: instruction no. 10;

TC8: instruction no. 11.

Step 3. Derivation of coverage elements. The coverage elements directly create test conditions:

CE1: instruction no. 2;

CE2: instruction no. 3;

CE3: instruction no. 4;

CE4: instruction no. 5;

CE5: instruction no. 6;

CE6: instruction no. 9;

CE7: instruction no. 10;

CE8: instruction no. 11.

The requirement of going through all the coverage elements is equivalent to the requirement of covering all the graph vertices from Pic. 6. It’s only a matter of going through the control path once to go through all of the graph vertices. It means that it requires only one test case to realize this path:

The path from Pic. 7 looks as follows: B1 -> B2 -> B3 -> B5 -> B6 -> B4 ->B3 -> B7 -> B8. This way, we achieved a 100% of statement coverage with one test case realizing the path.

Step 4. Defining test cases. For a statement coverage criterion, we need to cover as many coverage elements as possible with each new case.

Table 5. Test cases for the statement coverage

IdInputExpected outputCE coverageTC1T = [4,2]T = [2,4]CE1, CE2, CE3, CE4, CE5, CE6, CE7, CE8

Instructions no. 2, 3, 4 (CE2, CE3, CE4) will always perform regardless of the input value. The body of the loop for is carried out at least once (0 < 2–1), and the performed if instruction in line 5 covers CE4. The condition of the statement is fulfilled (4=T[0] > T[1] = 2) and line 6 is made (CE8). After increasing the i by 1 and returning to the beginning of the loop for, its condition is not fulfilled (1 <1 FALSE). That’s why the loop for ends its action and the program returns to line 9 (CE6). The next check includes the loop do-while condition in line 10 (CE7) which hasn’t been fulfilled (n = 1, a 1 >1 FALSE). That’s why the controlling comes out of the loop and transfers to line 11 where it ends the function operation covering CE8.

Decision testing and coverage

Decision testing requires each decision in the software to be “solved” at least once in a positive way (true) and at least once in a negative one (false).

Examples of decision statements are:

• a conditional instruction if: if (decision) then …;
• a condition in instruction while: while (decision) do …;
• a condition in instruction do-while: do … while (decision);
• an instruction switch-case;
• a loop condition in instruction for: for (initialization; decision, incrementation) do …

A decision is a point in a program in which the control flow has two or more alternative paths; a control flow graph loop with two or more branches.

A full decision coverage is achievable when each control process branch is used at least once during the tests. We can reach full decision coverage, choosing test cases data to meet these criteria.

Example no. 6

Let’s consider a simple program:

1. x=0
2. if (a>b) then
3. x = x+2
4. endif
5. y=50/x

Taking the value a so that a > b causes making all of the statement code lines, covering 100% of decisions. If the statement in line 3 is not made (i.e. a > b FALSE), then the variable x still has a value of 0 which leads to a prohibited operation — dividing by 0 in line number 5. Such an action can cause a failure or an unexpected performance of the program. So, using decision coverage criterion, we force the existence of at least one test case in which the value x in line number 2 is described as false which results in dividing by 0 in line 5 and detecting a failure.

Experience-based testing techniques — cases and examples

Guessing errors

It’s a technique which allows predicting the errors, defects and failures based on a tester’s intuition and experience. A tester’s job is to figure out the errors that might have been made by a developer and a way to reveal them.

Example no. 7

Let’s assume that there’s a validational requirement stating that a cell phone number needs to have 9 characters in the form of digits. Error guessing can look as follows:

• What if we enter a smaller or larger number than 9?
• What will be the result if the field for entering the number stays empty?
• What will be the result if we enter different characters than digits?

Exploratory testing

It’s a dynamic method with a large number of interactions between a tester and a tested software. A tester designs tests while they’re performed and uses information acquired during testing to design newer and better tests. Decisions which software use “path” to go are made abreast and in order to most possibly lead to a failure or an error.

As part of the exploratory tests, we perform sessions with a few steps:

1. Determined amount of time for performing the tests.
2. Establishing initial conditions.
3. Creating a list of comments emerged in the process of testing.

Example no. 8

An example of a website exploratory testing session.

Step 1. A “top-down” rule fits here perfectly. First, we need to ask the question “What’s the purpose of the tested website?”

Step 2. Creating the basic paths — getting to know the website’s purpose — allows us to deduce what an owner cares about and what we should pay special attention to while testing. Initially, we should focus on functional errors, because they are of the highest priority.

Step 3. Exploratory tests. It might be helpful to create a test description card.

An exemplary list of checkpoints during exploratory tests:

• Website’s menu — verifying if the menu displays and drop down correctly, if we can go to each of its elements and if they don’t overlap.
• Operation of links and redirects — checking the links/redirects by clicking each of them and observing if they work and lead to an adequate subpage.
• Website appearance, loading elements on the website and subpages — an overall review of the website to detect problems in its appearance, scrolling the website to look for overlapping elements and lacking fields.
• Handling of non-existent URL addresses — checking if the website’s errors are handled correctly by modifying their addresses. There are a few ways of modification in order to detect an error. We can talk about an error when the exception is displayed.
• Typos — verifying the text for typos and detecting them throughout the website.
• Players — if a website includes movie players, we should check if they work properly, performing actions such as: stopping, starting, resuming or rewinding.
• Headers testing — observing and clicking all the links.
• Footer testing — clicking all the links and checking how they work.
• Form data or other fields — we should test all the fields that allow entering data (function, overload, limitations).
• Privacy policy — does the website link to a privacy policy? The link should work correctly, and the policy should include data such as: the name of the website’s administrator, an address, and information of who is responsible for processing the data.

Exploratory testing requires a specific type of report. It can look as follows:

Id sessionCharter: Test…

Areas: OS | Windows 2000
Menu | View
Strategy | Function Testing (functional testing)
Strategy | Function Analysis (functional analysis)

Tester: Who tested

Start (start time): Date| dd.mm.yyyy hh:mm

Duration: e.g. short / long Goal: Functionality testing

Test notes: What has been achieved
Risks:Risks identified during testing (e.g: low efficiency of exploratory testing, tests are performed in the evening because of the testing team availability)

Bugs: A short description of bugs and a summary of the number of detected defects as well as the number of performed sessions

Issues (for further analysis)

Checklist-based testing

A checklist describes software’s features and properties that need to be checked. It’s usually focused on one topic, e.g. safety, standards, efficiency, data, etc. It can be updated along with gaining more experience while developing and testing the software.

Example no. 9

A checklist for verifying a website’s usability:

• Navigation — is it visible among the pages and easy to use?
• Labels — are the elements’ icons and descriptions intuitive and understandable?
• Number of objects — isn’t the website overloaded with too many elements?
• Referrals — are they visible and do they have correct titles?
• Title — do the main page and the subpages have correct titles?
• Structure — can we use the website without images, styles and layout? Does the information make sense when it’s read in the order it’s displayed?

A checklist helps in systematic testing. It makes the process easier and systematizes it, putting together all aspects of the software that need to be tested in one place.

Summing up

Each project’s different and has its own individual and unique characteristics. That’s why choosing the most appropriate techniques of testing depends on the context and a specific case. There’s no such thing as one, universal technique suitable for all kinds of projects. The techniques presented in this article are effective within certain limits and allow producing test cases that exhaust one specific coverage criterion. An experienced tester chooses adequate techniques and builds efficient tests on their basis.

“Some testing techniques are better in specific situations or at specific test levels, whereas others can be successfully used at every test level there is. To acquire the best results, testers usually create test cases blending several techniques.”

ISTQB