Open Test Framework  
Software Testing (Unit Test)

With the OTX development environment you can program in OTX. In order to ensure the quality of the programmed OTX code, so-called unit tests can be programmed in OTX. This is the task of the OTX UnitTest Extension.

This section gives a short glance about the fundamentals of software testing. Für einen tieferen Einblick in das Thema "Software Testen" wird auf die entsprechende Fachliteratur verwiesen, z.B. ISO 29119 "Software Testing".

  1. Basics of Software Testing
  2. Unit Test Example

Related Topics:

Note: The task of unit tests is to protect the programmed test logic against the expected behavior.

Note: The OTX UnitTest Extension was developed by EMOTIVE and is not part of the OTX standard. However, this has no impact on the standard compliance of the delivered OTX sequences, as the unit test cases are not saved together with the OTX test logic.

Note: The UnitTest Extension has no influence on the normal OTX test logic and is stored separately from it.

Basics of Software Testing

The so called "test pyramid" shows the different types for software verification.

There are essentially the following types of tests. The higher you get in the pyramid, the more expensive it gets and the longer the tests take.

  1. Unit Test
    The unit tests test the smallest possible, independent component.
  2. Integration tests
    Integration tests test the interaction between several components in a system, usually based on specific use cases such as flashing or coding.
  3. End-to-end tests (E2E)
    End-to-end tests test the entire system in an environment that is as realistic as possible.

A Unit Test tests the functionality of the smallest units of a software. The aim is to check each individual component independently and isolated from other data and external influences after each change and thus ensure the quality of the software.

Note: Unit Tests are also known as a regression tests.

A good unit test is based on the so called FIRST principle, see table below. It has to be fast and independently. It must always lead to the same result when repeated. It must be self-validating, meaning it must either pass or fail and the best thing to do, is to write the test before implementation.

Requirement Description
F-ast Fast test execution to run as often as possible
I-ndependent Independent to parallelize test execution
R-epeatable Must always lead to the same result when repeated
S-elf-Validating A test must either pass or fail
T-imely It is best to write the test before implementing the code (Test Driven Development)

Test Methods for Unit Tests

There are essentially two test methods for unit tests:

  1. Black Box Test
    The specification-oriented test method is also known as “black box test”, since it always tests against the specification. Black box tests are robust against changes and offer good protection against regression, but they always have to be created manually.
  2. White Box Test
    The white box test always looks at the internal structure of the code. In contrast to black box tests, white box tests can be generated automatically. But they are very vulnerable to changes.

Equivalence Classes

In OTX mostly black box tests are used. To write a good unit test the following two steps should be taken into account:

  1. Equivalence Classes
    Create classes of value ranges for the parameters for which expects the same behavior, the so-called equivalence classes. Then a representative for each class must be found and a test case must be written for it. The goal is high test coverage with as few test cases as possible.
  2. Border Analysis
    The border analysis is an equivalence class at the borders, since the errors are occur more often at the borders.

Unit Test Example

This should be illustrated using the practical example of an integer division with the OTX UnitTest extension in OTL syntax.

// Mark the following procedure as test procedure
[Test]
// Test case: Normal Division
[TestCase(dividend = 10, divisor = 2, expected quotient = 5)]
// Test case: Division with a negative sign
[TestCase(dividend = -10, divisor = -2, expected quotient = 5)]
[TestCase(dividend = -10, divisor = 2, expected quotient = -5)]
[TestCase(dividend = 10, divisor = -2, expected quotient = -5)]
// Test case: Division with remainder
[TestCase(dividend = 11, divisor = 3, expected quotient = 3)]
// Test case: Division at range borders
[TestCase(dividend = 9223372036854775807, divisor = 2, expected quotient = 4611686018427387903)]
[TestCase(dividend = -9223372036854775808, divisor = 2, expected quotient = -4611686018427387904)]
// Test case: Division from zero
[TestCase(dividend = 0, divisor = ValueList(1,-1,-9223372036854775808), expected quotient = 0)]
// Test case: Division with zero
[TestCase(dividend = ValueList(10,-10,0), divisor = 0, exception ArithmeticException)]
// Can be executed parallel
[Parallelizable]
IntegerDivision(in Integer dividend, in Integer divisor, out Integer quotient = 0)
{
quotient = dividend/divisor;
}

First, an OTX procedure with input parameters Dividend and Divisor and output parameter Quotient must be marked as Test Procedure. Within the procedure, the division is simply performed. In the first test case a normal division was described: 10 divided by 2 with the expected result 5. This corresponds to the Normal division equivalence class. The second equivalence class is the Negative Division for which three test cases be needed. The next equivalence class is the Division with Remainder: 11 divided by 3 gives 3 and not 4, since the remainder is always cut off. The next step is to go to the range borders and checks Division from Zero, which must always result in 0. Finally, it is checked whether an ArithmeticException is thrown when Dividing with Zero.

The attribute Parallelizable marks the test case as independent and thus parallelizable.

Note: A Test Procedure is like a normal Procedure and can contain any OTX code.