Exercise (COMPULSORY): (Unit) testing and CI¶

The aim in this week is to get familiar with the QtTest unit testing framework. In addition we get to know the CI environment used on the course. Note that while there is a lot of work in this exercise, you can focus on one thing at a time.

As the compulsory part of the exercise you must complete parts: Testproject ja GitLabCI. The submission is done with the help of version control. Remember to commit your work frequently.

The files needed in this task can be pulled from version control. They are in repository ``student_template_project https://course-gitlab.tuni.fi/tie-0240x-ohjelmointi-3-programming-3-2020-2021/student_template_project You must set it as a remote to your local repository. The codes needed in this exercise are in the directory EX3. Note that you do not have rights to update the repository.

.gitlab-ci.yml: CI/CD configuration at the root of the repository
EX3:
- WelcomeToTampere: A software project with a few initial components
  - WelcomeToTampere.pro: project file of the entire project
  - baddate.cc, baddate.hh, date.cc, date.hh, main.cc, morottaja.cc, morottaja.hh: the code files
- UnitTest: The unit test project
  - UnitTests.pro: the project file for the unit test project
  - Morottaja: testcode for the morottaja component
    - Morottaja.pro: project file for the unit test project
    - morottajatest.cc: the test code for the morottaja component

Note! The class to be tested Date may have bugs (either in code or in interface documentation/design). Fixing these bugs is not the purpose of this exercise.

Testproject¶

Add a project Date to the project UnitTest for unit testing the date class right click->New Subproject-> Other Project-> Qt Unit Test.

Add the source code files into the unit test project by adding their information into Date.pro

SOURCES +=
HEADERS +=

To make it possible for the compiler to find the files, add also

INCLUDEPATH +=
DEPENDPATH  +=

Morottaja.pro can act as a model.

Building and running projects¶

When you build the top level project, QtCreator builds both the "application" and unit tests. If you choose a sub-project and run from there, QtCreator runs the chosen sub-project.

Writing unit tests¶

In Qt's unit test framework, each test case is written as a member function in the test class. Test framework calls these member functions automatically in order. (Refer Qt Test framework documentation for more information.)

The idea in unit testing is that each test case tests a single property or member function (testing complex member functions should be divided into several test cases). Division makes it easier to see the location of a possible error more exactly in the output of the test run. In addition, test framework makes a test case to stop if an error occurs (further tests in the member function will not be run). However, the test run still continues and the other test member functions will be called.

Test class includes some example test cases to demonstrate how to use the framework. In short, the idea is to first create a date object, call an operation to be tested for it, and then ensure that the outcome of the operation is as it should be. This testing will be done with the test framework commands (QCOMPARE, QVERIFY, QFAIL), in order to log the errors. Documentation of testing macros can be found here .

In writing test case member functions, it is again good to use the services provided by QtCreator. Write the header of a member function in class header (class Datetest in this exercise), and then generate the body of the member function by choosing right click > Refactor > Add definition outside class.

Writing data driven unit tests¶

Use imagination and creativity in inventing good test cases (of course, without forgetting robust analytic mind). The class to be tested does not contain intentional bugs, but it has not been tested properly, so most probably there are bugs ... :-)

It is good to write test cases in a data-driven way. Test framework enables you to run the same test case with different test data (this is a typical situation). This is done by writing the test case as a member function of its own. In addition, you should write another member function ending with _data, which generates a test matrix with desired test cases.

There is an example of this, a test case named weekday. Member function weekday_data first determines the "input" types and names of the test case (QTest::addColumn), then it creates a test matrix, where each row is a named test case (QTest::newRow). The actual test member function weekday is called automatically for each row in the test matrix. The member function first reads the input data of the test matrix (QFETCH) and then it runs the normal test. Note that QFETCH automatically creates a variable, named as the test input, in the member function.

void Unittest::weekday()
{
   // This method tests a row of the test matrix generated by the method weekday_data.
   // It is called automatically for each row of the matrix

   // Fetching data from the matrix, variables are created automatically
   QFETCH(unsigned int, day);
   QFETCH(unsigned int, month);
   QFETCH(unsigned int, year);
   QFETCH(Date::Weekday, weekday);

   // Performing the test
   Date d(day, month, year);
   QVERIFY2(d.giveWeekday() == weekday, "Wrong weekday");
}

void Unittest::weekday_data()
{
    // This method defines the test matrix for the weekday test and generates the desired test cases there

    // Defining columns for the test matrix (types and names)
    QTest::addColumn<unsigned int>("day");
    QTest::addColumn<unsigned int>("month");
    QTest::addColumn<unsigned int>("year");
    QTest::addColumn<Date::Weekday>("weekday");

    // Generating test cases for the test matrix, 3u etc. are needed since the type is unsigned
    QTest::newRow("today")      << 3u  << 2u  << 2014u << Date::MONDAY;
    QTest::newRow("last Christmas") << 24u << 12u << 2013u << Date::TUESDAY;
    QTest::newRow("next May Day")  << 1u  << 5u  << 2014u << Date::THURSDAY;
    QTest::newRow("end of the year")<< 31u << 12u << 2013u << Date::TUESDAY;
    QTest::newRow("new year")  << 1u  << 1u  << 2014u << Date::WEDNESDAY;
}

For the data driven tests, somewhere in the source code file Q_DECLARE_METATYPE(Date::Weekday); must be defined. In addition to this example test, at least one additional test needs to be included.

GitLabCI¶

In the root of your repository, there is a file called .gitlab-ci.yml. It is used to configure GitLabCI. You can find information on CI/CD and configuring .gitlab-ci.yml under help .

Configuring the Pipeline¶

The pipeline konfiguration is based on jobs. A pipeline can consist of several jobs. They

are top-level elements for the contents of the pipeline
are named arbitrarily
must contain at least the script clause
are defined with constraints that state the conditions under which they should be executed
are not limites in number

An example of a job:

build_EX1:
  script:
    - cd EX1/DesignByContract
    - qmake
    - make
    - make clean

When defining a job, a group of parameters is used to define how the job should behave. The most important ones:

tags: s used to select specific CI runners from the list of all Runners that are allowed to run this project. On this course alwats qt.
image: Used to specify a Docker image to use for the job. On this course, defined already
stage: stage is defined for each job and relies on stages which is defined globally. With it jobs can be grouped under different stages. In this task the stages are build and test respectively.
artifacts: is used to specify a list of files and directories which should be

attached to the job. These are saved for investigation and can be passed between jobs. The path to the saved artifact paths: can be defined in addtion to e.g. expire_in to specify how long artifacts should be kept before they are deleted * dependencies: by default, all artifacts from all previous stages are passed to the job. The dependencies parameter can be used to define a limited list of jobs (or no jobs) to fetch artifacts from. Useful between build and test jobs.

CI for Date Tests¶

Open the file for edits. The goal is to add running the tests as a part of the CI-pipeline. Currently there are stages for compiling the weekly exercises and for running Morottaja's unit tests. Get to know the file and add Date's unit tests into it.

You can check if the syntax of you file is correct with GitLab CI linter https://course-gitlab.tuni.fi/tie-0240x-ohjelmointi-3-programming-3-2020-2021/repo_name/-/ci/lint where the repo_name is replaced by the name of your working repository. A linter is a program that can be used to analyze fixed format files, e.g. yaml, code files, to check that they follow the correct format.

Submission¶

Submit your work with a tag starting with EX3_submission.

A+ presents the exercise submission form here.