C++

Testing C++ solutions

We use googletest as a testing framework for C++ coding exercises. We also provide googlemock for more convenience methods to aid in testing.

Hello, World!

Before we go into details, let's see how you can use googletest to test the most basic C++ task: writing a function that returns "Hello, World!".

This is what a student might write:

#include <string>

std::string helloMessage() {
    return "Hello, World!";
}

And this is an example of what you could write to evaluate the student's code:

#include "gtest/gtest.h"

std::string helloMessage();

namespace {

class TestHello : public ::testing::Test {
};

TEST_F(TestHello, Message) {
    EXPECT_EQ("Hello, World!", helloMessage()) << "Your function should return: Hello, World!";
}

}  // namespace

Let's examine the above evaluation code.

First, we must include the testing framework header, "gtest/gtest.h".

Then we can either declare the interface that the student needs to implement, or we can provide it in a header file as part of the initial student files, which we would include in the evaluation file. We chose to declare it for this example.

Although optional, it's a good practice to write your testing code inside a namespace.

::testing::Test

Then we need to define a suite of unit tests as a class that inherits from ::testing::Test.

This groups together multiple unit tests and allows implementing common functionality that you want executed before and after each test, via the constructor and destructor respectively.

class TestSuite : public ::testing::Test {
protected:
    TestSuite() {
        // You can do set-up work for each test here.
    }

    virtual ~TestSuite() {
        // You can do clean-up work that doesn't throw exceptions here.
    }

    // Objects declared here can be used by all tests from this test suite.
};

TEST_F

Inside the class that you defined you can now write multiple unit tests by calling the following macro:

TEST_F(ClassName, UnitTestName) {
    /* Implementation. */
}

The unit tests that you define are the only functions that will be executed. You should not implement a main() function, and neither should your students.

Assertions

Assertions are the statements that actually verify that the student's code behaves as expected. The most common pattern is to call student's methods and assert that their return value is the one that you expect. If it is not, then the test will fail and you have the opportunity to provide a custom message to the student for each failed assertion, so that she can improve.

These are the most common ones:

EXPECT_EQ(expectedValue, actualValue);
EXPECT_TRUE(condition);
EXPECT_FALSE(condition);

If any of the expectations above aren't met, the test is marked as failed and feedback is given to the student. But the test continues execution until the end. If you instead decide that it's not worth continuing the test, you can stop execution as soon as a failure is recorded by using these alternative assertion macros:

ASSERT_EQ(obj1, obj2);
ASSERT_TRUE(condition);
ASSERT_FALSE(condition);

A good practice is to provide a custom feedback message to the student for every failed assertion. You can do this by using the << operator on the assertion macros, just like you would use it with std::cout.

There are more advanced assertion macros that allow you to test

Matchers (googlemock)

Google Mock is a complex framework that gives you more power when writing tests.

For testing simple student programs, you will not need to use mocks, or most of what this library provides. But there is one thing that you may want to use: matchers.

They make it super easy to assert various data types, like:

For example, with gmock's ASSERT_THAT macro and testing::ElementsAreArray function, the resulting message will indicate which specific array elements caused the test to fail.

To use Google Mock you just need to #include "gmock/gmock.h" in your evaluation file.

Standard output

In the previous example we had the students return Hello, World!, but actually a more common task is to have them print the same string to the standard output.

#include <iostream>

void say_hello() {
    std::cout << "Hello, World!";
}

How do we test this specific function?

#include <string>
#include <utility>

#include "gtest/gtest.h"
#include "helpers/iohelper.h"

void say_hello();

namespace {

class TestHello : public ::testing::Test {
};

TEST_F(TestHello, Message) {
    std::pair<std::string, std::string> output = CAPTURE_OUTPUT(say_hello());
    std::string stdout = output.first;
    EXPECT_EQ("Hello, World!", stdout) << "Your function should return: Hello, World!";
}

} // namespace

We implemented 2 macros that allow you to:

  • capture everything that students print to standard output and standard error

  • provide a string for standard input to read from

You can just include helpers/iohelper.h to get access to them.

#include <string>
#include <utility>

#include "helpers/iohelper.h"

std::pair<std::string, std::string> CAPTURE_OUTPUT(statement);
bool INJECT_INPUT(const std::string &content, statement);

CAPTURE_OUTPUT returns a pair of strings: the first one is the standard output and the second is the standard error.

statement can be any C++ statement. You can call a function with arguments like this:

CAPTURE_OUTPUT(sum(2, 3));

or even with if statements:

CAPTURE_OUTPUT(
    if (true) {
        say_hello();
    }
);

or multiple statements separated by ;:

std::string line1;
std::string line2;

INJECT_INPUT(std::string("abc\ndef"),
    std::cin >> line1;
    std::cin >> line2;
);

EXPECT_EQ("abc", line1);
EXPECT_EQ("def", line2);

Technical specs

  • gcc >= 5.4.0

  • C++14

Example 1: FizzBuzz

Given:

  • An integer number n

Your task is to:

  • Write a function that returns an array with the numbers from 1 to n with the following restrictions:

    • for multiples of 3 store "Fizz" instead of the number

    • for multiples of 5 store "Buzz" instead of the number

    • for numbers which are multiples of both 3 and 5 store "FizzBuzz"

Example:

fizzbuzz(15) == {
    "1", "2", "Fizz", "4", "Buzz",
    "Fizz", "7", "8", "Fizz", "Buzz",
    "11", "Fizz", "13", "14", "FizzBuzz"
}

Here's how a student might solve the problem:

#include <algorithm>
#include <vector>

std::vector<std::string> fizzbuzz(int n) {
    std::vector<int> range(n);
    std::iota(range.begin(), range.end(), 1);

    std::vector<std::string> values;
    values.resize(range.size());

    auto fizzbuzz = [](int i) -> std::string {
        if ((i%15) == 0) return "FizzBuzz";
        if ((i%5) == 0)  return "Buzz";
        if ((i%3) == 0)  return "Fizz";
        return std::to_string(i);
    };

    std::transform(range.begin(), range.end(), values.begin(), fizzbuzz);

    return values;
}

Here's how you might test the student's solution:

#include "gmock/gmock.h"
#include "gtest/gtest.h"

std::vector<std::string> fizzbuzz(int);

namespace {

class TestFizzBuzz : public ::testing::Test {
};

TEST_F(TestFizzBuzz, One) {
    std::vector<std::string> result = fizzbuzz(1);
    ASSERT_THAT(result, testing::ElementsAre("1"));
}

TEST_F(TestFizzBuzz, Fifteen) {
    std::vector<std::string> result = fizzbuzz(15);
    ASSERT_THAT(result, testing::ElementsAreArray({
        "1", "2", "Fizz", "4", "Buzz",
        "Fizz", "7", "8", "Fizz", "Buzz",
        "11", "Fizz", "13", "14", "FizzBuzz"
    }));
}

}  // namespace

Example 2: Title Case

Given:

  • A sentence with words separated by spaces

Your task is to:

  • Write a function that returns a copy of this sentence where all the words:

    • start with a capital letter

    • the rest of the letters are lower case

Note:

  • Your function should not modify the sentence given as argument.

Example:

normalize("This is SO  MUCH  FUN! ") == "This Is So  Much  Fun! "

Here's how a student might solve the problem:

#include <ctype.h>
#include <string>

int skip_whitespace(const std::string &sentence, size_t idx) {
    while (idx < sentence.length() && sentence[idx] == ' ') {
        idx += 1;
    }

    return idx;
}

int normalize_word(std::string &sentence, size_t idx) {
    if (idx < sentence.length() && sentence[idx] != ' ') {
        sentence[idx] = toupper(sentence[idx]);
        idx += 1;
    }

    while (idx < sentence.length() && sentence[idx] != ' ') {
        sentence[idx] = tolower(sentence[idx]);
        idx += 1;
    }

    return idx;
}

std::string normalize(const std::string &sentence) {
    std::string copy = sentence;
    size_t idx = 0;

    while (idx < copy.length()) {
        idx = skip_whitespace(copy, idx);
        idx = normalize_word(copy, idx);
    }

    return copy;
}

Here's how you might test the student's solution:

#include <string>

#include "gmock/gmock.h"
#include "gtest/gtest.h"

std::string normalize(const std::string &);

namespace {

class TestNormalize : public ::testing::Test {
};

TEST_F(TestNormalize, Character) {
    EXPECT_EQ("A", normalize("a")) << "Your function should convert a single character.";
    EXPECT_EQ("A", normalize("A")) << "Your function should convert a single character.";
}

TEST_F(TestNormalize, NoMutate) {
    std::string sentence = "a";
    normalize(sentence);
    EXPECT_EQ("a", sentence) << "Your function should not mutate the argument.";
}

TEST_F(TestNormalize, Word) {
    EXPECT_EQ("Abc", normalize("abc")) << "Your function should convert a single word.";
    EXPECT_EQ("Abc", normalize("ABC")) << "Your function should convert a single word.";
    EXPECT_EQ("Abc", normalize("aBC")) << "Your function should convert a single word.";
    EXPECT_EQ("Abc", normalize("Abc")) << "Your function should convert a single word.";
}

TEST_F(TestNormalize, Sentence) {
    EXPECT_EQ("Abc Def Ghi", normalize("abc def ghi")) << "Your function should convert a sentence.";
}

TEST_F(TestNormalize, MultipleWhitespace) {
    EXPECT_EQ("  Abc  Def Ghi ", normalize("  abc  def ghi ")) << "Your function should keep multiple whitespaces.";
}

}  // namespace
PreviousIntroductionNextC#

Last updated