Simple start to unit testing in C

(Remember, I am still a newb with C ... :) )

I did a search for "unit testing in C" and was a little disappointed, at least at first. The thing is, is that I want two things for the analytic database project:

1. The only dependency should be libc (well ... I might pull in a couple of really hard things, like a serializer or web server as I see fit)
2. That I write things with a short timespan with the option to throw stuff out.

I don't want to drag in dependencies and my testing needs are really simple. It's mostly an attempt to avoid just making a lot of files with main() functions just to test various features as I build them.

The first thing I thought was using assert() but it wasn't ideal. It would exit the program on failure when the first test fails. However, looking at the code, it wasn't that complicated. The "test framework" below might be glibc specific, but there is always stuff to sort out later if it ever mattered.

The other thing I wanted to do was to eventually use TAP parsers, so I made it vaguely like the output of simple ok/not ok messages. (test anything protocol).

The "testing framework"

Here it is:

#ifndef _TESTING_H
#define _TESTING_H


#include <assert.h>
#include <stdlib.h>

int total_success = 0;
int total_failure = 0;
#define TEST                                                            \
    ({                                                                  \
    setvbuf(stdout,(char *)_IONBF,0,0);                                 \
    int test_count = 0;                                                 \
    ((void)printf("%s\n", __ASSERT_FUNCTION));
#define TEST_END                                                               \
  test_count = 0;                                                              \
  });

#define test(test_name, e, msg)                                         \
  ((void)((e) ? __ok(test_name, ++test_count)                                  \
              : __notok(test_name, #e, __FILE__, __LINE__, msg,                \
                        ++test_count)))

#define __ok(test_name, count)                                                 \
    (++total_success, (void)printf("\tok %d - %s\n", count, test_name))
#define __notok(test_name, e, file, line, msg, count)                          \
    (++total_failure, (void)printf("\tnot ok - %d : %s - %s:%u: failed assertion `%s: %s'\n", \
                count, test_name, file, line, e, msg),                         \
   0)

#define show_total_success()                                                   \
  ((void)printf("Total successes: %d tests succeeded\n", total_success))
#define show_total_failure()                                                   \
  ((void)printf("Total failures: %d tests failed\n", total_failure))
#define finish_tests()                          \
    (exit(total_failure == 0 ? 0 : 1))

#endif

I started to write some tests with it.

#include "testing.h"
#include "htable.h"

static void test_simple() {
    TEST;

    test("Simple test", 0, "This is false!");

    TEST_END
}

static void test_hash_item_create() {
    TEST;

    hash_item* item = create_hash_item("test");
    test("Hashed value", item->key == hash_func("test", 4), "Wrong hash value");
    test("Another hash", item->key == hash_func("tesd", 4), "Wrong hash value");

    TEST_END
}

int main() {
    test_hash_item_create();

    show_total_success();
    show_total_failure();
    finish_tests();
}

The TEST and TEST_END macros really just create a block and wraps in a counter so it adds up the tests in the output. The rest is obvious.

A couple of features in the future could be to have a macro or function to randomize the order of the test functions. Another is to have a test suite generator (create a file with a main() function of all the extracted "test_" calls.

Here is sample output:

test_simple
    not ok - 1 : Simple test - htable_test.c:7: failed assertion `0: This is false!'
test_hash_item_create
    ok 1 - Hashed value
    not ok - 2 : Another hash - htable_test.c:17: failed assertion `item->key == hash_func("tesd", 4): Wrong hash value'
Total successes: 1 tests succeeded
Total failures: 2 tests failed

Special outputting

The other thing is that we don't do anything too special with the output. If I want to do anything more special, like color output, I can just write a shell script. It's a bit easier:

#!/bin/bash

FG_GREEN="$(tput setaf 2)";
FG_RED="$(tput setaf 1)";
FG_BOLD="$(tput bold)"
OFF="$(tput sgr0)"

IFS=$'\n'
res=`$1`
code=$?

echo "$res" | sed "s/^\(test_.*\)/${FG_BOLD}\1${OFF}/g;;s/^\tok/\t${FG_GREEN}${FG_BOLD}ok${OFF}/g;;s/not ok/${FG_RED}${FG_BOLD}not ok${OFF}/g"

if [[ $code == "139" ]]; then
    echo "${FG_BOLD}Exited with error code 139: Got a segfault!!${OFF}"
fi


exit $code

(Note to self: The tput setaf codes are in man terminfo.)

We have to remind ourselves that printf() will buffer output, so in the testing macros above with turn off buffering completely. This might affect tests though, so this is probably something to fix in the future.

With a little searching, I can pair this up with M-x compile in emacs with the fancy-compilation mode (https://codeberg.org/ideasman42/emacs-fancy-compilation) to get pretty output that prints out the ANSI colors.

The "Compilation exited abnormally" because I dragged the exit code of the test binary past the sed part and exited with that code instead. The finish_tests() macro decides what exit code to use here.

Conclusion

Great! Now I can get back to writing my hash table with the comfort of knowing I can write quick tests :) With that in place, I found my first segfault! (Didn't allocate a char* first in heap :D )