Work in progress.
Right now, the engine tests only function on *nix-like systems (including MacOS).
(Conversion to Windows will require, at the very least, looking at some of the file-handling code. If you want to fix this, start by looking in Engine.cc in the test/src directory!)
In addition to the engine requirements, you will need:
git(for getting the
gtest-parallelrepository, unless you grab the tarball from somewhere)
The Engine test suite now requires (at least) two additional pieces to run:
How this proceeds depends on your platform and whether or not you use a packaged distribution.
These instructions do this with
cmake. If you can manage it with
instead, more power to you ;)
This is the currently preferred way to do this, because everyone was doing it anyway and who am I to judge?
Thanks to Erik Smistad for this starting point (condensed from Getting Started with Google Test On Ubuntu):
Install the packages
libgtest-dev from the repository. This
will install the gtest source files to
/usr/src/gtest. You'll still need to
compile the code and link the library files to be able to use them.
Compile the source files:
cd /usr/src/gtest sudo cmake CMakeLists.txt sudo make
/usr/lib, hence the
sudo, but as long as it's in your library path, it shouldn't matter where you stick it):
sudo cp *.a /usr/lib
gtest from Macports
Macports will build the libraries for you.
In the next major section ("Building the Test Suite"), under
"Makefile and local.conf", set
local.conf (see instructions below)
/opt/local/lib is in your library path when compiling and
linking the tests.
mkdir -p ~/pep-src/googletest git clone https://github.com/google/googletest.git ~/pep-src/googletest
Switch into the source directory and find the subdirectory of
googletest (the source distribution of googletest also contains a
googlemock library which is not useful to us here)
include directories. Remember this
directory's path. For me it is
Edit the file
CMakeLists.txt in that directory, adding the
following line at the top:
set (CMAKE_CXX_STANDARD 11)
(If you don't, it won't compile, and I will shake my fist at you.)
cd ~/pep-src/googletest cmake CMakeLists.txt make
lib/*.a). You may leave them there without installing them, if you accept running a more complex command line than
later. If you want to install the library, read on.
Copy or symlink them to the library
location of your choice (make sure this is a directory that can be seen
during the test build process - i.e. one that's in one of the library paths
used in building. Mine are located in
local.conf under "Building the test suite" below -
In this scenario, I set
(i.e. the absolute path of where the
include directories were
above - for example,
Pick a source directory and put your
gtest-parallel source there
git clone https://github.com/google/gtest-parallel.git):
mkdir -p ~/pep-src/gtest-parallel git clone https://github.com/google/gtest-parallel.git ~/pep-src/gtest-parallel
This library is written in Python and does not require any actual build.
We'll deal more with this when preparing to compile the test suite.
local.conf in the top-level engine directory is where we stick all of the
Makefile overrides. The test Makefile contains some defaults for relevant
variables here, but if you need to override them, please either create or modify
local.conf in the top-level engine directory as needed. The relevant variables
GTEST_SRC_DIR: This is the directory where you compiled googletest above
GTEST_INC_DIR: This is where the include files for googletest are located
GTEST_PL: This is the full path to the python file for
(default presumes you cloned it under
src in your home directory, i.e. it is
local.conf included in
../doc/build-debian.md contains correct
definitions for these variables, assuming the user installed from sources under
~/pep-src as described here.
Presuming the above works and you installed every library, then from the top test
directory, simply run
Libraries which have not been installed into the
lib subdirectory of some
standard prefix will not be found automatically. But this problem is easy to
circumbent by setting environment libraries (on system using the GNU
LD_LIBRARY_PATH for dynamic libraries,
LIBRARY_PATH for static
For example, this should be sufficient to run the test suite under a debian
system following the instruction at
cd ~/pep-src/pep-engine LIBRARY_PATH=$LIBRARY_PATH:$HOME/pep-src/googletest/lib LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/pep-src/pep-engine/src:$HOME/pep-src/sequoia/target/release:$HOME/pep-src/libetpan/build/lib make test
Such variable definitions may be prepended to the command lines below for running individual tests or the entire test suite.
In case of test failures remember to build the database, on which the test suite depends, and to copy it into the system directory:
The database does not need to be rebuilt and reinstalled for every test suite
run: running the
dbinstall target once suffices.
Do one of:
make test OR
python3 <path to gtest-parallel.py> ./EngineTests
Note that for some test suites, this will, if something goes dreadfully wrong, mean that one test's failure may pollute another test. This generally means you have found a dastardly bug in the engine, but it can also be a test issue.
For example, for
lldb ./EngineTests -- --gtest_filter=TestSuiteName*
gdb --args ./EngineTests --gtest_filter=TestSuiteName*
For example, for
lldb ./EngineTests -- --gtest_filter=TestSuiteName.test_function_name
gdb --args ./EngineTests --gtest_filter=TestSuiteName.test_function_name
N.B. The gtest_filter can be globbed and will run all matching tests; if you want to run every test in a test suite, be sure to use TestSuiteName*.
(Different shells will require different quoting styles for this - YMMV.)
When debugging a failing test, use '--gtest_break_on_failure' to have gtest automatically break into the debugger where the assertion fails.
Compile tests with -DDEBUG_OUTPUT to (possibly) see output to cout. May only work in the tests run directly from ./EngineTests (with or without filter)
Script next on the agenda...
The normal run of the tests in parallel eats output. Try running the individual test case as above if you need to see a test case's output.