# Building and Running the Tests for the p≡p Engine
Work in progress... Currently have the debian build/run instructions in.
## Caveat, before you begin
Right now, the engine tests only function on \*nix-like systems (including MacOS). (Conversion to Windows will require, at the very least, looking at some of the file-handling code.) If you want to fix this, start by looking in Engine.cc in the test/src directory!
Right now, the engine tests only function on \*nix-like systems (including
MacOS).
*(Conversion to Windows will require, at the very least, looking at some of the
file-handling code. If you want to fix this, start by looking in Engine.cc
in the test/src directory!)*
## Requirements
In addition to the engine requirements, you will need:
* cmake
* python3
* `cmake``python3``git` (for getting the `gtest-parallel` repository, unless
* you grab the tarball from somewhere)
## Building the prerequisites
The Engine test suite now requires (at least) two additional pieces to run:
* `googletest`
* `gtest-parallel`
How this proceeds depends on your platform and whether or not you use a packaged
distribution.
These instructions do this with `cmake`. If you can manage it with `bazel`
instead, more power to you ;)
### Installing `googletest`
#### Packaged distributions
This is the currently preferred way to do this, because everyone was doing it
anyway and who am I to judge?
##### Debian and Ubuntu (and derivatives)
## Preparing to build
Thanks to Erik Smistad for this starting point (condensed from [Getting Started
The Engine test suite now requires (at least) two additional pieces to run - **googletest** and **gtest-parallel**. You will note that I give specific instructions about where to put these, because that is what I have tried and tested. That does NOT mean other things won’t work; I simply haven’t tried them. So without further ado…
1. Install the packages `cmake` and `libgtest-dev` from the repository. This
will install the gtest source files to `/usr/src/gtest`. You'll still need to
compile the code and link the library files to be able to use them.
### googletest
2. Compile the source files:
```
cd /usr/src/gtest
sudo cmake CMakeLists.txt
sudo make
```
**googletest** is an XUnit testing framework we are now using in place of cpptest. Unlike a lot of other testing frameworks, it’s recommended that you compile and link the test code directly within your project. These instructions do with with **cmake**. If you can manage it with **bazel** instead, more power to you ;)
3. Copy/symlink the libraries to the library location of your choice (here,
it's `/usr/lib`, hence the `sudo`, but as long as it's in your library path,
it shouldn't matter where you stick it):
```
sudo cp *.a /usr/lib
```
So. To get things started.
##### MacOS
In the directory of your choice (default, if you don’t want to change **local.conf** - specifically **GTEST\_DIR** - is the test directory (this one, presumably)):
I am totally guessing for now - this is a placeholder - that
macports gtest install will do the same. Will need to find the directories this
#### Downloading and compiling the source yourself
(Note that this hasn’t been tested in other directories, so I am presuming the Makefile works as is, but I could be wrong)
For now, don't.
### gtest-parallel
Or do, and document it for me.
### Again, in the directory of your choice (if you want to use the Makefile out of the box, you should, while still in the googletest directory, do the following):
If you were using the git repo and it was working before, please follow the
instructions above for Debian/Ubuntu, only with your source repository in mind
instead of `/usr/src`, and pay attention to the variables you'll need to set in
`local.conf` for the Makefile - they are different from before.
2. If using a different directory, please change **GTEST\_PL\_DIR** to indicate where **gtest-parallel.py** is located.
### Installing `gtest-parallel`
Pick a source directory and put your `gtest-parallel` source there
(e.g. via `git clone https://github.com/google/gtest-parallel.git`).
We'll deal more with this when preparing to compile the test suite.
## Building the test suite
### `Makefile` and `local.conf`
So `local.conf` in the top-level engine directory is where we stick all of the
Makefile overrides. The test Makefile contains some defaults for relevant
variables here, but if you need to override them, please either create or modify
`local.conf` in the top-level engine directory as needed. The relevant variables
are:
* `GTEST_SRC_DIR`: This is the directory where the gtest source you compiled
above is located (defaults to `/usr/src/gtest`)
* `GTEST_INC_DIR`: This is where the include files for gtest are located
(defaults to `$(GTEST_SRC_DIR)/include`)
* `GTEST_PL`: This is the full path to the *python file* for `gtest_parallel`
(default presumes you cloned it under `src` in your home directory, i.e. it is
`$(HOME)/src/gtest-parallel/gtest_parallel.py`)
### Building
Presuming the above works, then from the top test directory, simply run make.
# Running the Engine Test Suite
## Running the test suite
## To simply run the test suite and see what tests fail...
### To simply run the test suite and see what tests fail...
Do one of:
1. make test OR
2. python3 \<path to gtest-parallel.py\> ./EngineTests
1. `make test` OR
2. `python3 <path to gtest-parallel.py> ./EngineTests`
### To run individual test suites, especially for debugging purposes
Note that for some test suites, this will, if something goes dreadfully wrong,
mean that one test's failure may pollute another test. This generally means you
have found a dastardly bug in the engine, but it can also be a test issue.
## To run individual test suites, especially for debugging purposes
*Caveat lector*.
1. To run sequentially, IN THE SAME PROCESS: ./EngineTests --gtest_filter=TestSuiteName* (for example, for DeleteKeyTest: ./EngineTests DeleteKeyTest*)
2. To debug the same: lldb ./EngineTests -- --gtest_filter=TestSuiteName*
3. To run sequentially IN DIFFERENT PROCESSES: (FIXME - is this really the case?)
1. To run sequentially, *in the same process*: `./EngineTests
--gtest_filter=TestSuiteName*` (for example, for `DeleteKeyTest`:
`./EngineTests DeleteKeyTest*`)
2. To debug the same with lldb: `lldb ./EngineTests -- --gtest_filter=TestSuiteName*`
3. To debug with gdb: `gdb --args ./EngineTests --gtest_filter=TestSuiteName*`
### To run and/or debug individual test cases
1. To run: `./EngineTests --gtest_filter=TestSuiteName.test_function_name`
(for example, for `check_delete_single_pubkey` in `DeleteKeyTest`:
