|
|
@ -1,62 +1,145 @@ |
|
|
|
# Building the Engine Test Suite |
|
|
|
# Building and Running the Tests for the p≡p Engine |
|
|
|
|
|
|
|
Work in progress... Currently have the debian build/run instructions in. |
|
|
|
|
|
|
|
## Caveat, before you begin |
|
|
|
|
|
|
|
Right now, the engine tests only function on \*nix-like systems (including MacOS). (Conversion to Windows will require, at the very least, looking at some of the file-handling code.) If you want to fix this, start by looking in Engine.cc in the test/src directory! |
|
|
|
Right now, the engine tests only function on \*nix-like systems (including |
|
|
|
MacOS). |
|
|
|
|
|
|
|
*(Conversion to Windows will require, at the very least, looking at some of the |
|
|
|
file-handling code. If you want to fix this, start by looking in Engine.cc |
|
|
|
in the test/src directory!)* |
|
|
|
|
|
|
|
## Requirements |
|
|
|
|
|
|
|
In addition to the engine requirements, you will need: |
|
|
|
|
|
|
|
* cmake |
|
|
|
* python3 |
|
|
|
* `cmake` `python3` `git` (for getting the `gtest-parallel` repository, unless |
|
|
|
* you grab the tarball from somewhere) |
|
|
|
|
|
|
|
## Building the prerequisites |
|
|
|
|
|
|
|
The Engine test suite now requires (at least) two additional pieces to run: |
|
|
|
* `googletest` |
|
|
|
* `gtest-parallel` |
|
|
|
|
|
|
|
How this proceeds depends on your platform and whether or not you use a packaged |
|
|
|
distribution. |
|
|
|
|
|
|
|
These instructions do this with `cmake`. If you can manage it with `bazel` |
|
|
|
instead, more power to you ;) |
|
|
|
|
|
|
|
### Installing `googletest` |
|
|
|
|
|
|
|
#### Packaged distributions |
|
|
|
|
|
|
|
This is the currently preferred way to do this, because everyone was doing it |
|
|
|
anyway and who am I to judge? |
|
|
|
|
|
|
|
##### Debian and Ubuntu (and derivatives) |
|
|
|
|
|
|
|
## Preparing to build |
|
|
|
Thanks to Erik Smistad for this starting point (condensed from [Getting Started |
|
|
|
with Google Test On |
|
|
|
Ubuntu](https://www.eriksmistad.no/getting-started-with-google-test-on-ubuntu/)): |
|
|
|
|
|
|
|
The Engine test suite now requires (at least) two additional pieces to run - **googletest** and **gtest-parallel**. You will note that I give specific instructions about where to put these, because that is what I have tried and tested. That does NOT mean other things won’t work; I simply haven’t tried them. So without further ado… |
|
|
|
1. Install the packages `cmake` and `libgtest-dev` from the repository. This |
|
|
|
will install the gtest source files to `/usr/src/gtest`. You'll still need to |
|
|
|
compile the code and link the library files to be able to use them. |
|
|
|
|
|
|
|
### googletest |
|
|
|
2. Compile the source files: |
|
|
|
``` |
|
|
|
cd /usr/src/gtest |
|
|
|
sudo cmake CMakeLists.txt |
|
|
|
sudo make |
|
|
|
``` |
|
|
|
|
|
|
|
**googletest** is an XUnit testing framework we are now using in place of cpptest. Unlike a lot of other testing frameworks, it’s recommended that you compile and link the test code directly within your project. These instructions do with with **cmake**. If you can manage it with **bazel** instead, more power to you ;) |
|
|
|
3. Copy/symlink the libraries to the library location of your choice (here, |
|
|
|
it's `/usr/lib`, hence the `sudo`, but as long as it's in your library path, |
|
|
|
it shouldn't matter where you stick it): |
|
|
|
``` |
|
|
|
sudo cp *.a /usr/lib |
|
|
|
``` |
|
|
|
|
|
|
|
So. To get things started. |
|
|
|
##### MacOS |
|
|
|
|
|
|
|
In the directory of your choice (default, if you don’t want to change **local.conf** - specifically **GTEST\_DIR** - is the test directory (this one, presumably)): |
|
|
|
I am totally guessing for now - this is a placeholder - that |
|
|
|
macports gtest install will do the same. Will need to find the directories this |
|
|
|
goes in. Guessing `/opt/local/src`. |
|
|
|
|
|
|
|
1. git clone https://github.com/google/googletest.git |
|
|
|
2. cd googletest |
|
|
|
3. cmake . |
|
|
|
4. make |
|
|
|
#### Downloading and compiling the source yourself |
|
|
|
|
|
|
|
(Note that this hasn’t been tested in other directories, so I am presuming the Makefile works as is, but I could be wrong) |
|
|
|
For now, don't. |
|
|
|
|
|
|
|
### gtest-parallel |
|
|
|
Or do, and document it for me. |
|
|
|
|
|
|
|
### Again, in the directory of your choice (if you want to use the Makefile out of the box, you should, while still in the googletest directory, do the following): |
|
|
|
If you were using the git repo and it was working before, please follow the |
|
|
|
instructions above for Debian/Ubuntu, only with your source repository in mind |
|
|
|
instead of `/usr/src`, and pay attention to the variables you'll need to set in |
|
|
|
`local.conf` for the Makefile - they are different from before. |
|
|
|
It should work, but I haven't tested it yet. |
|
|
|
|
|
|
|
1. git clone https://github.com/google/gtest-parallel.git |
|
|
|
2. If using a different directory, please change **GTEST\_PL\_DIR** to indicate where **gtest-parallel.py** is located. |
|
|
|
### Installing `gtest-parallel` |
|
|
|
|
|
|
|
Pick a source directory and put your `gtest-parallel` source there |
|
|
|
(e.g. via `git clone https://github.com/google/gtest-parallel.git`). |
|
|
|
|
|
|
|
We'll deal more with this when preparing to compile the test suite. |
|
|
|
|
|
|
|
## Building the test suite |
|
|
|
|
|
|
|
### `Makefile` and `local.conf` |
|
|
|
|
|
|
|
So `local.conf` in the top-level engine directory is where we stick all of the |
|
|
|
Makefile overrides. The test Makefile contains some defaults for relevant |
|
|
|
variables here, but if you need to override them, please either create or modify |
|
|
|
`local.conf` in the top-level engine directory as needed. The relevant variables |
|
|
|
are: |
|
|
|
|
|
|
|
* `GTEST_SRC_DIR`: This is the directory where the gtest source you compiled |
|
|
|
above is located (defaults to `/usr/src/gtest`) |
|
|
|
* `GTEST_INC_DIR`: This is where the include files for gtest are located |
|
|
|
(defaults to `$(GTEST_SRC_DIR)/include`) |
|
|
|
* `GTEST_PL`: This is the full path to the *python file* for `gtest_parallel` |
|
|
|
(default presumes you cloned it under `src` in your home directory, i.e. it is |
|
|
|
`$(HOME)/src/gtest-parallel/gtest_parallel.py`) |
|
|
|
|
|
|
|
### Building |
|
|
|
|
|
|
|
Presuming the above works, then from the top test directory, simply run make. |
|
|
|
|
|
|
|
# Running the Engine Test Suite |
|
|
|
## Running the test suite |
|
|
|
|
|
|
|
## To simply run the test suite and see what tests fail... |
|
|
|
### To simply run the test suite and see what tests fail... |
|
|
|
|
|
|
|
Do one of: |
|
|
|
|
|
|
|
1. make test OR |
|
|
|
2. python3 \<path to gtest-parallel.py\> ./EngineTests |
|
|
|
1. `make test` OR |
|
|
|
2. `python3 <path to gtest-parallel.py> ./EngineTests` |
|
|
|
|
|
|
|
### To run individual test suites, especially for debugging purposes |
|
|
|
|
|
|
|
Note that for some test suites, this will, if something goes dreadfully wrong, |
|
|
|
mean that one test's failure may pollute another test. This generally means you |
|
|
|
have found a dastardly bug in the engine, but it can also be a test issue. |
|
|
|
|
|
|
|
## To run individual test suites, especially for debugging purposes |
|
|
|
*Caveat lector*. |
|
|
|
|
|
|
|
1. To run sequentially, IN THE SAME PROCESS: ./EngineTests --gtest_filter=TestSuiteName* (for example, for DeleteKeyTest: ./EngineTests DeleteKeyTest*) |
|
|
|
2. To debug the same: lldb ./EngineTests -- --gtest_filter=TestSuiteName* |
|
|
|
3. To run sequentially IN DIFFERENT PROCESSES: (FIXME - is this really the case?) |
|
|
|
1. To run sequentially, *in the same process*: `./EngineTests |
|
|
|
--gtest_filter=TestSuiteName*` (for example, for `DeleteKeyTest`: |
|
|
|
`./EngineTests DeleteKeyTest*`) |
|
|
|
2. To debug the same with lldb: `lldb ./EngineTests -- --gtest_filter=TestSuiteName*` |
|
|
|
3. To debug with gdb: `gdb --args ./EngineTests --gtest_filter=TestSuiteName*` |
|
|
|
|
|
|
|
### To run and/or debug individual test cases |
|
|
|
1. To run: `./EngineTests --gtest_filter=TestSuiteName.test_function_name` |
|
|
|
(for example, for `check_delete_single_pubkey` in `DeleteKeyTest`: |
|
|
|
`./EngineTests DeleteKeyTest.check_delete_single_pubkey`) |
|
|
|
2. To debug the same with lldb: |
|
|
|
`lldb ./EngineTests -- --gtest_filter=TestSuiteName.test_function_name` |
|
|
|
3. To debug with gdb: |
|
|
|
`gdb --args ./EngineTests --gtest_filter=TestSuiteName.test_function_name` |
|
|
|
|
|
|
|
# Creating new tests |
|
|
|
|
|
|
|
Script next on the agenda... |
|
|
|
|