We run tests using the github-actions infrastructure. The following steps are necessary to set-up a new runner machine (and connect a github repo to it).
Steps to add a new CI machine:
- Install github-runner software on a new test machine
- Give access to the benchmark repository
- Configure software for the
- Disable AppArmor
- Install a recent QEMU
- Do a test-run
- Start the new runner
Create a github-runner user first:
sudo useradd github-runner -m -s /bin/zsh
Add sudo capability for github-runner:
sudo visudo # github-runner ALL=(ALL) NOPASSWD: ALL
For better security with self-hosted code exeuction, make sure to enable
Settings -> Actions -> Runners -> Require approval for all outside collaboratorsin the github repo settings!
Other than that, follow the steps listed under
Settings -> Actions -> Runner -> Add runner:
sudo su github-runner cd $HOME << steps from Web-UI >>
When asked for labels, make sure to give it a machine specific tag. For example,
we currenly use the following labels
ryzen5 to indicate different machine type and the number of sockets/NUMA
nodes. Machines with identical hardware should have the same tag to allow
parallel test execution.
If you add a new machine label, make sure to also add it to
utils.pyin the CI website
Don't launch the runner yet with
run.sh (this happens further below in the doc).
Benchmark results are uploaded automatically to git.
Generate a key for accessing the repository or use an existing key on the github-runner account. Also add the user to the KVM group. Adding yourself to the KVM group requires a logout/reboot which we do in later steps.
sudo adduser github-runner kvm ssh-keygen
Then, add the pub key (
.ssh/id_rsa.pub) to the github CI account.
Install necessary software for use by the runner:
git clone firstname.lastname@example.org:vmware-labs/node-replicated-kernel.git nrk cd nrk/ bash setup.sh source $HOME/.cargo/env
The memcached benchmark uses the
memaslap binary that comes with
libmemcached but is not included in the Ubuntu libmemcached-tools deb package.
You'll have to install it manually from the sources:
cd $HOME sudo apt-get build-dep libmemcached-tools wget https://launchpad.net/libmemcached/1.0/1.0.18/+download/libmemcached-1.0.18.tar.gz tar zxvf libmemcached-1.0.18.tar.gz cd libmemcached-1.0.18/ LDFLAGS='-lpthread' CXXFLAGS='-fpermissive' CFLAGS='-Wno-errors -fpermissive' ./configure --enable-memaslap --with-pthread=yes make -j12 sudo make install sudo ldconfig which memaslap
An annoying security feature that blocks our DHCP server from starting for testing. You can set-up a rule for allowing this but it's easiest to just get rid of it on the CI machine:
sudo systemctl stop apparmor sudo systemctl disable apparmor sudo apt remove --assume-yes --purge apparmor # Unfortunately for apparmor and kvm group changes to take effect, we need to reboot: sudo reboot
After the reboot, verify that the nrk tests pass (this will take a while, but if it works CI will likely succeed too):
# Init submodules if not done so already: cd nrk git submodule update --init source $HOME/.cargo/env cd kernel RUST_TEST_THREADS=1 cargo test --features smoke -- --nocapture
Finally, launch the runner:
cd $HOME/actions-runner source $HOME/.cargo/env ./run.sh
cd $HOME/actions-runner sudo ./svc.sh install sudo ./svc.sh start
Check the runner status with:
sudo ./svc.sh status
Stop the runner with:
sudo ./svc.sh stop
Uninstall the service with:
sudo ./svc.sh uninstall
If the repo is migrated to a new location, the following settings should be mirrored:
- Under Settings -> Secrets: Add secret
WEBSITE_DEPLOY_SSH_KEYwhich contains a key to push the generated documentation to the correct website repository.
- Under Settings -> Options: Disable "Allow merge commits"
- Under Settings -> Branches: Add branch protection for
master, enable the following settings:
- Require pull request reviews before merging
- Dismiss stale pull request approvals when new commits are pushed
- Require status checks to pass before merging
- Require branches to be up to date before merging
- Require linear history
- Under Settings -> Actions -> Runners:
- "Require approval for all outside collaborators"