Using the load tester

This sample shows how to execute a load test. You need two clusters to run the sample. One cluster runs the Genvid stack and the other executes the load test.

See The Cloud Environment for information about running your game in the cloud.

This sample sends metrics to statsd. You can use the TICK Stack Metrics to collect and see them.

Create the test cluster

We assume that the tutorial sample is already running on the cloud for this demonstration.

The first step is to create a cluster to run your test.

  • Go to Bastion UI and create a cluster using the load_test source.
  • Click on the infrastructure link. (You can keep the default settings for now.)
  • Click on the Commands menu.
  • Click Apply and wait until it is done.

Initializing the load-test cluster

The first step is to initialize the cluster with the cluster-api. This lets you configure the cluster as it runs your game.

genvid-sdk -c testcluster upload-images cluster-api --update-config
genvid-sdk -c testcluster setup

You can now load the load tester into your cluster.

py loadtester.py build-cloud
py loadtester.py -c testcluster upload-images --update-config
py loadtester.py -c testcluster load

Optionally, you can use the TICK Stack Metrics sample in your cluster.

py tick.py -c testcluster download
py tick.py -c testcluster setup
py tick.py -c testcluster load

Using the load-test sample

The following command starts the test.

py loadtester.py -c testcluster start targetcluster

When the tests are started you can retrieve some information with:

py loadtester.py -c testcluster monitor targetcluster

This prints the health check of the targeted leaf services. To stop the test, run the command:

py loadtester.py -c testcluster stop

Looking at the metrics

Because load tests need metrics, the application sends data to statsd. You can view the metrics with the TICK Stack Metrics sample. Some important metrics to look at:

  • loadtest.session.total: The total number of connections.
  • loadtest.session.error: The number of errors while connecting.
  • loadtest.session.retry: The number of retried connections.
  • nomad.ip-xxxx.client.host.cpu.xxxx.cpux.total: The CPU usage of the machine running the test app.

Connection errors can occur for multiple reasons. The first one is that the machine CPU running the app is overloaded, which usually generates inconclusive results. The second is that the Genvid service is overloaded, which means that the system is at max capacity.

Warning

When running your test, high CPU usage causes inconclusive tests. You can change the number of connections in the config/loadtester.hcl file.