Starting a local cluster

Genvid Services depend on a few set of external services to run properly mainly Consul, Vault and Nomad Consul is used for storing configuration inside its key-value store, as well as registering, monitoring discovering the different services available. Vault is used to secure, store and tightly control access to tokens, passwords, certificates, API keys, and other secrets. Nomad is a scheduler. It helps to install, start and monitor different applications, on Linux and on Windows. We also install a small web interface to help you monitor and check on your cluster. Other services will likely be added in the future.

Although our default configuration depends on those services, Consul, Vault and Nomad were chosen for their very low dependencies on their infrastructure. Our application tries to keep our dependencies on them as minimal as possible so that they could easily be replaced to accommodate existing infrastructures.

In this section, we use the genvid-bastion command line tool from the Toolbox. Be sure to have it installed in your PATH as explained in Installation of the Toolbox.

Commands

To start a local cluster, you simply need to run the following command:

genvid-bastion install

This will setup the supervisor services, and run some Genvid Services on top of it, bastion-api. The supervisor services will use a working directory under ~/.genvid but you can change this location by setting GENVID_SERVICES_DATADIR to a valid directory.

Note

Although most of the data are written under the GENVID_SERVICES_DATADIR, the current implementation weren’t able to transfer everything there. Some data, like the terraform’s workdir, must be located next to the terraform configuratiion files under bastion-services/terraform under the installation directory.

Once the cluster is started, you can open the web interface by running:

genvid-bastion monitor

The reference for the other commands of genvid-bastion are available in genvid-bastion.

Ports used

A single cluster can be shared among multiple projects, but only one game can be run on it at a time. You cannot run more than one local cluster per machine, since their ports would conflict. The ports used by the local cluster are the following:

Application TCP Ports UDP Ports
Consul [1] 8300, 8301, 8302, 8400, 8500, 8600 8301, 8302, 8600
Vault [2] 8200  
Nomad [3] 4646, 4647, 4648 4648
Bastion [4] 8092  
Other 20000 - 60000  
[1]Consul ports used
[2]Vault Configuration
[3]Nomad Agent configuration
[4]See GENVID_BASTION_BINDING_PORT

The other applications ports are dynamically allocated by Nomad when scheduling an application to run on the cluster. Also, all services are configured to send metrics on udp://localhost:8125/ as explained in the section on Metrics.

Setting the local IP

By default local.py will bind the IP addresses to 127.0.0.1. In order to use a specific machine IP, you must stop the stop the cluster, set GENVID_DEFAULT_IP to the visible IP of your machine and restart the cluster. You will also need to set the CONSUL_HTTP_ADDR to the same IP:

genvid-bastion uninstall
# With Windows command prompt.  Used export with bash.
set GENVID_DEFAULT_IP=123.123.123.123
set CONSUL_HTTP_ADDR=123.123.123.123
genvid-bastion install