Genvid Cluster Servers¶
Supervisor Servers¶
The supervisor servers are usually a small group of micro servers that act as a point of registration and authority for all the other services. A single server is enough to run the cluster, but usually, 3 or more servers are preferred to achieve high availability of the cluster. The group of servers will keep the state of the orchestration services consistent using a gossip protocol with a simple majority, allowing half of the supervisor servers to go down without affecting the service. The principal tasks run by the servers are:
- High-Available Key-Value store for configuration.
- Secure Key-Value store that secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets (See Vault)
- Service registration exposed both through a DNS and HTTP API.
- Monitoring all services and reporting the health of the different worker instances.
- Scheduler service able to scale up and down the different services and restart them when they go down.
Given the criticality of the services running on them, the supervisor servers usually don’t run any other task, instead the supervisor servers delegate tasks to the worker servers.
Public and Internal Workers¶
All worker servers run a client version of the orchestration services, reporting to the supervisor servers. The worker servers are responsible for executing and monitoring the tasks scheduled by the supervisor servers. The difference between the public and internal workers is that only the public workers are accessible from an external, unregistered IP. The internal workers are only accessible to a limited set of pre-registered network range, as a security precaution.
Being on the edge of the network, the public workers usually require far more running instances than the internal workers. In the future, automation functionality will be added to the services to allow automatic provisioning and deallocation of servers when the load goes up and down. The internal workers have a far more stable load, usually related only to the game needs, although internal workers also run the Messaging Bus which has a load proportional to the number of clients, and a portion of the load supported by the public workers.