Generalized Streams

Genvid relies upon the concept of a Generalized Stream. In addition to providing audio and video streams (one for each camera), Genvid allows to specify an arbitrary number of supplemental game streams, which can effectively contain any type of data. This concept is similar to what modern video containers allow.

Every stream is uniquely identified using a streamID, an arbitrary UTF-8 string. The stream is filled with a series of data chunks, each properly timestamped. This allows association of every data element to a specific time, not only within a stream, but also across the various streams. For example, by sending player positions in a game stream, it is possible to highlight players in the video stream by drawing elements in an HTML overlay. With the proper effort, it is even possible to select players by clicking directly on the video stream.

Each stream can specify its data at a rate unrelated to the other ones. For example, it is not uncommon that audio systems refresh every millisecond, whereas video could be locked at 30 Hz (i.e. every 33 milliseconds). Similarly, game streams can be refreshed at unique intervals. As such, the list of players could be sent only once at the beginning of every game, while the player positions might need to be sent at the same rate as the rendering. Because Genvid associates every data with a timecode, it is possible to retrieve the proper active value for every one of the streams.

Generalized Streams

Fig. 25 Example of Generalized Streams

Fig. 25 shows an example of streams that a game could generate. Some notes regarding this example:

  • Audio and video data will be sent to the streaming service.
  • Audio data is sent more often than video data.
  • When the game starts, the player list is sent only once.
  • At every frame, the players’ positions and the viewport camera matrices are sent (for a 3D overlay).
  • Once in a while, a player gets killed.
  • The camera can switch during the match.
  • The game ends after the last kill.

Annotation Streams

An annotation is similar to a stream, but it does not represent a state. Instead we can see it as a fixed event in time. For example, it could represent a player kill or a critical hit. Annotations do not last in time.

The Genvid system ensures that an annotation won’t be missed. When jumping from one timecode to another, the system will return all the annotations present during this period.

Notifications

A notification is communicated to the browser as fast as possible and is not associated with a time code. For example, a notification could be used to tell the browser that the game is finished or to send other game results.