Generalized Streams

Genvid uses a Generalized Stream concept similar to what modern video containers allow. In addition to providing audio and video streams for each camera, you can specify an arbitrary number of supplemental game streams which can effectively contain any type of data.

You identify a stream with an arbitrary UTF-8 string, called the streamID. Each stream is filled with a series of timestamped data chunks. This allows association of every data element to a specific time, both within its stream and across other streams.

For example, if you send player positions in a game stream you could highlight players in the video stream via an HTML overlay. With some additional code, you could make it possible to select players by clicking directly on them in the video stream.

You can set data rates individually by stream. Because Genvid associates every data with a timecode, it retrieves the proper active value for every stream.

For example, audio systems often refresh every millisecond while video is often locked at 30 Hz (every 33 milliseconds). Similarly, you can refresh game streams at unique intervals. So, you could send the list of players one time at the beginning of every game while updating player positions at the same rate as the rendering.

Generalized Streams

Fig. 8 Example of Generalized Streams

Fig. 8 shows an example of streams that a game could generate. Some notes regarding this example:

  • We send audio and video data to the streaming service.
  • We send audio data more often than video data.
  • We send the player list once when the game starts.
  • At every frame, we send the players’ positions and the viewport camera matrices (for a 3D overlay).
  • Players occasionally die.
  • The camera can switch during the match.
  • The game ends after the last kill.

Annotation Streams

An annotation is like a stream, but it does not represent a state. It represents an event at a fixed point in time. For example, an annotation could be a player kill or critical hit. Annotations don’t last over time.

When jumping from one timecode to another, the Genvid system returns all annotations present during this period. This prevents the system from missing any annotations.


The system sends notifications to the browser as fast as possible and isn’t associated with a time code. For example, a notification could tell the browser that the game is finished or send game results.