Streams and Notifications¶
Genvid uses a concept similar to what modern video containers allow. In addition to providing audio and video streams for each camera, you can specify an arbitrary number of supplemental game streams which can effectively contain any type of data.
You identify a stream with an arbitrary UTF-8 string, called the streamID. Each stream is filled with a series of timestamped data chunks. This allows association of every data element to a specific time, both within its stream and across other streams.
For example, if you send player positions in a game stream you could highlight players in the video stream via an HTML overlay. With some additional code, you could make it possible to select players by clicking directly on them in the video stream.
You can set data rates individually by stream. Because Genvid associates every data with a timecode, it retrieves the proper active value for every stream.
For example, audio systems often refresh every millisecond while video is often locked at 30 Hz (every 33 milliseconds). Similarly, you can refresh game streams at unique intervals. So, you could send the list of players once at the beginning of every game while updating player positions at the same rate as the rendering.
Fig. 43 shows an example of streams that a game could generate. Some notes regarding this example:
- We send audio and video data to the streaming service.
- We send audio data more often than video data.
- We send the player list once when the game starts.
- At every frame, we send the players’ positions and the viewport camera matrices (for a 3D overlay).
- Players occasionally die.
- The camera can switch during the match.
- The game ends after the last kill.
An annotation is like a stream, but it does not represent a state. It represents an event at a fixed point in time. For example, an annotation could be a player kill or critical hit. Annotations are associated with a timecode and don’t last over time.
When jumping from one timecode to another, the Genvid system returns all annotations present during this period. This prevents the system from missing any annotations.
There is a limit of 256 annotations per stream, per frame. Any attempt to add annotations beyond this limit will result in those annotations being dropped. If your design requires a higher annotation frequency, you can either increase the stream framerate or reduce the rate at which annotations are submitted.