Website Integration

Because every game exposes Genvid differently, every game requires a customized website. This includes both the skinning aspect (HTML and CSS) as well as the functional side (JavaScript). You also need to run a web server hosting the web application for the browser.

Since there are multiple ways of creating websites, this section only covers steps common to the Genvid SDK.

Backend integration

The Genvid web client needs to access a single service, called the leaf, which provides a websocket server from which the client will access the game data. To get the connection URL for this service, two web requests must be sent to the Disco service API:

The APIs use a secret code for security, so they should only be called from a trusted website. In the tutorial website, we route them inside the routes/streams.t.ts file to the /streams and /channels/join URLs, respectively.

The disco service is registered with Consul and you can find its URL this way.

Note

Future versions of the API will expose more API for authenticating the user with a third party service and using this authentication for assigning the websocket, which is the main reason why this API is protected. For now, this functionality is disabled and no authentication is done in this version.

Commands and notifications integration

The command and notification services expose two categories of endpoints:

  • /commands which let you send commands directly to the game, and
  • /notifications which let you send notifications directly to the viewer.

These are powerful functionalities which we recommend you do not expose for public use.

See Command and notification service API for more information.

JavaScript API

Genvid provides two libraries:

  • genvid Contains the logic necessary to connect to the leaf, decode its stream, create a video player, and sync data and video.
  • genvid-math Provides several geometry computation utility functions that you may need to create a WebGL overlay for example. You can find examples using this lib in the Samples.

These two libraries each come in two different bundles:

  • NPM Package archive A gzipped tarball file that you can import as if it were a local NPM package. You can then import your dependencies using the ES6 syntax and bundle your project using the method of your choice. You will have access to type definitions.

    You can see an example of bundling using Webpack in the Unity sample.

  • UMD js module A UMD module that allows you to import the libraries in many ways, such as by exposing the global variables genvid and genvidMath. You won’t have access to the type definitions. All our client samples except Unity use this method.

Note

The preferred method to use the “genvid” and “genvid-math” dependencies is now by importing our NPM packages. We no longer advise you to include our type definitions directly in your source code. We still provide older type definitions for backward compatibility in the api/web/legacy_types folder. All the game integration samples except Unity illustrate this method. For a newer project, we strongly advise you to focus on the Unity sample.

Frontend integration

The first step in integrating Genvid into your frontend is to instantiate a genvidClient.IGenvidClient() object using the genvidClient.createGenvidClient():

let client = genvid.createGenvidClient(streamInfo, websocketURL, websocketToken, video_player_id);

The first parameter, streamInfo, corresponds to an genvidClient.IStreamInfo() structure, typically returned by the backend service from the POST /disco/stream/join call.

The next two parameters, websocketURL and websocketToken, are the two values specifying the websocket address and security token, respectively. They are provided by the backend through the POST /disco/stream/join call.

The last parameter, video_player_id, is a string referencing an HTML element that you want to use for the live streaming video player. When the IGenvidClient() creates the video player, it will replace this HTML element with the one from the live streaming service.

onVideoPlayerReady callback

When you create the player, IGenvidClient() calls the function specified with genvidClient.IGenvidClient.onVideoPlayerReady(). This is typically used to hook up the overlay.

client.onVideoPlayerReady( function(elem) { console.log('Create the overlay!'); } );

onAuthenticated callback

The onAuthenticated() callback tells the user when the IGenvidClient() successfully connects to the Genvid Services:

client.onAuthenticated( function(succeed) { if (succeeded) { console.log('Connected!'); } });

onStreamsReceived callback

The onStreamsReceived() callback tells you when you receive new stream data. It’s useful for decoding or analyzing the data before it’s rendered.

client.onStreamsReceived((streams) => { myGameOverlay.onStreamsReceived(streams); });

onNotificationsReceived callback

Notifications transfer information as fast as possible and are not related to a timecode. They are described with the IDataNotifications() interface.

client.onNotificationsReceived((notifications) => { myGameOverlay.onNotificationsReceived(notifications); });

onDraw callback

The onDraw() callback specifies a routine to call regularly (generally 30 times per second) to draw the overlay:

client.onDraw((frame) => { myGameOverlay.onDraw(frame); });

When you invoke the onDraw() callback, it receives an genvidClient.IDataFrame() instance as a parameter containing the timecode for this video frame, the data coming from the different streams, and information about the video composition. The frame data is organized in the following ways:

  • If you have several sources of data, you can obtain it inside the sessions property, which exposes streams and annotations data for each session.
  • If you only have one stream, you can also access to the streams and annotations in the root of the IDataFrame instance.

Composition data (an array of genvidClient.ISourceCompositionData()) is information about what transformations were applied to the different video streams. Some of the ways you can use it include knowing what pixel of the video overlay matches which video source or to performing a clipping operation to prevent an overlay related to one video source overlaying another video.

Streams and annotation behave differently but are both described with the structure genvidClient.IDataStreamFrame().

The current streams contain only the latest frame for each while the annotations hold all of the previous frames accumulated so far and they will not get repeated next time.

Genvid carries data in binary form using the field rawdata, which is a JavaScript ArrayBuffer(). You can interpret the data in any way, but Genvid provides a few utility routines which can help decoding.

For example, if the game sends JSON data encoded as UTF-8, the website code needs to decode the rawdata binary field as UTF-8, and then parse that resulting string as JSON.

onDraw(frame) {
   let stream = frame.streams["position"]
   let datastr = genvid.UTF8ToString(frame.rawdata);
   stream.user = JSON.parse(datastr);
   console.log(stream.user.myPosition);
}

Because Genvid sometimes repeats data (for example, when a stream has a low framerate value), we include a mechanism for avoiding decoding identical data multiple times. The IGenvidClient() includes an onStreamsReceived() which passes a genvidClient.IDataStreams() collection upon reception of the data. The data streams contain a collection of video streams and their frames which you can modify before they get integrated into the Genvid synchronization engine. For example, parsing a collection to detect an upcoming event or removing the collection entirely.

You could also modify the function to decode differently based on the streamId, as long as you’re consistent with the format used when sending data from the game process (see Game-Data Streaming).

Once everything is ready in the game’s website code, you can start the GenvidClient streaming:

client.start();

The IGenvidClient() automatically uses configured callbacks and handles synchronization between the streaming video and the game data sent in the onDraw().

onDisconnect callback

The onDisconnect() callback gets triggered upon a websocket closing. You can bind functionality to this callback and be alerted when a socket closes:

client.onDisconnect(() => {this.onDisconnectDetected();});

reconnect callback

The reconnect() callback establishes a new websocket connection. It receives new stream info, a new leaf URI, and a new token as parameters.

client.reconnect(info, uri, token);

Overlay

While Genvid doesn’t provide a strict overlay API, it does expose everything necessary in the IGenvidClient() for the overlay to work. We also expose WebGL utility-routines which we use for all of our samples.

The main entry point to the overlay is the callback set using onDraw(). On regular intervals, the specified callback is invoked, with a frame of data for all streams existing in the game.

This callback receives the latest game-data frame for every stream. You have full control of what to do with that data: render some 3D highlights in a WebGL canvas, tweak HTML elements to display current game stats, adjust button visibility to allow new game events from the spectator, etc.

To facilitate WebGL rendering, Genvid provides a genvidWebGL.IWebGLContext() class which simplifies repetitive tasks in WebGL. You can create it with the genvidWebGL.createWebGLContext() method.

Events

You can send events back to the Genvid Services through the IGenvidClient() instance. See sendEvent() and sendEventObject() for more information.

Google Chrome Autoplay Support

Google Chrome has a special policy for handling video playback. In our API, all video players that support it have the autoplay tag set. This enables starting video playback automatically on browsers like Firefox, as well as on websites with the Media Engagement Index set high enough.

If you want to enable automatic video-playback on most browsers, including Chrome, you can set the video to mute. You also need to ensure the overlay is hidden when the video is paused to avoid locking the user in front of a paused video. The following code is an example of how to do both.

client.onVideoPlayerReady( (elem) => {

    // Optional: Set to muted to autostart even on Chrome and also iOS.
    client.videoPlayer.setMuted(true);

    // Always safe to hide the overlay on startup.
    // The PLAYING event below will show it.
    myGameOverlay.hide();

    client.videoPlayer.addEventListener(genvid.PlayerEvents.PAUSE, () => {
      myGameOverlay.hide();
    });

    client.videoPlayer.addEventListener(genvid.PlayerEvents.PLAYING, () => {
      myGameOverlay.show();
    });
});