Tutorial Sample Application

To use the Genvid SDK, you have to integrate it in various sections of your application. This section shows how we integrate the SDK with the Tutorial Sample using code samples and line numbers to help you navigate the tutorial application.

Before You Begin

Setting up and running the Tutorial Sample on a Local Environment will help you better understand the integration steps in this section. Check out The Tutorial Sample to see how to do that.

Classes

Tutorial.cpp	This file contains the window initialization process, the DirectX rendering, the game interaction controls, and all of the Genvid SDK integration.
Cube.h	Header file for the cubes displayed in the game.
Cube.cpp	Class that contains all the transformation functions for the cubes.
DDSTextureLoader.h	Header file for loading a DDS texture and creating a Direct3D 11 runtime resource for it.
DDSTextureLoader.cpp	Class that contains all the functions for loading a DDS texture and creating a Direct3D 11 runtime resource for it.
JsonConversion.h	Header file for converting various data into JSON format to send via the stream.
JsonConversion.cpp	Class that converts various data types into JSON format to send via the stream.

Header and Declarations

We start by including the Genvid Native SDK header file and declaring a few static variables.

#include <genvid.h>

static const std::string sStream_Audio         = "Audio";
static const std::string sStream_Video         = "Video";
static const std::string sStream_GameData     = "GameData";
static const std::string sStream_Popularity = "Popularity";
static const std::string sStream_ColorChanged  = "ColorChanged";
static const std::string sStream_GameCopyright = "GameCopyright";

static const std::string sEvent_changeColor = "changeColor";
static const std::string sEvent_reset       = "reset";
static const std::string sEvent_cheer       = "cheer";

static const std::string sCommand_speed     = "speed";
static const std::string sCommand_direction = "direction";
static const std::string sCommand_reset     = "reset";

static std::string oldPopularityJSON;

static void GenvidSubscriptionCallback(const GenvidEventSummary * summary, void * userData);

static void GenvidSubscriptionCommandCallback(const GenvidCommandResult * summary, void * userData);

GenvidTimecode           gCurrentTc       = -1;
GenvidTimecode           gPrevTc          = -1;
GenvidTimecode           gFirstFrameTc  = -1;
system_clock::time_point gStartTime       = system_clock::now();
double                   gWorldTime       = 0.0;
float                    gGenvidFramerate = 30.0f; // uses default value
float                    gGameFramerate   = 30.0f; // uses default value
bool                     gEnableVSync    = true;

bool gSilent = false; // Global flag to kill all sounds.

XMVECTORF32 gBackgroundColor        = Colors::Blue;
XMVECTORF32 gBackgroundColorArray[] = {Colors::Blue, Colors::Green, Colors::Red, Colors::Orange, Colors::Bisque, Colors::CadetBlue, Colors::Indigo, Colors::Yellow};

XMFLOAT4 gColors[] = {
    {0.6f, 0.1f, 0.4f, 1.0f},
    {0.2f, 0.9f, 0.1f, 1.0f},
    {0.7f, 0.7f, 0.7f, 1.0f},
    {0.9f, 0.9f, 0.1f, 1.0f},
    {0.1f, 0.2f, 0.3f, 1.0f},
    {0.3f, 0.3f, 0.3f, 1.0f},
    {0.2f, 0.4f, 0.6f, 1.0f},
    {0.5f, 0.3f, 0.2f, 1.0f},
    {0.1f, 0.5f, 0.9f, 1.0f},
    {0.4f, 0.2f, 0.4f, 1.0f},
};

std::map<std::string, XMFLOAT4> sNameToColor = {
    {"rose", gColors[0]},
    {"green", gColors[1]},
    {"white", gColors[2]},
    {"yellow", gColors[3]},
    {"dark blue", gColors[4]},
    {"gray", gColors[5]},
    {"light blue", gColors[6]},
    {"orange", gColors[7]},
    {"blue", gColors[8]},
    {"purple", gColors[9]},
};

Initialization

After setting up DirectX in InitDevice(), we call InitGenvid() which does the following:

• Initializes the Genvid library.
• Defines all of the streams that we will need.
• Configures all the event and command subscriptions.

In this sample we have only one video stream and one audio stream.

We send the video IDXGISwapChain via Genvid_SetParameterPointer().

We create streams to send the game data to the servers: One for the popularity change and one for the the color change.

Finally, we create all the subscribes that we need to catch the events and commands coming from the webpage.

In this case, we catch 3 actions that can occur from the webpage:

• A color change.
• A position reset.
• A popularity change.

We also catch 3 actions that can occur from the webpage as a command:

• A speed change.
• A direction change.
• A position reset.

The function GenvidSubscriptionCallback() manages the event-catching portion which we display later. Because we send 3 types of events from the webpage, we need to call Genvid_Subscribe() 3 times with the proper names.

Note

The names we use while calling Genvid_Subscribe() must match the ones used on the webpage and in the map-reduce rules.

HRESULT InitGenvid()
{
    GenvidStatus gs;

    // Initialize the Genvid Native SDK.
    gs = Genvid_Initialize();
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // Create the audio stream.
    gs = Genvid_CreateStream(sStream_Audio.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // Specify auto-capture video source.
    gs = Genvid_SetParameterInt(sStream_Audio.c_str(), "Audio.Source.WASAPI", 1);
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // Create the video stream.
    gs = Genvid_CreateStream(sStream_Video.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    if (gGenvidFramerate > 0)
    {
        gs = Genvid_SetParameterFloat(sStream_Video.c_str(), "framerate", gGenvidFramerate);
        if (GENVID_FAILED(gs))
            return E_FAIL;
    }

#if GENVID_USE_DXGISWAPCHAIN
    // Specify auto-capture video source.
    gs = Genvid_SetParameterPointer(sStream_Video.c_str(), "Video.Source.IDXGISwapChain", g_pSwapChain);
    if (GENVID_FAILED(gs))
        return E_FAIL;
#endif

    // Create stream for game data.
    gs = Genvid_CreateStream(sStream_GameData.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    gs = Genvid_CreateStream(sStream_GameCopyright.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // this frame is receive one time only by each viewer when genvid client is connected
    const char copyright[] = "Copyright Genvid Technologies 2019";
    gs                     = Genvid_SubmitGameData(-1, sStream_GameCopyright.c_str(), copyright, (int)(sizeof copyright));

    if (GENVID_FAILED(gs))
        return E_FAIL;

    gs = Genvid_CreateStream(sStream_Popularity.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    gs = Genvid_CreateStream(sStream_ColorChanged.c_str());
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // Subscribe to events.
    gs = Genvid_Subscribe(sEvent_changeColor.c_str(), &GenvidSubscriptionCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;
    gs = Genvid_Subscribe(sEvent_reset.c_str(), &GenvidSubscriptionCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;
    gs = Genvid_Subscribe(sEvent_cheer.c_str(), &GenvidSubscriptionCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;

    // Subscribe to commands.
    gs = Genvid_SubscribeCommand(sCommand_speed.c_str(), &GenvidSubscriptionCommandCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;
    gs = Genvid_SubscribeCommand(sCommand_direction.c_str(), &GenvidSubscriptionCommandCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;
    gs = Genvid_SubscribeCommand(sCommand_reset.c_str(), &GenvidSubscriptionCommandCallback, nullptr);
    if (GENVID_FAILED(gs))
        return E_FAIL;

    return S_OK;
}

Streaming

Right before presenting the back buffer, we auto-capture the video frame (using the previously specified IDXGISwapChain) and send it to Genvid.

Next, we gather the following game data:

• Cube position.
• Orientation.
• Camera matrix.

Then we format the game data into JSON for the website.

We submit the game data with the same timecode as the video data using the utility routine GetGameDataJSON().

Then we gather the popularity data and verify if there were any changes. If there was a change, we perform a Genvid_SubmitNotification() with the updated popularity data formatted into JSON.

Finally, we call Genvid_CheckForEvents() to see if we received any events from the website.

GenvidStatus   gs = GenvidStatus_Success;
GenvidTimecode tc = Genvid_GetCurrentTimecode();

std::string gameData = GetGameDataJSON();
gs                   = Genvid_SubmitGameData(tc, sStream_GameData.c_str(), gameData.data(), (int)gameData.size());
assert(!GENVID_FAILED(gs));

// Send the popularity as a notification
std::string popularityJSON = GetPopularityJSON();
if (popularityJSON != oldPopularityJSON)
{
    gs = Genvid_SubmitNotification(sStream_Popularity.c_str(), popularityJSON.data(), (int)popularityJSON.size());
    assert(!GENVID_FAILED(gs) || gs == GenvidStatus_Disconnected);
}
oldPopularityJSON = popularityJSON;

gs = Genvid_CheckForEvents();
assert(!GENVID_FAILED(gs));

GenvidSubscriptionCallback

Calling Genvid_CheckForEvents() might invoke the function GenvidSubscriptionCallback(). This function catches the events sent from the webpage to the game. When invoked, we:

1. Evaluate the summary ID value to know which event was sent, then
2. We interpret the result key fields as parameters to the event.

//--------------------------------------------------------------------------------------
// Callback invoked when Genvid Events are received.
//--------------------------------------------------------------------------------------
void GenvidSubscriptionCallback(const GenvidEventSummary * summary, void * /*userData*/)
{
    if (summary->id == sEvent_changeColor)
    {
        // Change the color of the cube.
        const char * cubeName  = summary->results[0].key.fields[0];
        const char * cubeColor = summary->results[0].key.fields[1];
        // Finding the cube and change the color
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                std::map<std::string, XMFLOAT4>::iterator colorToFind = sNameToColor.find(cubeColor);
                if (colorToFind != sNameToColor.end())
                {
                    g_cubeList[i].SetColor(colorToFind->second);
                }
            }
        }
        OnUpdateColor();
    }
    else if (summary->id == sEvent_reset)
    {
        // Reset the cube position.
        const char * cubeName = summary->results[0].key.fields[0];
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                g_cubeList[i].ResetPosition();
                return;
            }
        }
    }
    else if (summary->id == sEvent_cheer)
    {
        // Handle cube cheering.
        const char * cubeName = summary->results[0].key.fields[0];
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                // GENVID - Get cheer min value.
                g_cubeList[i].Cheer(float(summary->results[0].values[0].value));
                // GENVID - Cheer value set.
                return;
            }
        }
    }
}

The summary structure is consistent with the map-reduce definition that we provided in config/events.json.

{
  "version": "1.7.0",
  "event": {
    "game": {
      "maps": [
        {
          "id": "changeColor",
          "source": "userinput",
          "where": {"key": ["changeColor", "<name>"], "name": "<color>", "type": "string"},
          "key": ["changeColor", "<name>", "<color>"], "value": 1
        },
        {
          "id": "reset",
          "source": "userinput",
          "where": {"key": ["reset"], "name": "<name>", "type": "string"},
          "key": ["reset", "<name>"], "value": 1
        },
        {
          "id": "cheer",
          "source": "userinput",
          "where": {"key": ["cheer"], "name": "<name>", "type": "string"},
          "key": ["cheer", "<name>"], "value": 1
        }
      ],
      "reductions": [
        {
          "id": "changeColor",
          "where": {"key": ["changeColor", "<name>", "<color>"]},
          "key": ["<name>", "<color>"],
          "value": ["$count"],
          "period": 250
        },
        {
          "id": "reset",
          "where": {"key": ["reset", "<name>"]},
          "key": ["<name>"],
          "value": ["$count"],
          "period": 250
        },
        {
          "id": "cheer",
          "where": {"key": ["cheer", "<name>"]},
          "key": ["<name>"],
          "value": ["$sum"],
          "period": 250
        }
      ]
    }
  }
}

GenvidSubscriptionCommandCallback

Calling Genvid_CheckForEvents() might also invoke the function GenvidSubscriptionCommandCallback(). This is where commands are interpreted.

The command result value contains an ID and a value which we parse into their own strings to use in the method.

void GenvidSubscriptionCommandCallback(const GenvidCommandResult * result, void * /*userData*/)
{
    std::string id(result->id);
    std::string value(result->value);
    if (id == "direction")
    {
        std::vector<std::string> elems;
        split(value, ':', elems);
        auto cubeName = elems[0];
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                float x = (float)std::atof(elems[1].c_str());
                float y = (float)std::atof(elems[2].c_str());
                float z = (float)std::atof(elems[3].c_str());
                g_cubeList[i].ChangeDirection(x, y, z);
                return;
            }
        }
    }
    else if (id == "speed")
    {
        std::vector<std::string> elems;
        split(value, ':', elems);
        auto cubeName = elems[0];
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                float speed = (float)std::atof(elems[1].c_str());
                auto  velocity = g_cubeList[i].GetLinearVelocity() * speed;
                g_cubeList[i].SetLinearVelocity(velocity);
                return;
            }
        }
    }
    else if (id == "reset")
    {
        auto cubeName = value;
        for (int i = 0; i < g_cubeTotal; ++i)
        {
            if (g_cubeList[i].GetName() == cubeName)
            {
                auto velocity = g_cubeList[i].GetInitialLinearVelocity();
                g_cubeList[i].ResetPosition();
                g_cubeList[i].ResetOrientation();
                g_cubeList[i].SetLinearVelocity(velocity);
                return;
            }
        }
    }
}

Video Streaming

The tutorial demonstrates two different approaches for video streaming: automatic and manual. Auto-capture is the default setting.

To activate the capture, call the Genvid_SubmitVideoData() function with null parameters.

gs = Genvid_SubmitVideoData(tc, sStream_Video.c_str(), nullptr, 0);
assert(!GENVID_FAILED(gs));

Some situations may require using manual capture, such as if you’re not using DirectX11 or you don’t want to use the rendering thread to push the data to Genvid.

To use manual capture, you must set up the video frame using the correct parameters.

assert(description.Format == DXGI_FORMAT_R8G8B8A8_UNORM);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.pixel_format", GenvidPixelFormat_R8G8B8A8);
assert(gs == GenvidStatus_Success);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.width", description.Width);
assert(gs == GenvidStatus_Success);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.height", description.Height);
assert(gs == GenvidStatus_Success);

Then you can capture your buffer and submit it to Genvid.

// Figure out what we need on the first run.
if (g_pVideoBuffer == nullptr)
{
    // Hardcoded for DXGI_FORMAT_R8G8B8A8_UNORM
    g_NumBytes   = description.Height * description.Width * 4;
    g_pVideoBuffer = new uint8_t[g_NumBytes];
}
// On the following runs, check just in case.
else
{
    // Hardcoded for DXGI_FORMAT_R8G8B8A8_UNORM
    uint32_t TempNumBytes = description.Height * description.Width * 4;
    if (TempNumBytes != g_NumBytes)
    {
        delete[] g_pVideoBuffer;

        g_pVideoBuffer = new uint8_t[TempNumBytes];
        g_NumBytes   = TempNumBytes;
    }
}

uint8_t * pDestination   = g_pVideoBuffer;
uint8_t * pSource        = (uint8_t *)resource.pData;
uint32_t  NumSourceBytes = resource.RowPitch;
// Hardcoded for DXGI_FORMAT_R8G8B8A8_UNORM
uint32_t NumDestinationBytes = description.Width * 4;
uint32_t NumSourceLines      = description.Height;

for (uint32_t i = 0; i < NumSourceLines; ++i)
{
    memcpy(pDestination, pSource, NumDestinationBytes);
    pSource += NumSourceBytes;
    pDestination += NumDestinationBytes;
}

gs = Genvid_SubmitVideoData(tc, sStream_Video.c_str(), g_pVideoBuffer, g_NumBytes);
assert(!GENVID_FAILED(gs));

g_pImmediateContext->Unmap(pNewTexture, subresource);

Termination

In the TermGenvid() function, we perform every action necessary to properly clean up the Genvid SDK.

We need to unsubscribe every stream, event, and command we originally subscribed to. Afterwards, we destroy all the streams and finish with a Genvid_Terminate().

void TermGenvid()
{
    // Cancel command subscriptions.
    Genvid_UnsubscribeCommand(sCommand_reset.c_str(), &GenvidSubscriptionCommandCallback, nullptr);
    Genvid_UnsubscribeCommand(sCommand_direction.c_str(), &GenvidSubscriptionCommandCallback, nullptr);
    Genvid_UnsubscribeCommand(sCommand_speed.c_str(), &GenvidSubscriptionCommandCallback, nullptr);

    // Cancel event subscriptions.
    Genvid_Unsubscribe(sEvent_cheer.c_str(), &GenvidSubscriptionCallback, nullptr);
    Genvid_Unsubscribe(sEvent_reset.c_str(), &GenvidSubscriptionCallback, nullptr);
    Genvid_Unsubscribe(sEvent_changeColor.c_str(), &GenvidSubscriptionCallback, nullptr);

    // Destroy the streams.
    Genvid_DestroyStream(sStream_ColorChanged.c_str());
    Genvid_DestroyStream(sStream_Popularity.c_str());
    Genvid_DestroyStream(sStream_GameData.c_str());
    Genvid_DestroyStream(sStream_GameCopyright.c_str());
    Genvid_DestroyStream(sStream_Video.c_str());
    Genvid_DestroyStream(sStream_Audio.c_str());

    // Terminate the Genvid Native SDK.
    Genvid_Terminate();
}

Other Files

demo.fx	File used for the shaders.
demo.rc	File used for the resources.
demo_ps.hlsl	File used for the pixel shaders.
demo_vs.hlsl	File used for the vertex shaders.
directx.ico	Icon used for the demo executable.
Tutorial.sln	Visual Studio project solution file.
Tutorial.vcxproj	Visual Studio project settings file.
Tutorial.vcxproj.filters	Visual Studio project filters file.