genvid.toolbox.SDK

class genvid.toolbox.SDK(cluster_id='local', **kwargs)

Bases: genvid.toolbox.aws.AWSTool, genvid.toolbox.config.ConfigTool, genvid.toolbox.consul_template.ConsulTemplateTool, genvid.toolbox.vault.VaultTool, genvid.toolbox.consul.ConsulTool

Common commands for setting up the SDK.

This class wraps the ClusterAPI and BastionAPI for setting up a new Genvid SDK into the current cluster.

NAME = 'sdk'
SDK_CONFIG_VERSION = '1.7.0'
cluster_id = None

The Cluster ID.

bastionAPI = None

Interface to the Bastion-API instance.

clusterAPI = None

Interface to the cluster Cluster-API instance.

cluster_vault_secret = None

Key in the bastion’s Vault where the cluster vault keys are saved.

get_remote_cluster_conn()

Return the cluster IP and private key.

get_remote_cluster_secrets()

Returns the cluster vault initialization secrets.

setup(bucket: str = None, path: str = '/', *, region=None)

Starts the base SDK services.

Parameters:
  • bucket – The name of the bucket. Defaults to the return of get_s3_bucket_id().
  • path – The prefix key in the bucket for the images.
clean()

Stops the base SDK services, removes jobs and clean the config.

get_sdk_config() → dict

Get the default SDK configuration.

get_config() → dict

Get the current cluster configuration.

set_config(config: dict = None)

Merge the specified configuration with the current one. :param config: The configuration to merge.

clean_config()

Clean the configuration.

remove_config(config: dict)

Remove values from the configuration. :param config: The configuration to remove values from.

static update_dict_no_create(source, dest)
classmethod merge_dict(source, destination)

Merge 2 dictionaries.

Parameters:
  • source – The source dictionary.
  • destination – The destination dictionary.
is_jobs_running()

Return True if a job is configured to be running.

start_jobs(jobs: typing.Union[typing.List[str], NoneType] = None)

Start all jobs or the specified job. :param jobs: The list of jobs to start.

stop_jobs(jobs: typing.Union[typing.List[str], NoneType] = None)

Stop all jobs or the specified job.

Parameters:jobs – The list of jobs to stop.
load_config_template(template: typing.Union[pathlib.Path, str], env: dict = None) → dict

Load a config template file.

Render a configuration template file through consul-template, and load it.

The environment variable GENVID_CONFIG_FILENAME and GENVID_CONFIG_DIRNAME will be set before being run to respectively the basename of the template and its directory.

New in version 1.20.0.

update_config(config: dict = None)

Update the specified configuration.

Parameters:config – The configuration to update.
Returns:The updated configuration.
default_images_sdk_path

Get default S3 key prefix for SDK images.

New in version 1.20.0.

upload_images_sdk(bucket=None, path=None, update_config: bool = False, *, region=None)

Upload the images for the SDK.

Parameters:
  • bucket (str) – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Changed in version 1.14.0: Default path changes from ‘/’ to ‘/images/sdk-{version}/’.

Changed in version 1.20.0: New region parameter following the conventions in AWSTool.

get_s3_bucket_id()

Get the S3 bucket ID for images.

Note

This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.

update_images_config(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', *, delete: bool = False, region: typing.Union[str, NoneType] = None)

Update the images configuration on the cluster.

Parameters:
  • bucket – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • delete – Do not update configuration, delete it instead.
  • region – Region containing the S3 bucket.

Warning

This method can only be called on a non-local cluster.

New in version 1.20.0.

upload_images(prefixes: typing.List[str], bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', update_config: bool = False, basedir: typing.Union[pathlib.Path, str, NoneType] = None, *, region: typing.Union[str, NoneType] = None)

Upload the images with the specified prefixes.

Note

The image names must match DockerTool.RE_IMAGE_IMAGEID.

Parameters:
  • prefixes – Only upload the files with these prefixes.
  • bucket (str) – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • basedir – Source directory (default: ARTIFACTS_DIR).
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.

    Changed in version 1.20.0:

  • Added the basedir parameter.

  • Augmented error handling.

  • New region parameter following the conventions in AWSTool.

  • Factored the update_config part of the method into its own method.

get_images_config(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', *, region: typing.Union[str, NoneType] = None) → dict

Retrieve images configurations from an S3 bucket.

Parameters:
  • bucket – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix in the bucket (default: ‘/images/custom’).
  • region (str) – Region containing the S3 bucket.

Important

The image names must match DockerTool.RE_IMAGE_IMAGEID to be selected.

Returns:Images configurations as a dict.

Changed in version 1.14.0: Changed the default path from ‘/’ to ‘/images/custom’.

Changed in version 1.20.0: New region parameter following the conventions in AWSTool.

clean_images(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', update_config: bool = False, *, region: typing.Union[str, NoneType] = None)

Remove all images from an S3 bucket.

Parameters:
  • bucket – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.

    Changed in version 1.20.0:

  • New region parameter following the conventions in AWSTool.

  • Now delegates to update_images_config() for update_config.

update_archive(archive: typing.Union[pathlib.Path, str], version: str = 'dev', destdir: typing.Union[pathlib.Path, str, NoneType] = None, clean: bool = True, keep: bool = False)

Update a binary archive to conform to the image format expected.

Parameters:
  • archive – The path to the archive file. The filename should always be in the form of name.archfmt.
  • version – Version of the archive.
  • destdir – Destination directory. Default to the same as the original archive.
  • clean – Clean old versions of the archive (the ones with the same name).
  • keep – Keep the original archives. You should almost always used it with a different destdir.

New in version 1.20.0.