genvid.toolbox.SDK

class genvid.toolbox.SDK(cluster_id='local', **kwargs)

Bases: genvid.toolbox.aws.AWSTool, genvid.toolbox.config.ConfigTool, genvid.toolbox.consul_template.ConsulTemplateTool, genvid.toolbox.vault.VaultTool, genvid.toolbox.consul.ConsulTool

Common commands for setting up the SDK.

This class wraps the ClusterAPI and BastionAPI for setting up a new Genvid SDK into the current cluster.

NAME = 'sdk'
SDK_CONFIG_VERSION = '1.7.0'
cluster_id = None

The Cluster ID.

bastionAPI = None

Interface to the Bastion-API instance.

clusterAPI = None

Interface to the cluster Cluster-API instance.

cluster_vault_secret = None

Key in the bastion’s Vault where the cluster vault keys are saved.

get_remote_cluster_conn()

Return the cluster IP and private key.

get_remote_cluster_secrets()

Returns the cluster vault initialization secrets.

setup(bucket: str = None, path: str = '/', *, region=None)

Starts the base SDK services.

Parameters:
  • bucket – The name of the bucket. Defaults to the return of get_s3_bucket_id().
  • path – The prefix key in the bucket for the images.
clean()

Stops the base SDK services, removes jobs and clean the config.

get_sdk_config() → dict

Get the default SDK configuration.

get_config() → dict

Get the current cluster configuration.

set_config(config: dict = None)

Merge the specified configuration with the current one. :param config: The configuration to merge.

clean_config()

Clean the configuration.

remove_config(config: dict)

Remove values from the configuration. :param config: The configuration to remove values from.

static update_dict_no_create(source, dest)
classmethod merge_dict(source, destination)

Merge 2 dictionaries.

Parameters:
  • source – The source dictionary.
  • destination – The destination dictionary.
is_jobs_running()

Return True if a job is configured to be running.

start_jobs(jobs: typing.Union[typing.List[str], NoneType] = None)

Start all jobs or the specified job. :param jobs: The list of jobs to start.

stop_jobs(jobs: typing.Union[typing.List[str], NoneType] = None)

Stop all jobs or the specified job.

Parameters:jobs – The list of jobs to stop.
load_config_template(template: typing.Union[pathlib.Path, str], env: dict = None) → dict

Load a config template file.

Render a configuration template file through consul-template, and load it.

The environment variable GENVID_CONFIG_FILENAME and GENVID_CONFIG_DIRNAME will be set before being run to respectively the basename of the template and its directory.

New in version 1.20.0.

update_config(config: dict = None)

Update the specified configuration.

Parameters:config – The configuration to update.
Returns:The updated configuration.
default_images_sdk_path
Returns:Default S3 key prefix for SDK images.

New in version 1.20.0.

upload_images_sdk(bucket=None, path=None, update_config=None, *, region=None)

Upload the images for the SDK.

Parameters:
  • bucket (str) – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Note

The default value of update_config is True unless the cluster is local.

Changed in version 1.14.0: Default path changes from ‘/’ to ‘/images/sdk-{version}/’.

Changed in version 1.20.0: New region parameter following the conventions in AWSTool.

get_s3_bucket_id()
Returns:Get the S3 bucket ID for images.

Note

This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.

upload_images(prefixes: typing.List[str], bucket=None, path='/images/custom', update_config=None, basedir: typing.Union[pathlib.Path, str, NoneType] = None, *, region=None)

Upload the images with the specified prefixes.

Note

The image names must match DockerTool.RE_IMAGE_IMAGEID.

Parameters:
  • prefixes – Only upload the files with these prefixes.
  • bucket (str) – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • basedir – Source directory (default: ARTIFACTS_DIR).
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Note

The default value of update_config is True unless the cluster is local.

Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.

Changed in version 1.20.0: * Added the basedir parameter. * Augmented error handling. * New region parameter following the conventions in AWSTool.

get_images_config(bucket=None, path='/images/custom', *, region=None) → dict

Retrieve images configurations from an S3 bucket.

Parameters:
  • bucket – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix in the bucket (default: ‘/images/custom’).
  • region (str) – Region containing the S3 bucket.

Important

The image names must match DockerTool.RE_IMAGE_IMAGEID to be selected.

Returns:Images configurations as a dict.

Changed in version 1.14.0: Changed the default path from ‘/’ to ‘/images/custom’.

Changed in version 1.20.0: New region parameter following the conventions in AWSTool.

clean_images(bucket=None, path='/images/custom', update_config=None, *, region=None)

Remove all images from an S3 bucket.

Parameters:
  • bucket – Bucket ID (default: get_s3_bucket_id()).
  • path – Key prefix to use in the bucket.
  • update_config (bool) – Update cluster images configuration.
  • region (str) – Region containing the S3 bucket.

Warning

update_config can only be set for non-local clusters.

Note

The default value of update_config is True unless the cluster is local.

Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.

Changed in version 1.20.0: New region parameter following the conventions in AWSTool.

update_archive(archive: typing.Union[pathlib.Path, str], version: str = 'dev', destdir: typing.Union[pathlib.Path, str, NoneType] = None, clean: bool = True, keep: bool = False)

Update a binary archive to conform to the image format expected.

Parameters:
  • archive – The path to the archive file. The filename should always be in the form of name.archfmt.
  • version – Version of the archive.
  • destdir – Destination directory. Default to the same as the original archive.
  • clean – Clean old versions of the archive (the ones with the same name).
  • keep – Keep the original archives. You should almost always used it with a different destdir.

New in version 1.20.0.