genvid.toolbox.SDK¶
-
class
genvid.toolbox.
SDK
(cluster_id='', **kwargs)¶ Bases:
genvid.toolbox.aws.AWSTool
,genvid.toolbox.config.ConfigTool
,genvid.toolbox.consul_template.ConsulTemplateTool
,genvid.toolbox.vault.VaultTool
,genvid.toolbox.consul.ConsulTool
Common commands for setting up the SDK.
This class wraps the
ClusterAPI
andBastionAPI
for setting up a new Genvid SDK into the current cluster.-
NAME
= 'sdk'¶
-
SDK_CONFIG_VERSION
= '1.7.0'¶
-
cluster_id
= None¶ The Cluster ID.
-
bastionAPI
= None¶ Interface to the Bastion-API instance.
-
clusterAPI
= None¶ Interface to the cluster Cluster-API instance.
-
cluster_vault_secret
= None¶ Key in the bastion’s Vault where the cluster vault keys are saved.
-
azure
¶
-
get_remote_cluster_conn
()¶ Return the cluster IP and private key.
-
get_remote_cluster_secrets
()¶ Returns the cluster vault initialization secrets.
-
setup
(bucket: str = None, path: str = '/', *, region=None)¶ Starts the base SDK services.
Parameters: - bucket – The name of the bucket. Defaults to the return of
get_s3_bucket_id()
. - path – The prefix key in the bucket for the images.
- bucket – The name of the bucket. Defaults to the return of
-
setup_remote
(cluster_id: str, server_ip: str, ssh_private_key_pem_file: str, ssh_user: str)¶
-
clean
()¶ Stops the base SDK services, removes jobs and clean the config.
-
get_sdk_config
() → dict¶ Get the default SDK configuration.
-
get_config
() → dict¶ Get the current cluster configuration.
-
set_config
(config: dict = None)¶ Merge the specified configuration with the current one. :param config: The configuration to merge.
-
clean_config
()¶ Clean the configuration.
-
remove_config
(config: dict)¶ Remove values from the configuration. :param config: The configuration to remove values from.
-
static
update_dict_no_create
(source, dest)¶
-
classmethod
merge_dict
(source, destination)¶ Merge 2 dictionaries.
Parameters: - source – The source dictionary.
- destination – The destination dictionary.
-
is_jobs_running
()¶ Return True if a job is configured to be running.
-
start_jobs
(jobs: typing.Union[typing.List[str], NoneType] = None)¶ Start all jobs or the specified job. :param jobs: The list of jobs to start.
-
stop_jobs
(jobs: typing.Union[typing.List[str], NoneType] = None)¶ Stop all jobs or the specified job.
Parameters: jobs – The list of jobs to stop.
-
load_config_template
(template: typing.Union[pathlib.Path, str], env: typing.Union[typing.Mapping[str, str], NoneType] = None) → typing.Mapping[str, str]¶ Load a configuration template file.
Render a configuration template file using
consul-template
and returns it as a dictionary.New in version 1.20.0.
-
Changed in version 1.21.0:
Adjusted to use the new
ConsulTemplateTool
.
-
update_config
(config: dict = None)¶ Update the specified configuration.
Parameters: config – The configuration to update. Returns: The updated configuration.
-
default_images_sdk_path
¶ Get default S3 key prefix for SDK images.
New in version 1.20.0.
-
upload_images_sdk
(bucket=None, path=None, update_config: bool = False, cloud_provider: typing.Union[str, NoneType] = None, *, region=None)¶ Upload the images for the SDK.
Parameters: - bucket (str) – Bucket ID (default:
get_s3_bucket_id()
). - path – Key prefix to use in the bucket.
- update_config (bool) – Update cluster images configuration.
- region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Default path changes from ‘/’ to ‘/images/sdk-{version}/’.
Changed in version 1.20.0: New
region
parameter following the conventions inAWSTool
.-
Changed in version 1.26.0:
Added parameter
region
to target another region for the S3 bucket where the image will be uploaded.If the bucket, named after the Bastion Id, does not exist, it will be created in the targeted
region
.If it already exists but in a different region, the underlying code will raise an exception.
By default, the region used to create the bucket will be the region of the current AWS profile.
It can also be overriden by the AWS_DEFAULT_REGION environment variable.
If the
region
parameter is None and the bucket region already exists but its region differs from the one of the profile or AWS_DEFAULT_REGION environment variable, it will simply be used.
- bucket (str) – Bucket ID (default:
-
get_s3_bucket_id
()¶ Get the S3 bucket ID for images.
Note
This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.
-
get_s3_bucket_with_bastion_id
(bastion_id: str)¶ Get the S3 bucket ID for images using the current AWS account and the Bastion ID.
Note
This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.
-
update_images_config
(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', *, delete: bool = False, region: typing.Union[str, NoneType] = None)¶ Update the images configuration on the cluster.
Parameters: - bucket – Bucket ID (default:
get_s3_bucket_id()
). - path – Key prefix to use in the bucket.
- delete – Do not update configuration, delete it instead.
- region – Region containing the S3 bucket.
Warning
This method can only be called on a non-local cluster.
New in version 1.20.0.
- bucket – Bucket ID (default:
-
upload_images
(prefixes: typing.List[str], bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', update_config: bool = False, basedir: typing.Union[pathlib.Path, str, NoneType] = None, cloud_provider: typing.Union[str, NoneType] = None, *, region: typing.Union[str, NoneType] = None)¶ Upload the images with the specified prefixes.
Note
The image names must match
DockerTool.RE_IMAGE_IMAGEID
.Parameters: - prefixes – Only upload the files with these prefixes.
- bucket (str) – Bucket ID (default:
get_s3_bucket_id()
). - path – Key prefix to use in the bucket.
- update_config (bool) – Update cluster images configuration.
- basedir – Source directory (default:
ARTIFACTS_DIR
). - region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.
-
Changed in version 1.20.0:
Added the
basedir
parameter.Augmented error handling.
New
region
parameter following the conventions inAWSTool
.Factored the
update_config
part of the method into its own method.
-
Changed in version 1.26.0:
Added parameter
region
to target another region for the S3 bucket where the image will be uploaded.If the bucket, named after the Bastion Id, does not exist, it will be created in the targeted
region
.If it already exists but in a different region, the underlying code will raise an exception.
By default, the region used to create the bucket will be the region of the current AWS profile.
It can also be overriden by the AWS_DEFAULT_REGION environment variable.
If the
region
parameter is None and the bucket region already exists but its region differs from the one of the profile or AWS_DEFAULT_REGION environment variable, it will simply be used.
-
create_nats_window_package
()¶ Creates nats window package if it does not exists for toolbox version.
-
get_images_config
(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', *, region: typing.Union[str, NoneType] = None) → dict¶ - Retrieve images configurations from an S3 bucket or azure container
- based on cloud provider.
Parameters: - bucket – Bucket ID .
- path – Key prefix in the bucket (default: ‘/images/custom’).
- region (str) – Region containing the S3 bucket/Azure Container.
Important
The image names must match
DockerTool.RE_IMAGE_IMAGEID
to be selected.Returns: Images configurations as a dict
.Changed in version 1.14.0: Changed the default path from ‘/’ to ‘/images/custom’.
Changed in version 1.20.0: New
region
parameter following the conventions inAWSTool
.
-
clean_images
(bucket: typing.Union[str, NoneType] = None, path: str = '/images/custom', update_config: bool = False, *, region: typing.Union[str, NoneType] = None)¶ Remove all images from an S3 bucket.
Parameters: - bucket – Bucket ID (default:
get_s3_bucket_id()
). - path – Key prefix to use in the bucket.
- update_config (bool) – Update cluster images configuration.
- region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.
-
Changed in version 1.20.0:
New
region
parameter following the conventions inAWSTool
.Now delegates to
update_images_config()
forupdate_config
.
- bucket – Bucket ID (default:
-
update_archive
(archive: typing.Union[pathlib.Path, str], version: str = 'dev', destdir: typing.Union[pathlib.Path, str, NoneType] = None, clean: bool = True, keep: bool = False)¶ Update a binary archive to conform to the image format expected.
Parameters: - archive – The path to the archive file. The filename should always
be in the form of
name.archfmt
. - version – Version of the archive.
- destdir – Destination directory. Default to the same as the original archive.
- clean – Clean old versions of the archive (the ones with the same name).
- keep – Keep the original archives. You should almost always used
it with a different
destdir
.
New in version 1.20.0.
- archive – The path to the archive file. The filename should always
be in the form of
-
load_nomad_job_template
(name, job_folder: typing.Union[pathlib.Path, str])¶ Read nomad job templates from the specific directory. Files only with extension of .nomad.tmpl are read.
Parameters: - name – Name of the file without extension.
- job_folder – Path to directory containing jobs.
Changed in version 1.26.0: Add support for
pathlib.Path
as job_folder.
-
load_config
(source: typing.Union[pathlib.Path, str], *, job_template_dir: typing.Union[pathlib.Path, str, NoneType], with_consul_template: bool, dryrun: bool = False) → dict¶ Loads the configuration file(s) (all files in a directory or a specific file) from the source.
Parameters: - source – Path to the configuration file/folder.
- with_consul_template – Execute consul-template on the configuration file(s) (if source is a directory) before using it only if it is set to true.
- job_template_dir – Path to Folder from where nomad templates should be loaded. If not mentioned, no jobs will be loaded.
- dryrun – If True, only parse the source and returns the configuration.
Returns: The configuration as a dictionary that can be used directly with
set_config()
.- :raises FileNotFoundError : If the job_template_dir is not None and
- is not a valid directory.
Changed in version 1.26.0: Fix support for
pathlib.Path
for arguments. Add load parameter
-
get_output_public_ips
() → list¶ Retrieves all public ip’s from output of terraform module.
Returns: List of all public IP Addressses
-
get_cloud_provider
() → str¶ Retrieves the cloud provider from output of terraform module.
If no cloud provider is found in terraform’s cluster output, aws is returned as default.
Returns: Cloud provider.
-
-
class
sdk.
SDK
¶ Implementation of
genvid.toolbox.SDK