gateway_provisioners.cli package

Submodules

class gateway_provisioners.cli.base_app.BaseApp(**kwargs)

Bases: JupyterApp

Base class containing parameters common to each provisioner.

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

static import_item(name)

Import and return bar given the string foo.bar. Calling bar = import_item("foo.bar") is the functional equivalent of executing the code from foo import bar.

Return type:

Any

:param namestring

The fully qualified name of the module/package being imported.

:returns modmodule object

The module that was imported.

install_dir

A trait for unicode strings.

install_files()

Installs applicable files into the target location.

kernel_spec_dir_name

A trait for unicode strings.

kernel_spec_install: bool = True

kernel_spec_manager

A trait whose value must be an instance of a specified class.

The value can also be an instance of a subclass of the specified class.

Subclasses can declare default classes by overriding the klass attribute

launcher_dir_name

A trait for unicode strings.

log_and_exit(msg, exit_status=1)

Logs the msg as an error and exits with the given exit-status.

resource_dir_name

A trait for unicode strings.

start()

Drive the kernel specification creation.

toree_jar_path: str | None = None

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

class gateway_provisioners.cli.base_app.BaseSpecApp(**kwargs)

Bases: RemoteProvisionerConfigMixin, BaseApp

add_optional_config_entries(config_stanza)

Adds optional configuration parameters to the ‘config’ stanza of ‘kernel_provisioner’.

This method is overridden by subclasses which should call super().add_optional_config_entries().

Return type:

None

aliases: StrDict = {'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'unauthorized-users': 'BaseSpecApp.unauthorized_users'}

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

display_name

The display name of the kernel - used by user-facing applications.

flags: StrDict = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

install_files()

Assembles kernel-specs, launchers and resources into staging directory, then installs as kernel-spec.

ipykernel_subclass_name

For Python kernels, the name of the ipykernel subclass.

kernel_name

Install the kernel spec into a directory with this name.

language

The language of the kernel referenced in the kernel specification. Must be one of ‘Python’, ‘R’, or ‘Scala’. Default = ‘Python’.

prefix

Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/

replace

If a kernel specification already exists in the destination, allow for its replacement.

user

Try to install the kernel spec to the per-user directory instead of the system or environment directory.

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

class gateway_provisioners.cli.base_app.BaseSpecSparkApp(**kwargs)

Bases: BaseSpecApp

aliases: StrDict = {'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'extra-spark-opts': 'BaseSpecSparkApp.extra_spark_opts', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'spark-home': 'BaseSpecSparkApp.spark_home', 'spark-init-mode': 'BaseSpecSparkApp.spark_init_mode', 'unauthorized-users': 'BaseSpecApp.unauthorized_users'}

extra_spark_opts

Specify additional Spark options.

flags: StrDict = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'spark': ({'BaseSpecSparkApp': {'spark': True}}, 'Install kernelspec for use with Spark.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

spark

Install kernelspec for use with Spark.

spark_home

Specify where the spark files can be found.

spark_init_mode

Spark context initialization mode. Must be one of [‘lazy’, ‘eager’, ‘none’]. Default = lazy.

class gateway_provisioners.cli.docker_specapp.DockerProvisionerApp(**kwargs)

Bases: JupyterApp

Application responsible for driving the creation of Docker-based kernel specifications.

aliases: StrDict = {}

description: str | Unicode[str, str | bytes] = 'Application used to create kernel specifications for use on Docker or Docker Swarm clusters\n    via the DockerProvisioner or DockerSwarmProvisioner kernel provisioners.'

flags: StrDict = {}

name: str | Unicode[str, str | bytes] = 'jupyter docker-spec'

start()

Start the whole thing

subcommands: dict[str, t.Any] | Dict[str, t.Any] = {'install': (<class 'gateway_provisioners.cli.docker_specapp.DockerSpecInstaller'>, 'Creates a Jupyter kernel specification for use within a Docker or Docker Swarm cluster.')}

version: str | Unicode[str, str | bytes] = '0.5.0.dev0'

class gateway_provisioners.cli.docker_specapp.DockerSpecInstaller(**kwargs)

Bases: BaseSpecApp

CLI for extension management.

aliases: StrDict = {'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'image-name': 'DockerSpecInstaller.image_name', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'unauthorized-users': 'BaseSpecApp.unauthorized_users'}

description: str | Unicode[str, str | bytes] = 'Creates a Jupyter kernel specification for use within a Docker or Docker Swarm cluster.'

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

examples: str | Unicode[str, str | bytes] = '\n    jupyter-docker-spec install --language=R --kernel-name=r_docker --image_name=foo/my_r_kernel_image:v4_0\n\njupyter-docker-spec install --swarm --kernel-name=python_swarm\n    '

flags: StrDict = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'swarm': ({'DockerSpecInstaller': {'swarm': True}}, 'Install kernelspec for use within a Docker Swarm cluster.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

image_name

The kernel image to use for this kernel specification. If this specification is enabled for Spark usage, this image will be the driver image. (GP_IMAGE_NAME env var)

image_name_env = 'GP_IMAGE_NAME'

launcher_name

A trait for unicode strings.

name: str | Unicode[str, str | bytes] = 'jupyter-docker-spec'

provisioner_name

A trait for unicode strings.

swarm

Install kernelspec for use within a Docker Swarm cluster.

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

class gateway_provisioners.cli.image_bootstrapapp.ImageBootstrapApp(**kwargs)

Bases: JupyterApp

Application responsible for driving the creation of Kubernetes-based kernel specifications.

aliases: StrDict = {}

description: str | Unicode[str, str | bytes] = 'Application used to bootstrap kernel images with the appropriate\nkernel launchers for use by Remote Provisioners.'

flags: StrDict = {}

name: str | Unicode[str, str | bytes] = 'jupyter image-bootstrap'

start()

Start the whole thing

subcommands: dict[str, t.Any] | Dict[str, t.Any] = {'install': (<class 'gateway_provisioners.cli.image_bootstrapapp.ImageBootstrapInstaller'>, 'Installs the bootstrap script and kernel launchers for use within a kernel image used by Remote Provisioners.')}

version: str | Unicode[str, str | bytes] = '0.5.0.dev0'

class gateway_provisioners.cli.image_bootstrapapp.ImageBootstrapInstaller(**kwargs)

Bases: BaseApp

CLI for extension management.

aliases: StrDict = {'config': 'JupyterApp.config_file', 'languages': 'ImageBootstrapInstaller.languages', 'log-level': 'Application.log_level'}

description: str | Unicode[str, str | bytes] = 'Installs the bootstrap script and kernel launchers for use within a kernel image used by Remote Provisioners.'

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

examples: str | Unicode[str, str | bytes] = '\n    jupyter-image-bootstrap install --languages=Python --languages=R\n\njupyter-image-bootstrap install --languages=Python --languages=Scala\n    '

flags: StrDict = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'generate-config': ({'JupyterApp': {'generate_config': True}}, 'generate default config file'), 'launchers-only': ({'ImageBootstrapInstaller': {'launchers_only': True}}, 'Only install kernel launchers, no bootstrap script.'), 'show-config': ({'Application': {'show_config': True}}, "Show the application's configuration (human-readable format)"), 'show-config-json': ({'Application': {'show_config_json': True}}, "Show the application's configuration (json format)"), 'y': ({'JupyterApp': {'answer_yes': True}}, 'Answer yes to any questions instead of prompting.')}

install_files()

Sets up the bootstrap-kernel.sh and corresponding kernel-launchers for use in kernel images

kernel_spec_install: bool = False

languages

The languages corresponding to the kernel-launchers to install into the kernel image. All values must be one of ‘Python’, ‘R’, or ‘Scala’.

launchers_only

Only install kernel launchers, no bootstrap script.

name: str | Unicode[str, str | bytes] = 'jupyter-image-bootstrap'

class gateway_provisioners.cli.k8s_specapp.K8sProvisionerApp(**kwargs)

Bases: JupyterApp

Application responsible for driving the creation of Kubernetes-based kernel specifications.

aliases: StrDict = {}

description: str | Unicode[str, str | bytes] = 'Application used to create kernel specifications for use on Kubernetes clusters.'

flags: StrDict = {}

name: str | Unicode[str, str | bytes] = 'jupyter k8s-spec'

start()

Start the whole thing

subcommands: dict[str, t.Any] | Dict[str, t.Any] = {'install': (<class 'gateway_provisioners.cli.k8s_specapp.K8sSpecInstaller'>, 'Creates a Jupyter kernel specification for use within a Kubernetes cluster.')}

version: str | Unicode[str, str | bytes] = '0.5.0.dev0'

class gateway_provisioners.cli.k8s_specapp.K8sSpecInstaller(**kwargs)

Bases: BaseSpecSparkApp

CLI for extension management.

aliases: StrDict = {'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'executor-image-name': 'K8sSpecInstaller.executor_image_name', 'extra-spark-opts': 'BaseSpecSparkApp.extra_spark_opts', 'image-name': 'K8sSpecInstaller.image_name', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'spark-home': 'BaseSpecSparkApp.spark_home', 'spark-init-mode': 'BaseSpecSparkApp.spark_init_mode', 'unauthorized-users': 'BaseSpecApp.unauthorized_users'}

crd

Install kernelspec for use with a Custom Resource Definition. When combined with –spark, will configure the SparkOperatorProvisioner for Spark Application CRDs.

description: str | Unicode[str, str | bytes] = 'Creates a Jupyter kernel specification for use within a Kubernetes cluster.'

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

examples: str | Unicode[str, str | bytes] = '\n    jupyter-k8s-spec install --language=R --kernel-name=r_k8s --image-name=foo/my_r_kernel_image:v4_0\n\njupyter-k8s-spec install --language=Scala --spark --kernel-name=scala_k8s_spark --display-name=\'Scala on Kubernetes with Spark\'\n\njupyter-k8s-spec install --spark --crd --display-name=\'Python SparkOperator"\n    '

executor_image_name

The executor image to use for this kernel specification. Only applies to Spark-enabled kernel specifications. (GP_EXECUTOR_IMAGE_NAME env var)

executor_image_name_env = 'GP_EXECUTOR_IMAGE_NAME'

flags: StrDict = {'crd': ({'K8sSpecInstaller': {'crd': True}}, 'Install kernelspec for use with a Custom Resource Definition.  When combined with --spark,\nwill configure the SparkOperatorProvisioner for Spark Application CRDs.'), 'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'spark': ({'K8sSpecInstaller': {'spark': True}}, 'Install kernelspec for use with Spark.  When combined with --crd,\nwill configure the SparkOperatorProvisioner for Spark Application CRDs.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'tensorflow': ({'K8sSpecInstaller': {'tensorflow': True}}, 'Install kernelspec for use with Tensorflow.'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

image_name

The kernel image to use for this kernel specification. If this specification is enabled for Spark usage, this image will be the driver image. (GP_IMAGE_NAME env var)

image_name_env = 'GP_IMAGE_NAME'

launcher_name

A trait for unicode strings.

name: str | Unicode[str, str | bytes] = 'jupyter-k8s-spec'

provisioner_name

A trait for unicode strings.

spark

Install kernelspec for use with Spark. When combined with –crd, will configure the SparkOperatorProvisioner for Spark Application CRDs.

tensorflow

Install kernelspec for use with Tensorflow.

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

class gateway_provisioners.cli.ssh_specapp.SshProvisionerApp(**kwargs)

Bases: JupyterApp

Application responsible for driving the creation of Ssh-based kernel specifications.

aliases: StrDict = {}

description: str | Unicode[str, str | bytes] = 'Application used to create kernel specifications for use on clusters via SSH\n    and the DistributedProvisioner kernel provisioner.'

flags: StrDict = {}

name: str | Unicode[str, str | bytes] = 'jupyter ssh-spec'

start()

Start the whole thing

subcommands: dict[str, t.Any] | Dict[str, t.Any] = {'install': (<class 'gateway_provisioners.cli.ssh_specapp.SshSpecInstaller'>, 'Creates a Jupyter kernel specification for use within a cluster of hosts via SSH.')}

version: str | Unicode[str, str | bytes] = '0.5.0.dev0'

class gateway_provisioners.cli.ssh_specapp.SshSpecInstaller(**kwargs)

Bases: BaseSpecSparkApp

CLI for extension management.

add_optional_config_entries(config_stanza)

Adds optional configuration parameters to the ‘config’ stanza of ‘kernel_provisioner’.

This method is overridden by subclasses which should call super().add_optional_config_entries().

Return type:

None

aliases: StrDict = {'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'extra-spark-opts': 'BaseSpecSparkApp.extra_spark_opts', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'remote-hosts': 'SshSpecInstaller.remote_hosts', 'spark-home': 'BaseSpecSparkApp.spark_home', 'spark-init-mode': 'BaseSpecSparkApp.spark_init_mode', 'spark-master': 'SshSpecInstaller.spark_master', 'unauthorized-users': 'BaseSpecApp.unauthorized_users'}

description: str | Unicode[str, str | bytes] = 'Creates a Jupyter kernel specification for use within a cluster of hosts via SSH.'

examples: str | Unicode[str, str | bytes] = "\n    jupyter-ssh-spec install --language R --spark --spark-master spark://192.168.2.5:7077 --spark-home=/usr/local/spark\n\njupyter-ssh-spec install --kernel-name ssh_python --remote-hosts=192.168.2.4 --remote-hosts=192.168.2.5\n\njupyter-ssh-spec install --language Scala --spark --spark_init_mode 'eager'\n    "

flags: StrDict = {'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'spark': ({'BaseSpecSparkApp': {'spark': True}}, 'Install kernelspec for use with Spark.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

name: str | Unicode[str, str | bytes] = 'jupyter-ssh-spec'

remote_hosts

List of host names on which this kernel can be launched. Multiple entries must each be specified via separate options: –remote-hosts host1 –remote-hosts host2

remote_hosts_default_value = 'localhost'

remote_hosts_env = 'GP_REMOTE_HOSTS'

spark_master

PORT’ or ‘yarn’.

Type:

Specify the Spark Master URL (e.g., ‘spark

Type:

//HOST

spark_master_default_value = 'yarn'

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

class gateway_provisioners.cli.yarn_specapp.YarnProvisionerApp(**kwargs)

Bases: JupyterApp

Application responsible for driving the creation of Yarn-based kernel specifications.

aliases: StrDict = {}

description: str | Unicode[str, str | bytes] = 'Application used to create kernel specifications for use on Hadoop Yarn clusters\n    via the YarnProvisioner kernel provisioner.'

flags: StrDict = {}

name: str | Unicode[str, str | bytes] = 'jupyter yarn-spec'

start()

Start the whole thing

subcommands: dict[str, t.Any] | Dict[str, t.Any] = {'install': (<class 'gateway_provisioners.cli.yarn_specapp.YarnSpecInstaller'>, 'Creates a Jupyter kernel specification for use within a Hadoop Yarn cluster.')}

version: str | Unicode[str, str | bytes] = '0.5.0.dev0'

class gateway_provisioners.cli.yarn_specapp.YarnSpecInstaller(**kwargs)

Bases: BaseSpecSparkApp

CLI for extension management.

add_optional_config_entries(config_stanza)

Adds optional configuration parameters to the ‘config’ stanza of ‘kernel_provisioner’.

This method is overridden by subclasses which should call super().add_optional_config_entries().

Return type:

None

aliases: StrDict = {'alt-yarn-endpoint': 'YarnSpecInstaller.alt_yarn_endpoint', 'authorized-users': 'BaseSpecApp.authorized_users', 'config': 'JupyterApp.config_file', 'display-name': 'BaseSpecApp.display_name', 'extra-dask-opts': 'YarnSpecInstaller.extra_dask_opts', 'extra-spark-opts': 'BaseSpecSparkApp.extra_spark_opts', 'ipykernel-subclass-name': 'BaseSpecApp.ipykernel_subclass_name', 'kernel-name': 'BaseSpecApp.kernel_name', 'language': 'BaseSpecApp.language', 'launch-timeout': 'BaseSpecApp.launch_timeout', 'log-level': 'Application.log_level', 'port-range': 'BaseSpecApp.port_range', 'prefix': 'BaseSpecApp.prefix', 'python-root': 'YarnSpecInstaller.python_root', 'spark-home': 'BaseSpecSparkApp.spark_home', 'spark-init-mode': 'BaseSpecSparkApp.spark_init_mode', 'unauthorized-users': 'BaseSpecApp.unauthorized_users', 'yarn-endpoint': 'YarnSpecInstaller.yarn_endpoint'}

alt_yarn_endpoint

The http url specifying the alternate YARN Resource Manager. This value should be set when YARN Resource Managers are configured for high availability. Note: If both YARN endpoints are NOT set, the YARN library will use the files within the local HADOOP_CONFIG_DIR to determine the active resource manager. (GP_ALT_YARN_ENDPOINT env var)

alt_yarn_endpoint_env = 'GP_ALT_YARN_ENDPOINT'

dask

Kernelspec will be configured for Dask YARN.

description: str | Unicode[str, str | bytes] = 'Creates a Jupyter kernel specification for use within a Hadoop Yarn cluster.'

detect_missing_extras()

Issues a warning message whenever an “extra” library is detected as missing.

Note that “extra” can also mean things like Apache Toree is not installed when the language is Scala, or Rscript is not available when the language is R.

examples: str | Unicode[str, str | bytes] = '\n    jupyter-yarn-spec install --language=R --spark-home=/usr/local/spark\n\njupyter-yarn-spec install --kernel-name=dask_python --dask --yarn-endpoint=http://foo.bar:8088/ws/v1/cluster\n\njupyter-yarn-spec install --language=Scala --spark-init-mode eager\n    '

extra_dask_opts

Specify additional Dask options.

flags: StrDict = {'dask': ({'YarnSpecInstaller': {'dask': True}}, 'Kernelspec will be configured for Dask YARN.'), 'debug': ({'Application': {'log_level': 10}}, 'set log level to logging.DEBUG (maximize logging output)'), 'impersonation-enabled': ({'YarnSpecInstaller': {'impersonation_enabled': True}}, 'Indicates whether impersonation will be performed during kernel launch.\n(GP_IMPERSONATION_ENABLED env var)'), 'replace': ({'BaseSpecApp': {'replace': True}}, 'If a kernel specification already exists in the destination, allow for its replacement.'), 'spark': ({'BaseSpecSparkApp': {'spark': True}}, 'Install kernelspec for use with Spark.'), 'sys-prefix': ({'BaseSpecApp': {'prefix': '/home/docs/checkouts/readthedocs.org/user_builds/gateway-provisioners/envs/latest'}}, 'Specify a prefix to install to, e.g. an env. The kernelspec will be installed in PREFIX/share/jupyter/kernels/'), 'user': ({'BaseSpecApp': {'user': True}}, 'Try to install the kernel spec to the per-user directory instead of the system or environment directory.'), 'yarn-endpoint-security-enabled': ({'YarnSpecInstaller': {'yarn_endpoint_security_enabled': True}}, 'Is YARN Kerberos/SPNEGO Security enabled (True/False).\n(GP_YARN_ENDPOINT_SECURITY_ENABLED env var)')}

get_substitutions(install_dir)

Gather substitution strings to inject into the templated files.

This method is overridden by subclasses which should first call super().get_substitutions().

Return type:

dict

impersonation_enabled

Indicates whether impersonation will be performed during kernel launch. (GP_IMPERSONATION_ENABLED env var)

impersonation_enabled_env = 'GP_IMPERSONATION_ENABLED'

name: str | Unicode[str, str | bytes] = 'jupyter-yarn-spec'

python_root

Specify where the root of the python installation resides (parent dir of bin/python).

validate_parameters()

Validate input parameters and prepare for their injection into templated files.

This method is overridden by subclasses which should call super().validate_parameters().

yarn_endpoint

If this value is NOT set, the YARN library will use the files within the local HADOOP_CONFIG_DIR to determine the active resource manager. (GP_YARN_ENDPOINT env var)

Type:

The http url specifying the YARN Resource Manager. Note

yarn_endpoint_env = 'GP_YARN_ENDPOINT'

yarn_endpoint_security_enabled

Is YARN Kerberos/SPNEGO Security enabled (True/False). (GP_YARN_ENDPOINT_SECURITY_ENABLED env var)

yarn_endpoint_security_enabled_default_value = False

yarn_endpoint_security_enabled_env = 'GP_YARN_ENDPOINT_SECURITY_ENABLED'

Module contents