Supported contexts
Context is the way to connect to the remote server. One needs to set context_type
to one of the following values:
LazyLocal
context_type
: LazyLocal
LazyLocal
directly runs jobs in the local server and local directory.
Since bash -l
is used in the shebang line of the submission scripts, the login shell startup files will be executed, potentially overriding the current environmental variables. Therefore, it’s advisable to explicitly set the environmental variables using envs
or source_list
.
Local
context_type
: Local
Local
runs jobs in the local server, but in a different directory. Files will be copied to the remote directory before jobs start and copied back after jobs finish.
Since bash -l
is used in the shebang line of the submission scripts, the login shell startup files will be executed, potentially overriding the current environmental variables. Therefore, it’s advisable to explicitly set the environmental variables using envs
or source_list
.
SSH
context_type
: SSH
SSH
runs jobs in a remote server. Files will be copied to the remote directory via SSH channels before jobs start and copied back after jobs finish. To use SSH, one needs to provide necessary parameters in remote_profile
, such as username
and hostname
.
It’s suggested to generate SSH keys and transfer the public key to the remote server in advance, which is more secure than password authentication.
Note that SSH
context is non-login, so bash_profile
files will not be executed outside the submission script.
Bohrium
context_type
: Bohrium
Bohrium is the cloud platform for scientific computing. Read Bohrium documentation for details. To use Bohrium, one needs to provide necessary parameters in remote_profile
.
HDFS
context_type
: HDFS
The Hadoop Distributed File System (HDFS) is a distributed file system. Read Support DPDispatcher on Yarn for details.
OpenAPI
context_type
: OpenAPI
OpenAPI is a new way to submit jobs to Bohrium. It using AccessKey instead of username and password. Read Bohrium documentation for details. To use OpenAPI, one needs to provide necessary parameters in remote_profile
.