Remote Scripts

Remote script actions allow you to execute a script on a remote server or servers as part of a job. These actions can be useful in the following situations:

  • Adding user accounts to newly provisioned servers
  • Executing Puppet run when server provisioning completes
  • Changing passwords when a server expires
  • Performing advanced configurations when the action is run as part of a blueprint deployment

Resource Technology Support Matrix

Running remote scripts on servers is supported as described in this support matrix. Be aware that some OSes and technologies require additional steps to set up, described later in this document.

Resource Technology *nix (SSH/SCP) 1 Windows (WinRM) 2 Via Resource Handler 3
Private Clouds:      
VMware vSphere YES YES YES 3
OpenStack YES YES NO
Hyper-V YES 4 YES 4 NO
Nutanix Acropolis YES YES NO
IPMI YES YES NO
Other Private Clouds NO NO NO
Public Clouds:      
AWS YES YES NO
Azure Classic NO NO NO
Azure YES YES NO
Google Compute YES YES NO
IBM SoftLayer YES YES NO
CenturyLink Cloud YES YES NO
Oracle Compute Cloud YES YES NO
Other Public Clouds NO NO NO
  • 1: *nix-based remote script execution relies on scp’ing scripts to the VM and then using ssh to execute them. See *nix Servers using SSH for setup instructions.
  • 2: Windows-based remote script execution relies on native WinRM. See Windows Servers using WinRM for setup instructions.
  • 3: VMware supports executing scripts on VMs using VMware Tools. See Configuring Remote Script Method for setup instructions.
  • 4: Hyper-V supports fetching the IP address of Windows VMs, but Linux VMs require special configuration depending on your OS. CloudBolt will fetch the IP address of the Hyper-V guest VM so long as Hyper-V knows it.

Writing Remote Scripts

Scripts can be written in any scripting language as long as that program is installed and can be executed on the guest VM. For Windows machines, this means the Execution Policy must be set in a way that allows the script to be run. The only caveat for *nix OS builds is that the script must start with a “shebang”. For example, a bash script needs to begin with #!/bin/bash or the more cross-compatible #!/usr/bin/env bash. Note that when using the #!/usr/bin/env approach, the desired program must be in the PATH. “Shebangs” are also used to pass program arguments, even in PowerShell. For instance to escalate execution priviledges at run time one can start a PowerShell script with: #!C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy UnRestricted -File.

Remote Scripts Parameterization

Whenever the script’s content is modified (either a new file is uploaded or the remote file at the specified URL changes), CloudBolt will scan the file contents for template style variables like {{ variable }} and will automatically create action inputs for each variable it discovers.

When using an action input that allows multiple values, you’ll need to specify how these values should be formatted using the “join” filter. For example, if you wanted a list of arguments separated with spaces, your script would look something like this:

echo {{ arguments|join:' ' }}

In addition to the script file itself, Remote Scripts also support parameterization of the Command Line Arguments attribute. Parameterized command line arguments are not automatically associated with action parameter inputs. See Action Input Parameters for more details on auto generated action inputs and Action Context for a complete reference list of objects available to Remote Scripts on different contexts.

Expected Output and Return Codes

CloudBolt expects each script to return a zero exit code when it is successful. If the script returns non-zero or the script otherwise fails to execute, the CloudBolt job will be marked as a failure.

Any output from a remotely run script will be logged to the job progress messages and the job log file.

Configuring Remote Script Method

By default, servers on a Resource Handler with a technology-specific mechanism for running scripts, such as VMware tools, will have their remote scripts run with that method. However, CloudBolt can be configured so that instead any server will have its remote scripts run with the generic method of WinRM for Windows and SSH for Unix-like servers. To change which mechanism is used for specific servers, add the Use Tech-Specific Script Execution parameter to servers, environments, etc. as appropriate and give it a value of False in order to use WinRM/SSH. To alter this behavior globally, you can toggle the TECH_SPECIFIC_SCRIPT_EXECUTION value in customer_settings.py between True and False. Setting it to False will cause all remote scripts, even on VMware, to execute using WinRM/SSH, unless the server has the parameter, whose value overrides the global setting.

When creating or editing a remote script, there is an option to configure it to run with sudo on *nix. This causes the entire script to run with elevated root privileges, and can be useful on VMs that do not allow direct use of the root user, as one example. Note that it only applies to scripts run with SSH, meaning that it does not affect VMware VMs when they are set to use VMware tools as explained above.

When remote scripts are run via SSH or VMware Tools, a temporary file is created on the target VM. By default, that temporary file is deleted after the script runs. However, leaving it on the target VM can be useful for debugging purposes, so that behavior can be controlled by toggling the Remove after running boolean on the script’s detail page.

For *nix VMs, the default directory where temporary files are written is /tmp/. For Windows VMs running remote scripts via VMware Tools, it is C:\Windows\Temp\. Both of those defaults can be modified in Admin > Miscellaneous Settings. Note: using a tilde ~/ in the path is not supported.

Authentication Requirements

CloudBolt must be able to authenticate with the remote guest in order to interact with it. This can be done with either a password or SSH key.

Passwords

By default, VMware and CenturyLink Cloud rely on passwords for accessing VMs. Authentication with these servers requires the password and sometimes the username of the server.

For a server that has already been provisioned, you can view the credentials it has set on its details page, and update them to match the VM’s actual configuration if necessary. Prior to provisioning, you can set the credentials on the template that will be used, or set the “Password” and potentially “Username” parameters on the group, the environment, or globally. These credentials will be associated with the server during provisioning, so they can be used to run remote scripts either at provision time or later. Note that the username must be provided if the username is not ‘ubuntu’ for Ubuntu servers, ‘root’ for all other *nix distributions, or ‘Administrator’ for Windows.

It is also possible to configure credentials on a particular remote script action itself, via its details page. Any username or password set on the action will override a username or password set on the server when that action is run.

When a remote script is run as a server action, it will also prompt for credentials so you can enter the desired username and password to run as right there. For an individual server action, any credentials stored on the remote script action or server in CloudBolt will be the default (with the script taking precedence), but you can use different ones. Entered credentials can be stored on the server for future use.

SSH Keys

By default, AWS, GCE and Oracle rely on SSH keys for accessing VMs. To authenticate with these servers and run scripts, CloudBolt needs to have the key material for each keypair used to build new servers. In the case of these Resource Handlers, CloudBolt gets a list of SSH keys from the handler, which can be selected when provisioning. To make the key material for these RH-specific keys available to CB, visit the SSH Keys tab of the Resource Handler and use the pencil icon to upload key material for the pertinent key.

CloudBolt also supports using arbitrary, generic/ global SSH keys for running remote scripts with SSH on VMs on any Resource Handler. The user is responsible for ensuring that a selected global key is valid for connecting to a given server; neither CloudBolt nor the Resource Handler will do anything to configure the VM to accept the key. These global keys can be managed from Admin > SSH Keys.

Whenever uploading key material to CloudBolt, make sure it is raw private key material that has not been encrypted with a passphrase.

When setting SSH keys as part of the credentials on either a template (to be propagated to servers at provisioning) or a previously-provisioned server, as long as it’s not Windows or a server that will run scripts with VMware tools, you will be able to choose from any of the global keys or the set of keys obtained from the Resource Handler and associated with the Environment, if any.

As with usernames and passwords, SSH keys can be associated with a particular remote script action, via its details page. Only global SSH keys can be chosen for consistent use with an action. A key set on a script will override a key set on the server when that script is run.

AWS Windows

Windows VMs on AWS are a special case where you can instead give a password for running remote scripts. This can be provided as described in the Passwords section. Either a password or SSH key may be given, depending on the VM and desired behavior. If a password is provided, CloudBolt will simply use it. If not, CloudBolt will attempt to obtain the Administrator password from EC2 using the provided SSH key, but that operation will only succeed if the VM is configured correctly in EC2 for get_password_data to be successful.

Google Compute Engine

There are a couple unique requirements to run remote scripts on GCE VMs:

  • In the GCE UI, it must be configured with some project-wide SSH keys that will apply to all new VMs. This list of SSH keys will be available to CB. Information from Google on how to do this can be found here.
  • The VM must have been provisioned from CB, with one of those SSH keys selected.
  • The appropriate username for the selected SSH key must be used.

For Windows VMs, a newly-provisioned server will not have a password set. However, a password can be generated and set on the server by running the Generate Windows Password on GCE server action, available in the Content Library.

OS-Specific Requirements

Some operating systems require specific configuration in the template or environment to work properly.

*nix Servers using SSH

In order to run remote scripts using SSH:

  • The VM’s IP address must be routable from the CloudBolt server. Often this involves use of a public IP on public cloud environments.
  • CloudBolt must know the IP address of the server, usually because it was provisioned with CloudBolt or the Resource Handler is able to provide it to CloudBolt.
  • TCP port 22 must be open. You can test this by SSH’ing from the CB server to your VM.
  • sshd must be installed and running on the VM.
  • openssh-clients (on CentOS/RHEL), openssh-client (on Debian/Ubuntu), or the equivalent package that provides the scp client must be installed on the VM.

Windows Servers using WinRM

Running scripts on Windows servers on most technologies uses WinRM (Windows Remote Management). CloudBolt can also be configured to run scripts on VMware Windows servers with WinRM rather than VMware tools, as described above.

  • The VM’s IP address must be routable from the CloudBolt server. Often this involves use of a public IP on public cloud environments.
  • CloudBolt must know the IP address of the server, usually because it was provisioned with CloudBolt or the Resource Handler is able to provide it to CloudBolt.
  • For any server using WinRM, firewalls and permissions within the guest must be configured to allow connections via WinRM.
  • From the guest Windows OS, launch “Local Group Policy user interface” by typing gpedit.msc in the run prompt.
  • Navigate to “Local Computer Policy > Computer Configuration > Administrative Templates > Windows Components > Windows Remote Management (WinRM) > WinRM Client”.
  • Enable “Trust the remote machine.” In the settings window, add the CloudBolt server name in the “trustedhostedlist” field.
  • Enable “Allow unencrypted traffic” under the “WinRM Client” menu.
  • Enable “Allow basic authentication” and “Allow unencrypted traffic” under the “WinRM Service” menu.
  • Additionally open the Control Panel, go to System and Security > Check Firewall Status > Advanced Settings > Inbound Rules and search for ‘windows remote management’ (HTTP-in). Double click it, go to advanced and make sure public is enabled/checked.
  • Open PowerShell in the guest Windows OS and run Configure-SMRemoting. Answer yes to all the prompts to make the VM fully ready for WinRM access.
  • Add the server to a security group that allows TCP connections on ports 5985 and 5986. In the case of an AWS server, that means going to the Amazon EC2 console and adding the server to an appropriate security group. In the case of Azure, those ports should be open by default in the server’s Network Security Group inbound security rules.

Using WinRM with Encryption

If you want CloudBolt to always use encryption when executing remote scripts with WinRM then you must add the following entry to your /var/opt/cloudbolt/proserv/customer_settings.py: ENCRYPT_WINRM = True. Please note that this setting affects all WinRM based remote script executions, including the interactions between CloudBolt and any Hyper-V resource handlers that might have been configured.

At this time, command line arguments are not supported when running scripts on Windows servers.

Examples

Below are some simple examples of remote script actions.

Expiring User Passwords

Replacing the value of cbuser to the desired username will expire that user’s password and require them to set a new one at next login.

#!/bin/bash

chage -d 0 cbuser

Joining a Windows Server to a Given OU

When run on a Windows server this script will be executed using Windows PowerShell. It will join the server to the given domain’s OU.

$domain = "lab.iad.cloudbolt.com"
$ou = "ou=cb_users,dc=lab,dc=iad,dc=cloudbolt,dc=com"
$duser = "LAB_USER\Administrator"
$dpwd = "AdminPasswd"
$creds = New-Object System.Management.Automation.PSCredential($duser, (ConvertTo-SecureString $dpwd -AsPlainText -Force))

Add-Computer -DomainName $domain -OUPath $ou -Credential $creds